How to load the weight weight from checkpoint while we didn't self.save_hyparams(.) ?

[HuangChiEn]

As the title, i wonder what I should do to only load the model weight as the starting point of finetuneing.
Firstly, I can not call self.hyparams(.) to save the model arguments, or it will stuck train.fit procedure without any error message. So, I need to first initialize the random weight modelA and modelB, and hope the checkpoint will automatically load the pretrained weight in it when I call Dummy_framework.load_from_checkpoint(.).
Otherwise, I will get the error about xxx.__init__() missing xxx required positional arguments: modelA, modelB, ...

Secondly, I try to trust the automatic saving mechanism (without any callback, any self.hyparams, torch.save something), and I can find the checkpoint under the folder. So, I don't know what the checkpoint contains, but I assume it contains everything including model weights.

when I execute the following code, it doesn't give any error and run smoothly, but I afraid the pre-trained weight doesn't be loaded in modelA and modelB.

dummy_modelA, dummy_modelB = get_rand_init_weight_models(cfger.model_params)
tmp = Dummy_framework.load_from_checkpoint(
    cfger.test_params['PATH'], modelA=dummy_modelA, modelB=dummy_modelB,
    **cfger.train_params  # dummy args
)
trainer.fit(tmp, dataloaders=tra_ld)

Since the torch-lightning document described :

# if you train and save the model like this it will use these values when loading
# the weights. But you can overwrite this
LitModel(in_dim=32, out_dim=10)

# uses in_dim=32, out_dim=10
model = LitModel.load_from_checkpoint(PATH)

# uses in_dim=128, out_dim=10
model = LitModel.load_from_checkpoint(PATH, in_dim=128, out_dim=10)

So, will the pretrained model be overrides by the random init modelA, modelB weights ?
As I did this :

Dummy_framework.load_from_checkpoint(cfger.test_params['PATH'], 
# I feed the dummy model into it, to prevent the error message I described in First point.
modelA=dummy_modelA, modelB=dummy_modelB
)

Besides, if I want to conduct finetuneing, how can I only load the pretrained weight (modelA, modelB) and discard the other info, including lr states, epochs states, ... ?

Any suggestion will be appreciated!!

( PS. the PL document in Checkpoint Loading seems not helpful in this case

[flashszn]

I have a similar question

#19312

waiting for answers.

[flashszn]

So, will the pretrained model be overrides by the random init modelA, modelB weights ?

I think no.
The lightning module load_from_checkpoint method is like this:

@_restricted_classmethod
def load_from_checkpoint(
    cls,
    checkpoint_path: Union[_PATH, IO],
    map_location: _MAP_LOCATION_TYPE = None,
    hparams_file: Optional[_PATH] = None,
    strict: bool = True,
    **kwargs: Any,
) -> Self:

and the discription of strict is: strict: Whether to strictly enforce that the keys in :attr:`checkpoint_path` match the keys returned by this module's state dict.
Since the your lightning module ckpt contains both two models (you can print your ckpt to check this), and default strict is True, the load_from_checkpoint method will strictly load every keys in :attr:`checkpoint_path` match the keys returned by this module's state dict.

[HuangChiEn]

thanks for your reply ~

[flashszn]

thanks for your reply ~

Nevermind. If the answer is right, you can mark it as this discussion's answer. ʕ ᵔᴥᵔ ʔ

[HuangChiEn]

Sorry for reply lately.
Sure, I'm glad to do that while leaving some minor additions.

---

In load_from_checkpoint (in class ModelIO), it will call _load_model_state finally. It will first init model and then model.call load_state_dict as torch old fashion ~
then we can ensure the pre-trained model weight saving in checkpoint with state_dict will be read in the model correctly ~

code trace :
https://github.com/Lightning-AI/pytorch-lightning/blob/b40de5464a953ff5866a255f4670d318bd8fd65a/pytorch_lightning/core/saving.py#L190C9-L192C71

    # class ModelIO(object)
    @classmethod
    def _load_model_state(cls, checkpoint: Dict[str, Any], *cls_args, strict: bool = True, **cls_kwargs):
      ...
      model = cls(*cls_args, **cls_kwargs)
      # load the state_dict on the model automatically
      model.load_state_dict(checkpoint['state_dict'], strict=strict)
      ...

also TKS @flashszn reply !!