The problem I am faced with is to be able to fit a series of models starting with progressively different inputs.
I am using the R interface to torch for this, which is based on pytorch. I have to use R for this because this neural network will work in collaboration with a mixture regression framework in R (flexmix) which implements an idea which does not seem to exist in the python ecosystem (concomitant models).
With that out of the way, the pattern I need to support is
- fit model 1 against inputs 1
- starting with the weights from model 1, fit model 2 against inputs 2 (which are not very different from inputs 1).
- continue up to inputs n
The R package luz (which is a wrapper around torch) provides a checkpointing facility: .html
But these checkpoints seem to be designed to support the case of auto resuming from failure, or inspecting intermediate models after training is complete.
What I need is a way to take a fitted model, and fit a new model using the fitted model weights as a starting point.
Ideally I could get an answer in R, but I will also accept an answer in python, if I can use it to work out what to do in R.
