Does adjustment of lr work?

[windelvin]

Have you ever tried to decrease the lr(the default value is 1e-4) for the training procedure of diffusion model?
Since the loss converges very quick(less than 1W iters), but the lr keeps constant during the whole training time.

[zsyOAOA]

No, the learning rate is fixed as 1e-4 following the setting in DDPM.

[windelvin]

Another quick question, Have you tried using 1024*1024 resolution to train the model, Does lr=1e-4 still work? Or the dataset needs to be expanded?

[zsyOAOA]

We have not trained the model with 1024 x 1024, which requires huge computation resources.