Hi @hansen7 ,
Thanks for sharing this great work with us.
I have a question regarding the output log during pretraining. The detail is here.
The question is, it seems that the loss did not drop too much or its range is varied from 0.03 to 0.04. Did this happen to your pretraining as well?
But visually, I could see some improvements in the plots folder.
Best regards
Hi @hansen7 ,
Thanks for sharing this great work with us.
I have a question regarding the output log during pretraining. The detail is here.
The question is, it seems that the loss did not drop too much or its range is varied from 0.03 to 0.04. Did this happen to your pretraining as well?
But visually, I could see some improvements in the
plotsfolder.Best regards