Rollercoaster Tycoon Gone Wrong

AI • Machine Learning • Tech on Instagram: "In grokking, the training cycle shows a striking gap between training and test behavior: the training loss drops very quickly as the model learns to fit the examples it sees, while the test loss stays high for a long time before suddenly improving. Why is this actually occurring? Researchers have come up with many hypotheses. A likely reason is that the model first relies on shortcut solutions or memorization that work only for the training data and do