Over-training large language models may make them harder to fine-tune

A small team of AI researchers from Carnegie Mellon University, Stanford University, Harvard University and Princeton University, all in the U.S., has found that if large language models are over-trained, it might make them harder to fine-tune. In their paper posted on the arXiv preprint server, the group compared the impact of different amounts of training on a single LLM.

Over the past couple of years, as AI researchers seek to enhance their products to make them more “intelligent,” many have been driven by the mantra that the more training a model is given, the better the model will be in the end. In this new study, the research team has found some evidence suggesting that there may be a point of diminishing returns with language model training.

The researchers came to this conclusion as they were testing the return when training two different versions of the LLM OLMo-1B. Under one scenario, they trained it using 2.3 trillion tokens, while in the other they used 3 trillion tokens. They then compared the scenarios by testing them with several benchmarks, such as ARC and AlpacaEval. In so doing, they found that the model trained with more tokens actually did worse when tested—up to 3% worse.

Surprised by their findings, they ran more tests and found similar results, suggesting that there is some point at which more training starts to make models less “intelligent.” The research team calls it “catastrophic overtraining,” and suggests it is due to what they describe as “progressive sensitivity.”

They further suggest that as the number of tokens rises, the more fragile a model becomes, which means that fine-tuning, which can be viewed as adding noise, starts to reverse the gains in improvement that were seen prior to the stress point.

To test their theory, they added Gaussian noise to some of the models, and found that doing so led to the same type of performance degradation they had witnessed earlier. They have named the point of no return, the “inflection point.” After that point, they suggest, any further training will reduce the stability of the model, making it more difficult to tune in ways that are useful for a desired set of applications.

The researchers conclude by suggesting that moving forward, developers of LLM models may have to make estimations regarding how much training is enough—or, find other types of methods that will allow for additional training with a more distant inflection point.

© 2025 Science X Network