there are dictionary compressors (decent compression, most common, fast), and statistical compressors (better compression, slower). Statistical compressors are much closer to LLMs in that an llm is learning statistics about the data too.
And yes, compression is history, that's what statistics are all about. Statistics can only measure the past to make a prediction about the future. And LLMs work in the same way. The context is the history, and given that history, it predicts the next token. An LLM can, almost trivially, be dropped into something like the PPM statistical compressor (it's just replacing one implementation of a markov chain with another).
Anything can only represent past measurements. Statistics is not an exception. But they don't make a prediction about the future. That comes from a model you have, and it often is implicit: "the linear trend from the last 12 months will hold in the next month" or whatever. So compression isn't by definition prediction. The other way around doesn't have to hold either, but in the case of LLMs it does.
a prediction is not necessarily correct. What compressors actually store is how wrong the prediction was, in fact, a guess takes exactly 0 bits to encode. It builds a st atistical model at runtime based on past encountered data. Exactly the same as llms, with the exception that llms remain fixed at runtime, instead of adapting to the data it is currently processing, so they can't adapt at runtime if the model does not suit the data
And yes, compression is history, that's what statistics are all about. Statistics can only measure the past to make a prediction about the future. And LLMs work in the same way. The context is the history, and given that history, it predicts the next token. An LLM can, almost trivially, be dropped into something like the PPM statistical compressor (it's just replacing one implementation of a markov chain with another).