ByteDance's Seed team submitted a paper in late September this year on the arXiv preprint platform examining the occasional "glitch" phenomenon in large models during long-context tasks, offering a technical explanation for its causes. The research finds that phase sensitivity introduced by chunked KV cache compression is directly related to the DeepSeek phenomenon.
DeepSeek Team Finds Phase Sensitivity Affects Long-Context Retrieval Performance
The DeepSeek team evaluated the base and post-trained versions of DeepSeek-V4-Flash and DeepSeek-V4-Pro, as well as the post-trained DeepSeek-V4.1-Flash. All of these models share an architectural approach using chunked KV cache compression, which compresses consecutive token windows into fewer cache entries at a fixed stride. This approach significantly reduces memory footprint and attention computation costs during long-context inference, making it popular among many models.
However, compression comes at a cost. The paper notes that this method introduces a new positional coordinate beyond the original positional system — the token's phase, meaning its position relative to the compression window boundary. In other words, the same information occupying a different "position" in the compressed cache leads to drastically different retrieval performance.
The team further discovered a systematic asymmetry: the same information is easily retrieved by the model at one phase, but becomes considerably difficult at another. They named this phenomenon of retrieval performance fluctuating periodically with phase "phase sensitivity."
More notable is the magnitude of this fluctuation. In large open-weight models employing such compression, long-context retrieval accuracy can differ by up to 40 percentage points between phases. This figure reveals a periodic weakness that average benchmark scores may mask — a seemingly excellent average report card is actually the result of repeated oscillation between highs and lows, and which phase a user actually encounters often depends on where the information lands in the context.