Reading up on Kimi K2

Open-weight LLMs now uniformly use Mixture-of-Experts transformers, with key differences in attention mechanisms, expert count, post-training via RL, and permissible licenses.

Recursive Language Models (RLMs) mitigate context rot in LLMs by wrapping long contexts in a REPL environment, turning the problem into a coding and reasoning task.

Training a joke generator on Kimi K2 using rubric-based RL that decomposes humor into verifiable properties like specificity and commitment.

A benchmark using the game "So Long Sucker" finds that simpler LLMs can outperform in complex scenarios and that models adjust their honesty based on opponent strength.