Yesterday, the research team behind Kimi released details of the Kimi K3 architecture, a 2.8‑trillion‑parameter open‑weight language model. The report describes K3 as the biggest open‑weight model currently available. The architecture extends the earlier Kimi Linear model and adds a LatentMoE component that compresses large linear layers, a design also used in Nemotron 3 Ultra. Attention mechanisms have been changed to multi‑head latent attention and a new Kimi Delta Attention, and residual connections now incorporate attention‑based weighting across layers, which the report says improves validation loss and downstream performance while raising training compute by about 4% and inference cost by roughly 2%. All RoPE positional embeddings have been removed in favor of NoPE, a shift that follows the earlier Kimi Linear release and differs from recent trends that retain RoPE in local attention layers. Native multimodal support is included, enabling image and text processing. The release also provides benchmark figures comparing inference and training performance at launch, and the authors note that additional training details are available in the technical report.