Kimi K3 Is Now Open-Weight: Inside Moonshot AI’s 2.8T-Parameter Frontier Model

Introduction

Moonshot AI has followed through on its promise to release the full weights of Kimi K3, its largest and most ambitious model to date.

The model is now available through Moonshot AI’s official Hugging Face and GitHub pages, together with a detailed technical report and supporting infrastructure projects.

Kimi K3 is a native multimodal Mixture-of-Experts model with 2.8 trillion total parameters, approximately 104 billion activated parameters per token, and a 1-million-token context window.

Moonshot positions it as an open-weight frontier model built for long-horizon coding, research, multimodal understanding, agentic knowledge work, and reasoning tasks that may span hundreds or thousands of steps.

The release is significant for two reasons.

First, the model itself pushes open-weight scale into the 3-trillion-parameter class.

Second, Moonshot has disclosed far more than a checkpoint. The public materials describe the architecture, post-training strategy, long-context reinforcement-learning infrastructure, expert-parallel training system, inference optimizations, benchmarks, and several long-running engineering case studies.

Kimi K3 Full Weights Are Now Available

Moonshot initially announced Kimi K3 with a promise to release the complete weights by July 27, 2026.

That release has now happened.

The official model is available through:

• Hugging Face
• GitHub
• ModelScope

The GitHub repository contains the README, Kimi K3 License, full technical report, architecture and benchmark assets, deployment guidance, and model-usage notes.

The Hugging Face model card provides the model configuration and access to the released weights.

This changes the model from a hosted frontier system with a planned open release into a model that researchers and qualified infrastructure teams can actually download, inspect, deploy, fine-tune, and adapt under the terms of the Kimi K3 License.

What “Open” Means for Kimi K3

Kimi K3 is best described as an open-weight model with publicly released code and documentation.

The license grants broad rights to use, copy, modify, fine-tune, deploy, create derivative works, redistribute, sublicense, and sell copies, subject to the license conditions.

However, the license contains additional commercial requirements.

For example, a company operating a qualifying Model-as-a-Service business with more than US$20 million in aggregate revenue over a consecutive 12-month period must enter into a separate agreement with Moonshot AI before using Kimi K3 or derivatives for commercial purposes.

The license also includes a display requirement for certain very large commercial products or services exceeding specified user or revenue thresholds.

Internal use and use through Moonshot AI’s official products or certified inference partners are treated differently.

Anyone deploying Kimi K3 commercially should therefore read the actual license rather than assuming it is identical to Apache 2.0, MIT, or another standard permissive license.

A 2.8T-Parameter Model with 104B Active Parameters

Kimi K3 is enormous in total capacity, but its Mixture-of-Experts design means that the full 2.8 trillion parameters are not activated for every token.

The official model card lists:

Specification	Kimi K3
Architecture	Mixture of Experts
Total parameters	2.8T
Activated parameters	104B
Layers	93
Dense layers	1
Routed experts	896
Experts selected per token	16
Shared experts	2
Vocabulary size	160K
Context length	1,048,576 tokens
Vision encoder	MoonViT-V2
Vision encoder parameters	401M
Quantization	MXFP4 weights / MXFP8 activations
Modalities	Text and image

The technical report gives a more precise architecture comparison with Kimi K2.

Compared with Kimi K2, K3 increases both model depth and expert sparsity while introducing new attention and residual mechanisms.

Moonshot says the combined architecture and training changes improve overall scaling efficiency by approximately 2.5× over Kimi K2.

That claim refers to how efficiently additional compute is converted into model capability, not to a universal 2.5× inference-speed improvement.

Native Multimodality and a 1-Million-Token Context Window

Kimi K3 is designed as a native multimodal model rather than a language model that receives a vision module only at the end of training.

Its vision encoder, MoonViT-V2, contains roughly 401 million parameters.

The technical report says MoonViT-V2 was trained from scratch through the model’s next-token prediction objective rather than initialized from a contrastively pretrained vision model such as SigLIP.

Moonshot reports that this approach remained stable during training and reached competitive visual performance.

The model supports a context length of:

1,048,576 tokens

That scale is particularly relevant for agent systems because a long-running coding or research task can accumulate repository contents, tool outputs, logs, screenshots, research material, intermediate plans, test failures, previous reasoning, and multiple rounds of corrections.

A million-token context does not eliminate the need for context management, but it allows K3 to preserve a much larger working history than traditional short-context systems.

Three Major Architecture Changes

The technical report attributes K3’s scale and stability to several architectural changes.

The three most visible are:

1. Hybrid KDA + Gated MLA attention.
2. Attention Residuals.
3. Stable LatentMoE.

1. Hybrid Attention: KDA + Gated MLA

Kimi K3 uses a hybrid attention design.

The official model card lists:

69 KDA layers + 24 Gated MLA layers

The architecture follows a repeating pattern in which multiple Kimi Delta Attention layers are combined with periodic Gated Multi-head Latent Attention layers.

Kimi Delta Attention

KDA is a linear-attention mechanism designed to reduce the memory and compute burden associated with very long contexts.

Instead of storing a conventional KV cache that grows with every token in the same way as standard full attention, KDA maintains a recurrent state.

That makes it attractive for million-token sequences.

Moonshot previously introduced KDA through the Kimi Linear research line, where the team reported lower KV-cache requirements and improved long-context decoding throughput compared with full attention under its evaluated conditions.

Gated MLA

K3 does not replace global attention completely.

Periodic Gated MLA layers preserve the model’s ability to retrieve information globally across the context.

The hybrid design therefore attempts to balance two goals:

• Efficient long-sequence processing.
• Strong global retrieval.

The idea is not that linear attention is always better than global attention, but that each mechanism handles a different part of the long-context problem.

2. Attention Residuals

Deep neural networks need to move information through many layers without gradually losing useful representations.

Kimi K3 introduces Attention Residuals, or AttnRes, to change how information is passed through depth.

Instead of relying only on a uniform sequential residual stream, later blocks can selectively retrieve representations produced by earlier blocks.

Conceptually, this gives the network a learned mechanism for deciding which earlier information should remain directly accessible.

The goal is to improve information flow across a 93-layer model without requiring every feature to survive through exactly the same residual path.

3. Stable LatentMoE

Kimi K3 significantly increases MoE sparsity.

The model contains:

896 routed experts

but activates only:

16 routed experts per token

It also includes two shared experts.

This produces a very large total model while keeping active compute much smaller than the full parameter count.

At this sparsity level, training stability and expert balance become difficult problems.

Moonshot’s Stable LatentMoE design includes several mechanisms intended to address them.

SiTU-GLU

Kimi K3 replaces SwiGLU with SiTU-GLU, an activation designed to avoid unbounded growth under extreme scaling.

Quantile Balancing

The system also uses a load-balancing approach based on dynamic routing bias rather than relying solely on a conventional auxiliary balancing loss.

The goal is to distribute routed tokens more evenly across experts without introducing excessive training interference.

Million-Token Agent Training

Kimi K3’s technical report places unusual emphasis on post-training for long-running agents.

The model was trained across three broad domains:

• General reasoning.
• General agent tasks.
• Coding agents.

It also supports multiple reasoning-effort levels:

• Low.
• High.
• Max.

The final model combines specialized policies through Multi-Teacher On-Policy Distillation, or MOPD.

Rather than producing separate permanent models for every domain and effort level, Moonshot trains specialized teachers and consolidates their behavior into one unified K3 policy.

AgentENV: Long-Horizon Sandboxed Training

The technical report describes AgentENV, an infrastructure layer built for persistent agent rollouts.

The system uses Firecracker-based microVM environments so an agent can perform long sequences of actions inside reproducible sandboxes.

A training task may involve:

1. Reading files.
2. Writing code.
3. Running commands.
4. Opening applications.
5. Taking screenshots.
6. Using simulated workplace tools.
7. Debugging failures.
8. Returning after long delays.
9. Continuing from previous environment state.

This differs from reinforcement learning built around short question-and-answer episodes.

An agent may operate over hundreds of tool calls and maintain state for a simulated period spanning multiple virtual days.

The source article describes simulated environments involving applications such as Slack, Notion, and Gmail.

The key training challenge is persistence: both the environment and the model’s long-context state must remain usable across lengthy rollouts.

Knowledge-Graph-Guided Task Synthesis

Long-horizon training also needs a large supply of difficult tasks.

Moonshot describes a task-generation pipeline organized around a hierarchical knowledge graph.

The graph covers areas such as computer science, artificial intelligence, coding, mathematics, physics, chemistry, biomedicine, and humanities.

Related concepts can be sampled together, relevant public material retrieved, and new evaluation or training tasks synthesized from that material.

The purpose is to create tasks that require actual tool use and multi-step reasoning rather than only memorized answers.

Infrastructure for a 3T-Class MoE Model

A 2.8T model with a million-token context cannot be trained and served efficiently through model architecture alone.

Moonshot’s technical report describes systems work covering KDA kernels, context parallelism, expert parallelism, memory management, long-horizon RL scheduling, prefix caching, speculative decoding, and serving admission control.

Several pieces have been made public.

MoonEP: Balanced Expert Parallelism

MoonEP is Moonshot’s open-source expert-parallel communication library.

In MoE training, routing can become badly imbalanced.

One rank may receive far more tokens than another because the router prefers certain experts. When that happens, the fastest devices wait for the slowest one.

MoonEP addresses the problem using dynamic redundant experts.

It can create temporary expert replicas based on current routing patterns so token workloads remain balanced across ranks.

The project describes its goal as keeping each rank at exactly the expected routed-token load while reducing communication overhead.

This matters at K3’s scale because 896 experts distributed across a large cluster create an enormous synchronization problem.

FlashKDA: High-Performance KDA Kernels

FlashKDA provides high-performance Kimi Delta Attention kernels implemented on top of CUTLASS.

The public repository currently lists requirements including:

SM90 or newer
CUDA 12.9 or newer
PyTorch 2.4 or newer

The repository includes correctness tests and benchmark data for supported hardware.

FlashKDA is designed to accelerate the attention mechanism that makes K3’s long-context architecture practical.

Native MXFP4 Quantization

Kimi K3 uses quantization-aware training from the supervised fine-tuning stage onward.

The official configuration lists:

MXFP4 weights
MXFP8 activations

This is different from taking a fully trained high-precision model and quantizing it only after training.

The model is trained to operate with these lower-precision formats as part of the pipeline.

That helps reduce memory and communication requirements, but it does not turn a 2.8T model into a normal desktop model.

Can You Run Kimi K3 Locally?

The weights are public, but “downloadable” does not mean “easy to run on a laptop.”

Kimi K3 contains 2.8 trillion total parameters.

Even with sparse activation and low-bit formats, serving the complete model requires large aggregate accelerator memory, high-bandwidth device interconnect, expert-parallel execution, a compatible inference engine, and efficient support for KDA, MLA, and MoE.

Moonshot’s official repository recommends:

• vLLM
• SGLang
• TokenSpeed through Moonshot’s deployment recipes

The ecosystem is still moving quickly. Support may depend on particular recipes, kernels, quantization modes, and hardware.

Before building a self-hosted environment, confirm the latest Kimi K3 deployment documentation rather than assuming standard Transformer-serving settings will work.

For most individuals and smaller teams, the hosted API is likely to be substantially easier than running the complete model.

Kimi K3 API and Reasoning Effort

Moonshot also offers Kimi K3 through its official API platform:

https://platform.kimi.ai/

The model identifier is:

kimi-k3

Kimi K3 uses reasoning and returns a reasoning_content field.

The official model card describes three reasoning-effort levels:

low
high
max

An important implementation detail is preserved thinking history.

For multi-turn agent workflows, Moonshot says developers should pass the complete previous assistant message back into the next request, including reasoning_content, content, and tool_calls.

Dropping part of that state can hurt continuity because K3 was trained to operate with preserved reasoning history.

Benchmark Performance

Moonshot’s technical report evaluates Kimi K3 across reasoning and knowledge, coding, agentic workflows, and vision.

The company’s own evaluation suite places K3 among the strongest available models and ahead of many open-weight and proprietary systems in selected tasks.

At the same time, the report explicitly says the model still trails the strongest proprietary systems overall, particularly Claude Fable 5 and GPT-5.6 Sol.

That caveat is important.

The claim is not that K3 wins every benchmark.

The more defensible conclusion is that Kimi K3 brings open weights into performance territory that, until recently, was associated mainly with frontier proprietary systems.

Several benchmark comparisons also depend on different agent harnesses, including Kimi Code, Claude Code, Codex, and official benchmark harnesses.

Agent benchmark scores should therefore not be interpreted as pure model-only measurements. The surrounding tools, prompts, context management, fallback behavior, and evaluator can materially affect results.

Long-Horizon Coding Case Studies

The report gives several examples intended to demonstrate that K3 can sustain engineering work well beyond short code-completion tasks.

These are case studies produced by the model developers and should be treated as such, not as universal performance guarantees.

MiniTriton: Building a GPU Programming System

Moonshot reports that Kimi K3 developed MiniTriton, a compact Triton-like GPU programming system.

The project included an intermediate representation, compiler passes, PTX generation, a frontend language, runtime components, GPU kernel optimization, and distributed training primitives.

Moonshot says K3 also optimized difficult kernels related to its own architecture and achieved substantial speedups on selected kernels.

This demonstrates the intended behavior of the model: inspect a complex codebase, profile it, write low-level code, benchmark the result, and iterate.

It does not mean every model-generated compiler or kernel should be trusted without human review and testing.

A 48-Hour Chip-Design Experiment

Another case study placed Kimi K3 in a sandbox for 48 hours and asked it to create an inference-chip prototype for a small model derived from similar architectural ideas.

The model used open-source electronic-design-automation tools and the Nangate 45 nm standard-cell library.

Moonshot reports that the resulting simulated design fit within a 4 mm² analytical area budget, closed timing at 100 MHz, included about 1.46 million standard cells, used 0.277 MiB of SRAM, and reached an RTL-simulated decode throughput above 8,700 tokens per second.

This was a proof of concept in simulation. It was not a manufactured production chip.

Research Coding and Astrophysics

K3 was also evaluated on a computational astrophysics reproduction task.

Moonshot says the model read and cross-checked more than 20 papers, implemented a full numerical pipeline, evaluated more than 300 equations of state, identified inconsistencies in published formulas, wrote more than 3,000 lines of Python, and produced an interactive HTML dashboard.

The reported autonomous work took about two hours, compared with an estimated one to two weeks for an experienced researcher.

For scientific work, independent validation remains essential. A model can generate a complete pipeline and still make subtle mistakes in assumptions, units, numerical methods, citations, or interpretation.

Cybersecurity Evaluation

The technical report also discusses cybersecurity-oriented evaluation.

Moonshot says Kimi K3 identified previously unknown Linux-kernel vulnerabilities during controlled security testing and that the findings were reviewed by human experts.

The significance is not that K3 should be used to run unsupervised offensive security operations.

It is that long-horizon coding models can increasingly read large codebases, trace low-level behavior, form hypotheses, test them, revise their approach, and continue after failed attempts.

Those capabilities can support defensive auditing and secure-code review, but they also make authorization, sandboxing, access control, and human oversight more important.

Cost and Efficiency

A model’s value is not determined by benchmark score alone.

Long-horizon agent systems may generate millions of tokens on one task.

Cost therefore depends on input length, output length, reasoning effort, cache hit rate, number of tool calls, retries, context management, and inference provider.

Moonshot’s technical report includes score-versus-cost comparisons for several agentic benchmarks.

These graphs show K3 as comparatively cost-efficient on some tested workloads.

They should not be read as a universal statement that K3 is always cheaper than every alternative.

For production planning, measure the cost of a successful completed task rather than only the price per million tokens.

Why the Open-Weight Release Matters

Kimi K3’s release matters beyond the ranking of one model.

It opens several areas of research and engineering that are difficult to study with API-only systems.

Researchers can inspect or modify the model weights, MoE behavior, KDA-based attention, long-context mechanisms, vision components, quantization behavior, serving systems, and agentic post-training strategies.

Infrastructure teams can test alternative serving approaches.

Enterprises can evaluate private deployment where the hardware economics make sense.

The community can build:

• New inference recipes.
• Quantized variants.
• Fine-tuned derivatives.
• Domain-specific systems.
• Evaluation harnesses.
• Model-routing tools.
• Hardware-specific optimizations.

Whether this produces an ecosystem comparable to earlier major open-model releases will depend on inference cost, accessible hardware, license terms, community tooling, and the quality of smaller derivatives.

Practical Checklist Before Deploying Kimi K3

1. Read the License

Confirm whether your use case is internal research, enterprise use, fine-tuning, embedded application use, public inference, or Model as a Service.

2. Confirm Hardware Support

Check accelerator architecture, HBM capacity, interconnect, CUDA version, kernel requirements, quantization support, device count, and expert-parallel topology.

3. Choose a Supported Inference Engine

Use current K3-specific recipes for vLLM, SGLang, or another officially supported serving path.

4. Test the Full 1M Context Separately

Measure prefill latency, decode latency, prefix-cache behavior, memory use, concurrency, throughput, and failure recovery.

5. Preserve Reasoning History

For agentic multi-turn workflows, follow Moonshot’s guidance and return the complete assistant message, including reasoning and tool-call state.

6. Validate Real Tasks

Benchmark your own repositories, research workflows, documents, visual tasks, tool use, and agent scenarios.

7. Measure Cost Per Completed Task

Token price alone can be misleading. Count retries, tool calls, output length, and human intervention.

FAQ

Is Kimi K3 open source?

Kimi K3’s full weights, repository, documentation, and technical report are public, and Moonshot describes it as an open-weight model. The model uses the custom Kimi K3 License rather than a standard license such as Apache 2.0, so commercial users should review the specific terms.

Where can I download Kimi K3?

The official weights are available from Moonshot AI on Hugging Face. The main source repository and technical report are available on GitHub.

How many parameters does Kimi K3 have?

Kimi K3 has approximately 2.8 trillion total parameters. Because it is a sparse Mixture-of-Experts model, about 104 billion parameters are activated for each token.

What is Kimi K3’s context window?

The official model supports up to 1,048,576 tokens of context. Long-context deployment still requires substantial memory and inference infrastructure, especially under concurrency.

Can Kimi K3 run on a consumer GPU?

The full 2.8T model is not realistically intended for a normal consumer GPU. Even with sparse activation and low-bit quantization, the complete weights and serving stack require large multi-accelerator infrastructure.

Which inference engines support Kimi K3?

Moonshot’s official repository recommends vLLM, SGLang, and TokenSpeed through K3-specific deployment guidance. Because the model is new and uses KDA, MLA, MoE, and MXFP4-related optimizations, verify the current supported versions before deployment.

What are MoonEP and FlashKDA?

MoonEP is Moonshot’s open-source expert-parallel communication library for balancing MoE token loads across ranks. FlashKDA provides high-performance Kimi Delta Attention kernels built on CUTLASS.

Does Kimi K3 outperform GPT-5.6 Sol and Claude Fable 5?

Not overall. Moonshot’s technical report says Kimi K3 reaches frontier-level performance and leads selected tasks, but still trails the strongest proprietary systems overall. Benchmark results also depend on agent harnesses, tools, reasoning settings, and evaluation methodology.

Related Tools

• Kimi K3 on Hugging Face: Official model card and full Kimi K3 weights.
• Kimi K3 on GitHub: Official repository containing the README, license, assets, and technical report.
• MoonEP: Moonshot’s expert-parallel communication library for dynamically balancing MoE workloads.
• FlashKDA: High-performance Kimi Delta Attention kernels built on CUTLASS.
• vLLM: Open-source inference and serving engine recommended in Kimi K3’s deployment guidance.
• SGLang: Open-source LLM and multimodal serving framework listed by Moonshot for K3 deployment.
• Kimi API: Moonshot AI’s hosted API platform with access to the kimi-k3 model.

Related Links

• Kimi K3 Official GitHub Repository: Main repository for architecture details, deployment notes, model usage, license, and technical assets.
• Kimi K3 Hugging Face Model Card: Official model summary and downloadable full weights.
• Kimi K3 Technical Report: Full research report covering architecture, training, infrastructure, evaluation, and case studies.
• Kimi K3 License: Official license governing model weights, code, derivative works, deployment, and certain commercial uses.
• MoonEP Repository: Open-source implementation of Moonshot’s balanced expert-parallel system.
• FlashKDA Repository: Open-source KDA kernel implementation and benchmark resources.
• Kimi K3 ModelScope Organization: Moonshot AI model distribution through ModelScope.

Summary

Moonshot AI has now released the full Kimi K3 weights, turning its 2.8T-parameter frontier model into a system that researchers and infrastructure teams can inspect, deploy, fine-tune, and extend under the Kimi K3 License.

The model combines 104B activated parameters, a 1M-token context window, native multimodality, hybrid KDA/Gated-MLA attention, Attention Residuals, Stable LatentMoE, quantization-aware training, and long-horizon agent post-training.

Just as important, the release exposes part of the systems engineering behind the model through projects such as MoonEP and FlashKDA. K3 remains an extremely demanding model to self-host, and its strongest results should be read in the context of harness choices and developer-run evaluations.

The real milestone is not simply that a 2.8T model exists—it is that a model at this scale is now available for the broader community to inspect, run, and build on.