ByteDance Reportedly Plans a 5T+ Parameter AI Model, With Later Reports Pointing to Up to 10T

Introduction

China’s frontier-model race is moving deeper into the trillion-parameter era.

Alibaba’s Qwen3.8-Max has reached 2.4 trillion parameters, while Moonshot AI’s Kimi K3 has pushed the publicly disclosed scale to 2.8 trillion parameters.

ByteDance may now be preparing for the next jump.

AIBase, citing a report from LatePost, said on August 7, 2026 that ByteDance was discussing the training of a foundation model with more than 5 trillion parameters. According to that report, the project was still at an early stage and there was no guarantee that the final model would be released.

The scale alone would make the project one of the largest publicly reported AI-model efforts in China.

There is, however, an important same-day update.

Later on August 7, Reuters reported, citing the Financial Times and people familiar with the matter, that ByteDance was training a model with as many as 10 trillion parameters and that the model was already in pre-training. ByteDance had not publicly commented on the report at the time.

The two reports are not necessarily incompatible. A project initially discussed as “above 5 trillion” could eventually target a larger configuration. But because ByteDance has not officially disclosed the architecture or parameter count, all figures in this article should be treated as reported plans rather than confirmed model specifications.

ByteDance Is Reportedly Moving Beyond the 5-Trillion-Parameter Mark

The original LatePost report described ByteDance as discussing a model with more than 5 trillion parameters.

That would put it well above the currently disclosed scale of several major Chinese frontier models.

Model	Publicly Reported Total Parameters	Status
Qwen3.8-Max	2.4T	Released by Alibaba
Kimi K3	2.8T	Released by Moonshot AI
Reported ByteDance model	More than 5T	Reported plan
Later FT/Reuters figure	Up to 10T	Reported, not officially confirmed

Moonshot AI officially describes Kimi K3 as a 2.8-trillion-parameter Mixture-of-Experts model with 104 billion activated parameters per token.

Reuters reported on August 3 that Alibaba’s Qwen3.8-Max contains 2.4 trillion total parameters.

If ByteDance ultimately trains and deploys a 5T- to 10T-parameter model, it would represent a substantial increase in total model scale.

That does not automatically mean the model would be two or three times more capable.

Parameter Count Is Not the Same as Intelligence

The AIBase source describes a 5-trillion-parameter model as being in the same general scale class as leading U.S. frontier systems such as GPT-5.6 or Anthropic’s latest high-end models.

That comparison needs an important qualification.

OpenAI and Anthropic do not publicly disclose the exact parameter counts of GPT-5.6, Claude Fable 5, or Claude Mythos 5.

Reuters made the same point in its August 7 coverage: direct size comparisons with leading U.S. systems are difficult because those laboratories do not publish their model parameter counts.

Parameter count is only one dimension of model capability.

Performance also depends on:

• Model architecture
• Active parameters per token
• Training-data quality
• Training-token count
• Data filtering
• Optimizer design
• Reinforcement learning
• Post-training
• Tool use
• Test-time compute
• Long-context architecture
• Inference efficiency

This is especially important for Mixture-of-Experts models.

A sparse MoE model may contain trillions of total parameters while activating only a much smaller subset for each token.

Kimi K3 is a good example.

Its official specification lists:

• 2.8T total parameters
• 104B activated parameters
• 896 routed experts
• 16 selected experts per token

So the total number of weights is not the same thing as the amount of computation used for every inference step.

ByteDance has not publicly disclosed whether the reported model uses a similar sparse architecture or how many parameters would be active per token.

A 5-Trillion-Parameter Model Would Require Enormous Training Compute

The original report illustrates the scale using NVIDIA H100 GPUs.

Its estimate suggests that training a 5-trillion-parameter model could require roughly:

• 100,000 H100-class GPUs for 347 days, or
• 1 million GPUs for about 35 days

Those two scenarios represent approximately the same order of total accelerator time:

Scenario	GPUs	Duration	Approximate GPU-Days
Long-duration cluster	100,000	347 days	34.7 million
Massive short-duration cluster	1,000,000	35 days	35 million

These figures should be understood as a rough scenario from the source, not as a universal engineering formula.

Actual training requirements depend heavily on:

• Architecture
• Sparse versus dense activation
• Token count
• Precision
• Model FLOPs utilization
• Checkpointing
• Network efficiency
• Hardware generation
• Training stability
• Data-pipeline performance
• Failed runs and restarts

NVIDIA’s official H100 specification lists up to 700W TDP for the SXM version and support for large-scale transformer training through Hopper’s Transformer Engine and NVLink infrastructure.

At hundreds of thousands of accelerators, the GPU cluster alone enters a scale where power, networking, cooling, and data-center capacity become first-order constraints.

Why “One Million GPUs” Does Not Mean ByteDance Would Train That Way

The AIBase source correctly notes that running one million H100-class accelerators simultaneously would be an extreme infrastructure configuration.

A more realistic project could spread training across a smaller but still enormous cluster.

The source suggests something closer to:

• 200,000 to 300,000 accelerators
• Running for roughly three to four months

That would still represent one of the largest model-training efforts ever attempted.

And pre-training is only one stage.

A frontier model also requires time and compute for:

1. Data preparation
2. Architecture experiments
3. Scaling studies
4. Pre-training
5. Checkpoint evaluation
6. Supervised post-training
7. Reinforcement learning
8. Safety testing
9. Tool and agent training
10. Serving optimization

The original AIBase article therefore estimated that the total process could take at least half a year and potentially closer to a year before a mature model becomes available.

The later FT report, as summarized by Reuters, said the model is already in pre-training and noted that this stage typically takes around three to six months before fine-tuning and release.

Both descriptions point to the same practical conclusion: a project at this scale is a long-running infrastructure effort, not a model that appears immediately after the first training cluster goes online.

ByteDance Has the Resources to Attempt It

Training at this scale is not only a research problem.

It is an infrastructure and capital problem.

ByteDance is one of the few Chinese technology companies with the financial resources, consumer distribution, cloud infrastructure, data-center capacity, and AI engineering teams needed to seriously attempt a project of this size.

The company’s official Seed organization already covers:

• Foundation-model pre-training
• Post-training
• Reinforcement learning
• High-performance inference
• Distributed training
• Heterogeneous hardware compilation
• Multimodal models
• Agent systems

ByteDance’s infrastructure team specifically describes its work as covering distributed training, reinforcement-learning systems, high-performance inference, and compiler technologies for foundation models.

Its recruitment materials also explicitly mention work on:

• Ultra-large-scale training
• Training stability
• Model FLOPs utilization
• Hardware-software co-design
• Multi-node inference
• Parallelism
• Scheduling optimization

These are precisely the systems problems that become critical when training models at trillion-parameter scale.

The Source’s GPU Inventory Numbers Are Estimates, Not Public Disclosures

AIBase also cites third-party estimates for the amount of AI compute available to several frontier labs.

The article mentions approximately:

• 2 million GPUs for OpenAI
• 620,000 for Anthropic
• Around 60,000 for DeepSeek

These numbers should not be treated as audited inventory figures.

OpenAI and Anthropic do not publicly maintain a real-time count of every accelerator available across their own facilities, cloud partners, and reserved capacity.

DeepSeek’s roughly 60,000-accelerator figure has circulated since earlier SemiAnalysis estimates and has been widely repeated in reporting. Those estimates included a mixture of A100, H100, H800, and H20 accelerators rather than one uniform fleet.

The broader point is more reliable than any exact inventory number:

Frontier-model development increasingly depends on access to very large pools of accelerator compute, and the gap between companies with hyperscale infrastructure and smaller labs can become substantial when model size rises into multiple trillions of parameters.

The H100 Comparison Is Only a Baseline

Using H100 equivalents makes the compute discussion easier to understand, but ByteDance would not necessarily rely on one accelerator type.

Large AI companies can combine:

• H100
• H200
• Blackwell-generation NVIDIA systems
• China-specific NVIDIA variants
• Domestic AI accelerators
• Custom hardware
• Multiple cloud and data-center regions

Newer accelerators can change the number of physical GPUs required for the same amount of training compute.

Software matters too.

Improvements in:

• Kernel efficiency
• Parallelism
• Expert routing
• Memory management
• Communication
• Checkpointing
• Quantization
• Data loading

can significantly change the relationship between total model size and training cost.

For this reason, “X model requires exactly Y GPUs” should always be treated as a scenario rather than a fixed law.

Why ByteDance Might Want a Model This Large

The source frames the project mainly as an attempt to push Doubao’s intelligence toward the global frontier.

That is likely only part of the motivation.

A larger foundation model could potentially support several parts of ByteDance’s AI ecosystem.

Doubao

Doubao is one of ByteDance’s major consumer AI products in China.

A stronger foundation model could improve:

• General reasoning
• Knowledge work
• Coding
• Search
• Long-context tasks
• Agent behavior
• Multimodal interaction

Seed Research

An ultra-large model would also give the Seed team a new platform for research into:

• Scaling laws
• Sparse architectures
• Reinforcement learning
• Agent training
• Model efficiency
• Memory
• Long-horizon reasoning

Volcano Engine

ByteDance can also commercialize model capabilities through enterprise and cloud services.

A frontier model therefore has possible value outside the consumer chatbot itself.

Bigger Models Still Have to Deliver Economic Returns

The final question in the AIBase article is the most important one.

Can a model that costs an extraordinary amount to train generate a comparable return?

A frontier lab does not only pay for the final training run.

Total spending can include:

• GPUs
• Data centers
• Power
• Networking
• Storage
• Research salaries
• Data preparation
• Failed experiments
• Post-training
• Inference
• Safety work
• Product integration

The model then has to create value through some combination of:

• Consumer subscriptions
• Advertising
• E-commerce
• Enterprise APIs
• Cloud services
• Productivity products
• Coding tools
• Agents
• Internal efficiency gains

Parameter scale becomes economically meaningful only if it translates into useful capability at an acceptable serving cost.

This is why the current generation of frontier models increasingly emphasizes scaling efficiency, not simply total parameters.

Kimi K3, for example, uses 2.8 trillion total parameters but activates 104 billion per token. Moonshot says its architecture improves scaling efficiency relative to its previous generation.

The real competition is therefore not:

Who has the most parameters?

It is closer to:

Who can convert the most compute into the most useful intelligence
at a sustainable training and inference cost?

What Is Confirmed and What Is Still a Report

Confirmed

• Kimi K3 has 2.8 trillion total parameters, according to Moonshot AI.
• Qwen3.8-Max has 2.4 trillion parameters, according to Alibaba and Reuters.
• ByteDance Seed works on large-scale pre-training, post-training, inference, and model infrastructure.
• Doubao is ByteDance’s consumer AI product.
• NVIDIA H100 is designed for large-scale transformer training and trillion-parameter AI workloads.

Reported but Not Officially Confirmed by ByteDance

• ByteDance is building a model above 5 trillion parameters.
• The eventual target could reach as high as 10 trillion parameters.
• The exact training cluster size.
• The amount of H100-equivalent compute required.
• The final release date.
• Whether the model will ultimately be released publicly.
• The model architecture and activated-parameter count.

Not Verifiable From Public Model Specifications

• That a 5T model is automatically “GPT-5.6-class.”
• That parameter count alone predicts model intelligence.
• Exact parameter counts for GPT-5.6 or Anthropic’s Fable/Mythos-class models.
• Exact current GPU inventories for OpenAI or Anthropic.

常见问题

Is ByteDance really training a 5-trillion-parameter model?

AIBase, citing LatePost, reported that ByteDance was discussing a model above 5 trillion parameters. Later the same day, Reuters cited the Financial Times as saying ByteDance was already pre-training a model that could reach up to 10 trillion parameters. ByteDance had not publicly confirmed the exact figure.

Will the model power Doubao?

The source expects the model to strengthen ByteDance’s Doubao ecosystem, but ByteDance has not officially announced how the reported model will be deployed. A frontier model could also support enterprise APIs, agents, research, and other ByteDance products.

Is a 5T model automatically better than Kimi K3 or Qwen3.8-Max?

No. Total parameter count does not directly determine model quality. Architecture, active parameters, training data, post-training, reinforcement learning, inference-time reasoning, and tool use can all be equally important.

How many parameters does Kimi K3 have?

Moonshot AI officially lists Kimi K3 at 2.8 trillion total parameters with 104 billion activated parameters. It uses a sparse Mixture-of-Experts architecture.

How many parameters does Qwen3.8-Max have?

Alibaba’s Qwen3.8-Max has 2.4 trillion total parameters, according to Alibaba and Reuters reporting. It is also based on a sparse model design rather than activating the full parameter count for every token.

How many H100 GPUs would a 5T model require?

The source gives a rough scenario equivalent to roughly 35 million H100 GPU-days, such as 100,000 H100s for 347 days or 1 million for about 35 days. Real requirements can differ dramatically depending on architecture, precision, utilization, token count, hardware, and training efficiency.

Does GPT-5.6 have 5 trillion parameters?

OpenAI does not publicly disclose GPT-5.6’s parameter count. Any direct numerical comparison between GPT-5.6 and a reported ByteDance model is therefore speculative.

When could ByteDance release the model?

No official release date has been announced. Reuters’ FT-based report says the model is in pre-training, a stage that can take several months before fine-tuning, evaluation, and deployment.

相关工具

• ByteDance Seed: ByteDance’s official research organization and model hub for foundation models, multimodal systems, and AI research.
• ByteDance Seed LLM: Official overview of ByteDance’s work on pre-training, post-training, inference, memory, learning, and foundation-model research.
• Doubao: ByteDance’s consumer AI assistant and one of the main product surfaces that could benefit from stronger foundation models.
• Kimi: Moonshot AI’s official product platform and access point for the Kimi model family.
• Qwen: Alibaba’s official Qwen model and product portal.
• NVIDIA H100: NVIDIA’s official specifications and training-performance information for the H100 accelerator.

Related Links

• Reuters: ByteDance Targets Mega AI Model: Same-day reporting that later put the possible model scale as high as 10 trillion parameters.
• ByteDance Seed Models: Official model catalog and research directions from the ByteDance Seed team.
• ByteDance Seed Infrastructure: Official overview of ByteDance’s distributed-training, inference, and AI-systems infrastructure work.
• Kimi K3 Technical Blog: Moonshot AI’s official explanation of Kimi K3’s 2.8T architecture and 104B activated parameters.
• Kimi K3 GitHub Repository: Official model repository and technical specifications for Kimi K3.
• Qwen Official Website: Alibaba’s official model portal and API entry.
• NVIDIA H100 GPU: Official H100 specifications and large-model training information.

Summary

ByteDance is reportedly preparing an ultra-large foundation model beyond the scale of today’s publicly disclosed Chinese models. AIBase and LatePost initially described the project as exceeding 5 trillion parameters, while later same-day Reuters/FT reporting said the model could reach up to 10 trillion and was already in pre-training.

A project of this size would demand an enormous amount of compute. The source’s illustrative H100 calculation is roughly equivalent to 35 million GPU-days, although real training requirements depend on architecture, active parameters, hardware generation, utilization, and training efficiency.

The larger issue is not simply whether ByteDance can build the biggest model. Total parameters are an incomplete measure of intelligence, especially for sparse Mixture-of-Experts systems.

The real test will be whether ByteDance can turn trillion-scale compute into a model that is meaningfully better, efficient enough to serve, and valuable enough to justify the infrastructure behind it.