Did AI Break NVIDIA’s CUDA Moat? What Infinity Actually Built in 10 Hours

Introduction

NVIDIA’s CUDA moat has been declared dead so many times that the phrase has almost become an industry ritual.

A new accelerator appears. A compiler reaches public beta. A cloud provider promotes its own AI chip. A programming abstraction promises portable kernels. Someone announces that developers no longer need to write CUDA by hand.

Then CUDA remains central to the next generation of AI infrastructure.

The latest challenge is more interesting because it comes from AI itself.

Infinity, an AI infrastructure startup founded by former Google Brain researcher Jeremy Nixon, says its Ignition agent brought important parts of an inference software stack to the unfamiliar d-Matrix Corsair accelerator far faster than a traditional engineering process.

The headline result was dramatic: within roughly 10 hours, Ignition had tensor-parallel matrix multiplication running across all 32 compute units and reached as much as 92% of the chip’s empirically measured compute ceiling.

That is a meaningful result. It is not the same as recreating CUDA in 10 hours.

Infinity’s own case study says the complete end-to-end Qwen3 inference path took about 10 days, with each required operation written for the new hardware. CUDA, meanwhile, is not one matrix-multiplication implementation or one model runtime. It is a mature platform containing compilers, runtimes, libraries, profiling and debugging tools, communication systems, framework integrations, documentation, and nearly two decades of production knowledge.

The better question is not whether AI copied CUDA overnight.

It is whether coding agents can dramatically reduce the time required to make a new AI chip useful.

The answer increasingly appears to be yes.

CUDA’s Moat Was Never Just the GPU

NVIDIA is best known for hardware: H100, Blackwell, Rubin, and the large systems built around them.

Its most durable advantage is the software surrounding that hardware.

CUDA stands for Compute Unified Device Architecture. NVIDIA describes it as a platform for accelerated computing, not merely a programming language.

The CUDA platform includes:

• A GPU programming model.
• A compiler toolchain.
• Runtime libraries.
• Driver interfaces.
• Highly optimized math and AI libraries.
• Debugging and profiling tools.
• Multi-GPU communication software.
• Framework integrations.
• Documentation, training, and code samples.
• A large developer and partner ecosystem.

At the low level, CUDA lets engineers write kernels that execute directly on NVIDIA GPUs.

Above that sit libraries such as cuBLAS, cuDNN, cuFFT, NCCL, TensorRT, and TensorRT-LLM. Above the libraries sit PyTorch, TensorFlow, JAX, inference servers, research code, and internal company systems.

A simplified view looks like this:

Applications and AI frameworks
↓
Optimized libraries and distributed systems
↓
Compilers, runtimes, profilers, debuggers, and kernels
↓
NVIDIA GPUs and interconnects

Organizations do not stay with NVIDIA only because they know CUDA syntax. They stay because models, tests, deployment systems, performance assumptions, debugging workflows, and production teams already depend on the complete stack.

Switching hardware can require much more than translating source code. It can require rebuilding confidence.

What Infinity Actually Did in 10 Hours

Infinity’s official case study is more precise than the viral headline.

The company worked with d-Matrix, a startup focused on generative-AI inference hardware. Its Corsair accelerator uses a different architecture and instruction set from NVIDIA GPUs. Infinity says the chip had essentially no public software history that a general-purpose coding model could simply reproduce from training data.

Ignition received hardware information and began generating the low-level software required to execute AI workloads.

Within approximately 10 hours, Infinity says the system achieved:

• Tensor-parallel matrix multiplication.
• Execution across all 32 Corsair compute units.
• Up to 92% “speed of light.”

Infinity defines speed of light as the fraction of the chip’s achievable compute peak realized by the generated kernel. The empirical ceiling in the published case study was itself approximately 90% of the theoretical peak.

This was a strong kernel bring-up result. It was not a complete CUDA-equivalent platform.

Ten Hours: High-Performance Matrix Multiplication

Matrix multiplication is central to transformer inference. It is also only one category of operation.

A full model requires additional work for attention, normalization, activation functions, routing, quantization, KV-cache management, sampling, tensor movement, state handling, and model serving.

Ten Days: End-to-End Model Inference

Infinity says that within about 10 days, Qwen3 was running end to end on Corsair. Its later funding announcement said three frontier models were running end to end within that period, with kernels written from scratch.

That longer timeline is still impressive.

Moving an unfamiliar accelerator from basic operations to a functioning model runtime in days rather than months can materially change the economics of launching new hardware.

Claim	More Precise Interpretation
“AI recreated CUDA in 10 hours”	Overstated
Matrix multiplication ran within 10 hours	Reported by Infinity
Performance reached up to 92% of the empirical ceiling	Reported by Infinity
Qwen3 ran end to end within about 10 days	Reported by Infinity
Infinity is building a universal inference library	Confirmed by Infinity
The full CUDA ecosystem was reproduced	No

Ignition Is an Automated Kernel and Inference Engineering Agent

Infinity calls Ignition an AI research agent.

Its purpose is to generate and improve the low-level software that turns model operations into efficient work on a specific chip.

The loop resembles an automated performance-engineering process:

Generate code
→ compile
→ run on hardware
→ test correctness
→ measure performance
→ diagnose bottlenecks
→ modify the implementation
→ repeat

Human engineers still provide the target, hardware information, constraints, acceptance criteria, and high-level direction. Ignition performs much of the repetitive search and optimization.

This type of task is well suited to agents because the environment produces unusually clear feedback.

A generated kernel can be evaluated against concrete questions:

1. Does it compile?
2. Does it run?
3. Is the numerical output correct?
4. Is it stable?
5. How much hardware performance does it achieve?
6. Did the latest change improve the result?

The hardware and tests provide ground truth.

Why Kernel Engineering Is a Good Agent Problem

Many software tasks have ambiguous acceptance criteria. Kernel optimization is difficult, but parts of it are measurable.

A valid implementation must meet two separate standards.

Correctness

The kernel must produce output within accepted numerical tolerances.

Performance

The kernel must use the target hardware efficiently enough to justify replacing an existing implementation.

An agent can generate hundreds of candidates, reject incorrect ones, benchmark the survivors, and keep refining the strongest paths.

The same pattern appears in NVIDIA’s CUDA-focused agents, INT21’s PTX Kernel Factory, DeepSeek’s TileKernels work, and research systems that generate Triton or TileLang kernels.

The recent change is that foundation models can now participate in more of the complete loop, rather than only filling in a few lines of syntax.

Infinity Raised $15 Million at a $100 Million Valuation

Infinity’s technical story has attracted investors.

The company announced a $15 million seed round** in July 2026 at a reported **$100 million post-money valuation. Touring Capital and Principal Venture Partners participated, along with chip-industry executives and researchers associated with major AI labs.

Infinity is building a model-aware inference software layer for hardware vendors and inference providers. Its stated workflow is effectively:

hardware specifications
→ generated production inference software

Infinity has also reported generating an NVIDIA inference engine from scratch that outperformed a matched vLLM configuration on Qwen3-8B by as much as 34.3%.

That result is company-reported and depends on hardware, batch settings, model configuration, and measurement methodology. It nevertheless shows that Infinity’s ambition is broader than basic chip bring-up.

DeepSeek TileKernels Needs Similar Precision

The original article also points to DeepSeek’s TileKernels repository.

TileKernels is a library of optimized GPU kernels written with TileLang, a Python-based domain-specific language for high-performance kernel development.

The repository includes operations for:

• Mixture-of-Experts gating.
• MoE routing.
• FP8 and FP4 quantization.
• Transpose.
• Engram gating.
• Manifold HyperConnection.
• PyTorch autograd wrappers.

DeepSeek says many of the kernels approach hardware limits for compute intensity or memory bandwidth, and some have been used internally.

TileLang reduces the amount of low-level CUDA C++ that engineers must write manually.

It does not, in its current form, show that DeepSeek has eliminated NVIDIA’s stack.

The official TileKernels requirements currently include:

NVIDIA SM90 or SM100 GPU
CUDA Toolkit 13.1 or newer
PyTorch 2.10 or newer
TileLang 0.1.9 or newer

The current library is designed for recent NVIDIA architectures. It weakens dependence on handwritten CUDA source code, not dependence on NVIDIA hardware or the CUDA toolkit.

TileLang Is Broader Than the Current DeepSeek Library

TileLang itself has a wider ambition.

The project describes a tiled programming model for GPU, CPU, and accelerator kernels using Pythonic syntax on top of TVM compiler infrastructure.

Its goal is to separate the data flow an engineer wants from the low-level schedule required to execute it efficiently.

TileLang has been adding multi-backend language support and hardware paths involving CUDA, ROCm, and Metal.

That portability could reduce switching costs between vendors, but only when the generated result is correct, stable, debuggable, and competitive with vendor libraries.

A kernel that runs everywhere but performs badly does not replace a production CUDA path.

Correct Code Can Still Be Hundreds of Times Too Slow

A July 2026 research paper examined the gap between kernel correctness and replacement quality in Triton and TileLang.

The authors found that kernels could pass numerical correctness tests while performing far below optimized baselines. One TileLang LayerNorm implementation reportedly ran more than 300 times slower than the PyTorch baseline despite passing correctness checks.

The paper is not an argument against TileLang. It is an argument for evaluating generated kernels on both axes:

correctness
+
hardware efficiency

Generating code is becoming faster. Proving that generated code is ready to replace mature production software remains difficult.

Inference Is the First Major Battleground

The challenge to CUDA is more credible in inference than in frontier-model training.

Training Prioritizes Maximum Scale and Stability

Large training runs can involve thousands or tens of thousands of accelerators operating together for weeks or months.

Training platforms must handle distributed communication, checkpointing, fault recovery, memory management, parallelism, reproducibility, and debugging across many devices.

A hardware or software failure can waste a large amount of compute. Organizations are therefore conservative about moving important training workloads away from proven systems.

CUDA, CUDA-X libraries, NCCL, PyTorch support, and NVIDIA’s integrated networking give the company a strong position here.

Inference Prioritizes Cost per Useful Answer

Inference serves models after they have been trained.

The relevant business metric is often closer to:

acceptable output quality
÷
total serving cost

Inference can be divided across individual accelerators, small clusters, large fleets, and specialized hardware.

A new chip does not need to replace NVIDIA everywhere. It may only need to prove that one model or workload runs cheaper, faster, with lower power, at better throughput, or with more predictable latency.

That narrower target gives specialist hardware a realistic entry point.

d-Matrix Is Built Around Generative-AI Inference

d-Matrix was founded in 2019 and focuses on inference acceleration. Its Corsair platform uses large amounts of on-chip SRAM and is designed to reduce the cost and energy of running generative models.

Infinity’s work addressed one of the classic problems for a new accelerator.

A chip may have promising hardware. Customers cannot use it efficiently until kernels, model execution, memory handling, quantization, serving logic, and integrations exist.

If agents compress that bring-up process from months into days, hardware startups can support new models closer to release and demonstrate hardware earlier.

This reduces one part of NVIDIA’s software advantage. It does not remove all of it.

Other Vendors Are Targeting the Same Opening

The inference market includes several challengers:

Vendor or Platform	General Position
Rebellions	Dedicated AI inference accelerators and systems
Cerebras	Wafer-scale systems for training and inference
AWS Inferentia	Amazon-designed inference chips using the Neuron SDK
Google TPU	Google accelerators supported through XLA and major frameworks
AMD Instinct	Data-center GPUs using the ROCm platform
d-Matrix	SRAM-centric generative-AI inference acceleration
NVIDIA	GPUs and the CUDA accelerated-computing stack

The pressure comes from software layers that let users deploy models without rewriting every operation manually, including AWS Neuron, Google XLA, AMD ROCm, Modular MAX and Mojo, TileLang, Triton, and AI-generated kernel systems.

The more work these layers automate, the less often an application developer needs to think directly about CUDA.

Cross-Chip Software Can Reduce Lock-In

CUDA’s lock-in is strongest when an application and its optimized kernels are tied closely to NVIDIA hardware.

A portable inference layer aims for this workflow:

model
→ portable execution layer
→ generated or selected hardware kernels
→ target accelerator

The real world is harder because chips have different memory hierarchies, numeric formats, interconnects, scheduling behavior, and supported operations.

High performance often still requires hardware-specific work.

Infinity’s thesis is that agents can generate that specialized work automatically.

CUDA’s Moat Is Not Only Existing Code

The 10-hour result does not reproduce the assets that make CUDA difficult to replace.

Those assets include:

Mature Libraries

Many organizations use vendor libraries rather than writing kernels directly. cuBLAS, cuDNN, TensorRT, NCCL, and other libraries encode years of optimization.

Debugging and Profiling

NVIDIA Nsight and related tools help engineers understand correctness, memory behavior, timelines, communication, and performance.

Framework Support

New model features are commonly integrated and optimized for NVIDIA hardware early.

Deployment Knowledge

Production teams understand how NVIDIA systems behave under load.

Documentation and Training

The ecosystem contains examples, courses, conference talks, community answers, and specialists.

Existing Investment

Companies have millions of lines of code, tests, procurement processes, and employee skills tied to the platform.

AI may reduce translation cost. It does not automatically remove organizational switching costs.

Verification May Become the Next CUDA Moat

Bing Xu, founder of INT21 and previously the founder of NVIDIA-acquired HippoML, argues that verification is the main bottleneck.

An agent can generate code quickly. Production teams still need evidence that it:

• Produces correct numerical output.
• Works across shapes and data types.
• Handles edge cases.
• Does not corrupt memory.
• Remains stable under concurrency.
• Performs well across workloads.
• Survives driver and hardware changes.
• Interacts correctly with the rest of the stack.

NVIDIA already owns deep collections of conformance tests, performance tests, reference implementations, simulators, profilers, compiler diagnostics, production workloads, and historical bug data.

Those assets make an agent more useful.

AI may therefore shift the moat from code generation toward the quality of the verification environment.

NVIDIA Is Building Its Own CUDA Agents

The technology challenging CUDA is also available to NVIDIA.

Business Insider quoted NVIDIA developer-ecosystem vice president Ankit Patel saying the company uses AI coding agents to develop CUDA faster and validate at greater scale.

NVIDIA has also presented a CUDA Intelligence strategy combining current CUDA knowledge, optimization expertise, cloud profiling, Nsight tools, benchmarks, and MCP-based services.

NVIDIA maintains ComputeEval, an open benchmark for AI-generated CUDA and CUDA Core Compute Libraries code.

The benchmark includes tasks involving Tensor Cores, shared memory, warp-level primitives, CUDA Graphs, Streams, and Events.

The race is therefore relative:

challengers’ rate of software catch-up
versus
NVIDIA’s rate of software improvement

Chris Lattner: The Hype Is Real but Overstated

Chris Lattner, co-founder and CEO of Modular, offers a more skeptical view.

He argues that coding agents provide an incremental improvement rather than an instant destruction of CUDA’s advantage.

The source article summarizes three reasons.

Writing Code Is Only One Part of Engineering

Production optimization determines whether a chip is economically useful. The final percentage of performance can require significant specialist work.

GPU Software Has Less Public Training Data

Application code is abundant online. High-end kernel and compiler engineering is a smaller field, and much of the strongest work remains private.

Existing Systems Still Need Migration

Technical feasibility does not erase qualification cost, operational risk, staff retraining, contracts, reliability requirements, or opportunity cost.

These points do not make agentic kernel engineering unimportant. They explain why a powerful demonstration does not immediately become market replacement.

The Moat Is Moving, Not Vanishing

The strongest interpretation is not “CUDA is dead.”

It is that one layer of CUDA’s historical advantage is becoming cheaper to reproduce.

Agents, DSLs, and automated compilers can reduce the time required to build kernels, runtimes, and model support for a new chip.

The most exposed layer is early inference enablement.

The most protected layers remain large-scale training, mature production libraries, verification, debugging, cluster orchestration, framework integration, existing customer workflows, and continuous optimization.

The strategic question may move from:

Who owns the largest amount of handwritten kernel code?

Toward:

Who owns the best automated generation, measurement,
verification, and optimization loop?

NVIDIA is well positioned because it already owns the hardware, tools, libraries, workloads, and feedback data. Challengers benefit because agents lower the cost of entry.

Both things can be true at once.

What the 10-Hour Result Really Changes

Infinity’s result changes expectations for hardware startups.

A new accelerator no longer has to assume that every useful kernel will be written manually by a small team of specialists.

An agent can potentially:

1. Read the architecture description.
2. Generate an initial kernel.
3. Compile and run it.
4. Compare output with a reference.
5. Measure hardware utilization.
6. Search alternative schedules.
7. Retain the best implementation.
8. Expand from operators to a complete model.

This can shorten the time between first silicon and useful model inference.

That reduction allows a chip vendor to demonstrate hardware earlier, support new models faster, reduce software staffing pressure, test more ideas, and compete in narrower inference markets.

The result does not erase CUDA. It makes the first step toward competing with CUDA less intimidating.

What to Watch Next

Independent Reproduction

Infinity’s published results come from the company and its hardware partner. External benchmarks would provide stronger evidence.

Model Coverage

A universal inference layer must handle many architectures, modalities, quantization formats, and serving patterns.

Production Reliability

A demo that runs end to end is different from software that operates for months under customer traffic.

Verification Scale

The key question is how agentic systems prove correctness and performance across a large test space.

Portability

TileLang and other DSLs become more strategic if one implementation can deliver strong results across NVIDIA, AMD, Apple, and additional accelerators.

NVIDIA’s Response

NVIDIA is actively building agents, benchmarks, compiler intelligence, and new CUDA abstractions. The incumbent is not standing still.

常见问题

Did an AI agent really recreate CUDA in 10 hours?

No. Infinity says Ignition generated tensor-parallel matrix-multiplication software for d-Matrix Corsair within 10 hours and reached up to 92% of the chip’s empirical compute ceiling. End-to-end Qwen3 inference took about 10 days, and the project did not reproduce CUDA’s complete ecosystem.

What is Infinity Ignition?

Ignition is Infinity’s AI research and engineering agent for generating, testing, debugging, and optimizing low-level inference software. It uses feedback from real hardware to improve kernels and model runtimes iteratively.

What is d-Matrix Corsair?

Corsair is a generative-AI inference platform from d-Matrix. Infinity used it as a target for automatically generated tensor-parallel operations and a full model-inference stack.

Does DeepSeek TileKernels replace CUDA?

No. TileKernels reduces the need to handwrite low-level CUDA kernels by using TileLang, but its current requirements include NVIDIA SM90 or SM100 hardware and CUDA Toolkit 13.1 or newer.

Why is inference more open to CUDA alternatives than training?

Inference can run on smaller systems and is often evaluated by cost, power, throughput, or latency for one model. Frontier training requires much larger clusters, mature communication systems, fault recovery, and proven stability.

What is CUDA’s biggest remaining moat?

CUDA’s moat includes mature libraries, framework integrations, debugging and profiling tools, distributed systems, documentation, production history, and existing customer code. Verification and continuous optimization may become even more important as code generation becomes cheaper.

Can AI-generated kernels be correct but too slow?

Yes. Research has shown that kernels can pass numerical correctness tests while performing far below optimized library implementations. Production evaluation needs both correctness and hardware-efficiency checks.

Is NVIDIA using AI coding agents too?

Yes. NVIDIA says it uses agents to accelerate CUDA development and validation, and it has publicly demonstrated CUDA-focused agent workflows, profiling integrations, and the ComputeEval benchmark.

相关工具

• NVIDIA CUDA: NVIDIA’s accelerated-computing platform, including compilers, runtimes, libraries, and developer tools.
• Infinity: The company building Ignition and model-aware inference software for new AI accelerators.
• DeepSeek TileKernels: DeepSeek’s MIT-licensed collection of optimized LLM kernels written with TileLang.
• TileLang: A Pythonic domain-specific language and compiler stack for high-performance kernels across multiple backends.
• INT21 PTX Kernel Factory: An agentic system for generating and improving low-level NVIDIA GPU kernels.
• AMD ROCm: AMD’s software platform for GPU computing, AI, and HPC.
• AWS Neuron: The SDK for deploying and optimizing models on AWS Trainium and Inferentia chips.
• Modular MAX: A hardware-oriented modeling and serving framework for portable high-performance AI execution.

Related Links

• Business Insider: AI Is Rewriting NVIDIA’s Software Moat: The primary report containing interviews with Infinity, NVIDIA, INT21, Modular, and Rebellions.
• Infinity d-Matrix Corsair Case Study: Infinity’s official account of the 10-hour matrix-multiplication result and 10-day end-to-end model bring-up.
• Infinity Research: Official case studies covering d-Matrix, generated inference software, and the company’s OMEGA research system.
• NVIDIA CUDA Platform: Official overview of CUDA’s compiler, runtime, libraries, tools, and ecosystem.
• DeepSeek TileKernels Repository: Official source code, requirements, testing commands, features, and MIT license.
• TileLang Paper: The research paper describing TileLang’s tiled programming model and compiler approach.
• NVIDIA ComputeEval: NVIDIA’s benchmark for evaluating modern CUDA programming tasks.

Summary

Infinity’s Ignition agent did not recreate NVIDIA CUDA in 10 hours. It generated high-performance tensor-parallel matrix multiplication for an unfamiliar d-Matrix accelerator in that time and reportedly reached up to 92% of the chip’s empirical compute ceiling. A complete Qwen3 inference path followed in about 10 days.

That result still matters. It shows that agents can compress early software bring-up for new AI chips, especially in inference, where customers care about cost per answer and can adopt specialized hardware for narrower workloads.

DeepSeek’s TileKernels and TileLang point in the same direction: more kernel work can be expressed through higher-level systems and optimized automatically. The current TileKernels release still depends on recent NVIDIA GPUs and CUDA, so it reduces handwritten CUDA work rather than eliminating the platform.

CUDA’s moat remains deep because it includes far more than source code. Libraries, validation, profiling, framework support, distributed training, production history, and existing organizational investment are difficult to reproduce.

AI has not crossed CUDA’s entire moat in 10 hours—but it may have lowered the cost of reaching the first wall, moving the long-term competition from code ownership toward automated generation, verification, and optimization.