An open-source AI accelerator named OpenTPU has been released by AI researchers, offering a complete hardware design stack that can be examined and modified from high-level algorithms down to physical implementation.

The project, hosted on GitHub, provides a full stack including hardware description language code, instruction set architecture, a cycle-accurate simulator, compiler tools, and host software for controlling a physical PCIe card. The accelerator runs on a Xilinx Kintex-7 FPGA within an Inspur YPCB-00338 development board featuring two DDR3 memory channels.

OpenTPU allows researchers to study how AI workloads map to hardware by executing models such as LFM2.5-230M, Qwen3-0.6B, and Gemma 4 variants directly on the FPGA card. Performance measurements show token generation rates ranging from 3.78 tokens per second for the largest Gemma 4 E4B model to 85.8 tokens per second for Qwen3-0.6B with 4-bit quantization, with memory bandwidth utilization between 82% and 94% of peak DDR3 capacity.

The system operates by executing model inference through a streamlined pipeline where the FPGA handles matrix operations and activation functions while the host CPU manages control flow. A key innovation is the use of a lightweight instruction set architecture that eliminates hidden scheduling, making performance characteristics fully traceable through cycle-accurate simulation.

Recent updates to the design have improved prefill performance by up to 2x compared to earlier versions while maintaining bit-for-bit numerical equivalence with the reference simulator. The latest production build (B, commit 79c5707a) runs at 133.33 MHz and achieves 91-94% of DDR3 peak bandwidth during decode operations.

OpenTPU also supports mixture-of-experts architectures where specialized sub-networks are streamed from host storage as needed, enabling inference for models larger than the card's 4GB memory capacity. Quantization techniques using 4-bit FP4 values with two-level scaling reduce memory requirements by approximately one-third while increasing decode speed by 40-45% compared to full 8-bit implementations.

The project includes tools for profiling execution through a visualizer called Lens, which displays instruction-level activity across different hardware units. Users can run the entire stack on a standard laptop using the open-source toolchain, with FPGA bitstream generation requiring only Python and Verilator for simulation.

OpenTPU is released under the Apache License 2.0, with documentation detailing the instruction set architecture, compiler workflows, and hardware integration. The developers encourage contributions through pull requests, though all modifications must preserve the existing test suite and performance validation methodology.