🔍 Read the full analysis: A Native TPU Path For AI With Hugging Face And TorchTPU on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face announced TorchTPU, a native PyTorch backend for using Google TPUs with supported Transformers, Diffusers and Kernels workflows. The company reports faster decoding for two models under a specific test setup, but has not supplied a full compatibility list or detailed benchmark methodology.
As detailed in the original TorchTPU analysis, Hugging Face has announced TorchTPU, a native PyTorch backend that lets supported workflows in its Transformers, Diffusers and Kernels libraries use Google Tensor Processing Units. The release gives developers a way to select a TPU through library interfaces, while the company’s reported speedups apply to specific kernels and test configurations rather than all PyTorch models or TPU workloads.
Hugging Face’s examples use familiar device-selection calls: a Transformers model can be loaded with device_map=”tpu”, while a Diffusers pipeline can be moved to a TPU with pipe.to(“tpu”), as in an all-in-one AI pipeline. The company says compiled execution is available as an option. The release also includes Pallas kernels tuned for TPUs, distributed training support and integration with torch.compile.
For larger models, Hugging Face says developers can split a model across chips during loading with a TensorParallelConfig, a useful approach in responsible AI operations. Its installation instructions target a TPU virtual machine and call for a CPU PyTorch wheel, rather than a CUDA build. These release details describe the intended setup; the announcement does not provide a complete list of supported models, operators or workflows.
Hugging Face reports that selected kernels from its Kernels Hub and Tokamax can improve decoding performance at larger batch sizes. On a TPU v6e-8 at batch size 64, the company reports decoding 21.5 times faster for Qwen3-30B-A3B and 12.8 times faster for Gemma 4 26B-A4B than each model’s default implementation. These are company-reported comparisons; the announcement does not include full measurement procedures or independent results. It also provides example image-generation timings on a TPU v6e-4, likewise as vendor-reported figures.
A Simpler Route to Google TPU Testing
TorchTPU addresses a practical hurdle for teams developing with PyTorch that want to evaluate Google TPU capacity. If supported models can use familiar Transformers and Diffusers interfaces, developers may have less TPU-specific code to maintain when trying the hardware. That could make it easier to compare TPU execution with other accelerator options, though the announcement does not establish that it will reduce costs or improve performance for a given deployment.
The potential benefit depends on model and operator coverage, reliability and the results teams see on their own workloads. Selecting a TPU as a device does not mean every PyTorch model or operation will run without changes. Nor do the reported speedups establish a general advantage: they refer to named models, a batch size of 64 and a specified TPU configuration, compared with each model’s default implementation.
As an affiliate, we earn on qualifying purchases.
From PyTorch XLA to TorchTPU
Before this announcement, Hugging Face says PyTorch users generally ran workloads on TPUs through PyTorch/XLA, also called torch_xla. The company describes that route as workable but says lazy tensor execution, graph recompilations after code changes and more difficult debugging could complicate development. Those are Hugging Face’s stated reasons for introducing a native backend, not an independent assessment of every XLA workflow.
Hugging Face previously offered Optimum TPU for inference and training with its models. The company says that package is no longer required to run Transformers and Diffusers models using the new native support. Google TPUs are specialized accelerators available through Google Cloud, from individual chips to larger pods; TorchTPU is intended to connect that hardware with Hugging Face’s PyTorch tools.
“With TorchTPU, a native PyTorch backend, TPU is now just another device.”
— Hugging Face
As an affiliate, we earn on qualifying purchases.
Compatibility and Benchmarks Await Testing
The announcement does not identify all supported and unsupported models, operators or workloads. It also provides no independent benchmark results, detailed measurement procedures or cost comparisons for the performance claims. Results may vary with model versions, batch sizes, TPU generations and software configurations, so the cited figures should not be treated as predictions for other deployments.
It is also unclear how much code adjustment or debugging may be needed for particular projects, and how TorchTPU’s operational behavior compares with existing TPU paths across a broad range of workloads. Hugging Face says it is running the Transformers test suite on TPU and working toward nightly TPU CI, but the announcement gives no date for that testing to become available or for a wider compatibility matrix to be published.
As an affiliate, we earn on qualifying purchases.
Nightly TPU Tests Are Planned
Hugging Face says it is working to add nightly testing for Transformers on TPU. Those runs could provide a clearer view of compatibility over time, but the company has not said when they will begin or what results will be made public. Developers evaluating the backend can consult the Diffusers TPU guide for distributed and compiled usage and follow the installation steps for TPU virtual machines.
The next useful evidence will be a published support picture, repeatable performance comparisons across representative workloads and updates on test coverage. Until then, teams will need to verify that their chosen models and operations work in their own TPU environment and measure performance under their own settings.
Tensor Processing Unit (TPU) for machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is TorchTPU?
TorchTPU is a PyTorch backend that Hugging Face says lets supported PyTorch workflows use Google TPUs as a device.
Which Hugging Face libraries are covered?
The announcement names Transformers, Diffusers and Kernels. It does not say that every model or operation in those libraries is supported.
Does TorchTPU replace PyTorch/XLA?
Hugging Face presents TorchTPU as a native backend and says its Optimum TPU package is no longer required for Transformers and Diffusers models using the new support. The announcement does not provide a full comparison with PyTorch/XLA or specify that every existing TPU workflow has been replaced.
How strong are the reported speedups?
Hugging Face reports 21.5-times-faster decoding for Qwen3-30B-A3B and 12.8-times-faster decoding for Gemma 4 26B-A4B on a TPU v6e-8 at batch size 64, compared with each model’s default implementation. The announcement does not give full benchmark methodology or independent confirmation, and the figures should not be generalized to other workloads.
What should developers watch for next?
Hugging Face says it is working toward nightly TPU testing for Transformers. A fuller compatibility list and more detailed benchmark information would help developers judge whether the backend fits their particular models and deployment needs.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
![Rails World 2026 Opening Keynote [Video]](https://designthinkingblog.com/wp-content/uploads/2026/09/rails-world-2026-opening-keynote-video-featured-560x300.jpg)
