🔍 Read the full analysis: Fine-Tuning AI Models For 3D Generation: A Guide on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face has published scripts and containers for adapting its TRELLIS.2 3D-generation model to custom asset datasets, locally or through Hugging Face Jobs. An example uses LoRA adapters and 188 footwear objects to target shoe shape, but the supplied material reports no completed evaluation or measured improvement.
Hugging Face has published scripts and containers that let developers fine-tune its TRELLIS.2 model on custom 3D assets, either on a local machine or through its cloud-based Jobs service. The accompanying example uses LoRA adapters and 188 footwear objects to test whether training can address inaccurate shoe soles, but the source material does not report a completed result or demonstrate that the model improved.
The example targets a specific weakness observed in the authors’ initial tests: generated shoes generally looked like shoes, but their soles were inaccurate. The proposed experiment uses a subset of 188 footwear objects from Google Scanned Objects to adapt the model toward that category. The post presents this as a training plan, not as evidence of a successful fix.
The scripts use low-rank adaptation, or LoRA, a method that adds small trainable adapter matrices while keeping the model’s original weights frozen. Hugging Face says this reduces the number of parameters being trained and the memory used for gradients and optimizer state. The workflow uses the PEFT library, and provides options for running preprocessing and training locally with Docker or through containerized Hugging Face Jobs.
For the footwear example, adapters are added to TRELLIS.2’s sparse-structure and shape-generation stages, which influence an object’s overall form and geometry. The texture stage is left unchanged: according to the post, the dataset has diffuse color maps but lacks the separate material maps needed to train that component, while the existing model’s colors and textures were considered satisfactory for the task. Users need a Hugging Face token and access to the gated DINOv3 model used to encode input images.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Workflow for Category-Specific 3D
The release gives developers a published route for adapting a general-purpose 3D-generation model to a narrower collection of objects or features. That could matter in work where a model needs more than a plausible appearance: geometry must fit the task in fields such as robotics, simulation and media production. A shoe that looks convincing from a distance may still be unsuitable if its sole shape is wrong.
The example also makes clear that fine-tuning involves choices about which parts of a generation pipeline to train. Focusing on structure and shape, rather than changing texture generation, is the authors’ rationale for this experiment. It may help teams target the component related to their problem, but the source provides no measurements showing that this selection saves a particular amount of compute or produces better results.
Providing containers and scripts for both local use and cloud jobs may make it easier for asset teams to try the workflow with their own data. Whether it is practical will depend on factors not settled in the post, including dataset preparation, hardware requirements and generalization to objects outside the training set.
Popular right now
De'Longhi Classic Signature Espresso Machine with Adjustable Milk Frother
Vitamix Touch & Go Advance 32-Ounce Bar Blender, White
Ninja Foodi 10-Qt DualZone XL Air Fryer, 6-in-1, Gray (DZ401)
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Produces 3D Assets
TRELLIS.2 uses a pipeline that turns an input image into a textured 3D mesh. As described in the source, it first removes the image background with BiRefNet and encodes the image with DINOv3-L. A sparse-structure flow model predicts a coarse shape, which guides a shape flow model that generates more detailed geometry. A separate texture flow model produces surface materials; the outputs are combined into a mesh that can be exported as a GLB file.
The post describes TRELLIS.2 as building on TRELLIS’s unified latent representation, which is intended to support different 3D output formats through decoders. The footwear experiment does not fine-tune every part of the pipeline: it targets the stages the authors associate with shape and geometry, while leaving texture generation untouched. This is a description of their chosen setup, not a comparison establishing that it is better than alternative training plans.
Hugging Face says developers can upload their mesh dataset to the Hugging Face Hub and adjust the scripts using a few arguments. The supplied account describes local Docker commands as well as cloud execution through Jobs. It does not provide a specific hardware configuration, training duration or cost estimate for the shoe example.
“The example targets footwear geometry with LoRA adapters and a subset of 188 shoes from Google Scanned Objects.”
— Hugging Face, as described in the supplied post
The Shoe Experiment Has No Results Yet
The supplied material does not say whether training has finished or whether the fine-tuned model produces more accurate soles. It includes no before-and-after examples, benchmark scores or independent evaluation. It also does not specify the test images, the criteria used to judge sole accuracy, or whether any assessment would involve people or automated measures.
Practical requirements remain unclear as well. The source gives no precise training time, compute configuration, cloud cost or detailed reproducibility results. Nor does it report how the adapted model performs on footwear outside the 188-object subset. These gaps mean the release establishes that a workflow is available, but not that this particular fine-tuning approach improves output quality or transfers reliably to new objects.
Run the Workflow and Test It
Developers can use the published repository to prepare their own mesh dataset, adjust the scripts and run preprocessing and training locally or through Hugging Face Jobs. They will need a Hugging Face token and access to gated DINOv3, along with hardware or cloud capacity suitable for the workload.
The next meaningful evidence would be a completed evaluation showing how fine-tuned outputs compare with the original model under a stated test method. Such a report could clarify whether the approach improves sole geometry, what resources training takes, and whether gains hold for footwear not represented in the training subset. The supplied source does not state when or whether those results will be published.
Key Questions
What did Hugging Face release?
It published scripts and containers for fine-tuning TRELLIS.2 on custom 3D assets. The workflow is designed to run locally or through Hugging Face Jobs.
What does the footwear example test?
It uses LoRA adapters and 188 footwear objects from Google Scanned Objects to explore whether fine-tuning can improve shoe shape, including inaccurate soles observed in initial tests.
Has the shoe model been shown to improve?
Not in the supplied material. It reports the experiment’s setup but gives no completed outcome, before-and-after comparison or measured improvement.
What access does the workflow require?
Users need a Hugging Face token and access to gated DINOv3, which the workflow uses to encode input images. They also need to prepare a suitable 3D asset dataset and provide local or cloud compute.
Why does the example leave texture generation unchanged?
The post says the footwear dataset has diffuse color maps but not the separate material maps needed to train the texture stage. It also says the base model’s colors and textures were already satisfactory for the task.
Primary source: Hugging Face · via ThorstenMeyerAI.com
