🔍 Read the full analysis: A Closer Look At Fine-Tuning 3D Generation Models on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face published scripts and container images for fine-tuning its TRELLIS.2 3D-generation model on custom asset datasets, either locally or through Hugging Face Jobs. The example uses LoRA adapters and 188 footwear objects to target shoe shape, including inaccurate soles, but the supplied material reports no completed evaluation or measured improvement.
Hugging Face has published scripts and container images for adapting its TRELLIS.2 model to custom 3D asset datasets, with an example focused on improving shoe geometry. The example trains LoRA adapters on 188 footwear objects from Google Scanned Objects, but the accompanying material does not report whether training improved the model’s results.
The workflow can run on a user’s own machine or through Hugging Face Jobs, a service for running containerized workloads on cloud hardware. According to the source post, developers can upload a mesh dataset to the Hugging Face Hub and adjust the scripts’ arguments for their own data. Use requires a Hugging Face access token and permission to use DINOv3, a gated model used to encode input images.
The example responds to an issue described in the post’s initial tests: TRELLIS.2 could generate shoes that looked plausible but had inaccurate soles. The proposed test uses a subset of 188 Google Scanned Objects footwear models to see whether fine-tuning can improve that category. This is a stated research aim, not a reported result; no completed before-and-after comparison or measured gain is provided.
The scripts use low-rank adaptation, or LoRA, through the PEFT library. LoRA adds small trainable adapter matrices while keeping the original model weights frozen. Hugging Face says this reduces the number of parameters being trained and the memory needed for gradients and optimizer state. The post describes options for running both data preprocessing and training locally or through Jobs, but does not provide a hardware requirement or cost estimate.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Route to More Specific 3D Output
General-purpose 3D generation can produce an object that looks right at a glance while missing geometry that matters to a particular use. Category-specific fine-tuning gives developers a way to test whether a model can better match a narrow class of objects or features, rather than relying only on its original training.
That distinction matters in fields such as robotics, simulation and media production, where the shape of an object may affect how it can be used, rendered or tested. The footwear example makes the practical question concrete: whether a model can represent soles more accurately, not simply produce a recognizable shoe. However, the available account establishes the motivation and method, not that the technique solves the problem.
The release also offers a packaged route for experimenting with a custom dataset across local and cloud environments. Its value for other teams will depend on factors not settled in the post, including asset preparation, compute demands and whether any improvement carries over to objects outside the training data.
Products Worth Considering
Samsung Odyssey OLED G5 27-Inch QHD Gaming Monitor
Jackery Explorer 2000 v2 Portable Power Station with Two 200W Solar Panels
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Bluetooth 5.3, USB-C Charging, and IPX4 Water Resistance
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Is Adapted
TRELLIS.2 generates a textured 3D mesh through separate stages, as described in the source post. The pipeline removes an image background with BiRefNet, encodes the image with DINOv3-L, predicts coarse structure, and then generates detailed geometry. A separate stage produces surface materials; the stages are combined into a mesh that can be exported as a GLB file.
For the footwear experiment, the scripts add LoRA adapters to the sparse-structure and shape flow models, which affect the object’s form. The texture stage is left unchanged. The post says the footwear data has diffuse color maps but lacks the separate material maps needed to train that component, and that the base model already produces satisfactory colors and textures for this task.
This component selection is the authors’ rationale for the experiment, not evidence that it outperforms other training choices. The supplied material also does not report a completed result from the earlier TRELLIS model or provide an evaluation that would establish how much TRELLIS.2 improves on it.
The Shoe Results Are Unreported
The supplied account does not say whether the footwear training run was completed or whether it improved sole geometry. It includes no before-and-after examples, quantitative benchmark or independent evaluation. It also does not specify a test set, a scoring method for judging sole accuracy, or how well a fine-tuned model performs on footwear outside the 188-object subset.
Practical requirements remain unclear as well. The post describes local GPU and cloud Jobs options but gives no hardware configuration, training time, cost or reproducibility results. The quality and preparation requirements for users’ own mesh datasets are not quantified. Without those details, the release shows how to run an experiment, but not the expected performance or cost of using the workflow.
Running and Evaluating the Workflow
Developers can use the published repository to adapt the scripts, upload a mesh dataset to the Hugging Face Hub, and run preprocessing and training locally or through Jobs. They will need a Hugging Face token, access to gated DINOv3 and suitable compute. The source post also describes a walkthrough, but the supplied material does not give further details about its scope.
The next substantive evidence would be a report of the footwear run, including comparable outputs from the base and fine-tuned models, the evaluation criteria, and the compute and data requirements. Until those details are available, the practical effect of fine-tuning on shoe geometry—and whether the approach generalizes to other categories—remains an open question.
Key Questions
What did Hugging Face publish?
Scripts and container images for fine-tuning TRELLIS.2 on custom 3D assets, with options to run locally or through Hugging Face Jobs.
What does the footwear example test?
It uses LoRA adapters and 188 footwear objects from Google Scanned Objects to test whether fine-tuning can improve shoe geometry, including soles described as inaccurate in initial tests.
Has the example been shown to improve results?
Not in the supplied material. It reports no completed comparison, benchmark or before-and-after results.
What access do users need?
Users need a Hugging Face token and access to gated DINOv3, as well as a prepared mesh dataset and suitable local GPU or cloud compute.
Primary source: Hugging Face · via ThorstenMeyerAI.com
