README

!

 Native and Compact Structured Latents for 3D Generation







https://github.com/user-attachments/assets/63b43a7e-acc7-4c81-a900-6da450527d8f

(Compressed version due to GitHub size limits. See the full-quality video on our project page!)

TRELLIS.2 is a state-of-the-art large 3D generative model (4B parameters) designed for high-fidelity image-to-3D generation. It leverages a novel "field-free" sparse voxel structure termed O-Voxel to reconstruct and generate arbitrary 3D assets with complex topologies, sharp features, and full PBR materials.


 ✨ Features

 1. High Quality, Resolution & Efficiency
Our 4B-parameter model generates high-resolution fully textured assets with exceptional fidelity and efficiency using vanilla DiTs. It utilizes a Sparse 3D VAE with 16× spatial downsampling to encode assets into a compact latent space.

| Resolution | Total Time | Breakdown (Shape + Mat) |
| :--- | :--- | :--- |
| 512³ | ~3s | 2s + 1s |
| 1024³ | ~17s | 10s + 7s |
| 1536³ | ~60s | 35s + 25s |

Tested on NVIDIA H100 GPU.

 2. Arbitrary Topology Handling
The O-Voxel representation breaks the limits of iso-surface fields. It robustly handles complex structures without lossy conversion:
   ✅ Open Surfaces (e.g., clothing, leaves)
   ✅ Non-manifold Geometry
   ✅ Internal Enclosed Structures

 3. Rich Texture Modeling
Beyond basic colors, TRELLIS.2 models arbitrary surface attributes including Base Color, Roughness, Metallic, and Opacity, enabling photorealistic rendering and transparency support.

 4. Minimalist Processing
Data processing is streamlined for instant conversions that are fully rendering-free and optimization-free.
    📂 Please refer to data_toolkit/README.md for detailed instructions on data preprocessing and dataset organization.

 2. Running Training

Training is managed through the train.py script, which accepts multiple command-line arguments to configure experiments:

 --config: Path to the experiment configuration file.
 --output_dir: Directory for training outputs.
 --load_dir: Directory to load checkpoints from (defaults to output_dir).
 --ckpt: Checkpoint step to resume from (defaults to the latest).
 --data_dir: Dataset path or a JSON string specifying dataset locations.
 --auto_retry: Number of automatic retries upon failure.
 --tryrun: Perform a dry run without actual training.
 --profile: Enable training profiling.
 --num_nodes: Number of nodes for distributed training.
 --node_rank: Rank of the current node.
 --num_gpus: Number of GPUs per node (defaults to all available GPUs).
 --master_addr: Master node address for distributed training.
 --master_port: Port for distributed training communication.


 SC-VAE Training


To train the shape SC-VAE, run:

sh
python train.py \
  --config configs/scvae/shape_vae_next_dc_f16c32_fp16.json \
  --output_dir results/shape_vae_next_dc_f16c32_fp16 \
  --data_dir "{\"ObjaverseXL_sketchfab\": {\"base\": \"datasets/ObjaverseXL_sketchfab\", \"mesh_dump\": \"datasets/ObjaverseXL_sketchfab/m