README

UniMate

One Unified Model to Animate Diverse Skeletons


  
  
  
  
  



  Linzhan Mou ·
  Jiahui Lei ·
  Zhiyang Dou ·
  Chenyue Cai ·
  Chaoyue Song ·
  Adam Finkelstein ·
  Szymon Rusinkiewicz


Princeton · UC Berkeley · MIT · NTU


    


---

 🔥 News

- [2026-10-05] Bring your own rig: rig_preprocess turns any rigged GLB / glTF / FBX, animated or not, into an asset UniMate can animate, with a review step for its joint labels and facing; the sampler takes dataset skeletons and your own assets by reference. UniML3D adds training-ready features, canonical rest-pose meshes and generic / detailed captions. 🛠️
- [2026-09-27] Preview checkpoints are released at HuggingFace; new checkpoints will be synced there. 📦
- [2026-09-06] The training and inference code is released. 🚀
- [2026-08-30] The UniML3D dataset and its data-processing pipeline are released. 🚀
- [2026-08-01] Our Interactive Demo is live — browse our animation results in 3D. 🎮
- [2026-07-18] UniMate is accepted to SIGGRAPH Asia 2026! 🎉

 🛠️ Environment Setup

All components share a single conda environment, specified in requirements.txt:

bash
conda create -n unimate python=3.10 -y
conda activate unimate
pip install "setuptools
    


The processed dataset is on the Hub as UniML3D: the exported clips with all captions and annotations, the training-ready features, and canonical rest-pose meshes (everything except the Truebones motions and meshes). See data_process/README.md for the full data processing pipeline that turns the raw assets into UniML3D (download → export → rendering → captioning → joint annotation → feature extraction → animation).

 🏋️ Training

Training reads the canonicalized clips under dataset/features//. Download them, or build them with stage 4 of the data-processing pipeline (the Truebones clips must be built from the purchased pack):

bash
hf download Linzhan/UniML3D --repo-type dataset --local-dir dataset \
    --include "features/" --exclude "features//videos/" --exclude "features//tpose/"


Runs are configured by the JSON files in configs/.


Config naming — {dataset}_{frames}frames_{attention}_{text_cond}.json

8 configs: 4 data combinations x 2 model variants, all at 60 frames.

| Prefix | Training data |
|--------|---------------|
| uniml3d_ | Full UniML3D dataset (Truebones + Mixamo + Objaverse) |
| truebones_ / mixamo_ / objaverse_ | A single source |

| Length | dataset.max_motion_length |
|--------|-----------------------------|
| 60frames | 60 frames per clip |

| Suffix | model.attention x model.text_cond |
|--------|---------------------------------------|
| _graph_adaln | graph x adaln — attention factored into spatial (per frame) and temporal (per joint) passes with graph-distance, edge-type and depth biases; the caption is folded into the adaLN modulation |
| _full_cross_attn | full x cross_attn — one attention over the flattened joint x time tokens; the caption enters every block as cross-attention keys/values |

The two axes are independent and