# Character Animation for Agents: rig and animate any 3D model on ZeroGPU (agent guide)

> Give a static 3D model (glb, gltf, obj or fbx) and a sentence; get back a rigged, animated GLB. Two models,
> one service: SkinTokens / TokenRig (ACM TOG 2026, MIT) predicts the skeleton and the skin weights, and UniMate
> (SIGGRAPH Asia 2026, CC BY-NC 4.0) generates a 60-frame (2 s, 30 fps) motion for that skeleton from text,
> with in-betweening, joint editing and multi-prompt expansion. Between the two, the rig becomes a reusable
> *asset* on this Space (joint labels, facing pair, canonical T-pose: the upstream rig_preprocess pipeline).
> One synchronous call does everything (POST /api/animate_mesh, about 40-90 s); every stage is also its own
> call, and every result is kept 24 h.

Base URL: https://townbox-character-animation-for-agents.hf.space
This is a public Hugging Face Space. Send `Authorization: Bearer $HF_TOKEN` (any Hugging Face account token)
with every request when you can: ZeroGPU time (tens of seconds per rig, a few seconds per motion) is then
billed to that account's daily quota instead of the small anonymous per-IP allowance, and never to the Space.
GET https://townbox-character-animation-for-agents.hf.space/api/identity shows who pays for the headers you send.

Interfaces (same functionality):
- REST: OpenAPI at https://townbox-character-animation-for-agents.hf.space/openapi.json, Swagger UI at https://townbox-character-animation-for-agents.hf.space/docs, service description at https://townbox-character-animation-for-agents.hf.space/api.
- MCP (streamable HTTP): https://townbox-character-animation-for-agents.hf.space/gradio_api/mcp/ (schema: https://townbox-character-animation-for-agents.hf.space/gradio_api/mcp/schema).
  Tools: animate_mesh, rig, prepare_asset, animate, inbetween, edit_joints, expand, list_assets, get_asset,
  list_examples, get_result, view_rig (PNG), view_animation (PNG), view_asset (PNG), service_info, guide.
- Gradio clients (Python gradio_client, JS @gradio/client): the same functions as endpoints (/animate_mesh, ...);
  schema at https://townbox-character-animation-for-agents.hf.space/gradio_api/info.

Job ids tell the kind of result: r_ (a rig), a_ (a motion), p_ (an animate_mesh run with its three stages).
Asset ids: unitree_go2, eagle, shark (bundled skeletons) and u_<12 hex> (prepared here; derived from the rigged
file and the options, so the same request returns the same asset).

## Workflow A: one call (REST)
    curl -s -X POST https://townbox-character-animation-for-agents.hf.space/api/animate_mesh -H "Authorization: Bearer $HF_TOKEN" -H 'content-type: application/json' \
      -d '{"example": "giraffe", "prompt": "An object walks forward.", "seed": 7}'
    # your own model: the file as the body, options as query parameters
    curl -s -X POST "https://townbox-character-animation-for-agents.hf.space/api/animate_mesh?filename=robot.glb&prompt=An%20object%20walks%20forward." \
      -H "Authorization: Bearer $HF_TOKEN" -H 'content-type: application/octet-stream' --data-binary @robot.glb
    # or {"mesh_url": "https://..."} in JSON, or a multipart `file` field with the options as form fields
-> 200 {"job_id": "p_...", "state": "done", "mode": "generate", "prompt": "...",
        "rig_job_id": "r_...", "asset_id": "u_...", "animation_job_id": "a_...",
        "asset_summary": {"n_joints", "facing_pair", "joint_labels", "flagged_joints"},
        "urls": {"animation_glb", "contact_sheet_png", "skeleton_mp4", "joints_json", "motion_npy", "rigged_glb",
                 "rig_json", "rig_preview_png", "rig_preview_glb", "skin_npz", "asset_preview_png", "asset_rest_glb"},
        "stages": {"rig": {...the r_ result...}, "asset": {...the asset...}, "animation": {...the a_ result...}},
        "timing": {"rig_s", "asset_s", "animation_s", "total_s", "gpu_s"}, "provenance": {...}, "next": [...]}
Download urls.animation_glb (the rigged mesh with the skeleton and the generated animation, glTF 2.0) and look
at urls.contact_sheet_png (6 skeleton frames) and urls.asset_preview_png (what the motion model was told:
T-pose, facing arrow, labels). "prompts" (2-6 captions, a list or separated by |) instead of "prompt" makes a
chained long motion (an expansion, see below). "rig_job_id": "r_..." starts from a rig made earlier;
"asset": "u_..." from a prepared asset (no GPU for rigging then). Options of every stage are accepted in the same
body (rig: num_beams, rig_seed, keep_skeleton, preserve_original, postprocess, influences; asset: labels,
face_right, face_left, body_axis, rest_rotation; motion: num_samples, cfg_scale, seed, overlap, preview).
The result's "next" list holds ready-made bodies for the follow-ups: in-betweening, joint editing, expansion on
the a_ job, another prompt on the same asset, a label / facing correction of the asset.

## Workflow B: the stages, one call each
1. Rig: POST https://townbox-character-animation-for-agents.hf.space/api/rig (MCP: rig). The mesh as the raw body (?filename=mesh.glb), a multipart `file`,
   or JSON {"example": "giraffe"} / {"mesh_url": "https://..."}; options num_beams (1-20, default 10), seed,
   keep_skeleton, preserve_original, postprocess, influences, top_k, top_p, temperature, repetition_penalty,
   preview. About 20-60 s of GPU.
   -> {"job_id": "r_...", "input": {"filename", "n_vertices", "n_faces", "had_skeleton"}, "options": {...},
       "rig": {"n_joints", "root", "max_depth", "leaves", "bone_length_mean", "skin": {...}},
       "urls": {"rigged_glb", "rig_json", "skin_npz", "preview_png", "preview_glb"}, "timing": {...}, "next": [...]}
2. Asset: POST https://townbox-character-animation-for-agents.hf.space/api/assets {"rig_job_id": "r_..."} (MCP: prepare_asset). No GPU, about 5-30 s of CPU.
   -> 201 {"id": "u_...", "n_joints", "joint_names", "joint_labels", "parents", "facing_pair": {"right", "left",
          "right_label", "left_label", "body_axis"}, "annotation": {"face_pair", "n_flagged", "flagged_joints",
          "notes", "label_source", "how_to_fix"}, "urls": {"preview_png", "annotation_preview_png",
          "annotation_json", "source", "log"}, "rest_pose_glb_url", "created": true, "next": [...]}
   Also accepted: a rigged glb/gltf/fbx of your own (raw body with ?filename=, multipart `file`, or
   {"mesh_url": ...}: one armature with a skinned mesh, any bone names), a zipped output of the UniMate
   repository's `data_process.rig_preprocess` (cond.npy + canonical GLB), and {"asset_id": "u_..."} with
   corrections (re-annotation; a new id comes back).
3. Animate: POST https://townbox-character-animation-for-agents.hf.space/api/animate {"asset": "u_...", "prompt": "An object walks forward.", "num_samples": 1,
   "cfg_scale": 3.0, "seed": 7} (MCP: animate). About 10 s wall, 7 s of GPU per sample.
   -> {"job_id": "a_...", "mode": "generate", "frames": 60, "samples": [{"index": 0, "urls": {"animation_glb",
       "motion_npy", "joints_json", "skeleton_mp4", "contact_sheet_png"}, "root_travel_xz", "root_height_range"}],
       "timing": {"condition_s", "gpu_s", "inference_s", "outputs_s"}, "provenance": {...}, "next": [...]}
   The bundled skeletons work without rigging: "asset": "unitree_go2" (quadruped robot, 18 joints), "eagle"
   (winged biped, 51 joints), "shark" (fish, 16 joints).
4. Refine: POST /api/inbetween, /api/edit (reference_job_id = an a_ job of the same asset) or /api/expand
   (the asset and 2-6 prompts). Same result shape as /api/animate plus the fields below.
5. Read a result again: GET https://townbox-character-animation-for-agents.hf.space/api/jobs/<job_id> (r_, a_ or p_; 24 h). GET https://townbox-character-animation-for-agents.hf.space/api/assets/<id>
   for an asset (assets live until the Space restarts; keep your rigged.glb and prepare it again if needed).

## Assets: what the motion model is told, and how to check it
UniMate is conditioned on the skeleton's topology, its canonical T-pose, a *label* per joint from one anatomical
vocabulary shared by all training skeletons (Hips, Spine, Neck, Head, Left Thigh, Right Upper Arm, Tail, Left
Wing, ...) and a *facing pair*, two joints that define which way the character faces (the right / left thighs
or shoulders; a head / tail pair with body_axis for snakes and fish). Preparing an asset runs the UniMate
repository's own rig_preprocess pipeline on this Space: Blender imports the rigged file, prunes helper bones,
converts to Y-up; the offline annotation rules label the joints (bone names when they carry meaning; for a
SkinTokens rig, whose bones are bone_0 ... bone_N, a geometric labeller reads the rest pose: symmetry, limbs,
legs near the floor, trunk, head, tail) and pick the facing pair from the labels; stage 4 canonicalizes the
rest pose (faces +Z, centred, diameter 2, grounded) and computes the topology condition; the canonical GLB is
rebuilt in that frame so generated motion drives the mesh without retargeting.
Check it (two things decide motion quality and are invisible in the files):
- urls.preview_png (MCP view_asset): left, the canonical T-pose with the facing pair (red: right, blue: left)
  and the grey forward arrow (+Z); middle, the front view (the face or chest must be this side, the character's
  right on the image's left); right, every joint with its label.
- annotation.n_flagged and annotation.flagged_joints: placeholder labels ("Bone"), labels outside the vocabulary,
  Left / Right limbs that lie the other way round from the facing pair; annotation.face_pair.check with reasons
  (no pair found: the asset keeps its authored facing; a pair of an unusual part). GET /api/assets/<id>/annotation
  (MCP get_asset) lists every joint with raw name, parent, label and the vocabulary.
Fix it: POST https://townbox-character-animation-for-agents.hf.space/api/assets {"asset_id": "u_...", "labels": {"bone_3": "Left Thigh", "bone_7": "Right Thigh"},
"face_right": "bone_7", "face_left": "bone_3"} (joint names are the rig's bone names; face_right is the joint on
the character's own right). "body_axis": true for a head / tail pair (face_right = head). "rest_rotation": "x180"
(or "x90", "x-90", "x90,y180") stands up a model authored upside down or lying on its back / front; the notes say
when the rest pose looks upside down. Or send a whole edited annotation.json as "annotation". A corrected copy
gets a new id; animate with that. Typical cases: a quadruped whose front legs were labelled as arms (the parser's
convention; fine for motion), a model facing sideways (give the thigh pair), a bad rig (many "Bone" labels and
no facing pair: rig again with another seed or more beams rather than relabelling).

## Parameters
Rigging (POST /api/rig; in /api/animate_mesh the rig's seed is rig_seed):
- num_beams (1-20, default 10): beam-search width of the autoregressive decoder; 10 is the authors' default,
  1 is fastest but skeletons degrade, 20 is slower and rarely better.
- seed / rig_seed (integer, default random): the same mesh, options and seed reproduce the rig.
- keep_skeleton (default false): the file already has an armature; keep it and predict only the skin weights.
- preserve_original (default false): transfer the predicted rig onto the untouched input mesh, so materials,
  textures, UVs and world scale survive into rigged.glb and into the animated GLB. Default false writes the mesh
  in the normalized frame (centred, scaled into [-1, 1]) without materials, which is what the model saw.
- postprocess (default false): voxel-based masking of the skin weights before normalization; a few seconds of
  CPU, can help with weights bleeding across nearby but unconnected parts.
- influences (1-8, default 4): bones kept per vertex in the GLB (the largest weights, renormalized).
- top_k (1-200, default 5), top_p (0.1-1.0, default 0.95), temperature (0.1-2.0, default 1.0),
  repetition_penalty (0.5-3.0, default 2.0): the decoder's sampling (the authors' settings).
Asset preparation (POST /api/assets, /api/animate_mesh): labels, face_right, face_left, body_axis, rest_rotation,
annotation (above).
Motion (POST /api/animate, /api/animate_mesh):
- asset (required for /api/animate): an id from GET /api/assets.
- prompt (required, <= 300 chars): a short caption in the training style. Start with "An object" and describe
  ONE motion, not the character: "An object walks forward.", "An object runs.", "An object jumps in place.",
  "An object turns around.", "An object sits down.", "An object waves its right hand.", "An object flaps its
  wings.", "An object swims and turns around.", "An object rears up on its hind legs.", "An object bites forward."
  The closer to that style, the better it is followed. Longer scene descriptions are followed worse.
- num_samples (1-4, default 1): independent samples of the same prompt (different noise). Samples vary; ask for
  several and keep the best (root_travel_xz and root_height_range in the result help to compare).
- cfg_scale (1.0-15.0, default 3.0): classifier-free guidance. Raise to 5-7 when the prompt is ignored; lower
  for smoother, calmer motion; 1.0 is unconditional (generate only).
- seed (integer, default random): same asset + prompt + seed + cfg_scale + num_samples reproduce the samples.
- preview (default true): also render the skeleton MP4 and contact sheet (about 1-2 s of CPU per sample).
Editing modes (the upstream options, all verified on the UniMate Space; the kept part of a reference motion is
pinned at every ODE step and the rest is denoised under the prompt; cfg_scale must be above 1.0; about 3 s of GPU
per sample for in-betweening and editing, 7 s per segment for expansion):
1. In-betweening: POST https://townbox-character-animation-for-agents.hf.space/api/inbetween (MCP: inbetween)
     {"reference_job_id": "a_...", "reference_sample": 0, "keep_frames": "0,-1", "prompt": "An object jumps in place."}
   keep_frames (default "0,-1": first and last frame): frame indices held exactly at the reference, comma-separated
   or a JSON list; negatives count from the end. The other frames are regenerated under the prompt. The result
   adds "reference", the resolved "keep_frames" and "generated_frames". Use it to change what happens between two
   poses, to make a clip loop (keep frame 0 and ask for a motion that returns to it), or to redo the middle of a
   segment of an expand result (reference_start = the segment's first frame).
2. Joint editing: POST https://townbox-character-animation-for-agents.hf.space/api/edit (MCP: edit_joints)
     {"reference_job_id": "a_...", "keep_joints": "Left Thigh, Right Thigh, Left Shin, Right Shin, Left Foot, Right Foot",
      "prompt": "An object waves its arms."}
   keep_joints (required): joint names or anatomical labels of the asset (GET /api/assets/<id>: joint_names and
   joint_labels), case-insensitive, comma-separated or a JSON list; a label such as "Left Wing" or "Tail" selects
   every joint carrying it; every requested name must match. Those joints keep the reference motion over all 60
   frames, the other joints are regenerated under the new prompt. Typical: keep the legs of a walk and ask for an
   upper-body action. The result adds "reference", "keep_joints" and "edited_joints".
3. Motion expansion: POST https://townbox-character-animation-for-agents.hf.space/api/expand (MCP: expand)
     {"asset": "u_...", "prompts": ["An object walks forward.", "An object turns around.", "An object sits down."], "overlap": 10}
   prompts: 2 to 6 captions, one 60-frame segment each, in order. overlap (1-59, default 10): the first `overlap`
   frames of each further segment are pinned to the last `overlap` frames of the previous one: 60 + (60 - overlap)
   x (prompts - 1) frames in one GLB, .npy and JSON (3 prompts: 160 frames, 5.3 s). num_samples x prompts is
   limited to 8 per call. The result adds "prompts", "overlap" and "segments" (frame ranges). Measured on the
   UniMate Space: the first regenerated frame after a seam can pop in orientation (33 deg at a trot-to-turn seam
   with overlap 10; larger overlaps made it worse): keep the default overlap, prefer prompts that follow
   naturally, re-run with another seed when a seam shows.
   reference_job_id, reference_sample (default 0) and reference_start (default 0; the first frame of the 60-frame
   window, useful on the longer expand results; a window that does not start at frame 0 is re-aligned to face +Z)
   select the reference of in-betweening and editing: any a_ result of the same asset (an animate_mesh result's
   animation_job_id).

## Outputs
Rig (r_ job): rigged.glb (glTF 2.0: mesh, armature with joint nodes bone_0 ... bone_{J-1}, bone_0 the root, skin
with at most `influences` bones per vertex; normalized frame unless preserve_original), rig.json (n_joints,
joint_names, parents (-1 = root), children, depth, root, leaves, max_depth, joints_gltf (rest positions, Y up),
joints_input_units_gltf, joints_blender, bone_length_mean, skin statistics: influences per vertex, dominant
weight, vertices with a dominant weight below 0.5, joints_with_no_skin (a sign of a spurious bone), frame),
skin.npz (vertices, faces, the full (N x J) float16 weights, joints, parents, joint_names; omitted above 48 MB),
preview.png (mesh with the skeleton, front and side; MCP view_rig), preview.glb (visible bones + a procedural
test wiggle, for looking only).
Asset (u_ id): rest.glb (GET /api/assets/<id>/rest.glb: the canonical rest-pose GLB the motions drive; Y up,
faces +Z, skeleton diameter 2, grounded), preview.png (three panels, above), annotation_preview.png (the rest
pose as exported with the chosen pair), annotation.json (the upstream review file), source (the rigged file the
asset was built from), rigprep.log.
Motion (a_ job, per sample): animation_glb (the asset's canonical mesh + skeleton + the generated animation,
glTF 2.0, one animation named after the prompt, 60 LINEAR keys at 30 fps; expand: 60 + 50 keys per extra prompt;
plays in any glTF viewer, Blender, three.js, Unity, Godot), motion_npy (NumPy (frames, J, 12) float32 UniMate
features: root-relative position 3, rotation 6D 6, local velocity 3 per joint; the root slot holds height, facing
and root velocity; exactly what `python -m unimate.inference.sample` writes), joints_json ({"fps", "frames",
"joint_names", "joint_labels", "parents", "positions": [frame][joint][x,y,z]} in the canonical frame),
skeleton_mp4 / contact_sheet_png (a rendered skeleton, red: root; the PNG shows 6 frames from 0 s to 2 s;
MCP view_animation).
Pipeline (p_ job): stages.rig, stages.asset, stages.animation (the three records above), a flat "urls" map and
"timing" (rig_s, asset_s, animation_s, total_s, gpu_s).

## Reading a result
- state is always "done" (calls are synchronous); errors come back as HTTP 4xx/5xx {"error", "hint"}.
- Rig quality: rig.n_joints and max_depth (a plain cube gets 100+ joints: the decoder always writes some rig),
  skin.joints_with_no_skin, the preview. Rig again with another seed or num_beams 10-20 when the skeleton looks
  wrong; the model saw humanoids, animals and articulated objects. The service retries once with the next seed
  when the decoder stops early (attempts in the result).
- Asset quality: annotation.n_flagged, facing_pair, the preview (above).
- Motion: root_travel_xz (ground distance the root moved; the skeleton is 2 units across) tells "walks forward"
  from "in place"; root_height_range; the contact sheet. timing.gpu_s is the ZeroGPU call (queue, worker start,
  inference); timing.inference_s the model alone. GET /api/gpu_check runs a 10 s probe that reports the GPU.
- provenance: both models' repos, revisions, checkpoints, code commits, licenses, and the asset pipeline.

## Errors and hints
- 400 no mesh / unsupported format / could not read the mesh (Blender 4.5 must import it; one triangle mesh at
  least) / keep_skeleton without an armature / out-of-range parameter / prompt empty or too long / prompts with
  fewer than 2 or more than 6 captions / num_samples x prompts above 8 / cfg_scale 1.0 in the editing modes /
  keep_joints names that match no joint (the hint lists the asset's names and labels) / labels or face_right
  naming joints the rig does not have (the error lists them) / a facing pair with no horizontal separation /
  could not prepare the asset (the message carries the pipeline's reason: several root bones, no skinned mesh,
  more than 71 joints).
- 404 unknown job, asset or example: jobs are kept 24 h, assets until the Space restarts.
- 413 upload or mesh_url larger than 60 MB.
- 500 "retry with another seed": the rigging decoder stopped before writing the skin tokens twice in a row.
  500 with "quota" or "GPU limit": your ZeroGPU quota is exhausted or no token reached the service; send
  `Authorization: Bearer $HF_TOKEN`, check GET /api/identity, or wait for the daily reset. 500 "no GPU was
  available": ZeroGPU had no free slice; retry in a moment.

## Limitations
- SkinTokens: joint names are generic (bone_i; keep_skeleton keeps the file's names), the skeleton is a single-
  rooted tree, very thin, disconnected or non-manifold parts can get odd bones or weights, shapes that are not
  characters still get an arbitrary skeleton. One mesh per call; multi-object files are merged.
- Asset preparation: labels of a SkinTokens rig come from geometry (symmetry, limbs, floor), so unusual bodies
  (many legs, no symmetry, a lying or upside-down rest pose) get placeholder labels or no facing pair; check the
  preview and correct, or give rest_rotation. The model takes at most 71 joints.
- UniMate: no contact model (feet can slide, hover or penetrate the ground); rare topologies and unusual motions
  can come out static, jittery or off-prompt; many motions and skeletons still fail, try several samples and
  phrasings and raise cfg_scale to 5-7 when a prompt is ignored. 60 frames per prompt (longer motions through
  /api/expand, up to 6 prompts, 310 frames); in-betweening and editing take their reference from a result of this
  service, not from uploaded motion files; rotations only (bone lengths stay those of the rest pose). The
  editing modes use a fixed-step Euler solver (50 steps, as upstream) instead of the adaptive dopri5 of plain
  generation.
- Results are kept 24 h, assets until the Space restarts; GPU time is billed to the caller's ZeroGPU quota.

## Examples
GET https://townbox-character-animation-for-agents.hf.space/api/examples lists the bundled meshes (giraffe: textured quadruped from the SkinTokens repository,
MIT; fox, cesium_man, rigged_figure: Khronos glTF sample characters, CC BY 4.0, with their original rigs removed),
the bundled skeletons (unitree_go2, eagle, shark: UniMate repository examples, MIT) and the real results this
Space produced from them with its own API (job ids, options, seeds, timings, joint counts, labels, file URLs
including motion_gif, an animated GIF preview of each generated motion) with the request that reproduces each. Files are served at https://townbox-character-animation-for-agents.hf.space/examples/<name>. They are a showcase only:
every API call generates fresh results.

## Provenance and licenses
- Rigging: Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging. Zhang, Pu, Guo,
  Cao, Hu. ACM TOG 2026, https://arxiv.org/abs/2602.04805. Code https://github.com/VAST-AI-Research/SkinTokens
  (MIT, commit 273b691d), vendored from the authors' Space https://huggingface.co/spaces/VAST-AI/SkinTokens at
  revision 1f4445ba (the file-based Blender worker; one line of its exporter fixed for meshes under 990
  vertices). Weights https://huggingface.co/VAST-AI/SkinTokens (MIT), revision 79736cad: grpo_1400.ckpt (TokenRig,
  595M parameters: Michelangelo mesh encoder + Qwen3-0.6B-architecture decoder) and the skin-token FSQ-CVAE.
  Qwen/Qwen3-0.6B (Apache 2.0) supplies only the architecture and tokenizer configuration. Blender: bpy 4.5.4
  (GPL; the Python wheel committed in the authors' Space) reads inputs, writes rigged GLBs and runs the asset
  preparation.
- Animation: UniMate: One Unified Model to Animate Diverse Skeletons. Mou, Lei, Dou, Cai, Song, Finkelstein,
  Rusinkiewicz. SIGGRAPH Asia 2026, https://arxiv.org/abs/2609.05415. Code https://github.com/Friedrich-M/UniMate
  (MIT), vendored at commit b78c7807: the inference path (unimate.inference.generate, motion_inbetweening,
  motion_editing, motion_expansion) and the asset pipeline (data_process.rig_preprocess and the stages it calls)
  are the repository's own, with two documented edits that drop the torch_geometric import. Weights
  https://huggingface.co/Linzhan/UniMate, folder unimate_uniml3d_f60_v3, checkpoint_step_150000.pt (EMA weights),
  revision 174fc755. License CC BY-NC 4.0: non-commercial research use; the training data keep their own terms
  (Truebones ZOO, Mixamo, Objaverse-XL), see the model card. Text encoder google/flan-t5-base (Apache 2.0),
  frozen. Mesh driving is a glTF re-implementation of scripts/run_animate_motion.sh (no Blender), checked against
  Blender's importer to 1e-5. The project page's 3D showcase is access-controlled and is not used here.
- Service code (app.py, engine_rig.py, engine_motion.py, rigprep.py, common.py, glb_animate.py, preview.py,
  index.html, this guide): this Space, MIT. Full table: PROVENANCE.md in the Space repository.