daanelson/motion_diffusion_model 📝❓🔢 → ❓

▶️ 14.2K runs 📅 Dec 2022 ⚙️ Cog 0.6.1 🔗 GitHub 📄 Paper ⚖️ License

action-to-motion animation human-motion json-output motion-synthesis text-to-motion text-to-video unconstrained-motion

About

A diffusion model for generating human motion video from a text prompt

Example Output

Prompt:

"the person walked forward and is picking up his toolbox."

Output

Performance Metrics

145.38s Prediction Time

323.59s Total Time

All Input Parameters

{
  "prompt": "the person walked forward and is picking up his toolbox.",
  "num_repetitions": 3
}

Input Parameters

prompt Type: stringDefault: the person walked forward and is picking up his toolbox.
output_format Default: animation: Choose the format of the output, either an animation or a json file of the animation data. The json format is: {"thetas": [...], "root_translation": [...], "joint_map": [...]}, where "thetas" is an [nframes x njoints x 3] array of joint rotations in degrees, "root_translation" is an [nframes x 3] array of (X, Y, Z) positions of the root, and "joint_map" is a list mapping the SMPL joint index to the corresponding HumanIK joint name
num_repetitions Type: integerDefault: 3: How many

Output Schema

Output

Example Execution Logs

### Sampling [repetitions #0]
1
2
created 1 samples
### Sampling [repetitions #1]
1
2
created 2 samples
### Sampling [repetitions #2]
1
2
created 3 samples
["the person walked forward and is picking up his toolbox." (00) | Rep #00 | -> sample00_rep00.mp4]
["the person walked forward and is picking up his toolbox." (00) | Rep #01 | -> sample00_rep01.mp4]
["the person walked forward and is picking up his toolbox." (00) | Rep #02 | -> sample00_rep02.mp4]
[ "the person walked forward and is picking up his toolbox." (00) | all repetitions | -> sample00.mp4]
[samples 00 to 00 | all repetitions | -> samples_00_to_00.mp4]

Version Details

Version ID: 3e2218c061c18b2a7388dd91b6677b6515529d4db4d719a6513a23522d23cfa7
Version Created: February 10, 2023

Run on Replicate →