haoheliu/audio-ldm 📝❓🔢 → 🖼️

▶️ 38.1K runs 📅 Feb 2023 ⚙️ Cog 0.6.1 🔗 GitHub 📄 Paper ⚖️ License

music-generation sound-effect-generation text-to-speech

About

Text-to-audio generation with latent diffusion models

Example Output

Output

Performance Metrics

25.44s Prediction Time

25.51s Total Time

All Input Parameters

{
  "text": "two starships are fighting in space with laser cannons",
  "duration": "5.0",
  "n_candidates": 3,
  "guidance_scale": 2.5
}

Input Parameters

text (required) Type: string: Text prompt from which to generate audio
duration Default: 5.0: Duration of the generated audio (in seconds). Higher duration may OOM.
random_seed Type: integer: Random seed for the model (optional)
n_candidates Type: integerDefault: 3: Return the best of n different candidate audios
guidance_scale Type: numberDefault: 2.5: Guidance scale for the model. (Large scale -> better quality and relavancy to text; small scale -> better diversity)

Output Schema

Output

Type: string • Format: uri

Example Execution Logs

DDIM Sampler:   0%|          | 0/200 [00:00<?, ?it/s]
DDIM Sampler:   0%|          | 1/200 [00:00<00:24,  8.17it/s]
DDIM Sampler:   1%|          | 2/200 [00:00<00:23,  8.31it/s]
DDIM Sampler:   2%|▏         | 3/200 [00:00<00:23,  8.44it/s]
DDIM Sampler:   2%|▏         | 4/200 [00:00<00:23,  8.51it/s]
DDIM Sampler:   2%|▎         | 5/200 [00:00<00:22,  8.49it/s]
DDIM Sampler:   3%|▎         | 6/200 [00:00<00:23,  8.36it/s]
DDIM Sampler:   4%|▎         | 7/200 [00:00<00:22,  8.43it/s]
DDIM Sampler:   4%|▍         | 8/200 [00:00<00:22,  8.47it/s]
DDIM Sampler:   4%|▍         | 9/200 [00:01<00:22,  8.47it/s]
DDIM Sampler:   5%|▌         | 10/200 [00:01<00:22,  8.46it/s]
DDIM Sampler:   6%|▌         | 11/200 [00:01<00:22,  8.47it/s]
DDIM Sampler:   6%|▌         | 12/200 [00:01<00:22,  8.47it/s]
DDIM Sampler:   6%|▋         | 13/200 [00:01<00:21,  8.52it/s]
DDIM Sampler:   7%|▋         | 14/200 [00:01<00:21,  8.53it/s]
DDIM Sampler:   8%|▊         | 15/200 [00:01<00:21,  8.51it/s]
DDIM Sampler:   8%|▊         | 16/200 [00:01<00:21,  8.47it/s]
DDIM Sampler:   8%|▊         | 17/200 [00:02<00:21,  8.50it/s]
DDIM Sampler:   9%|▉         | 18/200 [00:02<00:21,  8.28it/s]
DDIM Sampler:  10%|▉         | 19/200 [00:02<00:21,  8.32it/s]
DDIM Sampler:  10%|█         | 20/200 [00:02<00:21,  8.40it/s]
DDIM Sampler:  10%|█         | 21/200 [00:02<00:21,  8.46it/s]
DDIM Sampler:  11%|█         | 22/200 [00:02<00:21,  8.47it/s]
DDIM Sampler:  12%|█▏        | 23/200 [00:02<00:20,  8.45it/s]
DDIM Sampler:  12%|█▏        | 24/200 [00:02<00:20,  8.45it/s]
DDIM Sampler:  12%|█▎        | 25/200 [00:02<00:20,  8.45it/s]
DDIM Sampler:  13%|█▎        | 26/200 [00:03<00:20,  8.43it/s]
DDIM Sampler:  14%|█▎        | 27/200 [00:03<00:20,  8.47it/s]
DDIM Sampler:  14%|█▍        | 28/200 [00:03<00:20,  8.50it/s]
DDIM Sampler:  14%|█▍        | 29/200 [00:03<00:20,  8.50it/s]
DDIM Sampler:  15%|█▌        | 30/200 [00:03<00:19,  8.51it/s]
DDIM Sampler:  16%|█▌        | 31/200 [00:03<00:19,  8.48it/s]
DDIM Sampler:  16%|█▌        | 32/200 [00:03<00:19,  8.51it/s]
DDIM Sampler:  16%|█▋        | 33/200 [00:03<00:19,  8.48it/s]
DDIM Sampler:  17%|█▋        | 34/200 [00:04<00:19,  8.48it/s]
DDIM Sampler:  18%|█▊        | 35/200 [00:04<00:19,  8.37it/s]
DDIM Sampler:  18%|█▊        | 36/200 [00:04<00:19,  8.44it/s]
DDIM Sampler:  18%|█▊        | 37/200 [00:04<00:19,  8.45it/s]
DDIM Sampler:  19%|█▉        | 38/200 [00:04<00:19,  8.47it/s]
DDIM Sampler:  20%|█▉        | 39/200 [00:04<00:19,  8.44it/s]
DDIM Sampler:  20%|██        | 40/200 [00:04<00:18,  8.44it/s]
DDIM Sampler:  20%|██        | 41/200 [00:04<00:18,  8.48it/s]
DDIM Sampler:  21%|██        | 42/200 [00:04<00:18,  8.50it/s]
DDIM Sampler:  22%|██▏       | 43/200 [00:05<00:18,  8.45it/s]
DDIM Sampler:  22%|██▏       | 44/200 [00:05<00:18,  8.45it/s]
DDIM Sampler:  22%|██▎       | 45/200 [00:05<00:18,  8.45it/s]
DDIM Sampler:  23%|██▎       | 46/200 [00:05<00:18,  8.44it/s]
DDIM Sampler:  24%|██▎       | 47/200 [00:05<00:18,  8.49it/s]
DDIM Sampler:  24%|██▍       | 48/200 [00:05<00:17,  8.48it/s]
DDIM Sampler:  24%|██▍       | 49/200 [00:05<00:17,  8.47it/s]
DDIM Sampler:  25%|██▌       | 50/200 [00:05<00:17,  8.48it/s]
DDIM Sampler:  26%|██▌       | 51/200 [00:06<00:17,  8.45it/s]
DDIM Sampler:  26%|██▌       | 52/200 [00:06<00:17,  8.41it/s]
DDIM Sampler:  26%|██▋       | 53/200 [00:06<00:17,  8.45it/s]
DDIM Sampler:  27%|██▋       | 54/200 [00:06<00:17,  8.46it/s]
DDIM Sampler:  28%|██▊       | 55/200 [00:06<00:17,  8.37it/s]
DDIM Sampler:  28%|██▊       | 56/200 [00:06<00:17,  8.38it/s]
DDIM Sampler:  28%|██▊       | 57/200 [00:06<00:17,  8.41it/s]
DDIM Sampler:  29%|██▉       | 58/200 [00:06<00:16,  8.41it/s]
DDIM Sampler:  30%|██▉       | 59/200 [00:06<00:16,  8.42it/s]
DDIM Sampler:  30%|███       | 60/200 [00:07<00:16,  8.41it/s]
DDIM Sampler:  30%|███       | 61/200 [00:07<00:16,  8.45it/s]
DDIM Sampler:  31%|███       | 62/200 [00:07<00:16,  8.46it/s]
DDIM Sampler:  32%|███▏      | 63/200 [00:07<00:16,  8.43it/s]
DDIM Sampler:  32%|███▏      | 64/200 [00:07<00:16,  8.39it/s]
DDIM Sampler:  32%|███▎      | 65/200 [00:07<00:16,  8.40it/s]
DDIM Sampler:  33%|███▎      | 66/200 [00:07<00:15,  8.39it/s]
DDIM Sampler:  34%|███▎      | 67/200 [00:07<00:15,  8.41it/s]
DDIM Sampler:  34%|███▍      | 68/200 [00:08<00:15,  8.34it/s]
DDIM Sampler:  34%|███▍      | 69/200 [00:08<00:16,  8.01it/s]
DDIM Sampler:  35%|███▌      | 70/200 [00:08<00:15,  8.15it/s]
DDIM Sampler:  36%|███▌      | 71/200 [00:08<00:15,  8.24it/s]
DDIM Sampler:  36%|███▌      | 72/200 [00:08<00:15,  8.29it/s]
DDIM Sampler:  36%|███▋      | 73/200 [00:08<00:15,  8.32it/s]
DDIM Sampler:  37%|███▋      | 74/200 [00:08<00:15,  8.39it/s]
DDIM Sampler:  38%|███▊      | 75/200 [00:08<00:14,  8.42it/s]
DDIM Sampler:  38%|███▊      | 76/200 [00:09<00:14,  8.42it/s]
DDIM Sampler:  38%|███▊      | 77/200 [00:09<00:14,  8.41it/s]
DDIM Sampler:  39%|███▉      | 78/200 [00:09<00:14,  8.42it/s]
DDIM Sampler:  40%|███▉      | 79/200 [00:09<00:14,  8.39it/s]
DDIM Sampler:  40%|████      | 80/200 [00:09<00:14,  8.40it/s]
DDIM Sampler:  40%|████      | 81/200 [00:09<00:14,  8.44it/s]
DDIM Sampler:  41%|████      | 82/200 [00:09<00:13,  8.46it/s]
DDIM Sampler:  42%|████▏     | 83/200 [00:09<00:13,  8.49it/s]
DDIM Sampler:  42%|████▏     | 84/200 [00:09<00:13,  8.46it/s]
DDIM Sampler:  42%|████▎     | 85/200 [00:10<00:13,  8.45it/s]
DDIM Sampler:  43%|████▎     | 86/200 [00:10<00:13,  8.47it/s]
DDIM Sampler:  44%|████▎     | 87/200 [00:10<00:13,  8.47it/s]
DDIM Sampler:  44%|████▍     | 88/200 [00:10<00:13,  8.44it/s]
DDIM Sampler:  44%|████▍     | 89/200 [00:10<00:13,  8.46it/s]
DDIM Sampler:  45%|████▌     | 90/200 [00:10<00:13,  8.45it/s]
DDIM Sampler:  46%|████▌     | 91/200 [00:10<00:12,  8.44it/s]
DDIM Sampler:  46%|████▌     | 92/200 [00:10<00:12,  8.47it/s]
DDIM Sampler:  46%|████▋     | 93/200 [00:11<00:12,  8.48it/s]
DDIM Sampler:  47%|████▋     | 94/200 [00:11<00:12,  8.42it/s]
DDIM Sampler:  48%|████▊     | 95/200 [00:11<00:12,  8.43it/s]
DDIM Sampler:  48%|████▊     | 96/200 [00:11<00:12,  8.43it/s]
DDIM Sampler:  48%|████▊     | 97/200 [00:11<00:12,  8.40it/s]
DDIM Sampler:  49%|████▉     | 98/200 [00:11<00:12,  8.39it/s]
DDIM Sampler:  50%|████▉     | 99/200 [00:11<00:11,  8.44it/s]
DDIM Sampler:  50%|█████     | 100/200 [00:11<00:11,  8.46it/s]
DDIM Sampler:  50%|█████     | 101/200 [00:11<00:11,  8.46it/s]
DDIM Sampler:  51%|█████     | 102/200 [00:12<00:11,  8.43it/s]
DDIM Sampler:  52%|█████▏    | 103/200 [00:12<00:11,  8.40it/s]
DDIM Sampler:  52%|█████▏    | 104/200 [00:12<00:11,  8.40it/s]
DDIM Sampler:  52%|█████▎    | 105/200 [00:12<00:11,  8.45it/s]
DDIM Sampler:  53%|█████▎    | 106/200 [00:12<00:11,  8.44it/s]
DDIM Sampler:  54%|█████▎    | 107/200 [00:12<00:11,  8.44it/s]
DDIM Sampler:  54%|█████▍    | 108/200 [00:12<00:10,  8.46it/s]
DDIM Sampler:  55%|█████▍    | 109/200 [00:12<00:10,  8.43it/s]
DDIM Sampler:  55%|█████▌    | 110/200 [00:13<00:10,  8.39it/s]
DDIM Sampler:  56%|█████▌    | 111/200 [00:13<00:10,  8.36it/s]
DDIM Sampler:  56%|█████▌    | 112/200 [00:13<00:10,  8.29it/s]
DDIM Sampler:  56%|█████▋    | 113/200 [00:13<00:10,  8.33it/s]
DDIM Sampler:  57%|█████▋    | 114/200 [00:13<00:10,  8.35it/s]
DDIM Sampler:  57%|█████▊    | 115/200 [00:13<00:10,  8.37it/s]
DDIM Sampler:  58%|█████▊    | 116/200 [00:13<00:10,  8.38it/s]
DDIM Sampler:  58%|█████▊    | 117/200 [00:13<00:09,  8.40it/s]
DDIM Sampler:  59%|█████▉    | 118/200 [00:14<00:09,  8.43it/s]
DDIM Sampler:  60%|█████▉    | 119/200 [00:14<00:09,  8.38it/s]
DDIM Sampler:  60%|██████    | 120/200 [00:14<00:09,  8.35it/s]
DDIM Sampler:  60%|██████    | 121/200 [00:14<00:09,  8.34it/s]
DDIM Sampler:  61%|██████    | 122/200 [00:14<00:09,  8.36it/s]
DDIM Sampler:  62%|██████▏   | 123/200 [00:14<00:09,  8.35it/s]
DDIM Sampler:  62%|██████▏   | 124/200 [00:14<00:09,  8.36it/s]
DDIM Sampler:  62%|██████▎   | 125/200 [00:14<00:08,  8.39it/s]
DDIM Sampler:  63%|██████▎   | 126/200 [00:14<00:08,  8.41it/s]
DDIM Sampler:  64%|██████▎   | 127/200 [00:15<00:08,  8.45it/s]
DDIM Sampler:  64%|██████▍   | 128/200 [00:15<00:08,  8.41it/s]
DDIM Sampler:  64%|██████▍   | 129/200 [00:15<00:08,  8.38it/s]
DDIM Sampler:  65%|██████▌   | 130/200 [00:15<00:08,  8.41it/s]
DDIM Sampler:  66%|██████▌   | 131/200 [00:15<00:08,  8.43it/s]
DDIM Sampler:  66%|██████▌   | 132/200 [00:15<00:08,  8.41it/s]
DDIM Sampler:  66%|██████▋   | 133/200 [00:15<00:07,  8.39it/s]
DDIM Sampler:  67%|██████▋   | 134/200 [00:15<00:07,  8.38it/s]
DDIM Sampler:  68%|██████▊   | 135/200 [00:16<00:07,  8.39it/s]
DDIM Sampler:  68%|██████▊   | 136/200 [00:16<00:07,  8.40it/s]
DDIM Sampler:  68%|██████▊   | 137/200 [00:16<00:07,  8.41it/s]
DDIM Sampler:  69%|██████▉   | 138/200 [00:16<00:07,  8.39it/s]
DDIM Sampler:  70%|██████▉   | 139/200 [00:16<00:07,  8.39it/s]
DDIM Sampler:  70%|███████   | 140/200 [00:16<00:07,  8.39it/s]
DDIM Sampler:  70%|███████   | 141/200 [00:16<00:07,  8.40it/s]
DDIM Sampler:  71%|███████   | 142/200 [00:16<00:06,  8.38it/s]
DDIM Sampler:  72%|███████▏  | 143/200 [00:16<00:06,  8.39it/s]
DDIM Sampler:  72%|███████▏  | 144/200 [00:17<00:06,  8.38it/s]
DDIM Sampler:  72%|███████▎  | 145/200 [00:17<00:06,  8.35it/s]
DDIM Sampler:  73%|███████▎  | 146/200 [00:17<00:06,  8.39it/s]
DDIM Sampler:  74%|███████▎  | 147/200 [00:17<00:06,  8.38it/s]
DDIM Sampler:  74%|███████▍  | 148/200 [00:17<00:06,  8.37it/s]
DDIM Sampler:  74%|███████▍  | 149/200 [00:17<00:06,  8.34it/s]
DDIM Sampler:  75%|███████▌  | 150/200 [00:17<00:06,  8.33it/s]
DDIM Sampler:  76%|███████▌  | 151/200 [00:17<00:05,  8.31it/s]
DDIM Sampler:  76%|███████▌  | 152/200 [00:18<00:05,  8.33it/s]
DDIM Sampler:  76%|███████▋  | 153/200 [00:18<00:05,  8.32it/s]
DDIM Sampler:  77%|███████▋  | 154/200 [00:18<00:05,  8.27it/s]
DDIM Sampler:  78%|███████▊  | 155/200 [00:18<00:05,  8.23it/s]
DDIM Sampler:  78%|███████▊  | 156/200 [00:18<00:05,  8.27it/s]
DDIM Sampler:  78%|███████▊  | 157/200 [00:18<00:05,  8.30it/s]
DDIM Sampler:  79%|███████▉  | 158/200 [00:18<00:05,  8.32it/s]
DDIM Sampler:  80%|███████▉  | 159/200 [00:18<00:04,  8.34it/s]
DDIM Sampler:  80%|████████  | 160/200 [00:19<00:04,  8.37it/s]
DDIM Sampler:  80%|████████  | 161/200 [00:19<00:04,  8.38it/s]
DDIM Sampler:  81%|████████  | 162/200 [00:19<00:04,  8.37it/s]
DDIM Sampler:  82%|████████▏ | 163/200 [00:19<00:04,  8.36it/s]
DDIM Sampler:  82%|████████▏ | 164/200 [00:19<00:04,  8.33it/s]
DDIM Sampler:  82%|████████▎ | 165/200 [00:19<00:04,  8.32it/s]
DDIM Sampler:  83%|████████▎ | 166/200 [00:19<00:04,  8.35it/s]
DDIM Sampler:  84%|████████▎ | 167/200 [00:19<00:03,  8.33it/s]
DDIM Sampler:  84%|████████▍ | 168/200 [00:19<00:03,  8.35it/s]
DDIM Sampler:  84%|████████▍ | 169/200 [00:20<00:03,  8.35it/s]
DDIM Sampler:  85%|████████▌ | 170/200 [00:20<00:03,  8.34it/s]
DDIM Sampler:  86%|████████▌ | 171/200 [00:20<00:03,  8.33it/s]
DDIM Sampler:  86%|████████▌ | 172/200 [00:20<00:03,  8.37it/s]
DDIM Sampler:  86%|████████▋ | 173/200 [00:20<00:03,  8.39it/s]
DDIM Sampler:  87%|████████▋ | 174/200 [00:20<00:03,  8.38it/s]
DDIM Sampler:  88%|████████▊ | 175/200 [00:20<00:02,  8.36it/s]
DDIM Sampler:  88%|████████▊ | 176/200 [00:20<00:02,  8.37it/s]
DDIM Sampler:  88%|████████▊ | 177/200 [00:21<00:02,  8.36it/s]
DDIM Sampler:  89%|████████▉ | 178/200 [00:21<00:02,  8.37it/s]
DDIM Sampler:  90%|████████▉ | 179/200 [00:21<00:02,  8.37it/s]
DDIM Sampler:  90%|█████████ | 180/200 [00:21<00:02,  8.34it/s]
DDIM Sampler:  90%|█████████ | 181/200 [00:21<00:02,  8.35it/s]
DDIM Sampler:  91%|█████████ | 182/200 [00:21<00:02,  8.36it/s]
DDIM Sampler:  92%|█████████▏| 183/200 [00:21<00:02,  8.35it/s]
DDIM Sampler:  92%|█████████▏| 184/200 [00:21<00:01,  8.34it/s]
DDIM Sampler:  92%|█████████▎| 185/200 [00:22<00:01,  8.35it/s]
DDIM Sampler:  93%|█████████▎| 186/200 [00:22<00:01,  8.36it/s]
DDIM Sampler:  94%|█████████▎| 187/200 [00:22<00:01,  8.35it/s]
DDIM Sampler:  94%|█████████▍| 188/200 [00:22<00:01,  8.37it/s]
DDIM Sampler:  94%|█████████▍| 189/200 [00:22<00:01,  8.35it/s]
DDIM Sampler:  95%|█████████▌| 190/200 [00:22<00:01,  8.34it/s]
DDIM Sampler:  96%|█████████▌| 191/200 [00:22<00:01,  8.33it/s]
DDIM Sampler:  96%|█████████▌| 192/200 [00:22<00:00,  8.33it/s]
DDIM Sampler:  96%|█████████▋| 193/200 [00:22<00:00,  8.33it/s]
DDIM Sampler:  97%|█████████▋| 194/200 [00:23<00:00,  8.33it/s]
DDIM Sampler:  98%|█████████▊| 195/200 [00:23<00:00,  8.34it/s]
DDIM Sampler:  98%|█████████▊| 196/200 [00:23<00:00,  8.33it/s]
DDIM Sampler:  98%|█████████▊| 197/200 [00:23<00:00,  8.32it/s]
DDIM Sampler:  99%|█████████▉| 198/200 [00:23<00:00,  8.31it/s]
DDIM Sampler: 100%|█████████▉| 199/200 [00:23<00:00,  8.32it/s]
DDIM Sampler: 100%|██████████| 200/200 [00:23<00:00,  8.34it/s]
DDIM Sampler: 100%|██████████| 200/200 [00:23<00:00,  8.40it/s]

Version Details

Version ID: b61392adecdd660326fc9cfc5398182437dbe5e97b5decfb36e1a36de68b5b95
Version Created: February 3, 2023

Run on Replicate →