lucataco/apollo-7b 🔢🖼️📝 → 📝

▶️ 107.0K runs 📅 Dec 2024 ⚙️ Cog 0.13.6 🔗 GitHub 📄 Paper ⚖️ License

video-auto-captioning video-question-answering video-to-text

About

Apollo 7B - An Exploration of Video Understanding in Large Multimodal Models

Example Output

Prompt:

"Describe this video in detail"

Output

The video features an astronaut in a white spacesuit walking on the moon's surface. The background showcases a large, detailed moon against a starry sky. As the astronaut walks, they begin to run and eventually leap into the air, floating above the moon's rocky terrain. The scene transitions to the astronaut drifting away from the moon, with the lunar landscape and the moon itself visible in the background. The video concludes with the astronaut continuing to float in space, gazing at the moon.

Performance Metrics

3.13s Prediction Time

85.71s Total Time

All Input Parameters

{
  "top_p": 0.7,
  "video": "https://replicate.delivery/pbxt/M9kGHuJMeAKZs0eSbaEk6hCc7zqY4Tg94IxWwDpC5hRiuBPY/astro.mp4",
  "prompt": "Describe this video in detail",
  "temperature": 0.4,
  "max_new_tokens": 256
}

Input Parameters

top_p Type: numberDefault: 0.7Range: 0 - 1: Top-p sampling probability
video (required) Type: string: Input video file
prompt Type: stringDefault: Describe this video in detail: Question or prompt about the video
temperature Type: numberDefault: 0.4Range: 0.1 - 2: Sampling temperature
max_new_tokens Type: integerDefault: 256Range: 32 - 1024: Maximum number of tokens to generate

Output Schema

Output

Type: string

Example Execution Logs

/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
warnings.warn(
/root/.pyenv/versions/3.11.10/lib/python3.11/site-packages/torch/utils/checkpoint.py:61: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
warnings.warn(
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.

Version Details

Version ID: e282f76d0451b759128be3e8bccfe5ded8f521f4a7d705883e92f837e563f575
Version Created: December 16, 2024

Run on Replicate →