fofr/deprecated-batch-image-captioning ❓🔢📝🖼️✓ → 🖼️

▶️ 1.6K runs 📅 Aug 2024 ⚙️ Cog 0.9.14 🔗 GitHub ⚖️ License

image-to-text

Performance

23.0sTypical run time

~60sCold start (first call)

1.6KTotal runs

About

A wrapper model for captioning multiple images using GPT, Claude or Gemini, useful for lora training

Example Output

Output

Performance Metrics

23.04s Prediction Time

60.28s Total Time

All Input Parameters

{
"model": "gpt-4o-2024-08-06",
"max_dimension": 1024,
"system_prompt": "Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using \"or\". Precise composition is important. Avoid phrases like \"conveys a sense of\" and \"capturing the\", just use the terms themselves.\n\nGood examples are:\n\n\"Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine.\"\n\n\"A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series.\"\n\n\"An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.\"\n",
"caption_prefix": "",
"caption_suffix": "",
"message_prompt": "Caption this image please",
"openai_api_key": "[REDACTED]",
"image_zip_archive": "https://replicate.delivery/pbxt/LREOQCiXFRxVaSpwt2MYMwuwiEMIuiIw8YPm7rLLGPH94f57/Archive.zip",
"resize_images_for_captioning": true
}

Input Parameters

model Default: gpt-4o-2024-08-06: AI model to use for captioning. Your OpenAI or Anthropic account will be charged for usage, see their pricing pages for details.
max_dimension Type: integerDefault: 1024: Maximum dimension (width or height) for resized images
system_prompt Type: stringDefault: Write a four sentence caption for this image. In the first sentence describe the style and type (painting, photo, etc) of the image. Describe in the remaining sentences the contents and composition of the image. Only use language that would be used to prompt a text to image model. Do not include usage. Comma separate keywords rather than using "or". Precise composition is important. Avoid phrases like "conveys a sense of" and "capturing the", just use the terms themselves. Good examples are: "Photo of an alien woman with a glowing halo standing on top of a mountain, wearing a white robe and silver mask in the futuristic style with futuristic design, sky background, soft lighting, dynamic pose, a sense of future technology, a science fiction movie scene rendered in the Unreal Engine." "A scene from the cartoon series Masters of the Universe depicts Man-At-Arms wearing a gray helmet and gray armor with red gloves. He is holding an iron bar above his head while looking down on Orko, a pink blob character. Orko is sitting behind Man-At-Arms facing left on a chair. Both characters are standing near each other, with Orko inside a yellow chestplate over a blue shirt and black pants. The scene is drawn in the style of the Masters of the Universe cartoon series." "An emoji, digital illustration, playful, whimsical. A cartoon zombie character with green skin and tattered clothes reaches forward with two hands, they have green skin, messy hair, an open mouth and gaping teeth, one eye is half closed.": System prompt for image analysis
caption_prefix Type: stringDefault:: Optional prefix for image captions
caption_suffix Type: stringDefault:: Optional suffix for image captions
message_prompt Type: stringDefault: Caption this image please: Message prompt for image captioning
openai_api_key Type: string: API key for OpenAI
anthropic_api_key Type: string: API key for Anthropic
image_zip_archive (required) Type: string: ZIP archive containing images to process
google_generativeai_api_key Type: string: API key for Google Generative AI
resize_images_for_captioning Type: booleanDefault: true: Whether to resize images for captioning. This makes captioning cheaper

Output Schema

Output

Type: string • Format: uri

Example Execution Logs

Files extracted:
/tmp/outputs/2024-06-01--15-16-53-u-q3-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
/tmp/outputs/2024-06-01--15-16-53-u-q1-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
/tmp/outputs/2024-06-01--15-16-53-u-q4-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
/tmp/outputs/2024-06-01--15-16-53-u-q2-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
Number of images to be captioned: 4
===================================================
Processing 2024-06-01--15-16-53-u-q3-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
Resized from 928x1232 to 771x1024
Caption: Digital artwork of an abstract, cybernetic rabbit. The rabbit is composed of intricate, neon-like lines and patterns in blue and white, with a luminous, swirling design. Its eyes are glowing red, and it sits against a dark background. A spherical, luminous object hovers in the top right corner, casting a mystical glow.
===================================================
Processing 2024-06-01--15-16-53-u-q1-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
Resized from 928x1232 to 771x1024
Caption: Abstract digital illustration of a rabbit, featuring dynamic, swirling lines and vivid colors. The focus is on the rabbit's glowing red eyes and long ears, drawn in a sketchy style. The background is a dark, cosmic mix of deep blues and bright reds, creating a sense of mystery. Thin, energetic white lines surround the figure, adding motion and intensity.
===================================================
Processing 2024-06-01--15-16-53-u-q4-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
Resized from 928x1232 to 771x1024
Caption: Abstract digital painting featuring an intense, mischievous creature with large ears and glowing yellow eyes. The figure is surrounded by a chaotic swirl of red and blue strokes, giving a sense of movement and energy. The creature's wide grin and sharp features contribute to its menacing presence. Dark background contrasts with vibrant colors, enhancing the dramatic effect.
===================================================
Processing 2024-06-01--15-16-53-u-q2-fofr_pikachu_91215d95-1cb7-43c3-9448-5d97975efcf1.png
Resized from 928x1232 to 771x1024
Caption: Digital painting, abstract, vibrant. A stylized creature resembling a rabbit is depicted in dark blue and black tones. Bold, swirling yellow and white strokes create dynamic movement around the creature. The background is filled with intricate patterns, deep shadows, and highlights.
===================================================

Version Details

Version ID: d0adb15f4826881a68f1d82e0b10fe2ee1af536632dc8313f7f777ed8d264726
Version Created: August 13, 2024

Run on Replicate →