fofr/realvisxl-v3-multi-controlnet-lora 🖼️🔢📝❓✓ → 🖼️

▶️ 1.9M runs 📅 Jan 2024 ⚙️ Cog 0.8.6 🔗 GitHub 📄 Paper ⚖️ License
controlnet image-inpainting text-to-image

About

RealVisXl V3 with multi-controlnet, lora loading, img2img, inpainting

Example Output

Prompt:

"A detailed photo of an astronaut riding a unicorn through a field of flowers"

Output

Example outputExample output

Performance Metrics

9.33s Prediction Time
9.37s Total Time
All Input Parameters 
{
  "width": 768,
  "height": 768,
  "prompt": "A detailed photo of an astronaut riding a unicorn through a field of flowers",
  "refine": "no_refiner",
  "scheduler": "K_EULER",
  "lora_scale": 0.8,
  "num_outputs": 1,
  "controlnet_1": "soft_edge_hed",
  "controlnet_2": "none",
  "controlnet_3": "none",
  "guidance_scale": 7.5,
  "apply_watermark": false,
  "negative_prompt": "",
  "prompt_strength": 0.8,
  "sizing_strategy": "width_height",
  "controlnet_1_end": 1,
  "controlnet_2_end": 1,
  "controlnet_3_end": 1,
  "controlnet_1_image": "https://replicate.delivery/pbxt/JsfCQE8k1lsCinW1yo76yKMQe6R5MRt9WLL3H5T5Ypc5wasq/020e656d-0c71-45c3-a7f5-1facf7d52d4f.png",
  "controlnet_1_start": 0,
  "controlnet_2_start": 0,
  "controlnet_3_start": 0,
  "num_inference_steps": 30,
  "controlnet_1_conditioning_scale": 0.8,
  "controlnet_2_conditioning_scale": 0.8,
  "controlnet_3_conditioning_scale": 0.75
}
Input Parameters
mask Type: string
Input mask for inpaint mode. Black areas will be preserved, white areas will be inpainted.
seed Type: integer
Random seed. Leave blank to randomize the seed
image Type: string
Input image for img2img or inpaint mode
width Type: integerDefault: 768
Width of output image
height Type: integerDefault: 768
Height of output image
prompt Type: stringDefault: An astronaut riding a rainbow unicorn
Input prompt
refine Default: no_refiner
Which refine style to use
scheduler Default: K_EULER
scheduler
lora_scale Type: numberDefault: 0.6Range: 0 - 1
LoRA additive scale. Only applicable on trained models.
num_outputs Type: integerDefault: 1Range: 1 - 4
Number of images to output
controlnet_1 Default: none
Controlnet
controlnet_2 Default: none
Controlnet
controlnet_3 Default: none
Controlnet
lora_weights Type: string
Replicate LoRA weights to use. Leave blank to use the default weights.
refine_steps Type: integer
For base_image_refiner, the number of steps to refine, defaults to num_inference_steps
guidance_scale Type: numberDefault: 7.5Range: 1 - 50
Scale for classifier-free guidance
apply_watermark Type: booleanDefault: false
Applies a watermark to enable determining if an image is generated in downstream applications. If you have other provisions for generating or deploying images safely, you can use this to disable watermarking.
negative_prompt Type: stringDefault:
Negative Prompt
prompt_strength Type: numberDefault: 0.8Range: 0 - 1
Prompt strength when using img2img / inpaint. 1.0 corresponds to full destruction of information in image
sizing_strategy Default: width_height
Decide how to resize images – use width/height, resize based on input image or control image
controlnet_1_end Type: numberDefault: 1Range: 0 - 1
When controlnet conditioning ends
controlnet_2_end Type: numberDefault: 1Range: 0 - 1
When controlnet conditioning ends
controlnet_3_end Type: numberDefault: 1Range: 0 - 1
When controlnet conditioning ends
controlnet_1_image Type: string
Input image for first controlnet
controlnet_1_start Type: numberDefault: 0Range: 0 - 1
When controlnet conditioning starts
controlnet_2_image Type: string
Input image for second controlnet
controlnet_2_start Type: numberDefault: 0Range: 0 - 1
When controlnet conditioning starts
controlnet_3_image Type: string
Input image for third controlnet
controlnet_3_start Type: numberDefault: 0Range: 0 - 1
When controlnet conditioning starts
num_inference_steps Type: integerDefault: 30Range: 1 - 500
Number of denoising steps
disable_safety_checker Type: booleanDefault: false
Disable safety checker for generated images. This feature is only available through the API.
controlnet_1_conditioning_scale Type: numberDefault: 0.75Range: 0 - 4
How strong the controlnet conditioning is
controlnet_2_conditioning_scale Type: numberDefault: 0.75Range: 0 - 4
How strong the controlnet conditioning is
controlnet_3_conditioning_scale Type: numberDefault: 0.75Range: 0 - 4
How strong the controlnet conditioning is
Output Schema

Output

Type: array • Items Type: string • Items Format: uri

Example Execution Logs 
Using seed: 19498
Using given dimensions
resize took: 0.02s
Prompt: A detailed photo of an astronaut riding a unicorn through a field of flowers
Processing image with soft_edge_hed
controlnet preprocess took: 0.43s
Using txt2img + controlnet pipeline
Loading pipeline components...:   0%|          | 0/7 [00:00<?, ?it/s]
Loading pipeline components...: 100%|██████████| 7/7 [00:00<00:00, 15485.30it/s]
You have 1 ControlNets and you have passed 1 prompts. The conditionings will be fixed across the prompts.
  0%|          | 0/30 [00:00<?, ?it/s]
  3%|▎         | 1/30 [00:00<00:06,  4.63it/s]
  7%|▋         | 2/30 [00:00<00:06,  4.62it/s]
 10%|█         | 3/30 [00:00<00:05,  4.62it/s]
 13%|█▎        | 4/30 [00:00<00:05,  4.62it/s]
 17%|█▋        | 5/30 [00:01<00:05,  4.63it/s]
 20%|██        | 6/30 [00:01<00:05,  4.63it/s]
 23%|██▎       | 7/30 [00:01<00:04,  4.62it/s]
 27%|██▋       | 8/30 [00:01<00:04,  4.62it/s]
 30%|███       | 9/30 [00:01<00:04,  4.63it/s]
 33%|███▎      | 10/30 [00:02<00:04,  4.63it/s]
 37%|███▋      | 11/30 [00:02<00:04,  4.62it/s]
 40%|████      | 12/30 [00:02<00:03,  4.62it/s]
 43%|████▎     | 13/30 [00:02<00:03,  4.62it/s]
 47%|████▋     | 14/30 [00:03<00:03,  4.62it/s]
 50%|█████     | 15/30 [00:03<00:03,  4.62it/s]
 53%|█████▎    | 16/30 [00:03<00:03,  4.62it/s]
 57%|█████▋    | 17/30 [00:03<00:02,  4.62it/s]
 60%|██████    | 18/30 [00:03<00:02,  4.62it/s]
 63%|██████▎   | 19/30 [00:04<00:02,  4.62it/s]
 67%|██████▋   | 20/30 [00:04<00:02,  4.62it/s]
 70%|███████   | 21/30 [00:04<00:01,  4.62it/s]
 73%|███████▎  | 22/30 [00:04<00:01,  4.61it/s]
 77%|███████▋  | 23/30 [00:04<00:01,  4.62it/s]
 80%|████████  | 24/30 [00:05<00:01,  4.62it/s]
 83%|████████▎ | 25/30 [00:05<00:01,  4.62it/s]
 87%|████████▋ | 26/30 [00:05<00:00,  4.62it/s]
 90%|█████████ | 27/30 [00:05<00:00,  4.61it/s]
 93%|█████████▎| 28/30 [00:06<00:00,  4.61it/s]
 97%|█████████▋| 29/30 [00:06<00:00,  4.61it/s]
100%|██████████| 30/30 [00:06<00:00,  4.61it/s]
100%|██████████| 30/30 [00:06<00:00,  4.62it/s]
inference took: 6.76s
prediction took: 7.43s
Version Details
Version ID
90a4a3604cd637cb9f1a2bdae1cfa9ed869362ca028814cdce310a78e27daade
Version Created
January 5, 2024
Run on Replicate →