fermatresearch/sdxl-controlnet-lora 🔢🖼️📝❓✓ → 🖼️

▶️ 965.3K runs 📅 Oct 2023 ⚙️ Cog 0.8.6 🔗 GitHub
controlnet image-to-image text-to-image

About

'''Last update: Now supports img2img.''' SDXL Canny controlnet with LoRA support.

Example Output

Prompt:

"shot in the style of sksfer, a woman in alaska"

Output

Example output

Performance Metrics

20.41s Prediction Time
20.38s Total Time
All Input Parameters 
{
  "seed": null,
  "image": "https://replicate.delivery/pbxt/JiOTMCHj4oGrTTf8Pg2r7vyI8YdXc5jL2IDyC2SfhuggjYe6/out-0%20%281%29.png",
  "prompt": "shot in the style of sksfer, a woman in alaska",
  "refine": "base_image_refiner",
  "scheduler": "K_EULER",
  "lora_scale": 0.95,
  "num_outputs": 1,
  "lora_weights": "https://pbxt.replicate.delivery/mwN3AFyYZyouOB03Uhw8ubKW9rpqMgdtL9zYV9GF2WGDiwbE/trained_model.tar",
  "refine_steps": 20,
  "guidance_scale": 7.5,
  "apply_watermark": true,
  "condition_scale": 0.5,
  "negative_prompt": "",
  "num_inference_steps": 40
}
Input Parameters
seed Type: integer
Random seed. Leave blank to randomize the seed
image Type: string
Input image for img2img or inpaint mode
prompt Type: stringDefault: An astronaut riding a rainbow unicorn
Input prompt
refine Default: base_image_refiner
Whether to use refinement steps or not
img2img Type: boolean
Use img2img pipeline, it will use the image input both as the control image and the base image.
strength Type: numberDefault: 0.8Range: 0 - 1
When img2img is active, the denoising strength. 1 means total destruction of the input image.
scheduler Default: K_EULER
scheduler
lora_scale Type: numberDefault: 0.95Range: 0 - 1
LoRA additive scale. Only applicable on trained models.
num_outputs Type: integerDefault: 1Range: 1 - 4
Number of images to output
lora_weights Type: string
Replicate LoRA weights to use. Leave blank to use the default weights.
refine_steps Type: integerDefault: 10
For base_image_refiner, the number of steps to refine
guidance_scale Type: numberDefault: 7.5Range: 1 - 50
Scale for classifier-free guidance
apply_watermark Type: booleanDefault: true
Applies a watermark to enable determining if an image is generated in downstream applications. If you have other provisions for generating or deploying images safely, you can use this to disable watermarking.
condition_scale Type: numberDefault: 1.1Range: 0 - 2
The bigger this number is, the more ControlNet interferes
negative_prompt Type: stringDefault:
Input Negative Prompt
num_inference_steps Type: integerDefault: 30Range: 1 - 500
Number of denoising steps
Output Schema

Output

Type: array • Items Type: string • Items Format: uri

Example Execution Logs 
Using seed: 44186
loading custom weights
weights already in cache
Loading fine-tuned model
Does not have Unet. assume we are using LoRA
Loading Unet LoRA
Prompt: shot in the style of <s0><s1>, a woman in alaska
Original width:1024, height:1024
Aspect Ratio: 1.00
new_width:1024, new_height:1024
txt2img mode
  0%|          | 0/40 [00:00<?, ?it/s]
  2%|▎         | 1/40 [00:00<00:14,  2.72it/s]
  5%|▌         | 2/40 [00:00<00:13,  2.73it/s]
  8%|▊         | 3/40 [00:01<00:13,  2.73it/s]
 10%|█         | 4/40 [00:01<00:13,  2.73it/s]
 12%|█▎        | 5/40 [00:01<00:12,  2.73it/s]
 15%|█▌        | 6/40 [00:02<00:12,  2.73it/s]
 18%|█▊        | 7/40 [00:02<00:12,  2.73it/s]
 20%|██        | 8/40 [00:02<00:11,  2.73it/s]
 22%|██▎       | 9/40 [00:03<00:11,  2.73it/s]
 25%|██▌       | 10/40 [00:03<00:11,  2.72it/s]
 28%|██▊       | 11/40 [00:04<00:10,  2.72it/s]
 30%|███       | 12/40 [00:04<00:10,  2.72it/s]
 32%|███▎      | 13/40 [00:04<00:09,  2.72it/s]
 35%|███▌      | 14/40 [00:05<00:09,  2.72it/s]
 38%|███▊      | 15/40 [00:05<00:09,  2.72it/s]
 40%|████      | 16/40 [00:05<00:08,  2.72it/s]
 42%|████▎     | 17/40 [00:06<00:08,  2.71it/s]
 45%|████▌     | 18/40 [00:06<00:08,  2.71it/s]
 48%|████▊     | 19/40 [00:06<00:07,  2.71it/s]
 50%|█████     | 20/40 [00:07<00:07,  2.71it/s]
 52%|█████▎    | 21/40 [00:07<00:06,  2.71it/s]
 55%|█████▌    | 22/40 [00:08<00:06,  2.72it/s]
 57%|█████▊    | 23/40 [00:08<00:06,  2.71it/s]
 60%|██████    | 24/40 [00:08<00:05,  2.71it/s]
 62%|██████▎   | 25/40 [00:09<00:05,  2.71it/s]
 65%|██████▌   | 26/40 [00:09<00:05,  2.71it/s]
 68%|██████▊   | 27/40 [00:09<00:04,  2.71it/s]
 70%|███████   | 28/40 [00:10<00:04,  2.72it/s]
 72%|███████▎  | 29/40 [00:10<00:04,  2.71it/s]
 75%|███████▌  | 30/40 [00:11<00:03,  2.71it/s]
 78%|███████▊  | 31/40 [00:11<00:03,  2.71it/s]
 80%|████████  | 32/40 [00:11<00:02,  2.71it/s]
 82%|████████▎ | 33/40 [00:12<00:02,  2.71it/s]
 85%|████████▌ | 34/40 [00:12<00:02,  2.71it/s]
 88%|████████▊ | 35/40 [00:12<00:01,  2.71it/s]
 90%|█████████ | 36/40 [00:13<00:01,  2.71it/s]
 92%|█████████▎| 37/40 [00:13<00:01,  2.71it/s]
 95%|█████████▌| 38/40 [00:13<00:00,  2.71it/s]
 98%|█████████▊| 39/40 [00:14<00:00,  2.71it/s]
100%|██████████| 40/40 [00:14<00:00,  2.71it/s]
100%|██████████| 40/40 [00:14<00:00,  2.72it/s]
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:00<00:01,  4.32it/s]
 33%|███▎      | 2/6 [00:00<00:00,  4.30it/s]
 50%|█████     | 3/6 [00:00<00:00,  4.30it/s]
 67%|██████▋   | 4/6 [00:00<00:00,  4.28it/s]
 83%|████████▎ | 5/6 [00:01<00:00,  4.28it/s]
100%|██████████| 6/6 [00:01<00:00,  4.28it/s]
100%|██████████| 6/6 [00:01<00:00,  4.29it/s]
Version Details
Version ID
3bb13fe1c33c35987b33792b01b71ed6529d03f165d1c2416375859f09ca9fef
Version Created
January 31, 2024
Run on Replicate →