← All guides

Depth Estimation API

A depth estimation API takes a single 2D image and predicts how far every pixel is from the camera. It powers background blur, 3D parallax effects, AR occlusion, robotics perception, and depth-conditioned image generation (ControlNet-style workflows).

The fundamental split: relative depth models rank pixels (nearer vs farther), while metric depth models output real-world distances. Choosing wrong is the most common integration mistake.

Quick Recommendation

For most visual-effects uses, start with chenxwh/depth-anything-v2 — the current-generation model, fast in example runs (~3s), with a model_size dial to trade speed against quality.

Use cjwbw/zoedepth (4.6M+ runs) when you need metric depth — actual distances, not just ordering. It combines relative and metric training, with a model_type input to pick indoor/outdoor-tuned checkpoints.

Use cjwbw/midas when you want the long-standing baseline that much of the depth ecosystem (including ControlNet preprocessing) was built around.

Keyword-Matched Use Cases

Search intent What the user likely needs Best starting point
Depth estimation API Depth map from one image Depth Anything V2
Metric depth from single image Real distances (meters) ZoeDepth
Depth map for ControlNet Relative depth conditioning image MiDaS or Depth Anything V2
Background blur / portrait mode Nearer-vs-farther separation Depth Anything V2
3D photo / parallax effect Dense relative depth Depth Anything V2
AR / robotics distance sensing Calibrated metric depth ZoeDepth (right model_type)

Model Comparison

Model Runs on Replicate Example speed Depth type Key control
chenxwh/depth-anything-v2 2.6M+ ~3s Relative model_size (speed vs quality)
cjwbw/zoedepth 4.6M+ ~8s Metric + relative model_type (indoor / outdoor / mixed)
cjwbw/midas 827K+ ~4s Relative model_type (checkpoint variants)

Relative vs metric — the decision that matters

If your use case only needs to know what is in front of what — blur, parallax, depth-conditioned generation — relative depth is enough, and Depth Anything V2 is the modern default.

If your code does arithmetic on distances — "is this object within 2 meters?", AR occlusion, measurement — you need metric depth, and ZoeDepth is the purpose-built option. Its model_type matters: indoor-trained checkpoints mis-scale outdoor scenes and vice versa, so match it to your environment (or use the mixed variant).

All three are single-image models: they infer depth from visual cues, so accuracy varies with scene familiarity. Mirrors, glass, and unusual scales remain hard for every monocular model.

Practical Decision

Default to Depth Anything V2 for effects and conditioning workflows — newest architecture, fastest example runtime, quality dial. Switch to ZoeDepth the moment real distances enter the requirements, and pick its model_type to match indoor vs outdoor scenes. Keep MiDaS in mind for ecosystem compatibility where a pipeline expects MiDaS-style output.

FAQ

What is the difference between relative and metric depth?

Relative depth ranks pixels by distance without units; metric depth outputs real-world distances. Effects need relative; measurement, AR, and robotics need metric.

Which depth estimation model is most accurate?

For relative depth, Depth Anything V2 is the current-generation pick. For metric depth, ZoeDepth — provided its model_type matches your scene (indoor vs outdoor).

Can I get depth from a single photo?

Yes — all three models are monocular: one image in, dense depth map out. No stereo pair or LiDAR required.

Which depth model should I use with ControlNet?

MiDaS-style relative depth is the classic ControlNet conditioning input; Depth Anything V2 output works in the same role with better quality.