Depth Sensing Cameras: How Machines Understand the Physical World

From warehouse robots and autonomous forklifts to humanoid robots and drones, intelligent machines are moving rapidly out of controlled environments and into the real world. To operate safely and autonomously out there, these systems need more than images. They need an understanding of distance, space, movement and the relationships between objects.

That capability is depth perception. Depth sensing cameras are how machines acquire it, building a three-dimensional understanding of their environment that makes navigation, obstacle avoidance, object manipulation, scene reconstruction and spatial AI applications possible.

As Physical AI becomes a major driver of innovation, depth perception has become one of the foundational technologies powering intelligent automation, and the choice of depth sensing technology is one of the more consequential decisions in an embedded vision system design.

There is no single approach that suits every application. This article explains how the main depth sensing technologies work, what each is genuinely good and bad at, and how to choose between them.

Why depth matters

A conventional camera captures a flat, two-dimensional representation of a scene. That image can carry a great deal of information, but it does not directly tell you how far anything is from the camera. A robot that cannot measure distance cannot judge whether a pallet is within reach, whether a person has stepped into its path, or whether a gap is wide enough to drive through.

With reliable depth data, a system can navigate autonomously, detect and avoid obstacles, locate and identify objects, perform manipulation tasks, build 3D maps of an environment, and track people and objects in motion. Without it, robotics and autonomous systems stay confined to structured, predictable spaces.

There is no single depth sensing technology that suits every application. In many systems it is useful to combine approaches, so the strengths of one cover the weaknesses of another.

The rise of Physical AI

The first wave of AI worked on language, data and digital information. The next wave works on the physical world.

Physical AI describes systems that perceive, understand and act in real environments. Unlike traditional automation, which follows predefined instructions, these systems continuously interpret their surroundings and make decisions in real time. Autonomous mobile robots, industrial and humanoid robots, agricultural machinery, autonomous drones, smart logistics systems and automated inspection platforms all belong to this category.

For all of them, depth perception is a foundational input layer. It supplies the spatial information that navigation, manipulation, obstacle avoidance and scene understanding are built on. As AI moves out of data centers and into warehouses, factories, farms and public spaces, reliable depth sensing stops being an advanced feature and becomes a requirement.

The main depth sensing technologies

Four approaches account for most depth sensing in machine vision today: stereo vision, Time-of-Flight, structured light and LiDAR. Each measures distance differently, and each carries a different set of trade-offs in range, accuracy, speed, cost and environmental tolerance.

Stereo depth cameras

Stereo depth cameras, including the FRAMOS D400e cameras shown here, are useful for real-time applications, including providing guidance for autonomous guided vehicles, like this forklift. (image courtesy of Phase 3 Automation Ltd).

Stereo depth cameras, also known as stereoscopic cameras, work the way human binocular vision does. Two or more lenses or image sensors are set a few centimeters apart. Software in the camera’s processing unit detects the same feature in each sensor. That feature sits at a slightly different position in each image, and the software uses the resulting offset, known as disparity, to calculate the depth of that point through triangulation.

Most stereo depth cameras also use active sensing. A patterned light projector helps the matching algorithm find corresponding points on flat or featureless surfaces, where there would otherwise be nothing to correlate. A blank wall is a hard problem for passive stereo, and the projector solves it.

These cameras typically use near-infrared sensors that can see the projected infrared pattern alongside visible light. Many stereo depth cameras also include an RGB sensor, so color information can be overlaid on the resulting depth map. That combination of geometry and color in one device is part of why stereo suits AI inference workflows so well: the model gets both what the object is and where it sits in space.

Strengths. Stereo works indoors and outdoors, which sets it apart from most of the alternatives. It has no moving parts. It produces dense, real-time depth maps rather than sparse point measurements. It supports multi-camera deployments. And it delivers depth and RGB together.

Limitations. Detecting and correlating features across two sensor images is computationally expensive. More importantly, stereo has a limited effective operating range, which depends on the baseline, meaning the separation between the two main image sensors, and on the resolution of those sensors. As objects get further from the camera, the separation between corresponding features becomes too small for the sensors to resolve. Compact stereo depth cameras are typically effective at ranges up to around 6 meters, and a wider baseline extends that.

Typical applications: mobile robot navigation, obstacle avoidance, guidance for autonomous guided vehicles such as warehouse forklifts, bin picking, human-robot collaboration, agricultural and outdoor robotics.

Time-of-Flight cameras

Time-of-Flight cameras measure distance by actively illuminating a scene and measuring how long the reflected light takes to return. There are two distinct implementations, each with its own advantages.

Direct Time-of-Flight (dToF). Like LiDAR, direct ToF cameras scan the scene with pulses of invisible infrared laser light and observe what is reflected back. The distance to each point is computed from the time a pulse takes to travel out and return.

Direct ToF cameras use a specific kind of sensing pixel called a Single Photon Avalanche Diode, or SPAD. These pixels detect the sudden spike in photons when a light pulse returns and record the interval. SPAD pixels are comparatively large and are read out in groups as the laser scan progresses, which is why direct ToF cameras tend to be fairly low resolution. In exchange they are compact, relatively inexpensive, and well suited to applications where high resolution and real-time performance are not required.

Indirect Time-of-Flight (iToF). Indirect ToF cameras use diffuse infrared laser light from one or more emitters to illuminate the entire scene in a series of modulated pulses, with the emitters pulsing at high frequency.

Rather than timing each pulse directly, iToF cameras record and compare the phase shift of the waveform as recorded in each pixel of the sensor. Comparing how much the waveform has shifted in each pixel gives the distance to the corresponding point in the scene. Because the whole scene is captured at once, indirect ToF determines the distance to every point in a single shot rather than scanning.

Strengths. ToF cameras typically have no moving parts. This is true of all indirect ToF cameras, which use diffuse illumination, though some direct ToF cameras use MEMS chips or other moving parts to steer the laser. All ToF cameras are compact, lightweight and relatively inexpensive, and depending on the power their emitters need, small enough to embed in very small devices including phones.

Because they provide their own illumination, ToF cameras work in very low light or complete darkness. Their accuracy is better than any other depth sensing technology except structured light, in the range of 1 mm to 1 cm depending on the operating range. Indirect ToF cameras in particular deliver high-fidelity depth at up to 640 x 480 pixels, and because they capture the scene in a single frame they run at up to 60 frames per second, which makes them well suited to high-speed and real-time work.

Limitations. In brightly lit conditions and outdoors, ambient light can wash out the light from the laser emitters. Indirect ToF cameras can also be confused by highly reflective and retroreflective surfaces. And all ToF cameras can be disrupted by light from other ToF cameras operating in the same field of view, which matters if you plan to run several units in one space.

For applications that need to work outdoors, or that require multiple depth cameras covering the same area, stereo depth cameras are often the more effective choice.

Typical applications: package sorting and dimensioning on conveyors, pick-and-place machine vision, object recognition on assembly lines, AGV navigation, access control, gesture recognition, human-machine interfaces.

Structured light cameras

A structured light camera uses a projector to illuminate the scene with a known light pattern, such as stripes, bars or points. By observing how that pattern is distorted by the surfaces it lands on, the camera computes the depth and contours of objects in the scene.

Structured light is closely related to stereo. Both depend on a baseline, here the offset between the light projector and the camera lens, to triangulate the depth of each point in the reflected pattern.

Some structured light cameras rapidly scan a scene with phase-shifted patterns to resolve contours that a single pass would miss. Some 3D scanner products combine a projector with stereo cameras for additional precision.

Strengths. Accuracy. Structured light produces depth maps with precision up to 100 micrometers, well beyond what any other approach here offers. That makes it the standard choice in quality control, for inspection work and for verifying the accuracy of 3D models and manufactured parts.

Limitations. Precision falls away quickly at longer ranges, so structured light is useful only at very short working distances. The depth maps are computationally expensive and slow to produce relative to the alternatives, which makes the technology unsuitable for real-time applications and best suited to stationary subjects. Ambient light interferes with the projected pattern, so structured light cameras are generally used indoors where lighting can be controlled.

Typical applications: precision inspection, quality control, metrology, 3D scanning.

LiDAR sensors

LiDAR, short for Light Detection and Ranging, uses a focused laser emitter that scans back and forth, projecting a raster pattern of light points across the scene.

Each time a pulse is emitted, the sensor records the interval between emission and the moment the light is reflected back. That interval gives the distance to the target, based on the speed of light. A completed scan produces a point cloud, anywhere from a few hundred to many thousands of individual points, positioned in three dimensions. LiDAR systems can also run in a continuously streaming mode, and the data can be assembled into a 3D map of an area.

LiDAR systems vary enormously by application. Because they use a collimated, focused laser beam, they can be effective at extremely long ranges of up to several hundred meters, while small, low-power LiDAR sensors are also used for short-range depth sensing.

One detail worth knowing when specifying a system: LiDAR sensors typically use infrared lasers at one of two wavelengths, 905 nanometers or 1550 nanometers. The shorter wavelength is less readily absorbed by atmospheric water and suits long-range surveying. The longer wavelength is more compatible with eye-safe operation, which matters for robots working around people.

Strengths. Range well beyond the alternatives, good performance across a wide variety of lighting conditions, and accurate mapping of large environments.

Limitations. Like all active sensing technologies, LiDAR is sensitive to ambient light outdoors. Scanning systems are bulkier, draw more power and produce sparser data at close range than a stereo or ToF camera.

Typical applications: autonomous vehicles, surveying, infrastructure mapping, outdoor robotics, large-scale navigation.

Comparison of depth sensing technologies

Property	Structured Light	Stereo Vision	LiDAR	dToF	iToF
Principle	Observes distortions in projected pattern	Compares features in two stereo images	Measures transit time of reflected light from an object	Measures transit time of reflected light from an object	Measures phase shift of modulated light pulses
Software complexity	Very high	High	Low	Low	Medium
Relative cost	High	Low	Varies	Low	Medium
Accuracy	Micrometer to millimeter	Centimeter	Depends on range	Millimeter to centimeter	Millimeter to centimeter
Operating range	Low, but scalable	Around 6 m	Very scalable	Scalable	Scalable
Low light	Good	Weak	Good	Good	Good
Outdoor	Weak	Good	Good	Fair	Fair
Scan speed	Slow	Medium	Slow	Fast	Very fast
Compactness	Medium	Low	Low	High	Medium
Power consumption	High	Scalable, low	Scalable, high	Medium	Scalable, medium

Choosing the right technology

The right choice depends on operating range, required accuracy, environmental conditions, processing budget, system cost and how much integration effort the project can absorb. As a general guideline:

Application	Common fit
Autonomous mobile robots	Stereo vision
Warehouse automation	Stereo vision, iToF
Humanoid robots	Stereo vision
Industrial inspection	Structured light
Autonomous vehicles	LiDAR with stereo vision
Drones	Stereo vision
Smart manufacturing	Stereo vision, iToF
Digital twins	Stereo vision, LiDAR

Increasingly, systems combine more than one technology to get the strengths of each.

Applications driving growth

Robotics and automation. Robots need continuous awareness of their surroundings, and depth data provides it. It underpins autonomous navigation, obstacle avoidance, bin picking, palletizing, object recognition on assembly lines and safe collaboration with people on the same floor. For autonomous robots, this is the difference between a machine that follows a route and one that understands a space. As automation moves into work that was previously manual, depth perception is becoming a standard capability rather than an advanced one.

Autonomous mobile robots. AMRs depend on real-time environmental understanding more than almost any other category. Depth cameras let them detect obstacles, plan routes, avoid collisions, handle dynamic environments and operate safely around people. Crucially, reliable depth information lets a mobile robot work without dedicated infrastructure, so no floor markers, guide rails or predefined paths are required. Autonomous forklifts in an automated warehouse are a good example: the guidance system has to work in an aisle that changes shape every hour.

Humanoid robotics. Humanoids are among the fastest growing areas of Physical AI and among the most demanding. To function they need to understand object locations, human movement, floor conditions, free space and hand-to-object interaction, all at once and all while moving. Depth sensing supplies the spatial awareness that everything from walking to manipulation depends on.

Logistics and warehouse automation. Depth cameras produce high-resolution data at high speed, with iToF running at frame rates up to 60 fps. A single camera mounted over a conveyor can sort packages by size, measuring in all three dimensions, as fast as the conveyor runs. That has driven wide adoption across logistics, for sorting and for letting robots handle, stack and palletize packages accurately.

Mobility and automotive. Depth sensing appears both outside and inside the vehicle. Outside, it provides situational awareness for driver assistance features such as automatic parking assist, and for autonomous navigation. Inside, it is used for attention tracking to confirm the driver is watching the road, for gesture recognition, and for occupant monitoring.

Retail and consumer systems. Because depth cameras are not confused when people stand close together, they suit people counting, tracking and flow analysis, and they support checkout systems that need to recognize gestures and whether someone is holding an object. The same properties made depth sensing useful in consumer devices, where compact modules provide focus assistance in low light and body tracking for immersive interfaces.

Drones, inspection and AI-powered vision. Depth perception underpins indoor drone navigation, obstacle avoidance, 3D mapping and infrastructure inspection, including operation in GPS-denied environments. In industrial inspection it supports dimensional measurement, quality control and accurate 3D reconstruction of real assets. And combined with deep learning at the edge, it enables edge AI vision, covering scene understanding, semantic mapping, object segmentation, human awareness and safety monitoring. This is the basis of spatial AI: machine perception that reasons about a space rather than only recognizing what is in it.

That last combination is where the field is heading. Pairing machine learning with depth data produces systems that understand not only what they are looking at, but where it sits in physical space, and that improves recognition and depth estimation in exactly the difficult conditions where single-mode vision struggles.

The future of depth perception

Demand for machine perception continues to accelerate as robotics, factory automation, autonomous systems and Physical AI move into mainstream deployment. The systems being designed now need richer spatial understanding, more autonomy, and tighter integration between perception and AI processing.

Among the available approaches to robot vision, stereo vision has emerged as one of the most versatile and scalable for 3D vision systems. Its ability to deliver real-time 3D understanding across a wide range of environments, indoors and out, without moving parts and with RGB data alongside depth, makes it a foundational technology for the next generation of robotics.

Active technologies are improving too. Sensor designs increasingly include on-chip processing to suppress unwanted ambient light, which extends the usable accuracy and range of ToF systems in bright environments and outdoors, historically their weakest point.

For teams building these systems, choosing a depth sensing technology is no longer only a vision decision. It sets the ceiling on how capable the machine can be.

Where FRAMOS fits

FRAMOS builds complete embedded camera modules and industrial depth cameras for exactly these applications, with imaging engineering, R&D and manufacturing under one roof in Europe.

For 3D perception in production environments, our industrial depth cameras bring stereo depth into an IP66 housing with GigE and PoE connectivity, M12 connectors and multi-camera synchronization, so a system that worked on the bench keeps working on a moving platform. For autonomous robot programs specifically, we cover both navigation and task vision, including FSM:GO camera modules where a validated 2D module is the better answer.

If you are weighing depth sensing options for a specific application, talk to our camera experts. We will work through the requirements with you rather than sell you a camera you do not need.

Talk to an Expert