The physics of irreversible states. How Control Barrier Functions guarantee safety

When a large language model hallucinates, the cost of the error is negligible: a developer presses backspace, or a user re-generates the prompt. The digital space is intrinsically forgiving, operating on virtual tokens where failure is merely an computational inconvenience.

But when a physical foundation model hallucinates while commanding a multi-ton robotic manipulator, the cost is immediately physical. A joint exceeds its torque limit, an end-effector collides with its environment, or a high-velocity movement violates a keep-out zone. In physical AI, "alignment" is not a set of conversational guidelines, it is the deterministic boundary between operation and destruction.

For years, the robotics industry has attempted to solve safety through the brute-force scaling of neural network parameters, relying on reinforcement learning penalties to "teach" the model to avoid catastrophic states. This is a dangerous category error. Probabilistic safety is not safety at all. To build truly deployable physical autonomy, we must treat safety not as a learned behavior, but as a mathematical invariant.

In machine learning, it is customary to train policies by assigning negative rewards to undesirable behaviors. If a robot collides with an obstacle during a training rollout, the policy receives a severe mathematical penalty. Over millions of iterations, the neural network learns a policy that minimizes the likelihood of this penalty.

However, deep neural networks are, at their core, high-dimensional interpolators. They output continuous probability distributions over action spaces. Even if a model is 99.99% confident in a safe trajectory, the remaining 0.01% represents a non-zero probability of executing an action that results in an irreversible physical state.

In classical mechanics, many physical failures are thermodynamically irreversible: a bent metal link, a shattered gear, or an overheated motor cannot be corrected by a subsequent positive reward. In the physical world, we cannot afford to learn safety by experiencing failure. Safety must be enforced externally, deterministically, and with absolute mathematical guarantees at the very boundary of hardware execution.

To guarantee safety, we must move away from soft penalties and look to the classical mathematics of dynamical systems, specifically Set Invariance and Control Barrier Functions (CBFs).

Let the continuous-time physical system of the robot be modeled as a control-affine system:

where $x \in \mathcal{C}$ represents the state of the robot (its position, velocity, and torque in configuration space), and $u \in \mathcal{U}$ is the control input commanded by a planning policy.

We define a closed, safe set $\mathcal{C}_{safe} \subset \mathcal{C}$ as the superlevel set of a continuously differentiable scalar function $h(x)$:

The boundary of this safe set, where the robot is precisely on the edge of a collision or joint limit, is denoted as $\partial \mathcal{C}_{safe} = \{ x \in \mathcal{C} : h(x) = 0 \}$.

Our objective is to guarantee that if the robot starts within this safe set, it remains there for all future time:

According to Nagumo's Theorem, a cornerstone of dynamical systems theory, a closed set is forward invariant under a vector field if and only if the vector field at the boundary points back into the set.

For control systems, this means the control input $u$ must satisfy a constraint that prevents the state from crossing the boundary. This is formalized by the Control Barrier Function inequality:

Applying the chain rule, we can expand $\dot{h}(x, u)$ in terms of the system dynamics:

Using Lie derivatives to represent the directional derivative of the safety function $h$ along the vector fields $f$ and $g$, we write this elegantly as:

This inequality defines a half-space of safe control inputs. As long as the command $u$ lies within this half-space, the system is mathematically guaranteed to remain within $\mathcal{C}_{safe}$ indefinitely. If the robot is far from the boundary ($h(x)$ is large), the constraint is highly permissive, allowing the planning policy complete freedom. But as the robot approaches the boundary ($h(x) \to 0$), the constraint restricts the allowable control inputs, forcing the system to decelerate or steer away, regardless of what the neural network commands.

The engineering elegance of this mathematical formulation lies in its modularity. We do not need to restrict the neural network's architecture or alter its weights to ensure safety. Instead, the safety filter acts as a pure geometric projection.

If the neural network proposes a nominal command $u_{nom}$, we can project this command onto the safe half-space defined by our Control Barrier Functions. This is a minimum-deviation optimization problem:

Because the constraints are linear with respect to the control input $u$, this optimization is convex. It can be solved deterministically at the hardware boundary with minimal computational overhead.

If the proposed command $u_{nom}$ is safe, the projection has no effect, allowing the system to leverage the full intelligence of the neural network. If the command would cause a violation, the projection outputs the mathematically minimal correction that preserves set invariance. The network is allowed to "think" freely, but the hardware remains physically incapable of executing a dangerous command.

The future of industrial physical AI cannot rely on the hope that a neural network will not fail. In environments where robots share spaces with humans and interact with expensive infrastructure, hope is not an engineering metric.

True progress in robotics requires a division of labor. We must leverage the incredible semantic understanding and pattern recognition of deep foundation models to propose complex, intelligent behaviors. But we must bound that intelligence within the rigorous, absolute constraints of physical geometry and control theory.

At Xolver, our work is anchored by this division. By separating probabilistic reasoning from deterministic safety invariance, we don't just build smarter models, we build systems that can be trusted to survive the complexity of the real world.