I began writing this series when I started learning sensor fusion on my own.
It wasn’t just once or twice, I kept losing competitions because I couldn’t decide which sensor to trust. Some sensors drifted, others were noisy, and occasionally two sensors measuring the same quantity would disagree completely.
The obvious solution seemed to be “use them together,” but that immediately raises the question: when sensors disagree, how does a robot decide which measurement is closer to reality?
Around then I started hearing about Kalman filters and sensor fusion. I didn’t understand much at first, but I realised this was the field that tackled exactly these problems, so I decided to learn it properly.
This is not a mathematical tutorial. There are already many excellent resources, I’ll link some later. Instead, I want to write about the practical things I struggled with:
Some of what I say may still be incomplete or slightly wrong, I’m still learning.
You don’t need advanced mathematics to begin. It helps to have:
When I started learning sensor fusion I already knew a bit about PID and feedback systems, which turned out to be more helpful than I expected.
One of the first useful ideas is to think in terms of state. Examples of state variables include:
Systems are often classified as open-loop or closed-loop. In an open-loop system you command an action (for example, “move forward at 1 m/s”) and assume the system follows it. In practice that rarely holds: motors behave differently under load, wheels slip, battery voltage changes, and external disturbances occur.
In a closed-loop system we measure the system while controlling it and correct errors continuously. PID is a classic example: measure the current state, compare against the desired state, and apply corrections.
There are two main types of interaction with the environment:
To understand sensor fusion and filters properly, there are a few probabilities we should be aware of.
stateDiagram-v2
X<sub>t</sub> --> X<sub>t+1</sub>
U<sub>t</sub> --> X<sub>t</sub>
X<sub>t</sub> --> Z<sub>t</sub>
X<sub>t+1</sub> --> X<sub>t+2</sub>
U<sub>t+1</sub> --> X<sub>t+1</sub>
X<sub>t+1</sub> --> Z<sub>t+1</sub>
State-space model showing how control inputs influence state transitions and how states generate observations over time. —
| One is the state transition probability: $$p(x_t | x_{t-1},u_t)$$ |
The state transition probability specifies how control actions affect the next state, given the current state. This is closely related to Markov Decision Processes (MDPs), since sensor fusion builds on similar assumptions.
The other important probability is the measurement probability: \(𝑝(𝑧_𝑡∣𝑥_𝑡)\)
This describes how likely we are to get a particular measurement given the current state. In practice, we know the measurement but are trying to infer the hidden state using filters.
In the diagram above, you can see a Markov model representing these probabilities.
Finally, there is the belief over state, our confidence about the system’s current state given all past measurements and control actions: \(bel(x_t)=p(x_t∣z_{1:t},u_{1:t})\)
After learning about these probabilities, I came across Bayes filters, which can also be used for sensor fusion.
From what I understand, Bayes filters do something like this: using every control action and measurement up to a certain time, they compute the belief over all possible states, and then incorporate the latest measurement to update that belief.
Then there are Gaussian filters, which include the famous Kalman filter. While the Bayes filter is defined for the general case, Gaussian filters assume that the underlying distributions are normal (Gaussian).
It’s difficult to give a complete and accurate description of both Bayes filters and Gaussian filters in a single blog post, so I highly recommend the book Probabilistic Robotics by Sebastian Thrun, Wolfram Burgard, and Dieter Fox for a deeper understanding.