How a maze-solving AI learns
No coding or maths needed. Nobody tells the agent where to go. It tries things, gets rewards and penalties, and slowly works out the best route. This is called reinforcement learning.
1 · The cast
- The agent is the glowing square. It starts in the top-left corner.
- The goal is the pulsing gold target in the bottom-right.
- Walls are the bright green lines. The agent can only move up, right, down or left.
- An episode is one attempt: from the start until the agent reaches the goal (or runs out of steps).
2 · Rewards: how it knows what's good
- +100 for reaching the goal.
- −1 for every step. Wasting time costs points, so shorter routes win.
- −5 for bumping into a wall (you'll see a red ring flash).
The agent's only aim is to collect as many points as it can.
3 · Explore or exploit?
At every step the agent flips a weighted coin:
- Explore — try a random move to find out something new.
- Exploit — take the move it currently believes is best.
The trail behind the agent uses these colours, so you can see which kind of decision made each part of the path. The chance of exploring is epsilon (ε). It starts high and shrinks every episode: curious at first, confident later.
4 · The memory: the Q-table
For every cell, the agent keeps four numbers: how good is up, right, down, left from here? That table is the Q-table. After each move it nudges one number toward what it just learned:
new Q = old Q + α × ( reward + γ × best next Q − old Q )
- α (learning rate) — how far to nudge. Too high and it forgets; too low and it's slow.
- γ (discount) — how much future points count. High γ lets the goal's reward "leak" backwards along the route.
5 · Try it: step through learning yourself
This is a tiny 5×5 maze. Press step and read what happens. Turn on the heatmap and arrows to see the memory fill in.
6 · Reading the lab
Heatmap — purple cells have low value, yellow-green cells have high value. Watch a bright glow spread backwards from the goal.
Fog — dotted cells have never been visited. The agent knows nothing about them.
Policy arrows — the move the agent would pick in each cell, if it never explored.
Learned route — the marching white line is the greedy route from the start. If it ends with a red ✕ the agent is stuck or looping.
Policy score — shortest possible length ÷ the learned route's length. 100% means a perfect route. While the route still dead-ends or loops, you only get a little partial credit (up to 25%) for how far along it got.
Cell inspector — hover or tap a cell to see its four Q-values.
7 · Graphs
- Policy score should climb toward 100%.
- Steps per episode should fall toward the dashed gold line (the shortest route). It stays above it while ε is high because of random moves.
- Reward rises as steps get shorter and bumps stop.
8 · Experiments to run
- Set ε decay to 0.999 and watch how long random moves linger.
- Set γ to 0.5: the agent can't "see" the goal from afar and learns badly.
- Set α to 0.99 versus 0.05 and compare the graphs.
- Switch layout to loops. Now there are shortcuts and several routes; the best one is harder to find.
- Use the same seed for every test so the comparison is fair.