World Model in Code · Part 2 of 3
How 49 Pixels Become Eight Numbers
DE Auf Deutsch lesenThe dynamics model isn't meant to work with full images. So an autoencoder first builds a small internal data format, and in the process learns which parts of the image actually need to be preserved.
Encoder and decoder share the bottleneck
The encoder takes in 49 grayscale values and outputs eight numbers. The decoder tries to reconstruct 49 grayscale values from those eight numbers.
Encoder: 49 → 24 → 8
Decoder: 8 → 24 → 49Both are ordinary small neural networks. The function netz builds the layers, vorwaerts computes their output, and rueckwaerts propagates the error back onto the weights. The project writes these parts itself so that no library hides the mechanics.
Why 24 neurons first?
The 24 neurons form an intermediate layer. They still see all 49 pixels and can combine them into features before the eight values compress the state. In this grid world, those could be combinations that distinguish the moving agent from walls and the goal, or that narrow down its position on the grid.
The network here is fully connected. So no single one of the 24 neurons reliably stands for one particular image pattern. During training they split the task among themselves. The tanh activation after 49 → 24 and 24 → 8 gives the encoder two computation steps in which such combinations can emerge.
A direct encoder 49 → 8 would be smaller and might already be enough for this map. It would have only one weighted computation step down to the bottleneck. Whether the intermediate layer is needed is decided here by comparison: if the smaller variant reconstructs the agent's position just as reliably, it's the better-fitting architecture.
Why eight values?
Eight is a parameter of this experiment, not a fixed rule for world models. Fewer values would compress more strongly and could lose important information. More values would give the network more room, but make the bottleneck less effective.
GROESSE = len(KARTE)
PIXEL = GROESSE * GROESSE # for this 7×7 map: 49
LATENT = 8
VERSTECKT_AE = 24
encoder = netz([PIXEL, VERSTECKT_AE, LATENT],
["tanh", "tanh"], zufall)
decoder = netz([LATENT, VERSTECKT_AE, PIXEL],
["tanh", "sigmoid"], zufall)The eight numbers also aren't named fields. Value 1 isn't automatically the x-coordinate, and value 2 isn't the y-coordinate. Training is free to distribute the information however it likes. What matters is whether the decoder and the dynamics model can work with it.
The average image gets subtracted out
Walls and the goal sit in the same spot in every training image. The agent only changes a single tile. The average image separates these two kinds of information: for each of the 49 positions, it computes the mean grayscale value.
Position Bild jetzt Durchschnitt nach dem Abzug
Wand oben links 1,0 1,0 0,0
Freies Feld 0,0 0,0 0,0
Feld mit Agent 0,3 0,006 0,294The wall disappears from the input because its value never changes. A free tile with the agent standing on it stays clearly visible: the agent is only rarely there, so the average at that position is close to zero. These are exactly the deviations the encoder is meant to capture in its eight numbers.
def zentriere(modell, bild):
return [wert - mittel
for wert, mittel in zip(bild, modell["mittel"])]
def kodiere(modell, bild):
z, _ = vorwaerts(modell["encoder"],
zentriere(modell, bild))
return zzip(bild, modell["mittel"]) walks both lists tile by tile and computes current value − average for each. Only the encoder's input gets centered. The decoder is still trained against the original image. It can reproduce the fixed walls from its own weights; the eight latent values mainly have to take care of the part that actually moves.
Training runs against the input image
For each image, the encoder runs first, then the decoder. The error is the difference between the reconstruction and the original. During the backward pass, the decoder weights are adjusted first, then the encoder weights.
z, cache_e = vorwaerts(encoder, x)
rekonstruktion, cache_d = vorwaerts(decoder, z)
fehler = [(aus - soll) * 2.0 / PIXEL
for aus, soll in zip(rekonstruktion, bild)]
dz = rueckwaerts(decoder, cache_d, fehler)
rueckwaerts(encoder, cache_e, dz)The decoder is essentially a training aid. It isn't needed for the later planning. There, the dynamics model works directly with the latent state. The decoder only makes a dreamed state visible again so we can check it.
Why the first metric was misleading
The reconstruction error is the average of the squared deviations across all 49 pixels. The agent occupies only one of them. If it's missing entirely while all other 48 pixels are correct, the average stays small:
richtiger Wert am Agentenfeld: 0,3
Ausgabe ohne Agent: 0,0
(0,3 − 0,0)² / 49 = 0,00184An error of roughly 0.0018 can therefore mean: walls, goal, and empty floor look correct, but the agent is missing. For a normal image that would often count as a small error. For this world model, it means exactly the information that's supposed to carry forward after an action is missing.
Epoche 5 Fehler 0.00146 Agent gefunden 38,3 %
Epoche 6 Fehler 0.00076 Agent gefunden 85,3 %
Epoche 7 Fehler 0.00022 Agent gefunden 100,0 %That's why the program additionally measures the agent's position. It looks for the agent's tile in both the original and the reconstruction and only counts a hit when both tiles match. At epoch 5, the mean pixel error already stood at 0.00146, yet only 38.3 percent of agents were placed correctly. Only the second metric shows whether the latent state is actually usable for the dynamics model.
What gets handed off to Part 3
After training, kodiere(modell, bild) can translate any image into eight values. The next part appends four more values for the action and learns from that the state after the move, the reward, and whether the episode ends.