Z is a real-time strategy game by the Bitmap Brothers from 1996. Two teams of robots, red and blue, fight over twelve flagged territories on the demo map. Whoever holds more territory gets faster production from their factories. The winner is whoever breaks into the enemy fort.
Zeta is my port of the game, written in C++, which loads the original game data. I tried to keep the gameplay 1:1 with the original. That accuracy became the foundation of the neural network project. When the game behaves the same way every time, you can learn in it.
In the video below, two small neural networks play against each other. This is not ChatGPT or anything like it. It is a few thousand numbers that learned a single thing: deciding, inside the game, where to send a squad of soldiers. They knew nothing about how the game is played beforehand. They learned it by playing, several thousand matches in a row.
Why this works at all
The 1996 game computes its world at a fixed rhythm. Zeta keeps that rhythm exactly, and on top of that it is deterministic. The same start (seed) and the same orders always produce the same match, down to the last pixel. That has two consequences.
-
The game can run without a window. Nothing is drawn, no sound plays, it just computes. A whole mission that takes ten minutes in real time finishes in 1.5 s. I ran six matches side by side, and twenty-four of them were done in 18 s. Without that, training would have taken months.
-
Every match can be replayed. All it takes is writing down the seed and every order. Run the match again and it comes out the same. That is also how the video was made: it is not an animation, it is a game that really happened, played once more with a camera on it.
How the network “sees” the game
The network does not look at the screen. Every two seconds the game sends a text report: which territory belongs to whom, where its own squads stand and how many soldiers and how much health they have, where the enemies are, where abandoned vehicles and guns lie, how the buildings are doing. It is the same information a player has on the minimap, nothing more.
The network answers with orders. And here is the trick that keeps the whole project honest: the network’s order is exactly what a player does with the mouse. “Select this squad and click on this point of the map.” The game then decides for itself what that means: go there, take the flag, attack the robot or the building, or enter the enemy fort. The game has no idea that a neural network is holding the mouse. :)
What one decision looks like
Every two seconds, every squad of soldiers gets to choose from 17 options:
- keep doing what it is doing (1 option)
- go to one of the twelve territories (12)
- attack one of the four nearest enemy squads (4)
The program describes each option with 55 numbers: a few numbers about the state of the whole game (how much territory each side holds, say), numbers about the squad itself (what kind of soldiers, how many, how much health, where it stands, whether it is moving or fighting) and numbers about the target (how far away and in which direction, who owns it, how much strength both sides have there).
The neural network takes those 55 numbers and returns one number: a score. The network has two hidden layers of 64 “neurons” and 7,809 weights in total. That is its entire brain.
The seventeen scores are then turned into probabilities (softmax: the better the score, the better the odds) and the squad draws one option. The draw is deliberate. If the network always took the best score, it would play the same way every time and never discover anything new.
Alongside it runs a second, smaller network (6,273 weights) that estimates from the overall state how the match will end. Nobody in the game listens to it; it serves during training, and in the video it draws the “who is winning” chart.
How it learned
Generation 0: First I wrote a simple scripted player, a handful of rules in a handful of lines: send an idle squad to the nearest foreign flag, attack an enemy inside your own territory, and when you hold more territory than your opponent, attack their fort. The network then watched 48 matches and learned to imitate the script. The result was a weak player, but no longer a random one.
Then learning by reward. From this point on, nobody taught the network anything. It just played, and after every match it got feedback: points for extra territory, points for its soldiers’ health compared with the opponent’s, and a big reward for a win or a penalty for a loss. Thousands of times in a row.
Who did it play against? The original AI from 1996 (ported into Zeta from the original, and able to play either colour), the script, an opponent that does nothing, itself, and in the third round its own older versions too.
| generation | matches in total (approx.) |
|---|---|
| gen 1 | 1,600 |
| gen 2 | 4,000 |
| gen 3 | 5,600 |
Everything ran on one ordinary computer, a few hours in total.
How good they are
So as not to guess from a handful of matches, I held a tournament: six players, each against each, 32 different starts, always from both colours. 960 matches in total. From the results I computed an Elo rating, the same system as in chess, with the original AI as the zero point at 1000.
| player | Elo |
|---|---|
| network gen 2 | 1183 |
| network gen 3 | 1162 |
| scripted player | 1082 |
| network gen 1 | 1032 |
| original AI (1996) | 1000 |
| network gen 0 (script) | 820 |
The difference between gen 2 and gen 3 is within the uncertainty (about ±28 points), so the third round did not measurably help. Both networks, however, beat the original AI and the hand-written script.
How the video was made
The match in the video is one match from the tournament: gen 2 as red against gen 3 as blue. I replayed it the same way once more and, while doing so, saved every decision of the network: all 17 options, their probabilities, the drawn choice and what was going on inside the network. Then once more, this time drawing the whole map into a video.
The music is the game’s original music. The main-menu track is the intro, the battle track plays under the match, the victory track at the end, all played through an emulated OPL3 sound chip. The sounds are the ones the game itself played wherever the camera happened to be looking. Gunfire, explosions and the soldiers’ radio. The game only voices the radio for the red side.
What it is not
On the limits:
- The network does not invent its own moves; it picks from the 17 options I designed. Likewise the 55 numbers that describe an option are hand-picked. The network only learned how to choose.
- It knows a single map, the mission from the demo. On another one it would have to learn again.
- The reward is shared by the whole team, so the network can hardly tell which squad decided well and which badly. That is the weakest point and the first thing to improve.
- The original game stays untouched. The agents are a tool next to the game, not part of it.
What could come next
A better reward for individual squads. Exporting the trained weights into the game, so a human can play against the network as an optional opponent. And a page where the tournament matches can be browsed on a tactical map.