Peter Wang and Nico Christie · September 25, 2026

Warcraft III as an RL environment

Code on GitHub

StarCraft has received a lot of research love over the past decade (s2client-proto, PySC2 and AlphaStar, to name a few), but Warcraft III, the game that filled my childhood with magic and wonder, has been rendered frozen in Northrend. So Nico and I present wc3env, an open-source gym environment for Warcraft III: The Frozen Throne.

With the release of Jev, I also spent the last three days asking how well a frontier language model (Opus 5.5), when paired with a fast, smart-ish zero-shot classifier (Jev), can play a real-time strategy game. While Jev is surprisingly good at isolated fights, Opus+Jev fail in free-form 1v1 matches. I'll go through why below.

The environment

Blizzard never open-sourced the Warcraft III engine, and writing an open engine that behaves exactly like the original would be a far bigger project. So wc3env works on the real game. It injects a small DLL into Warcraft III (the 1.29.2 Legacy build) before the game runs its first instruction. Using about 90 known addresses inside that exact executable, the DLL detours the game's own functions through its code, which lets it freeze and advance the game clock, read every unit's state, and issue orders exactly as a player's clicks would. Python talks to it over a local pipe. Of course, an official API from Blizzard would be preferable, but this is more than performant enough to run experiments and train on (see the comparison with PySC2 below).

WARCRAFT III PROCESS Your agent Python wc3hook.dll clock · state · orders Game engine unmodified actions observation

Like other gym environments, you step it with an action (which can be empty) and get back an observation:

step.py
from wc3env import GameConfig, WC3Env

with WC3Env(GameConfig(map="(2)EchoIsles.w3x", step_ms=1000)) as env:
    obs = env.reset()
    # find a Peasant (type id hpea) and move it 400 units west
    peasant = next(u for u in obs["units"] if u["type_id"] == "hpea")
    move = {"unit_id": peasant["unit_id"], "command": "move",
            "arguments": {"x": peasant["x"] - 400, "y": peasant["y"]}}
    obs, done, info = env.step([move])  # advance one second of game time

An observation is exactly what a human player can see, fog of war included: every visible unit with its buffs, full detail on your own (orders, cooldowns, items), resources, and the events since the last step. Actions are also what a human player does: move, attack, build, train, research, learn a hero skill, cast, use, drop or buy an item, and revive a hero. Together they cover almost all of Warcraft III. There's no built-in reward yet, so you have to define your own. Terrain and pathing are not observed, because building placement and unit pathing are both left to the game engine.

How it compares with StarCraft II's API and PySC2:

wc3envPySC2 / s2client-proto
APIWC3Env (reset, step); GameSession for several agents on one clock; StepPool for many gamesSC2Env (reset, step)
Agents per game2 to 8 tested, plus built-in AI at easy, normal or insane1 or 2, plus built-in AI; two agents means two game processes
ConcurrencyOne process per game, any number of agents inside it. 4 games: 41× real time in total; 8 games: 73×One process per agent. 2 games: 82× in total; 4 games: 128×; 8 games did not fit in 16 GB of memory
Speed15× real time for one game, frozen between steps; real-time mode too43× real time for one game; real-time mode too
Launch3–7 s to a playable game15–24 s
ObservationsRaw unit state behind fog of war; no terrain or pixelsRaw units, feature layers or rendered RGB pixels, including terrain
RenderingOn or off; windows stay in the backgroundHeadless on Linux, with hardware or software rendering
Resets1.7 s, inside the running process; a fresh process every 32 episodes3.7 s
Seeds and replaysSeeded games end in the same state every run; native .w3g replaysSeeded games end in the same state every run; replays need the exact game version
MemoryAbout 225 MB per game1.4–1.7 GB per game
LinuxWine in DockerOfficial headless builds
Game versionsOne build: Legacy TFT 1.29.2Many; Linux packages ship past versions
OfficialNo: an injected DLL, offline onlyYes: Blizzard and DeepMind

Limit: you need your own licensed copy of the game. The repository contains no game files, and it is offline only. Both columns were measured on the same laptop (Intel Core Ultra 5 325, 8 cores, 16 GB) with the same test: an idle agent against the hardest non-cheating built-in AI (Warcraft III insane on Echo Isles, StarCraft II very hard on Simple96), rendering off, one-second steps, five game minutes, raw observations only. StarCraft II ran through PySC2 4.0 on game version 5.0.16. More in the architecture notes.

The agent

The good: Jev in a straight fight

wc3env the fight, every step Question builder code · legal options Jev a probability per option unit state per unit type the top choice is issued as an order, ~0.3 s later

Jev doesn't take images, so we turn game state into text that is compact, clear and agent-friendly. About once a second, each unit type in the fight gets one request, with a question for each of its units (every hero is its own type). A request has four parts (see below for the Far Seer):

The highest-probability option is chosen and sent to the game as an order. Most of the context engineering went into how the state of battle is featurized. For example, each unit's health comes with how much it lost in the last five seconds and which enemies hit it, and each attack option states the target's role, its distance, and how many of our units already attack it. Jev is not perfect and has a limited understanding of Warcraft, so we also introduce some fixed rules that are handled in code instead of by Jev, like computing a priority list of deserving units for healing, or automatically picking up close-by items.

To test how well Jev micros, I pitted it against the game's own computer control, with identical armies for each race. As you can see below, Jev won all the duels, and usually it was not close. These were typical rounds, not cherry-picked. Sampled highlights are bookmarked below, and the panel in each video shows every unit's current action for closer inspection. While not close to a pro gamer's level, I was impressed.

The bad: full games

wc3env real time Macro (Opus 5.5) the whole game, every ~7 s Micro (Jev) one call per unit type, ~1 per second text report build, train, research groups + objectives unit state unit orders

For full games, Opus 5.5 serves as the macro brain that controls the economy, the build order, neutral creeping, scouting and, of course, when to fight and when to leave. It runs at the latency of Opus, which is roughly one turn every 7 seconds. When combat is needed, Opus puts units into groups and hands each group off to Jev to control, with a clear objective.

Each Opus turn has three parts (see below for a Macro call):

A game costs about $5 against Warcraft's Insane AI. It loses, and often it is painful for me to watch. See the blunders below.

The failures can be attributed to several fundamental issues.

Intelligence and knowledge

Opus' decisions are often sensible but wrong: dishing out damage on creeps but wasting mana doing so, creeping too often instead of engaging the enemy, sacrificing units to protect heroes when both could live. These suggest a fairly naive view of Warcraft III. The solution is not prompting, since prescriptive advice tends to fail in the many scenarios nobody foresaw.

Moreover, decision quality appears to degrade when there are more than 10 prior observation-action pairs in context. This creates a tension. Keeping more turns gives Opus more context, but also more accumulated "diffs" to reason through, and its decisions get worse. Keeping fewer turns gives it a clearer head, but it loses track of what it set in motion. For example, it often forgets that Peasants turned into Militia should go back to their posts once a fight is done. Better context engineering may ameliorate this, but fundamentally the tension exists because Opus is not smart enough to reason about states through accumulated diffs. I have often had similar frustrations with coding agents after a file has accumulated many changes (I often just ask the agent to reread it), but games take this problem to a whole new level.

Jev suffers from the same issues. It takes in a very limited number of tokens, doesn't do well with larger contexts, and often makes suboptimal choices, especially when the right choice depends on history. For example, a unit will often attack one target, step back, attack another and move again, even when prompted to not switch targets often.

Space and time

There is no substitute for spatial intelligence. Terrain, pathing and army formations are crucial variables in fights and are in constant flux, so battle decisions must take them into account and be recomputed every second. Neither Opus nor Jev can parse visual input well (if at all), especially with many prior frames in its context. And even if Opus could, it would be too slow.

My guess is that a generalist language model would need to see the battlefield directly, decide in under a second, and reason well over the last 10 to 20 frames, all at once.

What's next

RTS games like Warcraft III are interesting benchmarks for general models because they combine long-term planning, hidden information and spatial reasoning, with time as a crucial factor, much like how decisions are made in the real world. The second direction is RL on top of an LLM. AlphaStar learned from up to 200 years of real-time play per agent, but starting from a model that already understands the rules might need far less.