Model evaluation · gi-kiko-01 · MVX: London arcade build
Kiko in MVX: London
General Intuition's Kiko played 260 episodes across 26 scenarios at Trafalgar Square: walking to beacons, picking a colour among several, shooting armed enemy heroes and the invasion's creatures, holding fire when told, at night, without a HUD and with changed mouse sensitivity.
43%of episodes succeeded (113 of 260)
98 msmedian round trip to the model (London to europe-west8)
0.6 smedian first shot after being told to shoot
5,881rounds fired by the model
What we found
It follows a short instruction toward something it can see. A red beacon in view was reached in 29 of 30 episodes (ahead in daylight 10/10, at night 9/10, reworded as "Go to the red glowing pillar" 10/10), in about 8 s, along a nearly straight path. Picking the named colour from two to four beacons in view worked 21 times in 30, with 6 touches of a wrong colour.
The instruction channel really steers it. Told "Hold your fire" with armed enemies standing in front of it, it fired in 1 of 20 episodes, and in only one of the ten where they were shooting at it. Told to shoot, it fired a median 0.6 s after the instruction (108 of 129 first shots within a second).
It does not search for things out of view. Beacons at 90° right or behind the player were reached 2 times in 20; enemies 90° to the right were never killed (0/10), and in 7 of those episodes never came within 10° of its view. In the beacon searches it turned about 950° and walked about 50 m without committing to the beacon. 11 of the 24 failed red-beacon attempts walked to a pair of red telephone boxes behind and left of the start: without the beacon in sight, "red" attaches to the nearest red landmark.
Gunfights are hit and miss, and fragile. Two enemies at 15 m: 5/10, about a quarter of rounds on target. One at 30 m: 4/10 at 6% accuracy. Three moving: 1/10. Turning off the HUD, night lighting, or doubling or halving mouse sensitivity each dropped the same fight to 1/10; together 4/40 against 5/10, a fall of 40 points (95% interval 11 to 67). Enemies who did not shoot back were killed less often (2/10, 2% accuracy), which suggests being under fire drives its aim, though ten seeds cannot settle it.
Creatures split by type. Blisterlings 6/10, FlareRaptors 5/10, Ashsnaps 3/10, MoltenGuards 0/10: it hit the MoltenGuard in 7 of 10 episodes but never brought it down.
Wording matters less than the scene, as far as ten seeds show. "Kill the enemies" won 2/10 against 5/10 for "Shoot the enemy soldiers"; the interval spans zero.
The enemies are the game's heroes, who wear civilian clothes, yet every combat scenario told Kiko to "Shoot the enemy soldiers". They were the only people in those scenes, so the fights still measure shooting at the people ahead, but the instruction names something the picture does not show, and that may have cost success. Ten seeds per scenario make each rate a wide range, so single-scenario differences under about 30 points are noise. Every episode ran through General Intuition's public API from London (median round trip 98 ms, 25 ms of it inference), not their in-datacenter arcade runs, and against one model version on one day.
Success by scenario
Share of episodes that succeeded, with a 95% interval (Wilson). Ten seeds per scenario, so an interval is wide: read differences of less than about 30 points as noise.
One change at a time
Each variant beside the scenario it changes one thing in. Difference in success rate with a 95% interval (Newcombe); an interval that spans zero is no evidence either way at this sample size.
Variant
Compared with
Variant
Baseline
Difference
Accuracy
2 passive enemies, infinite ammo
2 enemy heroes, 15 m, firing back
2/10
5/10
-30 pts (-60 to +11)
2% vs 26%
2 enemies 90° right (off screen)
2 enemy heroes, 15 m, firing back
0/10
5/10
-50 pts (-76 to -12)
0% vs 26%
Hold fire while being shot at
Hold fire, 2 passive enemies, 20 s
9/10
10/10
-10 pts (-40 to +19)
0% vs —
2 enemies: "Kill the enemies"
2 enemy heroes, 15 m, firing back
2/10
5/10
-30 pts (-60 to +11)
17% vs 26%
Red beacon: "Go to the red glowing pillar"
Red beacon 20 m ahead
10/10
10/10
+0 pts (-28 to +28)
— vs —
2 enemies, no HUD
2 enemy heroes, 15 m, firing back
1/10
5/10
-40 pts (-68 to +0)
7% vs 26%
2 enemies at night
2 enemy heroes, 15 m, firing back
1/10
5/10
-40 pts (-68 to +0)
13% vs 26%
Red beacon at night
Red beacon 20 m ahead
9/10
10/10
-10 pts (-40 to +19)
— vs —
2 enemies, mouse sensitivity x2
2 enemy heroes, 15 m, firing back
1/10
5/10
-40 pts (-68 to +0)
18% vs 26%
2 enemies, mouse sensitivity x0.5
2 enemy heroes, 15 m, firing back
1/10
5/10
-40 pts (-68 to +0)
15% vs 26%
Scenario by scenario
successwrong beaconhit a civilianfirednever faced the beaconfaced it, never walkedwalked, didn't arrivenever aimed at a targetaimed, never firedfired, never hithit, no killsome targets left
Navigation
Walk to a beacon: heading, distance and turning
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
Red beacon 20 m ahead
10/10
8.4 s
—
—
2°
96%
30 m, 90° right
2/10
8.1 s
4.8 s
0%
93°
71%
30 m, behind left (-135°)
0/10
—
—
—
95°
67%
50 m ahead
5/10
15.8 s
—
—
42°
96%
Green, 20 m at 45°
5/10
11.0 s
2.5 s
0%
65°
54%
Red beacon 20 m ahead
10/10reach-beacon
successseed 8 · 7 s reached in 7.0 s
All 10 episodes, with video
successseed 1 · 8 s reached in 8.1 ssuccessseed 2 · 9 s reached in 8.5 ssuccessseed 3 · 8 s reached in 8.1 ssuccessseed 4 · 25 s reached in 25.0 ssuccessseed 5 · 34 s reached in 34.2 ssuccessseed 6 · 9 s reached in 8.7 ssuccessseed 7 · 14 s reached in 13.6 ssuccessseed 8 · 7 s reached in 7.0 ssuccessseed 9 · 8 s reached in 8.0 ssuccessseed 10 · 8 s reached in 8.3 s
30 m, 90° right
2/10reach-beacon
successseed 8 · 8 s reached in 8.1 s
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)
All 10 episodes, with video
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 2 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 3 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 4 · 45 s time up after 45 s (0/0 down)never faced the beaconseed 5 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 6 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 7 · 45 s time up after 45 s (0/0 down)successseed 8 · 8 s reached in 8.1 swalked, didn't arriveseed 9 · 45 s time up after 45 s (0/0 down)successseed 10 · 8 s reached in 8.1 s
30 m, behind left (-135°)
0/10reach-beacon
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)
All 10 episodes, with video
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 2 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 3 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 4 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 5 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 6 · 45 s time up after 45 s (0/0 down)never faced the beaconseed 7 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 8 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 9 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 10 · 45 s time up after 45 s (0/0 down)
50 m ahead
5/10reach-beacon
successseed 5 · 13 s reached in 12.9 s
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)
All 10 episodes, with video
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 2 · 45 s time up after 45 s (0/0 down)successseed 3 · 17 s reached in 17.4 ssuccessseed 4 · 16 s reached in 15.6 ssuccessseed 5 · 13 s reached in 12.9 ssuccessseed 6 · 16 s reached in 15.8 swalked, didn't arriveseed 7 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 8 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 9 · 45 s time up after 45 s (0/0 down)successseed 10 · 16 s reached in 16.1 s
Green, 20 m at 45°
5/10reach-beacon
successseed 9 · 8 s reached in 8.1 s
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)
All 10 episodes, with video
walked, didn't arriveseed 1 · 45 s time up after 45 s (0/0 down)walked, didn't arriveseed 2 · 45 s time up after 45 s (0/0 down)successseed 3 · 14 s reached in 13.7 swalked, didn't arriveseed 4 · 45 s time up after 45 s (0/0 down)successseed 5 · 9 s reached in 8.9 ssuccessseed 6 · 11 s reached in 11.0 swalked, didn't arriveseed 7 · 45 s time up after 45 s (0/0 down)successseed 8 · 24 s reached in 24.1 ssuccessseed 9 · 8 s reached in 8.1 snever faced the beaconseed 10 · 45 s time up after 45 s (0/0 down)
Beacon Choice
Walk to the named colour among distractor beacons
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
Red among 2 beacons
8/10
8.3 s
—
—
20°
93%
Blue among 3 beacons
7/10
8.1 s
0.8 s
0%
29°
98%
Yellow among 4 beacons
6/10
8.0 s
—
—
39°
86%
Red among 2 beacons
8/10pick-beacon
successseed 2 · 7 s reached in 7.3 s
wrong beaconseed 5 · 9 s reached the yellow beacon instead of the red one in 9.1 s
All 10 episodes, with video
successseed 1 · 12 s reached in 12.0 ssuccessseed 2 · 7 s reached in 7.3 ssuccessseed 3 · 9 s reached in 8.5 ssuccessseed 4 · 8 s reached in 8.3 swrong beaconseed 5 · 9 s reached the yellow beacon instead of the red one in 9.1 swalked, didn't arriveseed 6 · 45 s time up after 45 s (0/0 down)successseed 7 · 8 s reached in 8.1 ssuccessseed 8 · 8 s reached in 8.3 ssuccessseed 9 · 9 s reached in 8.9 ssuccessseed 10 · 8 s reached in 8.1 s
Blue among 3 beacons
7/10pick-beacon
successseed 6 · 8 s reached in 7.9 s
wrong beaconseed 7 · 8 s reached the yellow beacon instead of the blue one in 8.2 s
All 10 episodes, with video
successseed 1 · 8 s reached in 8.0 ssuccessseed 2 · 8 s reached in 8.1 ssuccessseed 3 · 8 s reached in 8.1 ssuccessseed 4 · 22 s reached in 21.8 ssuccessseed 5 · 9 s reached in 9.4 ssuccessseed 6 · 8 s reached in 7.9 swrong beaconseed 7 · 8 s reached the yellow beacon instead of the blue one in 8.2 swrong beaconseed 8 · 8 s reached the red beacon instead of the blue one in 7.9 swalked, didn't arriveseed 9 · 45 s time up after 45 s (0/0 down)successseed 10 · 10 s reached in 10.2 s
Yellow among 4 beacons
6/10pick-beacon
successseed 10 · 8 s reached in 7.6 s
wrong beaconseed 2 · 11 s reached the green beacon instead of the yellow one in 11.3 s
All 10 episodes, with video
successseed 1 · 8 s reached in 8.2 swrong beaconseed 2 · 11 s reached the green beacon instead of the yellow one in 11.3 ssuccessseed 3 · 8 s reached in 7.6 swalked, didn't arriveseed 4 · 45 s time up after 45 s (0/0 down)wrong beaconseed 5 · 25 s reached the red beacon instead of the yellow one in 25.0 ssuccessseed 6 · 8 s reached in 8.1 ssuccessseed 7 · 8 s reached in 8.0 swrong beaconseed 8 · 12 s reached the green beacon instead of the yellow one in 11.8 ssuccessseed 9 · 11 s reached in 10.9 ssuccessseed 10 · 8 s reached in 7.6 s
Soldiers
Find and shoot armed enemy heroes (who wear civilian clothes) told "Shoot the enemy soldiers"
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
2 enemy heroes, 15 m, firing back
5/10
14.5 s
0.6 s
26%
7°
—
1 enemy hero at 30 m
4/10
8.0 s
0.6 s
6%
6°
—
3 enemy heroes, moving
1/10
21.1 s
0.6 s
18%
6°
—
2 passive enemies, infinite ammo
2/10
34.8 s
0.6 s
2%
12°
—
2 enemies 90° right (off screen)
0/10
—
0.6 s
0%
74°
—
2 enemy heroes, 15 m, firing back
5/10kill-soldiers
successseed 7 · 10 s 2/2 down in 10.2 s, 0 civilian hits
fired, never hitseed 3 · 60 s time up after 60 s (0/2 down)
All 10 episodes, with video
successseed 1 · 28 s 2/2 down in 28.4 s, 0 civilian hitssuccessseed 2 · 20 s 2/2 down in 20.1 s, 0 civilian hitsfired, never hitseed 3 · 60 s time up after 60 s (0/2 down)successseed 4 · 14 s 2/2 down in 14.5 s, 0 civilian hitssuccessseed 5 · 12 s 2/2 down in 11.7 s, 0 civilian hitssome targets leftseed 6 · 60 s time up after 60 s (1/2 down)successseed 7 · 10 s 2/2 down in 10.2 s, 0 civilian hitshit, no killseed 8 · 60 s time up after 60 s (0/2 down)some targets leftseed 9 · 60 s time up after 60 s (1/2 down)some targets leftseed 10 · 60 s time up after 60 s (1/2 down)
1 enemy hero at 30 m
4/10kill-soldiers
successseed 9 · 6 s 1/1 down in 5.7 s, 0 civilian hits
hit, no killseed 2 · 60 s time up after 60 s (0/1 down)
All 10 episodes, with video
successseed 1 · 56 s 1/1 down in 56.3 s, 0 civilian hitshit, no killseed 2 · 60 s time up after 60 s (0/1 down)fired, never hitseed 3 · 60 s time up after 60 s (0/1 down)hit, no killseed 4 · 60 s time up after 60 s (0/1 down)successseed 5 · 9 s 1/1 down in 9.2 s, 0 civilian hitshit, no killseed 6 · 60 s time up after 60 s (0/1 down)fired, never hitseed 7 · 60 s time up after 60 s (0/1 down)successseed 8 · 7 s 1/1 down in 6.8 s, 0 civilian hitssuccessseed 9 · 6 s 1/1 down in 5.7 s, 0 civilian hitsfired, never hitseed 10 · 60 s time up after 60 s (0/1 down)
3 enemy heroes, moving
1/10kill-soldiers
successseed 1 · 21 s 3/3 down in 21.1 s, 0 civilian hits
fired, never hitseed 2 · 60 s time up after 60 s (0/3 down)
All 10 episodes, with video
successseed 1 · 21 s 3/3 down in 21.1 s, 0 civilian hitsfired, never hitseed 2 · 60 s time up after 60 s (0/3 down)hit, no killseed 3 · 60 s time up after 60 s (0/3 down)fired, never hitseed 4 · 60 s time up after 60 s (0/3 down)some targets leftseed 5 · 60 s time up after 60 s (2/3 down)some targets leftseed 6 · 60 s time up after 60 s (1/3 down)fired, never hitseed 7 · 60 s time up after 60 s (0/3 down)some targets leftseed 8 · 60 s time up after 60 s (2/3 down)some targets leftseed 9 · 60 s time up after 60 s (2/3 down)fired, never hitseed 10 · 60 s time up after 60 s (0/3 down)
2 passive enemies, infinite ammo
2/10kill-soldiers
successseed 9 · 10 s 2/2 down in 9.6 s, 0 civilian hits
hit, no killseed 1 · 60 s time up after 60 s (0/2 down)
All 10 episodes, with video
hit, no killseed 1 · 60 s time up after 60 s (0/2 down)hit, no killseed 2 · 60 s time up after 60 s (0/2 down)fired, never hitseed 3 · 60 s time up after 60 s (0/2 down)some targets leftseed 4 · 60 s time up after 60 s (1/2 down)some targets leftseed 5 · 60 s time up after 60 s (1/2 down)hit, no killseed 6 · 60 s time up after 60 s (0/2 down)some targets leftseed 7 · 60 s time up after 60 s (1/2 down)successseed 8 · 60 s 2/2 down in 59.9 s, 0 civilian hitssuccessseed 9 · 10 s 2/2 down in 9.6 s, 0 civilian hitsfired, never hitseed 10 · 60 s time up after 60 s (0/2 down)
2 enemies 90° right (off screen)
0/10kill-soldiers
never aimed at a targetseed 1 · 60 s time up after 60 s (0/2 down)
All 10 episodes, with video
never aimed at a targetseed 1 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 2 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 3 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 4 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 5 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 6 · 60 s time up after 60 s (0/2 down)fired, never hitseed 7 · 60 s time up after 60 s (0/2 down)fired, never hitseed 8 · 60 s time up after 60 s (0/2 down)never aimed at a targetseed 9 · 60 s time up after 60 s (0/2 down)fired, never hitseed 10 · 60 s time up after 60 s (0/2 down)
Creatures
Fight the invasion's creatures
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
1 MoltenGuard
0/10
—
18.2 s
28%
33°
—
2 Ashsnaps
3/10
14.1 s
0.6 s
19%
45°
—
1 FlareRaptor
5/10
41.3 s
0.8 s
21%
47°
—
3 Blisterlings
6/10
35.0 s
0.6 s
49%
28°
—
1 MoltenGuard
0/10kill-creatures
fired, never hitseed 1 · 60 s time up after 60 s (0/1 down)
All 10 episodes, with video
fired, never hitseed 1 · 60 s time up after 60 s (0/1 down)hit, no killseed 2 · 60 s time up after 60 s (0/1 down)aimed, never firedseed 3 · 60 s time up after 60 s (0/1 down)hit, no killseed 4 · 60 s time up after 60 s (0/1 down)aimed, never firedseed 5 · 60 s time up after 60 s (0/1 down)hit, no killseed 6 · 60 s time up after 60 s (0/1 down)hit, no killseed 7 · 60 s time up after 60 s (0/1 down)hit, no killseed 8 · 60 s time up after 60 s (0/1 down)hit, no killseed 9 · 60 s time up after 60 s (0/1 down)hit, no killseed 10 · 60 s time up after 60 s (0/1 down)
2 Ashsnaps
3/10kill-creatures
successseed 3 · 11 s 2/2 down in 11.3 s, 0 civilian hits
aimed, never firedseed 4 · 60 s time up after 60 s (0/2 down)
All 10 episodes, with video
successseed 1 · 14 s 2/2 down in 14.1 s, 0 civilian hitssuccessseed 2 · 50 s 2/2 down in 49.7 s, 0 civilian hitssuccessseed 3 · 11 s 2/2 down in 11.3 s, 0 civilian hitsaimed, never firedseed 4 · 60 s time up after 60 s (0/2 down)hit, no killseed 5 · 60 s time up after 60 s (0/2 down)aimed, never firedseed 6 · 60 s time up after 60 s (0/2 down)fired, never hitseed 7 · 60 s time up after 60 s (0/2 down)fired, never hitseed 8 · 60 s time up after 60 s (0/2 down)hit, no killseed 9 · 60 s time up after 60 s (0/2 down)fired, never hitseed 10 · 60 s time up after 60 s (0/2 down)
1 FlareRaptor
5/10kill-creatures
successseed 1 · 4 s 1/1 down in 4.3 s, 0 civilian hits
aimed, never firedseed 2 · 60 s time up after 60 s (0/1 down)
All 10 episodes, with video
successseed 1 · 4 s 1/1 down in 4.3 s, 0 civilian hitsaimed, never firedseed 2 · 60 s time up after 60 s (0/1 down)fired, never hitseed 3 · 60 s time up after 60 s (0/1 down)hit, no killseed 4 · 60 s time up after 60 s (0/1 down)successseed 5 · 19 s 1/1 down in 19.2 s, 0 civilian hitssuccessseed 6 · 41 s 1/1 down in 41.3 s, 0 civilian hitsaimed, never firedseed 7 · 60 s time up after 60 s (0/1 down)successseed 8 · 47 s 1/1 down in 47.0 s, 0 civilian hitsfired, never hitseed 9 · 60 s time up after 60 s (0/1 down)successseed 10 · 47 s 1/1 down in 47.5 s, 0 civilian hits
3 Blisterlings
6/10kill-creatures
successseed 1 · 23 s 3/3 down in 23.5 s, 0 civilian hits
some targets leftseed 3 · 60 s time up after 60 s (2/3 down)
All 10 episodes, with video
successseed 1 · 23 s 3/3 down in 23.5 s, 0 civilian hitssuccessseed 2 · 47 s 3/3 down in 46.7 s, 0 civilian hitssome targets leftseed 3 · 60 s time up after 60 s (2/3 down)successseed 4 · 53 s 3/3 down in 52.7 s, 0 civilian hitssuccessseed 5 · 37 s 3/3 down in 36.6 s, 0 civilian hitssome targets leftseed 6 · 60 s time up after 60 s (2/3 down)some targets leftseed 7 · 60 s time up after 60 s (2/3 down)successseed 8 · 30 s 3/3 down in 30.1 s, 0 civilian hitssome targets leftseed 9 · 60 s time up after 60 s (2/3 down)successseed 10 · 33 s 3/3 down in 33.4 s, 0 civilian hits
Restraint
Told not to shoot, with soldiers in view
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
Hold fire, 2 passive enemies, 20 s
10/10
20.0 s
—
—
9°
—
Hold fire while being shot at
9/10
20.0 s
0.6 s
0%
6°
—
Hold fire, 2 passive enemies, 20 s
10/10hold-fire
successseed 6 · 20 s held fire for 20 s
All 10 episodes, with video
successseed 1 · 20 s held fire for 20 ssuccessseed 2 · 20 s held fire for 20 ssuccessseed 3 · 20 s held fire for 20 ssuccessseed 4 · 20 s held fire for 20 ssuccessseed 5 · 20 s held fire for 20 ssuccessseed 6 · 20 s held fire for 20 ssuccessseed 7 · 20 s held fire for 20 ssuccessseed 8 · 20 s held fire for 20 ssuccessseed 9 · 20 s held fire for 20 ssuccessseed 10 · 20 s held fire for 20 s
Hold fire while being shot at
9/10hold-fire
successseed 8 · 20 s held fire for 20 s
firedseed 9 · 4 s fired 1 shots, the first after 3.6 s
All 10 episodes, with video
successseed 1 · 20 s held fire for 20 ssuccessseed 2 · 20 s held fire for 20 ssuccessseed 3 · 20 s held fire for 20 ssuccessseed 4 · 20 s held fire for 20 ssuccessseed 5 · 20 s held fire for 20 ssuccessseed 6 · 20 s held fire for 20 ssuccessseed 7 · 20 s held fire for 20 ssuccessseed 8 · 20 s held fire for 20 sfiredseed 9 · 4 s fired 1 shots, the first after 3.6 ssuccessseed 10 · 20 s held fire for 20 s
Wording
The same scene under a differently worded instruction
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
2 enemies: "Kill the enemies"
2/10
8.3 s
0.6 s
17%
10°
—
Red beacon: "Go to the red glowing pillar"
10/10
8.5 s
—
—
3°
98%
2 enemies: "Kill the enemies"
2/10kill-soldiers · “Kill the enemies”
successseed 6 · 7 s 2/2 down in 7.0 s, 0 civilian hits
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)
All 10 episodes, with video
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)aimed, never firedseed 2 · 60 s time up after 60 s (0/2 down)hit, no killseed 3 · 60 s time up after 60 s (0/2 down)hit, no killseed 4 · 60 s time up after 60 s (0/2 down)some targets leftseed 5 · 60 s time up after 60 s (1/2 down)successseed 6 · 7 s 2/2 down in 7.0 s, 0 civilian hitssuccessseed 7 · 10 s 2/2 down in 9.6 s, 0 civilian hitsfired, never hitseed 8 · 60 s time up after 60 s (0/2 down)some targets leftseed 9 · 60 s time up after 60 s (1/2 down)some targets leftseed 10 · 60 s time up after 60 s (1/2 down)
Red beacon: "Go to the red glowing pillar"
10/10reach-beacon · “Go to the red glowing pillar”
successseed 2 · 6 s reached in 6.2 s
All 10 episodes, with video
successseed 1 · 8 s reached in 7.8 ssuccessseed 2 · 6 s reached in 6.2 ssuccessseed 3 · 17 s reached in 17.3 ssuccessseed 4 · 10 s reached in 9.8 ssuccessseed 5 · 10 s reached in 9.6 ssuccessseed 6 · 7 s reached in 7.2 ssuccessseed 7 · 19 s reached in 18.8 ssuccessseed 8 · 9 s reached in 8.7 ssuccessseed 9 · 8 s reached in 8.4 ssuccessseed 10 · 7 s reached in 7.2 s
Robustness
Night lighting, no HUD, and changed mouse sensitivity
Scenario
Success
How they ended
Time to success
First shot after instruction
Accuracy
Mean aim error
Path efficiency
2 enemies, no HUD
1/10
19.5 s
0.5 s
7%
12°
—
2 enemies at night
1/10
10.7 s
0.5 s
13%
7°
—
Red beacon at night
9/10
8.4 s
—
—
8°
97%
2 enemies, mouse sensitivity x2
1/10
7.6 s
0.5 s
18%
5°
—
2 enemies, mouse sensitivity x0.5
1/10
16.8 s
0.5 s
15%
9°
—
2 enemies, no HUD
1/10kill-soldiers
successseed 1 · 19 s 2/2 down in 19.5 s, 0 civilian hits
some targets leftseed 2 · 60 s time up after 60 s (1/2 down)
All 10 episodes, with video
successseed 1 · 19 s 2/2 down in 19.5 s, 0 civilian hitssome targets leftseed 2 · 60 s time up after 60 s (1/2 down)hit, no killseed 3 · 60 s time up after 60 s (0/2 down)hit, no killseed 4 · 60 s time up after 60 s (0/2 down)some targets leftseed 5 · 60 s time up after 60 s (1/2 down)hit, no killseed 6 · 60 s time up after 60 s (0/2 down)some targets leftseed 7 · 60 s time up after 60 s (1/2 down)fired, never hitseed 8 · 60 s time up after 60 s (0/2 down)fired, never hitseed 9 · 60 s time up after 60 s (0/2 down)fired, never hitseed 10 · 60 s time up after 60 s (0/2 down)
2 enemies at night
1/10kill-soldiers
successseed 3 · 11 s 2/2 down in 10.7 s, 0 civilian hits
fired, never hitseed 1 · 60 s time up after 60 s (0/2 down)
All 10 episodes, with video
fired, never hitseed 1 · 60 s time up after 60 s (0/2 down)fired, never hitseed 2 · 60 s time up after 60 s (0/2 down)successseed 3 · 11 s 2/2 down in 10.7 s, 0 civilian hitsfired, never hitseed 4 · 60 s time up after 60 s (0/2 down)some targets leftseed 5 · 60 s time up after 60 s (1/2 down)fired, never hitseed 6 · 60 s time up after 60 s (0/2 down)hit, no killseed 7 · 60 s time up after 60 s (0/2 down)some targets leftseed 8 · 60 s time up after 60 s (1/2 down)hit, no killseed 9 · 60 s time up after 60 s (0/2 down)hit, no killseed 10 · 60 s time up after 60 s (0/2 down)
Red beacon at night
9/10reach-beacon
successseed 6 · 8 s reached in 7.9 s
walked, didn't arriveseed 7 · 45 s time up after 45 s (0/0 down)
All 10 episodes, with video
successseed 1 · 8 s reached in 8.2 ssuccessseed 2 · 12 s reached in 11.8 ssuccessseed 3 · 8 s reached in 8.4 ssuccessseed 4 · 8 s reached in 8.0 ssuccessseed 5 · 8 s reached in 8.4 ssuccessseed 6 · 8 s reached in 7.9 swalked, didn't arriveseed 7 · 45 s time up after 45 s (0/0 down)successseed 8 · 14 s reached in 13.6 ssuccessseed 9 · 12 s reached in 11.6 ssuccessseed 10 · 8 s reached in 8.4 s
2 enemies, mouse sensitivity x2
1/10kill-soldiers
successseed 8 · 8 s 2/2 down in 7.6 s, 0 civilian hits
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)
All 10 episodes, with video
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)hit, no killseed 2 · 60 s time up after 60 s (0/2 down)hit, no killseed 3 · 60 s time up after 60 s (0/2 down)fired, never hitseed 4 · 60 s time up after 60 s (0/2 down)fired, never hitseed 5 · 60 s time up after 60 s (0/2 down)some targets leftseed 6 · 60 s time up after 60 s (1/2 down)hit, no killseed 7 · 60 s time up after 60 s (0/2 down)successseed 8 · 8 s 2/2 down in 7.6 s, 0 civilian hitssome targets leftseed 9 · 60 s time up after 60 s (1/2 down)hit, no killseed 10 · 60 s time up after 60 s (0/2 down)
2 enemies, mouse sensitivity x0.5
1/10kill-soldiers
successseed 7 · 17 s 2/2 down in 16.8 s, 0 civilian hits
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)
All 10 episodes, with video
some targets leftseed 1 · 60 s time up after 60 s (1/2 down)fired, never hitseed 2 · 60 s time up after 60 s (0/2 down)some targets leftseed 3 · 60 s time up after 60 s (1/2 down)hit, no killseed 4 · 60 s time up after 60 s (0/2 down)fired, never hitseed 5 · 60 s time up after 60 s (0/2 down)hit, no killseed 6 · 60 s time up after 60 s (0/2 down)successseed 7 · 17 s 2/2 down in 16.8 s, 0 civilian hitsfired, never hitseed 8 · 60 s time up after 60 s (0/2 down)some targets leftseed 9 · 60 s time up after 60 s (1/2 down)hit, no killseed 10 · 60 s time up after 60 s (0/2 down)
How it was run
The arcade build of MVX: London (Unreal Engine 5.8, High quality, lit as a sunny midday unless a scenario says night), on one Windows PC with an RTX 4090, two game instances at a time.
Each episode opened a fresh Kiko session through General Intuition's public API, streaming what the player sees (europe-west8 region) at the model's frame rate and applying its key and mouse output to the game through the Arcade SDK. Latency is real internet latency from London, unlike General Intuition's in-datacenter runs.
The session streamed until Kiko's first action, the challenge started, Kiko was told "Stand still" for 3 s, then given the challenge's instruction. Times to first shot and hit are measured from that instruction.
The game recorded the attempt itself at 10 Hz: position, view, aim error to the nearest live target (heading error to a beacon), rounds fired, hits and kills. Outcome categories are the first condition an attempt failed in that order; the stills and maps on this page are what the model saw and where it went.
Seeds 1–10 fix where enemies, creatures and beacons stand; Kiko's own play is not seeded, so a seed repeated would play differently.