Since the Atari era, video games changed significantly in as much as they no longer tend to be built around trying to "kill" the human player and end their game. This structure made sense in a video game arcade (you die, you put in another quarter, the more often this happens without you losing interest the more money the game makes) and it got inherited into most early video games _because_ of the arcades, but it's not actually fun.
In most modern games the human will invariably "win" eventually, so the "try to get most points before dying" approach makes less sense in general. But, _Speed running_ has become a very popular activity even in games you may not think of as being about "speed". I will be interested to see multi-purpose AIs take on speed running for two reasons:
1. The evaluation function is easier for the machine AND the human assessing it to understand, especially for "Any%" runs in most games, where the rule is basically you don't care how the game is supposed to be played, you just want to go from the start to the end in the least time using the controls (e.g. Mario Kart speed runs are often mostly about convincing the game that your shorter route counts as a "lap" of the track).
2. TAS (Tool Assisted Speed-running) is already a thing, TAS players don't compete with live human players, they choose and play back a precise sequence of inputs to cause the game to play out in a particular way, so they can do hundreds of frame-precise movements, or even influence hidden internal variables (with the effect that seemingly "random" behaviour from a game often proves to be deterministic). An AI can do everything the TAS player can, moment by moment, but it reacts to changes, which a TAS run cannot, on the other hand the TAS players often have deep insight into the implementation details of the game which would not be available to a general purpose AI player.
That's actually an interesting point - TAS tools should make it possible to make any game into an AI training environment.
The one thing I would say about that is that speedrunning is a really, really difficult thing to do with a gradient descent approach; you can get really optimized with a particular strat, but getting substantially faster often requires drastically different strategies - which can only be discovered with a very broad knowledge of the game's systems, in areas which you will likely never see if your only experience with the game is speedrunning one particular strat exclusively. And exploration of the game's space during a speedrun would kill your time.
> speedrunning is a really, really difficult thing to do with a gradient descent approach
That's kind of what makes it interesting, no? You could argue that speedrunning requires not just observing a change in score from a change in inputs, but understanding, symbolically manipulating, and exploiting game mechanics.
A lot of human speed-runners exploit glitches to achieve record times and even more so with TAS runs, where you can exploit glitches that require superhuman input sequences. It would be very interesting to see an AI discover and take advantage of such glitches in video games to improve its speedruns.
Actually, just the idea of having an AI discovering glitches at all is an impressive use of AI.
I like the simple idea behind arcade games. Last week I played a free flipper game on a ps4. I enjoyed it. On the other side, modern games feel like hybrid bastardization of movies with near zero gameplay improvement over games 10 year ago.
In most modern games the human will invariably "win" eventually, so the "try to get most points before dying" approach makes less sense in general. But, _Speed running_ has become a very popular activity even in games you may not think of as being about "speed". I will be interested to see multi-purpose AIs take on speed running for two reasons:
1. The evaluation function is easier for the machine AND the human assessing it to understand, especially for "Any%" runs in most games, where the rule is basically you don't care how the game is supposed to be played, you just want to go from the start to the end in the least time using the controls (e.g. Mario Kart speed runs are often mostly about convincing the game that your shorter route counts as a "lap" of the track).
2. TAS (Tool Assisted Speed-running) is already a thing, TAS players don't compete with live human players, they choose and play back a precise sequence of inputs to cause the game to play out in a particular way, so they can do hundreds of frame-precise movements, or even influence hidden internal variables (with the effect that seemingly "random" behaviour from a game often proves to be deterministic). An AI can do everything the TAS player can, moment by moment, but it reacts to changes, which a TAS run cannot, on the other hand the TAS players often have deep insight into the implementation details of the game which would not be available to a general purpose AI player.