Google’s Apparent Plan to Tackle Lag on Stadia, Explained
Google’s Stadia game-streaming service is a fascinating leap into the future of gaming. But in a world of varying internet speeds, restrictive data caps, and the worrying loss of “ownership,” it has a few challenges to overcome. Stadia engineers think they can at least conquer the input lag problem with what Google calls “negative latency.”
That may sound like an oxymoron, but (contrary to the memes that have already risen about it) Google isn’t claiming that your on-screen character is going to actually act before you press the button. Instead of your game playing itself like Lineage II: Revolution, Google is just trying to close the gap between your computer and their server.
“There’s always going to be some latency in them sending the data to your machine, so they must have some solution in mind to allow them to pipeline the data,” explains Garrett Cooper, computer engineer and indie game developer speaking to IGN. By sending that data to the user as fast as possible, they can help reduce some of that latency hence the somewhat confusing “negative latency” buzzword.
Google makes it sound like this is revolutionary tech, but input prediction is nothing new.
There are several ways Google could pipeline that data, but predicting user inputs is a big one. Speaking to Edge magazine in an interview (which PCGamesN reports on here, if you don’t have a copy), Stadia’s VP of engineering Madj Bakar explains that in a year or two the service could “have games that are running faster and feel more responsive in the cloud than they do locally, regardless of how powerful the local machine is.”
That’s a bold claim, so let’s step back and talk about how this predictive tech might work. Instead of actually pressing the buttons for you, it’s likely Google is going to pre-render certain frames in the background to account for different possible actions you might take. With a bit of machine learning added to the mix (not to mention some data collection), they could even predict the likelihood of specific actions over others, to make these predictions more accurate. Once you press the button on your controller, they open Schrödinger’s box and serve the pre-rendered frame that corresponds to that button press. For example, if Stadia can tell you’re in the middle of a multi-button combo, it might start rendering your next attack animation before you actually hit that button but wait to show it to you until it actually registers you inputting the command.
Google makes it sound like this is revolutionary tech, but input prediction is nothing new. Similar tricks date back to the days of Duke Nukem 3D and QuakeWorld, and have been used in plenty of online games since then. Fighting game enthusiasts use middleware like GGPO to reduce latency when playing online, and retro emulators are starting to build predictive input features to achieve a more CRT-like experience.
To Google’s credit, though, 3D games have to account for a lot more potential inputs than 2D retro sidescrollers. Tony Cannon, the creator of the aforementioned GGPO middleware, explains to us how his solution compares to what Stadia could achieve: “GGPO predicts the actions of other players in the game and computes the most likely outcome [of their inputs] immediately. If that outcome is incorrect, it rolls back to past to fix the error and quickly simulates forward to the new, more accurate present,” he says. “Based on what I’ve read, Stadia is most likely computing many possible futures and sending them all to the client, along with the current frame. If the next client input matches any one of the predictions, it can render that frame immediately, avoiding a round-trip to the server.”
In other words, Stadia’s servers wouldn’t need to wait for your input—your console could already have a few frames stored locally, showing you the right one as soon as you press a button. To be clear, we’re speculating here, but we already know that Microsoft began testing a very similar method of predicting input years ago, and its testers reported noticeably better experiences compared to typical game streaming.
“The problem they have to solve,” says Cooper, “is what to do when they predict wrong. Much like when you seek to a different part of a movie on Netflix, the algorithm can’t account for that and will need to re-buffer. In Stadia terms, this probably means resetting the game state on their server to the time on your machine that you pressed the button, and then sending you the new frames as soon as possible.” Unfortunately, when Stadia has to play catch-up in this way, it could result in a degraded user experience. Maybe the image will get a little blurry for those few frames, or perhaps the screen might “rubber-band” to the correct state. This correction would likely happen in a matter of milliseconds, so casual gamers may not notice but eagle-eyed veterans could, and that alone is probably enough to keep folks like me playing locally-rendered games rather than opting into Google’s streaming service.
The other big downside, of course, is the horsepower needed to render multiple outcomes before they happen. “It’s going to be more expensive to compute, render, and encode multiple futures compared to doing just one,” says Cannon. “On the other hand, the computation is done in the cloud, so the increased demand would primarily impact how much it costs Google to operate the service rather than hardware requirements of the device owned by the consumer.” So if Google throws enough silicon at the problem, they could decrease the latency between your controller and the screen significantly but it won’t necessarily be perfect.
As for being “more responsive in the cloud,” well… it’s possible, but it depends on a lot of factors. “If your local system can’t render 4k frames itself as fast as it can download 4k video, then it’ll feel faster,” says Cooper. Though this is more likely if developers create more complex games with this in mind using a ton of physics, or other computationally-heavy effects. “I’m sure there’d be games that will feel better local, but for a potential new breed of AAA games, it might only be possible [with the Cloud].”
So color me skeptical that this will play out the way Google makes it sound like it will at least in the immediate future. Local consoles aren’t going anywhere just yet, and unless Google’s creating Stadia-exclusive titles that take full advantage of this tech and avoid almost all of its potential pitfalls, developers will still build games with the lowest common denominator in mind.
But from the tech side of things, it’s a captivating advancement nonetheless.