How videogames are being used for cycling safety research
Check out the shelves of your local videogames store and you’ll find any number of titles that simulate the experience of driving a car. Games that simulate riding a bike are considerably rarer. One such game, however, is the critically acclaimed 2013 title Grand Theft Auto V (GTA V).
In GTA V, players can find (and yes, steal) a variety of bikes as they explore the open-world “sandbox” that is the city of Los Santos. Cycling also plays a role in one of the game’s main storylines.
But beyond the fun of blasting around a fictionalised version of Los Angeles by bike — and it is fun — cycling in GTA V might actually be useful in the real-world too: as a simulator that can be used by cycling safety researchers.
At the Delft University of Technology in the Netherlands, a group of researchers have been testing GTA V to see if it could be used to help understand the decisions that road users make in hazardous situations. The group is headed by Pavlo Bazilinskyy who is both a cyclist (he’s part of a triathlon club in Delft) and a long-time gamer. It was Bazilinskyy’s history with modifying (or “modding”) an earlier game in the GTA series that sparked the idea for this recent research project.
“When I was a kid I used to mod Grand Theft Auto: San Andreas, a few versions back. It was quite easy and fun,” he told CyclingTips. “Now I’m doing my research with driving simulators which are often quite slow and painful to work with. And I remembered my experience with the game. So I offered it to my group and they ate it up.
“This is the first iteration. We’re trying to find out if it’s possible to use the game as a platform for science research and it seems to be possible.”
Bazilinskyy and his team modded GTA V to create a self-contained, in-game environment which would allow them to see how people reacted to various on-road incidents. Twenty-seven study participants were invited to guide one of GTA V’s protagonists through a short course in Los Santos, encountering 12 scripted traffic incidents along the way.
Some of the incidents required the user’s input to prevent the rider from crashing — a car pulling out from a parking spot unexpectedly, and a driver opening their door into the cyclist’s path — while other incidents simply required the rider to pay attention to avoid potential danger.
While completing the trial, the participants wore an eye-tracking device to allow researchers to see where the participants were looking as they encountered each traffic incident.
“We combine an eyetracker and the game, so we were able to see where people look in the game, in real time,” Bazilinskyy said. “So that’s unique — it hasn’t been done before. And we also made the foundation for future research because we now have developed a set of tools which can allow to make such scenarios as we have demonstrated.
“We have 12 cases of cycling behavior in traffic and that’s the first step in line, I hope, of many projects in this direction.”
The benefits of GTA V
The use of eye-tracking devices in road safety research isn’t new. They’ve been used for years now to determine how effective road users are at detecting hazardous situations around them, both in real-world tests and with simulators. So what is it about GTA V that gives it research potential as a cycling simulator?
For a start, traditional driving (and cycling) simulators are tough to work with, expensive to modify, and often lack “perceptual fidelity” — that is, a sense of realism. By contrast, the physics engines that power modern videogames, not to mention the graphical capabilities of the machines that run them, allow for a much more realistic, immersive experience. And there are other benefits to GTA V.
“The huge world of 100 kilometers square or so gives some possibility as well,” said Bazilinskyy. “And the flexibility of the game … if we put enough effort we can probably even put [the entire city of] Delft, where I am right now, in the game as it is. The flexibility of the game is the biggest strength.”
This isn’t the first time GTA V has been used for academic research. Earlier this year, researchers from the U.S. used the GTA V world to teach AI to recognise street signs — a vital part of the puzzle when it comes to creating smart and safe automated vehicles.
“To train the network, the authors develop a method to automatically collect labeled data from Grand Theft Auto 5, a video game,” the study’s abstract reads. “Using this method, they collect a dataset of 1.4 million images with and without stop signs across different environments, weather conditions, and times of day. Convolutional neural network trained and tested on this data can detect 95.5% of the stops signs within 20 meters of the vehicle with a false positive rate of 5.6% and an average error in distance of 1.2m to 2.4m.”
Despite these and other research applications for the GTA V environment, there are pitfalls to consider. Not least copyright concerns from the game’s publisher, Rockstar Games.
“I think we stated in the very last sentence of our paper, at the moment Rockstar is a bit … they’re trying to sue a couple of guys over the use of the game. So it’s a bit of a grey area,” Bazilinskyy said. “It’s kind of a shame which I think is not really correct. But the other guys, the ones who have used GTA for example to train AI — AI for autonomous driving — those guys have done it quite large-scale.
“I think they put hundreds of thousands of images from the game … online. I think that pissed off Rockstar. So we are trying to be a bit smaller scale than that …”
So what did Bazilinskyy and his team achieve with their study? In short: they confirmed that GTA V does seem to have potential as a cycling simulator. Part of the reason for that is the ability to combine in-game data with eye-tracking data — something not offered by traditional simulators.
With the GTA V plus eye-tracking setup, a user’s gaze data can be mapped, in real-time, against in-game variables such as the rider’s speed and the direction they’re heading, plus the speed of other vehicles. This can provide a deeper understanding of the way cyclists perceive and interact with the world around them, and the physics involved in those interactions.
The GTA V setup can also allow simulations that are considerably more interactive than traditional vehicle simulators. Rather than forcing the user to simply follow an entirely pre-determined routine, as if on rails, the combination of eye-tracking plus the GTA V game allows the possibility of triggering different events based on the user’s gaze. Perhaps making eye contact with a driver at a confusing intersection prompts the driver to stop, pushing the simulation in a different direction?
Bazilinskyy’s study is really just the beginning. It was designed to test the capabilities of the eye-tracking and GTA V combination and now that’s proven effective, much more can be done.
“I think the next [step], and what I’m really excited about, is we want to see if it’s possible to use GTA V in a coupled simulator where you combine two games, basically as a network, and then you put two people into different PCs and then you let them drive and cycle and then you see how they behave with each other,” said Bazilinskyy, who comes to this research from the field of autonomous driving. “That’s really hard and difficult to do with traditional driving simulators — it’s really expensive. And now I want to see if that’s possible in GTA basically.
“I think that’s the next step which we’ll do in a few months from now.”
As in Bazilinskyy’s initial study, the cyclist- and driver-participants in the proposed study would both be following a scripted routine featuring a series of interactions with other road users, including each other.
Bazilinskyy is also keen to use GTA V to investigate how people behave while riding e-bikes, a growth market in many countries around the world, including Bazilinskyy’s.
“In the Netherlands quite some people now move to e-bikes and it’s not really understood well how they behave in traffic as well,” he said. “So maybe that could be another area which we’ll research. But we have to define the exact topic yet.”
There’s also potential when it comes to increasing the realism of the simulator setup.
“In order for the participants to control the bicycle in GTA V we used a game controller,” Bazilinskyy and colleagues wrote in their conference paper. “It may be beneficial to develop a setup that links GTA V with a stationary bicycle platform, similar to those of existing cycling simulators.”
While Bazilinskyy and his team are only just finishing the groundwork, it’s clear there’s great potential in this space. It would seem that GTA V isn’t just one of the most critically acclaimed videogames of all time, it also has great promise in the world of cycling safety research, helping researchers better understand how cyclists react to and behave around other road users.