31 de maio de 2019
Getting AI to control giant speeding cars with lasers and machine guns using Unreal Engine
In this article, I’ll dig into how working with Unreal Engine 4 helped us develop a neat solution for creating compelling AI opponents.
Decision making at 200 KPH…
Dark Future’s AI had to solve an interesting set of challenges:
- Firstly, it needed to let vehicles navigate a stretch of highway often filled with debris, other cars, tight turns, ravines, and more.
- It then had to do all of that at high speeds (and we mean high speeds... like 200 KPH (124 MPH) when things get going!)
- It also had to do both of those while fighting player-controlled vehicles, utilizing mounted machine guns, rocket launching turrets, mines, and more.
- And finally, it has to do ALL of the above while being just predictable enough to enable the player to strategize against them, without feeling boring or repetitive.
It’s a daunting prospect, to be sure!
But before getting to solving these sorts of problems, they have to just be able to drive. Dark Future uses a modified version of Unreal Engine 4’s vehicle simulation code, which is implemented from the PhysX vehicle simulation, so the cars have a very detailed driving model.
That’s not AI of course, so I won’t go into detail here, but suffice it to say - our cars work like cars - they have a throttle, steering, and so on.
So, the cars can be driven. That’s good, the ‘how’ is easy: Throttle is a value from -1 to +1, and the same for steering. Cool.
Lessons in controlling a high-speed car ladened with lasers
It was clear once we got thinking about these problems that the built-in Nav Mesh functionality that UE4’s AI has would not suit our vehicles. It’s superb for characters that can turn on the spot, but stretching the system to work for vehicles would not be ideal. We needed something to help make the steering behaviors work correctly.
While steering behaviors are great for large amounts of less detailed entities, if you spend a lot of quality time with a few larger actors navigating an environment using steering behaviors, you start to see the cracks. And if you don’t see the cracks, it’s probably because the code is thousands of lines long, with many tightly connected behaviors that have to talk to each other directly in order to work out the context for the decisions that the behaviors reach.
Our lead programmer at the time, Abhishek Sagi, found a solution in a chapter of Game AI Pro 2 called “Context Steering: Behavior-Driven Steering at the Macro Scale” written by Andrew Fray, who superbly outlines a solution his team came to for a AAA racing game.
I won’t go into a lot of detail about Context Steering (as you can - and should - read it all here on his blog) but to briefly explain... Context Steering is similar in principle to a steering behavior implementation, whereby a number of isolated behaviors make calculations, which are then combined to produce a result. In the Context Steering system, instead of the behaviors each reaching their own decision and producing a result, they provide the context for the decision they would have made, so that the system has significantly more information with which to make its final decision.
Making the most of multithreading
So we set about prototyping our implementation of what we refer to as the Context Behavior System (CBS). For this, we started a new project using the “Vehicle Advanced” template, threw together our own map with some obstacles, and gave the car a set throttle.
We then wrote a quick and rough CBS using a single Blueprint. This Blueprint ended being absolutely enormous - too big even for a screenshot. The picture below shows our prototype vehicle attempting to follow a spline, while chasing a target in the distance, while avoiding the blocks.
Great! Now that our cars could steer, we took the prototype Blueprint out of the test project, and imported it into Dark Future, hooked it up to the cars, and it’s all good!
As you might imagine, this process is hugely expensive. Realistically, we couldn’t afford to have more than four vehicles operating using CBS in our level simultaneously without curtailing the effectiveness of the system to such a degree as to render the whole thing pointless. This would not do.
However, we had a solution. Context Steering lends itself extremely well to being multithreaded, due to its nature of having small, isolated behaviors with set amounts of data required in order to provide their context. So, using UE4’s multithreading API, we wrote a solution where the data set for each vehicle could be generated on the game thread, then offloaded to a worker thread, which would run the data through the behaviors, process the contexts they returned, and compute the end steering result to pass back to the vehicle.
Now we could easily have 20 or even 30 vehicles in the level, all running exactly the same code as before, but with a negligible cost from CBS. Naturally, the other costs for having 30 highly detailed, fully simulated vehicles in the level at once is still prohibitive, let alone the extraordinary difficulty of playing the game with 30 enemy cars roaming about the place, but sometimes it’s the principle that counts!
Using CBS to navigate behavior trees
After weeks worth of incredibly challenging work, our AI is capable of... turning left or right. Which is wonderful!
Now it needs to figure out where it might want to go, who it might want to chase, what road to navigate, who to shoot, when to shoot, etc. For this, we turned to UE4’s Behavior Tree system.
Behavior Trees are a widely used method for driving decision-making for AI. Despite the fact that we would not use the nifty built-in features for navigation, the Behavior Tree system still has everything we need, and more!
At this point, we had three behaviors in the CBS:
- Avoidance, which traces the environment to find things to stay away from.
- Chase, which wants to head towards a given point.
- Road Follow, which wants to... well y’know.
Writing a number of custom Blueprint tasks for the AI to perform, I was quickly able to utilize the CBS for a number of things.
When not in combat, we deactivate the Chase, feed the ideal road into the target road for Road Follow, and allow the AI to be “cautious” about obstructions by increasing the length of traces for the Avoidance. When in combat, we simply activate the Chase behavior and feed it the target vehicle's location, then proceed into logic for firing weapons at the appropriate time.
We also decided to make use of the Environment Query System, or EQS, which is a neat way of querying the game environment and running a set of tests against the results. Most commonly, this is used in partnership with the Navigation system, where you might query a set of Nav points, and test them for things like distance and Line of Sight. In our case, we wrote a custom query generator which looks for vehicles on a given team (player, hostile AI, civilian AI, etc), and scores the results based on proximity and direction.
EQS queries can be run to look for a single result, or to return all results, so this query can double up as simple enemy detection as well as ideal target picking. What’s even neater than that is that EQS queries are time-sliced - they run over a series of frames - so you can run fairly complicated queries without too much of a performance hit, as long as you don’t need instant results.
Teaching AI all about formations
Now that our AI can drive and fight, it became readily apparent that the AIs needed coordination. Each car roaming around on their own having a grand old time wouldn’t be good enough when the mission gameplay calls for coordinated attacks on convoys. We needed a hierarchical approach to solve this problem, whereby the Driver AI could handle the busy work of driving and shooting, and a Group AI, which owns a series of Drivers, could tell them where to go and who to shoot.
This involved a few key changes - instead of Driver AIs locating and engaging their own targets, they inform their Group AI that a target has been found. The Group keeps a list of all known targets and can prioritize and assign targets back to Drivers. In a standard situation this might play out like the Drivers were deciding for themselves anyway, but having the Group enables our designers to put limits on the number of Drivers engaging a single target at any given time, as well as overriding other engagements in favor of a particular target.
It also enables formation driving. The Group assigns a particular vehicle to be the Lead, and the others to follow, giving them an ideal clock facing (5 o’clock, 7 o’clock) and distance. The other Drivers then run a custom service on their Behaviors Tree (Services are essentially functions that run on a custom tick rate and only execute when a task in their branch of the tree is executing), which computes the ideal location and feeds that to the Chase behaviour in CBS.
This feature opened the door to a lot of great variation in gameplay situations. Formations were good for keeping a few cars driving together, sure, but they were easily extended to be Hostile Formations, and since they are so simple to define, we were able to set up many preset formations to use depending on what type of vehicles are in the Group and what types of weapons they have. To give an example - if the group has a vehicle with strong rear armour and a turret-mounted weapon, and a vehicle with powerful ramming weapons, it may try to assume a “lead-and-follow” formation where the armored car goes in front to provide the player a tempting target, while the ramming vehicle rams the player’s vehicle from behind.
In this image, you can see two AI-controlled vehicles (left, bottom) attempting to assume a formation around the player's vehicle (centre), with a third off-screen bringing up the rear.
Creating Dark Future has created some really unique challenges and issues, but working with UE4 has helped see us through the worst of them, and we’ve ended up with a game we are really proud of.
If you want to have a look at the game in action, you can check out our new trailer. Dark Future: Blood Red States releases on Thursday May 16 and you can find more details on our Steam page.