Skip to content

Home

Jump to bottom Edit New page
mcoms edited this page May 22, 2024 · 6 revisions

This document will cover the overall design of the AI system used in Project Borealis. The first section briefly explains the necessary theory behind the AI system and provides different in-depth learning resources. The second section presents an overview on the AI system and how to tune its parameters. Finally, the last section explains how to implement new "considerations", which determine when behaviors should be executed by the AI system.

IAUS Overview

Project Borealis' AI system is a custom implementation of the Infinite Axis Utility System (IAUS). IAUS is based on Utility Theory, where the basic premise is that AI characters score their possible actions, and pick from the top scoring options. IAUS has three components: considerations, contexts, and behaviors.

A consideration takes a single input, like self health, and outputs a score using a response curve. A response curve is a mathematical function, and is the basis of tuning AI in this set up. More detail on response curves will be given in the following section.

A context is the target of a behavior. It can be an enemy, neutral, ally, or the AI character itself.

A behavior is the routine that the AI character will perform. Each behavior has a list of considerations which are evaluated per context and multiplied together to get the final score. The behavior and context combination with the highest final score will be selected. Additionally, the behavior that the AI character is currently performing receives an inertia bonus, increasing the score by a constant factor, which prevents flip flopping between behaviors.

For more information about IAUS and Utility Theory, we recommend the following learning resources:

You can also read Dave Mark's blog for more insight and info.

IAUS in UE Behavior Trees

Project Borealis has a custom implementation of IAUS within UE's Behavior Tree system. This allows us to easily use the full feature-set of Unreal's AI system, while having the flexibility and expressiveness of a Utility AI system.

As an example, we will look at the Combine Overwatch soldier Behavior Tree, as it covers most of the features of the AI system:

Note that below the Behavior Tree's root there is a Utility node with multiple children representing the different behaviors of the AI character (Retreat, Move Closer, Throw Grenade, etc.). Behaviors are composed of decorators representing considerations, and the Utility node is charge of evaluating them to determine which behavior should be executed.

The Target Selector decorator on the Utility node allows the designer to adjust some parameters of the Utility system:

Inertia Weight: Currently, this value does not represent actual "inertia" in Utility Theory, but a time duration that limits how often a new behavior should be chosen. In other words, this acts like a cooldown to switch between behaviors. In the future, this will be changed to represent the time duration for the inertia bonus.

Blackboard Key: The blackboard key where currently selected context is stored.

Typically, you will not need to change these from their initial set up.

Now, let's look at the Retreat behavior:

This is composed of the considerations (decorators) Target Prioritization, Health, and Distance, and a service node that finds a position to cover from attacks. The behavior settings can be configured in the details panel of the "Retreat" node.

The Target flags tell which contexts to evaluate this behavior with:

  • Target Self: the AI character itself.
  • Target Friendly: characters within the same team or with a friendly relationship to it.
  • Target Neutral: characters with no relationship to the AI character's own team.
  • Target Hostile: characters with an enemy relationship to the AI character's own team.

This means that a behavior may have multiple characters to be evaluated with. Therefore, IAUS will select the behavior and context combination with the highest final score.

Initial Weight: Determines the starting score to multiply consideration scores by. Essentially, it is a limit to how high this score can go, as 0.95 * 1 * 1 * 1 = 0.95, so even if all considerations have a high score, it will be limited to 0.95.

Node Name: The name of the behavior. In this instance, the property can be used to override the default consideration name of “Health” for this consideration type.

Now, let's look at the Health consideration on the Retreat behavior:

In the consideration's details panel, you can see a preview of the current response curve and adjust it:

  • The X axis of this curve is the input from Minimum to Maximum. Typically, this range is determined automatically by the consideration type, but can be tuned if necessary. Any input outside the range will be clamped to within bounds.
  • The Y axis is the output score.

There are several response curves to choose from, each with its own settings:

You can drag the parameters around to tune your response curve easily, and see the effect of each adjustment in the preview image.

In this example, Target Self is checked, which is a specific property of the Health consideration. It means that the AI character checks its own health, rather than the context's health. We are scoring high for lower amounts of health, and start to drop off at around 30% health. This means that when the AI character has lower health, the Retreat behavior will be scored higher, and thus may be selected more often.

For more information about response curve design, we recommend the following learning resources:

Finally, you can add new considerations by right clicking the behavior and selecting "Add Decorator". Note that putting decorators that are not considerations does not make sense in the context of IAUS, and may crash the Engine at runtime.

Implementing Considerations

Note: These considerations are not available in the base plugin at this time, but can serve as a basis for your own game.

Implementing new considerations in C++ is the basis of creating new behaviors for our AI characters in Project Borealis. To explain how to do so, we will dissect and analyze the consideration IsMovementObstructed, which is used in the Zombie's behavior Break Obstacle.

This consideration is defined by the class UIAUSConsideration_IsMovementObstructed, and its purpose is to detect if there is a destructible object in front of the AI character obstructing its movement. If that is true, the consideration returns a score based on the normalized distance between the AI character and the object, and also stores the object in a given blackboard key; otherwise, the consideration returns a score of 0.

Let's start by explaining the header file, following a similar order in which things are defined:

Most considerations inherit from UIAUSAxisInput_Range. This class provides two float instance variables, Minimum and Maximum, which are used to normalize the consideration's input and can be adjusted from the editor. More detail on that will be given at the end of the section. Considerations that do not need such normalization should instead extend the class UIAUSBTDecorator_Consideration.

#include "IAUS/Public/Considerations/IAUSAxisInput_Range.h"

UCLASS(Meta = (DisplayName = "Is Movement Obstructed Consideration", Category = "Considerations"))
class PROJECTBOREALIS_API UIAUSConsideration_IsMovementObstructed : public UIAUSAxisInput_Range
{
	...
};

A consideration needs a constructor in the following cases:

  • It inherits from UIAUSAxisInput_Range and has to set a default value for Minimum or Maximum, or
  • It has FBlackboardKeySelector instance variables and it is desired to filter the types of blackboard keys that the user can select from the editor.
public:
	UIAUSConsideration_IsMovementObstructed();

Then, if the consideration has to write or read values from the blackboard, it is necessary to override the function InitializeFromAsset and to declare the required blackboard key selector instance variables of type FBlackboardKeySelector.

public:
	virtual void InitializeFromAsset(UBehaviorTree& Asset) override;

private:
	/** Blackboard key where the object obstructing the pawn's movement will be stored */
	UPROPERTY(EditAnywhere, Category = Blackboard)
	FBlackboardKeySelector ObstructingObjectKey;

Finally and most importantly, all considerations must override the function Score that evaluates the Context passed as parameter and returns an appropriate score.

public:
	virtual float Score(const struct FIAUSBehaviorContext& Context) const override;

Now let's jump to the implementation file:

The constructor sets the default value of Maximum to 2000. All distance-based considerations in Project Borealis set the default value of Maximum to 2000 (for no reason in particular, I believe), and Minimum is left to 0, but they can be adjusted from the editor if necessary. Both values will be used in the Score function to normalize the distance between the AI character and the object obstructing its movement. In addition, the constructor filters the instance variable ObstructingObjectKey so that only blackboard keys of type APBWorldObjectPhysicsDestructible can be selected from the editor. Filtering key types is completely optional but is a good practice, specially if the blackboard has a large number of keys.

UIAUSConsideration_IsMovementObstructed::UIAUSConsideration_IsMovementObstructed()
{
	Maximum = 2000.0f;

	ObstructingObjectKey.AddObjectFilter(this, GET_MEMBER_NAME_CHECKED(UIAUSConsideration_IsMovementObstructed, ObstructingObjectKey),
										 APBWorldObjectPhysicsDestructible::StaticClass());
}

Then, as the name suggests, the function InitializeFromAsset initializes the blackboard keys for its correct use.

void UIAUSConsideration_IsMovementObstructed::InitializeFromAsset(UBehaviorTree& Asset)
{
	Super::InitializeFromAsset(Asset);

	const UBlackboardData* const BlackboardData = GetBlackboardAsset();
	if (ensure(BlackboardData))
	{
		ObstructingObjectKey.ResolveSelectedKey(*BlackboardData);
	}
}

Finally comes the implementation of the Score function, the heart of the consideration.

float UIAUSConsideration_IsMovementObstructed::Score(const FIAUSBehaviorContext& Context) const
{
	...
}

First, let's take a look a the definition of the Context passed as parameter:

struct FIAUSBehaviorContext
{
	class IAUSEvaluator* Evaluator;
	class AActor* Actor;
	class AAIController* AIController;
	float TotalScore;
	int32 BehaviorIndex;
};

From all these variables, only Actor and AIController will be used by Score. The rest are used by the internal implementation of IAUS and should be disregarded. It is important to understand that AIController is the controller of the AI character that is executing the Behavior Tree, while Actor is the character that IAUS is currently evaluating this behavior with. In this example, Break Obstacle (the behavior using this consideration) has the flag Target Hostile set, so Actor could be any of the characters with an enemy relationship to the AI character's own team (for example, the player).

It is important to validate the pointers that will be dereferenced inside the function. In particular, Context.Actor, Context.AIController, Context.AIController->GetPawn(), and GetWorld() could be invalid during level streaming, and Context.AIController->GetBlackboardComponent() could be invalid if the Behavior Tree is corrupted.

	const UWorld* const World = GetWorld();
	if (!IsValid(Context.AIController) || !IsValid(World))
	{
		return 0.0f;
	}

	APawn* const Pawn = Context.AIController->GetPawn();
	UBlackboardComponent* const BlackboardComponent = Context.AIController->GetBlackboardComponent();
	if (!IsValid(Pawn) || !IsValid(BlackboardComponent))
	{
		return 0.0f;
	}

Without going into unnecessary details, this function does a collision check to find all destructible objects that are in front of the AI character. Then, it does the following:

	...

	for (const FOverlapResult& OverlapResult : OverlapResults)
	{
		... // Filter out objects that do not meet certain conditions

		// An obstructing object was found
		BlackboardComponent->SetValueAsObject(ObstructingObjectKey.SelectedKeyName, DestructibleObject);

		const float Input = FVector::Distance(Pawn->GetActorLocation(), DestructibleObject->GetBlastMeshComponent()->GetChunkCenterWorldPosition(0));
		const float Normalized = (Input - Minimum) / (Maximum - Minimum);

		return ResponseCurve->ComputeValue(Normalized);
	}

	BlackboardComponent->SetValueAsObject(ObstructingObjectKey.SelectedKeyName, nullptr);

	return 0.0f;

Note that, if an obstructing object is found, it is stored in the blackboard key ObstructingObjectKey, the distance between the AI character and the other actor is computed and normalized using Minimum and Maximum, and that normalized input is used to compute and return the score using the consideration's response curve. Otherwise, ObstructingObjectKey is emptied and a score of 0 is returned.

EQS

Finally, let’s look at the EQS service on the Retreat Behavior.

A service runs while a behavior is active. EQS (Environment Query System) is an Unreal Engine 4 system for navigation, which runs a Query on a set of points selected by a generator, which is then filtered or scored by tests. Typically, you will use this to select an ideal location in the world for the AI to go to. You do not need to use EQS if you’d like to go straight to a known location, like in the case of the zombie, which simply uses a Move To TargetActor node in the behavior routine.

This guide will not cover the system in its entirety, so visit the EQS documentation to get more information after this example.

In the details panel, you can configure the EQS service.

  • Query Template: The Environment Query to use for this service (will discuss in a later section).
  • EQS Blackboard Key: The Blackboard key which contains the Environment Query to use (alternative option to Query Template), allowing for dynamic Query selection.
  • Query Config: Key-value store for parameters passed into the Query
  • Run Mode: What results to use from the Query.

  • Single Best Item: Uses the item with the highest score.
  • Single Random Item from best x%: Uses a random item within the top x% items.
  • All Matching: Uses a random item which passes filters, ignoring score.

Now, let’s view the Query:

The generator for this Query is a Pathing Grid. You can select your own generator by dragging from the Root node:

You can adjust many properties in the Pathing Grid generator:

  • Grid Half Size: This is how far in each direction the pathing grid will be generated in a square around “Generate Around”.
  • Space Between: The spacing between each point in the pathing grid. Increasing this will make the set of points more sparse, good for selecting only a few locations where it doesn’t matter to be precise (regrouping behavior, battle preparation, etc.) Making it more dense is good in small areas while in combat.
  • Generate Around: The Environment Query Context to generate the pathing grid around.

The Item context is an internal built-in query and should not be used.

The built-in Querier context is the AI character performing the Query. Lastly, we currently have one context implemented in our project: AllPlayers, which gets all players (should only be one). It is implemented in Blueprint, and you can create your own Blueprint child class of EnvQueryContext to create your own. Here is the implementation for AllPlayers:

Projection Data provides advanced control of the navigation used for this generator.

You can create a new navigation filter, and adjust settings per navigation area. You can use navigation areas as markup, to include or exclude from the navigation filter, as well as set costs for the area:

The next component is tests of a generator. These score and/or filter the set of points from the generator.

The PathExist _test _checks if a path exists from the querier to the item (a point in the grid).

It filters out those which do not have a valid path.

Next, let’s see this distance test:

These settings find the 3D distance (Test Mode) from players (Distance To), and only select points which are at least 2000cm away (Float Value Min) from the player, and at most 5000cm (Float Value Max) away from the player. Points which are further away are scored higher, so that the AI prefers being further away from the player when retreating.

Add a custom footer
Add a custom sidebar
Clone this wiki locally