Behavior tree (artificial intelligence, robotics and control)

A behavior tree is a mathematical model of plan execution used in computer science, robotics, control systems and video games. They describe switchings between a finite set of tasks in a modular fashion. Their strength comes from their ability to create very complex tasks composed of simple tasks, without worrying how the simple tasks are implemented. Behavior trees present some similarities to hierarchical state machines with the key difference that the main building block of a behavior is a task rather than a state. Its ease of human understanding make behavior trees less error prone and very popular in the game developer community. Behavior trees have been shown to generalize several other control architectures.[1][2] Mathematically, they are directed acyclic graphs.

Behavior tree modelling the search and grasp plan of a two-armed robot.

Background

Behavior trees originate from the computer game industry as a powerful tool to model the behavior of non-player characters (NPCs).[3][4][5][6] They have been extensively used in high-profile video games such as Halo, Bioshock, and Spore. Recent works propose behavior trees as a multi-mission control framework for UAV, complex robots, robotic manipulation, and multi-robot systems.[7][8][9][10][11][12] Behavior trees have now reached the maturity to be treated in Game AI textbooks,[13][14] as well as generic game environments such as Pygame and Unreal Engine (see links below).

Behavior trees became popular for their development paradigm: being able to create a complex behavior by only programming the NPC's actions and then designing a tree structure (usually through drag and drop) whose leaf nodes are actions and whose inner nodes determine the NPC's decision making. Behavior trees are visually intuitive and easy to design, test, and debug, and provide more modularity, scalability, and reusability than other behavior creation methods.

Over the years, the diverse implementations of behavior trees kept improving both in efficiency and capabilities to satisfy the demands of the industry, until they evolved into event-driven behavior trees.[15][5] Event-driven behavior trees solved some scalability issues of classical behavior trees by changing how the tree internally handles its execution, and by introducing a new type of node that can react to events and abort running nodes. Nowadays, the concept of event-driven behavior tree is a standard and used in most of the implementations, even though they are still called "behavior trees" for simplicity.

Key concepts

A behavior tree is graphically represented as a directed tree in which the nodes are classified as root, control flow nodes, or execution nodes (tasks). For each pair of connected nodes the outgoing node is called parent and the incoming node is called child. The root has no parents and exactly one child, the control flow nodes have one parent and at least one child, and the execution nodes have one parent and no children. Graphically, the children of a control flow node are placed below it, ordered from left to right.[16]

The execution of a behavior tree starts from the root which sends ticks with a certain frequency to its child. A tick is an enabling signal that allows the execution of a child. When the execution of a node in the behavior tree is allowed, it returns to the parent a status running if its execution has not finished yet, success if it has achieved its goal, or failure otherwise.

Control flow node

A control flow node is used to control the subtasks of which it is composed. A control flow node may be either a selector (fallback) node or a sequence node. They run each of their subtasks in turn. When a subtask is completed and returns its status (success or failure), the control flow node decides whether to execute the next subtask or not.

Selector (fallback) node

Figure I. Graphical representation of a fallback composition of N tasks.

Fallback nodes are used to find and execute the first child that does not fail. A fallback node will return immediately with a status code of success or running when one of its children returns success or running (see Figure I and the pseudocode below). The children are ticked in order of importance, from left to right.

In pseudocode, the algorithm for a fallback composition is:

1 for i from 1 to n do
2     childstatus ← Tick(child(i))
3     if childstatus = running
4         return running
5     else if childstatus = success
6         return success
7 end
8 return failure

Sequence node

Figure II. Graphical representation of a sequence composition of N tasks.

Sequence nodes are used to find and execute the first child that has not yet succeeded. A sequence node will return immediately with a status code of failure or running when one of its children returns failure or running (see Figure II and the pseudocode below). The children are ticked in order, from left to right.

In pseudocode, the algorithm for a sequence composition is:

1 for i from 1 to n do
2     childstatus ← Tick(child(i))
3     if childstatus = running
4         return running
5     else if childstatus = failure
6         return failure
7 end
8 return success

Mathematical state space definition

In order to apply control theory tools to the analysis of behavior trees, they can be defined as three-tuple.[17]

${\displaystyle T_{i}=\{f_{i},r_{i},\Delta t\},}$

where ${\displaystyle i\in \mathbb {N} }$ is the index of the tree, ${\displaystyle f_{i}:\mathbb {R} ^{n}\rightarrow \mathbb {R} ^{n}}$ is a vector field representing the right hand side of an ordinary difference equation, ${\displaystyle \Delta t}$ is a time step and ${\displaystyle r_{i}:\mathbb {R} ^{n}\rightarrow \{R_{i},S_{i},F_{i}\}}$ is the return status, that can be equal to either Running ${\displaystyle R_{i}}$, Success ${\displaystyle S_{i}}$, or Failure ${\displaystyle F_{i}}$.

Note: A task is a degenerate behavior tree with no parent and no child.

Behavior tree execution

The execution of a behavior tree is described by the following standard ordinary difference equations:

${\displaystyle x_{k+t}(t_{k+1})=f_{i}(x_{k}(t_{k}))}$

${\displaystyle t_{k+1}=t_{k}+\Delta t}$

where ${\displaystyle k\in \mathbb {N} }$ represent the discrete time, and ${\displaystyle x\in \mathbb {R} ^{n}}$ is the state space of the system modelled by the behavior tree.

Sequence composition

Two behavior trees ${\displaystyle T_{i}}$ and ${\displaystyle T_{j}}$ can be composed into a more complex behavior tree ${\displaystyle T_{0}}$ using a Sequence operator.

${\displaystyle T_{0}={\mbox{sequence}}(T_{i},T_{j}).}$

Then return status ${\displaystyle r_{0}}$ and the vector field ${\displaystyle f_{0}}$ associated with ${\displaystyle T_{0}}$ are defined (for ${\displaystyle {\mathcal {S}}_{1}}$) as follows:

${\displaystyle r_{0}(x_{k})={\begin{cases}r_{j}(x_{k})&{\text{ if }}x_{k}\in {\mathcal {S}}_{1}\\r_{i}(x_{k})&{\text{ otherwise }}.\end{cases}}}$

${\displaystyle f_{0}(x_{k})={\begin{cases}f_{j}(x_{k})&{\text{ if }}x_{k}\in {\mathcal {S}}_{1}\\f_{i}(x_{k})&{\text{ otherwise }}.\end{cases}}}$

See also

• Decision tree
• Hybrid system
• Subsumption architecture

References

1. Colledanchise, Michele; Ögren, Petter (2017). "How Behavior Trees Modularize Hybrid Control Systems and Generalize Sequential Behavior Compositions, the Subsumption Architecture, and Decision Trees". IEEE Transactions on Robotics. 33 (2): 372–389. doi:10.1109/TRO.2016.2633567. S2CID 9518238.
2. Colledanchise, Michele; Ögren, Petter (2018). Behavior Trees in Robotics and AI: An Introduction. CRC Press. arXiv:1709.00084. doi:10.1201/9780429489105. ISBN 978-1-138-59373-2. S2CID 27470659.
3. Isla, D. (2005). "Handling complexity in the Halo 2 AI". Game Developers Conference (Vol. 12).
4. Isla, D. (2008). Halo 3-building a better battle. Game Developers Conference 2008.
5. Agis, Ramiro A.; Gottifredi, Sebastian; García, Alejandro J. (2020). "An event-driven behavior trees extension to facilitate non-player multi-agent coordination in video games" (PDF). Expert Systems with Applications. 155 (1): 113457. doi:10.1016/j.eswa.2020.113457.
6. Lim, C. U.; Baumgarten, R.; Colton, S. (2010). "Evolving behaviour trees for the commercial game DEFCON" (PDF). Applications of Evolutionary Computation. Berlin: Springer. pp. 100–110. doi:10.1007/978-3-642-12239-2_11. ISBN 978-3-642-12238-5.
7. Ögren, Petter (2012). "Increasing Modularity of UAV Control Systems using Computer Game Behavior Trees" (PDF). AIAA Guidance, Navigation and Control Conference, Minneapolis, Minnesota. pp. 13–16.
8. Colledanchise, Michele; Marzinotto, Alejandro; Ögren, Petter (2014). "Performance Analysis of Stochastic BTs" (PDF). Robotics and Automation (ICRA), 2014 IEEE International Conference on. doi:10.1109/ICRA.2014.6907328. S2CID 14719083.
9. Marzinotto, Alejandro; Colledanchise, Michele; Smith, Christian; Ögren, Petter (2014). "Towards a Unified BTs Framework for Robot Control" (PDF). Robotics and Automation (ICRA), 2014 IEEE International Conference on.
10. Klöckner, Andreas. "Interfacing BTs with the World Using Description Logic." In AIAA Guidance, Navigation and Control Conference, Boston, MA. 2013.
11. Klöckner, Andreas (2013). "Behavior Trees for UAV Mission Management". GI-Jahrestagung. pp. 57–68.
12. Bagnell, J. Andrew; Cavalcanti, Felipe; Cui, Lei; et al. (2012). "An integrated system for autonomous robotics manipulation" (PDF). Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE. pp. 2955–2962. doi:10.1109/IROS.2012.6385888. hdl:20.500.11937/14608. ISBN 978-1-4673-1736-8. S2CID 419179.
13. Millington; Funge (2009). Artificial Intelligence for Games. CRC Press. ISBN 978-0-12-374731-0.
14. Rabin, S. (2014). Game AI Pro. CRC Press. ISBN 978-1-4665-6596-8.
15. Champandard, Alex J.; Dunstan, Philip (2012). "The Behavior Tree Starter Kit" (PDF). Game AI Pro: Collected Wisdom of Game AI Professionals. pp. 72–92.
16. craft ai (2015). "BT 101 – Behavior Trees grammar basics".
17. Colledanchise, Michele; Ögren, Petter (2014). "How Behavior Trees Modularize Robustness and Safety in Hybrid Systems" (PDF). In Intelligent Robots and Systems (IROS), 2014 IEEE/RSJ International Conference on. IEEE.

External links

• ROS behavior tree library
• Unreal Engine 4 behavior tree documentation
• Behavior trees for AI: How they work
• Behavior Trees: Simple yet Powerful AI for your Robot