Barton

Project Specs:
Academic Project | Unreal 5.4.4 | 20-Person Interdisciplinary Team | ChatGPT 3.5

Overview

On this project I was a technical designer and AI programmer. I was in charge of the action planning system within this game which utilizes a Large Language Model (LLM). This included designing a way to get the AI to understand what actions the player wants Barton to take in the game world. Along with Barton completing the action that was asked.

My Design/prototyping process

First, I prototyped an extremely bare bones version of this system.
- It functioned by having a list of key words, parsing through what the player says, and if a word matched then Barton would take that action.
- This worked as a simple prototype, but did have a lot of issues due to the AI not understanding what actions Barton was taking in the game world so often times there was this disconnect with what Barton’s responses were.

Spacer

Second, I implemented a way for more AI to be included in this system.
- For example, after asking Barton if they can do this task, a prompt will be fed to the AI asking if the player wants Barton to take a specific task and will respond with yes or no.
- This worked only sometimes, because it was using the AI to answer the question it was very unpredictable and didnt always give the accurate response I was looking for, so I did not end up going this route.
  - Though, I am really happy that I tested with this system and had the experience programming with the AI and learning more about it.

Spacer

Third, I made a system that utilized parsing because that gave the most accurate results while notifying the AI in the background what actions Barton was taking so the AI better understood what was going on in the game world.
- I first made an interaction trigger that kept track of all the interactions and inside of it.
  - This allowed Barton to only interact with what was inside the trigger.
- If Barton was inside the AI and you asked him to complete a command, they would do so. While if Barton was outside they would say that interaction was too far away.
  - This was all done through parsing and notifing the AI what was going on when the player was asking Barton to do a task.

Reflection

This project was the first time I ever used Unreal! For a bit, it was difficult to try and get my tasks done while also learning the engine.
- Though, the more I worked on it and looked at resources online and talked to my other teammembers I was able to figure it out and got a lot better at understanding how Unreal works!
Trying to get the AI to understand what was going on in the game world was a really interesting technical design challenge!
- I went through a lot of iterations for the action planning system, that took a lot of time and testing to get it in a spot I was happy and proud of. 🙂