I Taught a Stick-Figure NPC to Dodge Bullets With a Neural Network I Wrote From Scratch

Nicely constructed NPCs in games often rely on artificial intelligence to make decisions. However, this AI is pre-programmed and does not learn or adapt on its own. The author wanted to create an NPC that could learn and improve its abilities through experience, without any predetermined programming. To achieve this, the author built two NPCs using reinforcement learning.

The first NPC is a stick figure that learns to dodge incoming bullets in a game environment. The second NPC is a combat-ready character that learns to duel a player. Both NPCs are developed from scratch using a neural network and a reward system. Reinforcement learning (RL) is a method of training an agent when the correct answer is unknown.

Instead of receiving pre-labeled data, the agent is given a reward for desired outcomes. The NPC receives input about its surroundings (state), makes a choice (action), and receives feedback (reward). The goal is to maximize the total reward over time by learning the best actions to take in different situations. The neural network used for the bullet-dodging NPC consists of two layers: four inputs representing the position and velocity of the nearest bullet, and four outputs estimating the value of moving in each direction.

The network learns by updating its weights based on the difference between predicted and actual rewards. By constantly learning from its experiences, the NPC can improve its performance over time. The article provides details on how these NPCs are implemented using JavaScript, including the neural network structure, reward system, and the reinforcement learning algorithm.