How should dynamic, interruptible multitasking structure look like?

Question

I am trying to make an AI that plays a game. Actually, I'm playing around with both StarCraft and MineCraft which are completely different kinds of games. As well, I use different programming languages for both, and one is multi-threaded (java), other event-driven single-threaded (Node.js).

Despite all those differences, I face the same problem: How can I make an AI that can handle tasks in sensible order, but also interrupt the tasks in cases of need and return to them later? How to make a sensible task chain (where one task causes another subtask)?

Examples of above can be:

• In minecraft, you're digging a block and you're attacked by a monster
• In starcraft you're leading an attack and a critical position in your base is attacked
• In minecraft, task of chopping trees is caused by the task building the house - the bot must remember why is he doing what is he doing.
• In starcraft, certain buildings are just built in order to unlock types of soldiers

I just was playing around with the idea, but I just kept writing and deleting completely stupid code...

This is what I have now, but it makes no sense. I can see many situations that will not be solved using this model.

/***
 *  Task pseudo class. After finishing the task, this task notifies the parent task.
 *  Arguments:
 *    action - function callback that will execute. The callback must call the finished() method in the end. The callback will receive
 *             this task's object as a first argument.  
 **/   
function Task(action, parent, name) {
  this.action = action;
  this.after = after;
  this.name = name||"Void task";
}

Task.prototype.start = function() {
  console.log("Task started...");
  try {
    this.action(this);
  }
  catch(e) {
    console.log("The task action has thrown an error: "+e);
  }
}
Task.prototype.finished = function() {
  console.log("Task over.");
}
//Override this
Task.prototype.pause = function() {};

I am afraid I have a lot of reading to do about this. But are there some general known ideas I can start with?

Answer 1

You are building a handcrafted AI Agent, often called an Intelligent Agent (IA); its 'environment' is the game, its 'sensors' are programmatic functions that gather input, its 'actuators' are also functions that output the keyboard/controls the game processes as input.

To the best of my knowledge, if you are writing a handcrafted AI, a state-machine (SM) is probably the best way to start; and will likely get you the biggest result for your efforts.

Handcrafting a SM would let you encode priorities into the states and/or state-transitions. A danger signal could cause a transition to a defense state, then when the danger has been dealt with, you can resume the previous state. Alternatively you could just transition to the next highest priority state. This could leave an objective unfinished, but this may be acceptable. For example it may not be efficient to path back to where you were if the dangerous encounter displaced your AI's position to another location; i.e. re-evaluating what state to transition based on the current context is probably the correct thing to do.

---

Handcrafting an Agent will only get you so far, in essence you will transferring your expertise and knowledge to the agent. Because of this you will only be able to transfer some fraction of your knowledge to the agent. This process will be difficult, lengthy and error-prone, and your agent will never play 'better' than you, since it only has some subset of your knowledge about the game.

The alternative to hand-crating an agent, is to build a Learned Agent, i.e. one based on Machine Learning.

A learning system needs an objective function, also called a loss function, which needs to encode many things:

• goals
• survival
• rewards
• penalties
• ...

The learning system will try to maximize the objective function; basically the objective function tells the agent how well it is doing.

There are many learning algorithms to choose from to run your learning system, but there are only a few categories:

• Supervised learning
• Unsupervised learning
• Semi-supervised learning

Some learning algorithms include:

• Reinforcement learning (Q learning)
• Neural Networks
• Support vector machines

These methods are not mutually exclusive, ex. one can implement reinforcement learning with a neural network. And these methods/algorithms need to be utilized/combined in some non-trivial way to create an agent. One might even want to combine a learned solution with a handcrafted one; i.e. use learned functions to improve and/or guide your handcrafted solution, for example using a classifier to help choose state transitions.

I included this section so you can have a starting point into researching a Learned solution to your problem.

Answer 2

Your event handlers should be fast, like starting one game action and registering and event handler to do the next thing when that action completes. No event handler should be waiting game time for a game action to complete. In MineCraft when you start mining a block, you ask for an event when the mining finishes, and you might get an event before then that a monster is attacking. In Star Craft you give a unit an order to move/attack/build something, and ask for an event when that task is complete. While waiting for that event to occur you might get a different event that your base is under attack, or another unit has completed its task and needs further orders.

By organizing your high level actions into really fast low level actions triggered by events, you allow events relevant to other high level actions to come in the middle.