2

AIXI is a theoretical formulation for artificial general intelligence. While AIXI is computationally intractable, there are approximations of AIXI like https://arxiv.org/abs/0909.0801 that are computable.

Some time ago, I stumbled upon theoretical artificial general intelligence, and I would like to know what are the practical differences between approximations of AIXI and deep reinforcement learning methods.

Petercommand Hsu
  • 21
  • 3
  • Can you explain what you mean by "practical differences"? AIXI is a theoretical framework, whereas deep learning is a practical framework without theoretical grounding. – Yuval Filmus Jul 12 '17 at 15:00
  • By practical differences, I want to know that when I am writing a computer program to play games (say, the go game), what are the considerations I have to make to choose between approximations of AIXI and deep reinforcement learning methods. I know that people rarely use AIXI-related methods, but I want to know if there are any circumstances where one might want to use AIXI approximations instead of deep reinforcement learning methods when actually implementing agent AIs. – Petercommand Hsu Jul 12 '17 at 15:18

2 Answers2

1

It's a bit of an apples and oranges comparison. AIXI is a proposal for how generalized learning should be structured, while neural networks are a specific learning implementation.

That being said, you probably wouldn't want to use an AIXI implementation for a known task such as game AI. Game AI is a specific learning task, so it's better and easier to use an AI specifically tailored to that task. The AlphaGo AI, for example, combined a deep learning neural network for finding interesting looking moves with a positional analysis engine for figuring out the strength of those moves. That sort of architecture works great for learning to play Go, but it's not general.

ckersch
  • 163
  • 5
0

Deep reinforcement learning is reinforcement learning where you use a deep neural network as the memory component (in RL terminology the Q-function). More details can e.g. be found here: https://www.intelnervana.com/demystifying-deep-reinforcement-learning/

Concerning RL and AIXA: "The general reinforcement learning problem is to construct an agent that, over time, collects as much reward as possible from an initially unknown environment. The AIXI agent (Hutter 2005) is a formal, mathematical solution to the general reinforcement learning problem." Source: https://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/viewFile/1722/2048

The problem is that AIXA is not computable so approximations of AIXA are necessary.

Putting it all together:
Deep reinforcement learning is a particular implementation of an AIXA approximation.

Personal note: I think it is no coincidence that the author of the abovementioned paper and several other papers in this area (Joel Veness) is now working for Google DeepMind...

vonjd
  • 141
  • 3