1

References from Wikipedia:

Q-learning can be used to create new creative solutions, combining different behaviors, characteristics, reactions, facts, qualities, advantages, of materials, substances, etc., and using these as actions in the action space, and thus being able to find a solution to meet a given need or specific characteristic, such as finding possibilities for new recipes, combining ingredients so the cake has flavors that the user likes most, or based on physical or chemical reactions and interactions, creating materials that have certain characteristics, such as better resistance the given situation?

In summary, can Q-learning be used for these types of more complex problems, which involve finding new creative solutions, involving combinations of various actions, and thus being able to create new products, materials, etc.? Is this something possible for Q-learning?

will The J
  • 267
  • 6

1 Answers1

2

Your imagination about Q-learning even including possibly all RL methods to learn really creative solutions in the sense of AGI seems too far-fetched, as intuitively this needs additional higher level abstract reasoning and inferencing capabilities and production expert systems at least which Q-learning cannot model and it cannot even deal with non-Markov processes or time series in theory.

Additionally, even you can use neural nets to approximately deal with large (infinite) state and action spaces, reward can be very hard and tricky for hard problem requiring creative solutions where the reward may be subjective and different for each agent, such as the case to find a creative solution to unify quantum mechanics and general relativity in theoretical physics where some scientists favor explanatory power of real hidden variables, some favor the reward of instrumental predictive power and falsifiability, while others many favor completely different reward of the unified grand theory.

Finally Q-learning implicitly assumes a stationary environment where the transition dynamics and reward distributions don't change over time, however, in creative tasks the definition of a good solution might evolve such as some social application thus adapting to dynamic environments becomes essential while current standard Q-learning cannot handle.

cinch
  • 2,082
  • 1
  • 4
  • 9
  • I have one more question about this: What if I use a very simple creative problem, where the value of the rewards would not change, and instead of using standard Q-learning, I assume that the states are infinite, with a quantity of actions that I knew could be combined to reach the final objective. And, if in this scenario, I know how to correctly define the appropriate rewards and punishments for the problem in question. Could this algorithm be able to find a combination of actions that reach the final goal? And in your opinion, what could be the biggest challenge in tasks of this type? – will The J Nov 17 '23 at 11:10
  • 1
    It'll become much more suitable for RL then, especially with policy gradient methods instead of Q-learning as you cannot store an infinite Q-table for your potentially infinite states. If the existent creative policy is extremely nonlinear then you'd need deep RL to estimate value functions in its advantage to update the policy. And in difficult creative situations like Chess or Go, additionally you need some heuristic search like MCTS to speed up the learning process. These are the main tricks currently known. – cinch Nov 17 '23 at 17:33
  • thanks for the explanations! – will The J Nov 17 '23 at 22:16
  • Please accept any answer you feel satisfied for many of your recent batch of questions if possible, thanks! – cinch Nov 17 '23 at 22:36
  • Of course, I'll do it! – will The J Nov 17 '23 at 22:40