MLG 029 Reinforcement Learning Intro

Notes and resources: ocdevel.com/mlg/29

Try a walking desk to stay healthy while you study or work!

Reinforcement Learning (RL) is a fundamental component of artificial intelligence, different from purely being AI itself. It is considered a key aspect of AI due to its ability to learn through interactions with the environment using a system of rewards and punishments.

Links:

• openai/baselines
• reinforceio/tensorforce
• NervanaSystems/coach
• rll/rllab
• Differential Computers

Concepts and Definitions

• Reinforcement Learning (RL):
  • RL is a framework where an "agent" learns by interacting with its environment and receiving feedback in the form of rewards or punishments.
  • It is part of the broader machine learning category, which includes supervised and unsupervised learning.
  • Unlike supervised learning, where a model learns from labeled data, RL focuses on decision-making and goal achievement.

Comparison with Other Learning Types

• Supervised Learning:
  • Involves a teacher-student paradigm where models are trained on labeled data.
  • Common in applications like image recognition and language processing.
• Unsupervised Learning:
  • Not commonly used in practical applications according to the experience shared in the episode.
• Reinforcement Learning vs. Supervised Learning:
  • RL allows agents to learn independently through interaction, unlike supervised learning where training occurs with labeled data.

Applications of Reinforcement Learning

• Games and Simulations:
  • Deep reinforcement learning is used in games like Go (AlphaGo) and video games, where the environment and possible rewards or penalties are predefined.
• Robotics and Autonomous Systems:
  • Examples include robotics (e.g., Boston Dynamics mules) and autonomous vehicles that learn to navigate and make decisions in real-world environments.
• Finance and Trading:
  • Utilized for modeling trading strategies that aim to optimize financial returns over time, although breakthrough performance in trading isn't yet evidenced.

RL Frameworks and Environments

• Framework Examples:
  • OpenAI Baselines, TensorForce, and Intel's Coach, each with different capabilities and company backing for development.
• Environments:
  • OpenAI's Gym is a suite of environments used for training RL agents.

Future Aspects and Developments

• Model-based vs. Model-free RL:
  • Model-based RL involves planning and knowledge of the world dynamics, while model-free is about reaction and immediate responses.
• Remaining Challenges:
  • Current hurdles in AI include reasoning, knowledge representation, and memory, where efforts are ongoing in institutions like Google DeepMind for further advancement.