Reinforcement Learning
I want to go through the Wikipedia series on Machine Learning and Data mining. Data mining is the process of extracting and discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems.
References
Related
- Intelligent Agent
- In intelligence and artificial intelligence, an intelligent agent (IA) is an agent that perceives its environment, takes actions autonomously in order to achieve goals, and may improve its performance with learning or acquiring knowledge.
- Q-learning
- Q-learning is a model-free reinforcement learning algorithm that teaks an agent to assign values to each action it might take, conditioned on the agent being in a particular state. It does not require a model of the environment, and it can handle problems with stochastic transitions and rewards without requiring adaptations.
- Exploration-exploitation Dilemma
- The exploration-exploitation dilemma, also known as the explore-exploit takeoff, is a fundamental concept in decision making that arises in many domains. It is depicted as the balancing act between two opposing strategies.
- Markov Decision Process
- Also called a stochastic dynamic program or stochastic control problem, is a model for sequential making when outcomes are uncertain.
- Dynamic Programming
- Dynamic programming is both a mathematical optimization method and an algorithmic paradigm. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields.
Notes
Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent should takes actions in a dynamic environment in order to maximize a reward signal. Reinforcement learning is one of the three basic machine learning paradigms, alongside supervised learning and unsupervised learning.
Q-learning at its simplest stores data i tables. This approach becomes infeasible as the number of states/actions increases, as the probability of the agent visiting a particular state and performing a particular action diminishes.
The focus of reinforcement learning is to find the balance between exploration (or uncharted territory) and exploitation (or current knowledge) with the goal of maximizing the cumulative reward (the feedback of which might be incomplete or delayed). The search for this balance if known as the exploration-exploitation dilemma.
The environment is typically stated in the form of a Markov Decision process (MDP), as many reinforcement leaning algorithms use dynamic programming techniques. The main difference between classical dynamic programming methods and reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the Markov decision process, and they target large MDPS where exact methods become infeasible.
Basic reinforcement learning is modeled as a Markov Decision process:
- A set of environment and agent states (the state space),
- A set of actions (the action space), , of the agent
- , the transition probability (at time ) from state to state under action
- , the immediate reward after transition from to under action
The purpose of reinforcement learning is for the agent to learn an optimal (or near-optimal) policy that maximizes the reward function or other user-provided reinforcement signal that accumulates from immediate rewards. This is similar to processes that appear to occur in animal psychology.
The goal of reinforcement learning is to learn a policy:
that maximized the cumulative reward. Formulating the problem as a Markov decision process assumes the agent directly observes the current environmental state; in this case, the problem is said to have full observability. If the agent only has access to a subset of states, or if the observed states are corrupted by noise, the agent is said to have partial observability, and formally the problem must be formulated as a partially observable Markov decision process. When the agent's performance is compares that of an agent that acts optimally, the difference in performance yields the notion of regret. In order to act near optimally, the agent must reason about long-term consequences of its actions, although immediate reward associated with this might be negative. Thus, reinforcement learning is particularly well-suited to problems that include a long-term versus short-term reward trade-off.
Comments
You have to be logged in to add a comment
User Comments
There are currently no comments for this article.