For questions related to the concept of Markov decision process (MDP), which is a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision-maker. The concept of MDP is useful for studying optimization problems solved via dynamic programming and reinforcement learning.
Questions tagged [markov-decision-process]
201 questions
2
votes
1 answer
Remove already reached targets from the system to enable reaching other targets?
This may be a very fundamental question, but somehow I can't decide.
I have a graph and the user can take several actions while traversing it and there are multiple points with rewards. When I execute the MDP process, it ends up finding the first…
Ferda-Ozdemir-Sonmez
- 133
- 5
2
votes
1 answer
Markov Decision Processes with variable epoch lengths
I am working on modeling a transportation problem as an MDP. Multiple trucks move material from one node to various other nodes in a network. However, the time it takes a truck to travel between any 2 nodes is different based on distance, and…
Katie
- 21
- 2
1
vote
1 answer
When is it non-Markovian?
Several months ago, I was writing for class. I claimed an environment was non-Markovian because it would take several states to de-alias some positions in the grid world. I was corrected that it was n-step Markovian. Something that didn't occur…
foreverska
- 724
- 1
- 16
0
votes
0 answers
Does fixing an action in the state transition function of an MDP yield a transition matrix?
I stumbled across this seemingly elementary question while studying the theory of Markov Decision Processes. Suppose $\mathcal{M} = (\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R})$ is an MDP and fix an action $a \in \mathcal{A}$. Does $(i, j)…
Othman El Hammouchi
- 101
- 1