Markov Decision Theory
Project course, 4 hours per week in the first quarter. The second quarter is used to work on projects which are presented and discussed at the end of the quarter.
Lecturer
Søren Glud Johansen
Content
A short presentation of Renewal/Reward Processes and Markov Chains will form the first lectures of the course. Then follows a thorough presentation of the theory about Markov Decision Processes, including some applications. Further applications will be discussed, based on reports made by the students.
Prerequisites
Computer Science, Mathematical Programming and
Probability Theory 1 and
Probability theory 2.
It is acceptable to follow
Probability theory 2 simultaneously with
Markov decision theory Assessment
A written report which is also presented orally. This report can be accepted as a bachelor project and will then be assessed according to the 13 scale.
Literature
H.C. Tijms:
Stochastic Models: An Algorithmic Approach, John Wiley & Sons 1994 (Chapters 1, 2 and 3).
S.M. Ross:
Introduction to Stochastic Dynamic Programming, Academic Press, 1983 (Chapters 2, 3, 4 and 5).
Various articles and technical reports.
ECTS-credits
10.
Quarter
Spring 2004.