Part of Advances in Neural Information Processing Systems 13 (NIPS 2000)
Justin Boyan, Michael Littman
We describe an extension of the Markov decision process model in which a continuous time dimension is included in the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public trans(cid:173) portation and telescope observation scheduling.