The paper “Finding the K best policies in finite-horizon Markov decision processes” has been published as a research report at the Danish Informatics Network in the Agriculture Sciences (DINA), The Royal Veterinary and Agricultural University (see under publications).
Daniele Pretolani had the opportunity to come to Denmark for a month to visit my former supervisor Kim Allan Andersen and me. We are using the opportunity to write three companion papers from my thesis. One concerning finding the K shortest hyperpaths using reoptimization, one concerning finding the K best strategies in a stochastic time-dependent network and finally, a paper on bicriterion problems in stochastic time-dependent networks. The papers will be available under publication as soon as possible.
During my work at the Biometry Research Unit I had so discussions with Anders Ringgaard Kristensen about possible applications where directed hypergraphs can be used to model Markov decision processes. As a result he offered me a position as a assistent professor at the Department of Large Animal Sciences, Royal Veterinary and Agricultural University from 19. July to 30. September on a project concerning risk-aversion in forestry.
I am currently working on a project concerning optimal replacement strategies for sows with Erik Jørgensen. The problem is modelled using a multi-level hierarchic Markov decision process. The original model was developed by Anders Ringgaard Kristensen (KVL) whom I have had fruitful discussions with. I have realized that directed hypergraphs actually can be used to model finite-horizon Markov decision processes. They provide us with an efficient way of storing the process and by finding the shortest hyperpath we actually can fnd the optimal policy. Moreover, it should be possible the find the K best policies.