Finite-state approximations to discounted and average cost constrained Markov decision processes

Name: Finite-state approximations to discounted and average cost constrained Markov decision processes
Author: Saldı, Naci

İsim	Finite-state approximations to discounted and average cost constrained Markov decision processes
Yazar	Saldı, Naci
Basım Tarihi:	2019-07
Basım Yeri	- IEEE
Konu	Constrained Markov decision processes (MDPs), Finite-state approximation, Quantization, Stochastic control
Tür	Süreli Yayın
Dil	İngilizce
Dijital	Evet
Yazma	Hayır
Kütüphane:	Özyeğin Üniversitesi
Demirbaş Numarası	0018-9286
Kayıt Numarası	ce710a75-3604-4c68-89a6-a5373c1ef504
Lokasyon	Natural and Mathematical Sciences
Tarih	2019-07
Örnek Metin	In this paper, we consider the finite-state approximation of a discrete-time constrained Markov decision process (MDP) under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted cost problem, we prove the asymptotic convergence of the optimal value of the finite-state model to the optimal value of the original model. With further continuity condition on the transition probability, we also establish a method to compute approximately optimal policies. For the average cost, instead of using the finite-state linear programming approximation method, we use the original problem definition to establish the finite-state asymptotic approximation of the constrained problem and compute approximately optimal policies. Under Lipschitz-type regularity conditions on the components of the MDP, we also obtain explicit rate of convergence bounds quantifying how the approximation improves as the size of the approximating finite-state space increases.
DOI	10.1109/TAC.2018.2890756
Cilt	64

Kaynağa git Özyeğin Üniversitesi

Aramaya Dön

Özyeğin Üniversitesi

Kaynağa git

Finite-state approximations to discounted and average cost constrained Markov decision processes

Yazar Saldı, Naci

Basım Tarihi 2019-07

Basım Yeri - IEEE

Konu Constrained Markov decision processes (MDPs), Finite-state approximation, Quantization, Stochastic control

Tür Süreli Yayın

Dil İngilizce

Dijital Evet

Yazma Hayır

Kütüphane Özyeğin Üniversitesi

Demirbaş Numarası 0018-9286

Kayıt Numarası ce710a75-3604-4c68-89a6-a5373c1ef504

Lokasyon Natural and Mathematical Sciences

Tarih 2019-07

Örnek Metin In this paper, we consider the finite-state approximation of a discrete-time constrained Markov decision process (MDP) under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted cost problem, we prove the asymptotic convergence of the optimal value of the finite-state model to the optimal value of the original model. With further continuity condition on the transition probability, we also establish a method to compute approximately optimal policies. For the average cost, instead of using the finite-state linear programming approximation method, we use the original problem definition to establish the finite-state asymptotic approximation of the constrained problem and compute approximately optimal policies. Under Lipschitz-type regularity conditions on the components of the MDP, we also obtain explicit rate of convergence bounds quantifying how the approximation improves as the size of the approximating finite-state space increases.

DOI 10.1109/TAC.2018.2890756

Cilt 64