In probability theory and machine learning, the multi-armed bandit problem is a problem in which a decision maker iteratively selects one of multiple fixed choices when the properties of each choice are only partially known at the time of allocation, and may become better understood as time passes. A fundamental aspect of bandit problems is that choosing an arm does not affect the properties of the arm or other arms.

Michael N. Katehakis is a Professor of Management Science at Rutgers University. He is noted for his work in Markov decision process, Gittins index, the multi-armed bandit, Markov chains and other related fields.