Bandit optimization is a real-time decision-making strategy, named after the "multi-armed bandit" problem of choosing which slot machine to play when each has an unknown payout rate, that balances trying new options against sticking with options already known to perform well. At each decision point, a system has to weigh exploring an option it hasn't tried enough to be confident about against exploiting the option it currently believes is best, all while operating with limited and incomplete information.
This exploration-exploitation tradeoff shows up throughout applied machine learning: recommendation systems use bandit algorithms to decide whether to show a user a familiar type of content or test something new, and online advertising platforms use them to allocate impressions across ad variants while still learning which one converts best. Bandit methods are especially useful in settings where waiting to collect a full, complete dataset before acting isn't practical, since they update and improve decisions continuously as new outcomes come in.
