Cosmic Bandits: Exploration vs. Exploitation in Cosmological Surveys

Various cosmological observations consist of prolonged integrations over small patches of sky. These include measurements of B-modes in the CMB, the power spectrum of 21-cm fluctuations during the epoch of reionization and deep-field imaging by telescopes such as HST/JWST, among others. However, since these measurements are often hindered by spatially-varying foreground noise, the observational sensitivity may be improved considerably by finding the region of sky cleanest of foregrounds. The best strategy thus involves a tradeoff between exploration (to find lower-foreground patches) and exploitation (through prolonged integration). But how to balance this tradeoff efficiently? This problem is akin to the multi-armed bandit (MAB) problem in probability theory, wherein a gambler faces a series of slot machines with unknown winning odds and must develop a strategy to maximize his/her winnings with some finite number of pulls. While the optimal MAB strategy remains to be determined, a number of machine-learning algorithms have been developed in an effort to maximize the winnings.Focusing on measurements of primordial gravitational waves, we will demonstrate how to construct adaptive survey strategies based on heuristic methods to solve the MAB problem. Other possible implementations and pertaining issues will also be discussed.