We show that the Fermi Bubbles and the Galactic haze are naturally interpreted as inverse-Compton and synchrotron emission from the same spectrum of cooling cosmic ray electrons. This avoids the ad-hoc spectral features and superfluous acceleration mechanisms typically assumed. The model implies that the bubbles originated 2-3 Myr ago, the mean magnetic field in the haze region is ~3\muG, and the radiation field in the bubble is dominated by starlight with mean energy density >2eV/cm^3. These parameters imply that the bubbles accelerate cosmic ray protons (iron nuclei) up to energies of ~5·10^{17}eV (~10^{19}eV). In contrast, a hadronic model fails to naturally account for the gamma-ray spectrum of the bubbles, in particular once the haze-emitting electrons are taken into account.