One of the proposed qubits for quantum information is the hyperfine ground state of a neutral atom or ion. If all Zeeman sublevels are used it is called a qudit. A possible method to perform quantum state tomography of such an atom is by measuring the polarization moments ρ^κ. However, a standard light absorption method for measuring the κ=0 moment (which is proportional to the hyperfine population) would be inaccurate in this case, since the absorption is sensitive to the hyperfine population distribution between the various Zeeman sublevels. A high fidelity measurement is therefore required. Here we demonstrate a method for precision probing of the κ=0 moment, using atom-light interaction in vapor. We analyze theoretically and experimentally the existence of a magic frequency for which the absorption of a linearly polarized light beam by vapor alkali-metal atoms is independent of the population distribution among the Zeeman sub-levels and the angle between the beam and a magnetic field. The phenomenon originates from a peculiar cancelation of the contributions of higher moments of the atomic density matrix, and is described using the Wigner-Eckart theorem and inherent properties of Clebsch-Gordan coefficients.