We investigate theoretically the dynamics of colonization of a territory by a population of individuals which exhibit random births and deaths. As a prototypical example we consider a variant of the Verhulst model, supplemented with an Allee effect and immigration. At the level of deterministic theory, the Allee effect introduces an unstable fixed point at an intermediate population size. At low immigration rate, the typical population size prior to colonization is small. This makes it impossible to use standard techniques, such as WKB approximation or Fokker-Planck equation. We circumvent this difficulty by matching a recursive solution of the Master equation at small population sizes with a standard boundary-layer analysis around the unstable fixed point. This procedure allows us to accurately calculate the long-lived quasi-stationary probability distribution of population sizes, and the mean time to colonization, for a wide range of the immigration rate. At sufficiently high immigration rate, our results coincide with WKB results obtained previously. At lower immigration rates we obtain new results, significantly different from previous ones.