The strong Nernst effect observed in the pseudogap state of the cuprate superconductors has been attributed to the motion of vortices in the presence of an applied temperature gradient. However, previous analytical treatments of the effect have not related it directly to the dynamics of vortices. Here we present a self contained analytical study of diamagnetism and transport in a film with superconducting phase fluctuations, formulated in terms of vortex dynamics within the Debye-H\"uckle approximation. We find that the diamagnetic and Nernst signals decay strongly with temperature in a manner which is dictated by the vortex core energy. We use the theory to interpret measurements of underdoped cuprates and show that the core energy scales roughly with the critical temperature. The onset of the Nernst effect lies in a regime which can not be described in terms of a liquid of well defined vortices and involves substantial amplitude fluctuations.