In the limit of zero Reynolds number, it is well known that the hydrodynamic interaction between two rigid spheres in an unbounded fluid does not produce relative translation. We study the symmetry-based criteria which govern such an effective interaction for more general cases of two arbitrarily shaped objects and confined geometries. We show that the breaking of inversion symmetry by boundaries of the system accounts for the interactions between two spheres, as observed in experiments. We provide new predictions for interactions in other object configurations near obstacles. In addition, we examine the time-dependent relative translation of two self-aligning objects, where the interplay between the orientational interaction and the translational one, in most cases, leads over time to repulsion between the two objects.