While nitrogen doping has been investigated extensively in silicon, there is only limited information on its interaction with vacancies in germanium, despite most point defect processes in germanium being vacancy controlled. Thus, spin polarized density functional theory calculations are used to examine the association of nitrogen with lattice vacancies in germanium and for comparison in silicon. The results demonstrate significant charge transfer to nitrogen from nearest neighbour Ge and strong N-Ge bond formation. The presence of vacancies results in a change in nitrogen coordination (from tetrahedral to trigonal planar) though the total charge transfer to N is maintained. A variety of different nitrogen vacancy clusters are considered all of which demonstrated strong binding energies. Substitutional nitrogen remains an effective trap for vacancies even if it has already trapped one vacancy.