Use of nutrients recycled from societal waste streams in agriculture is part of the circular economy, and in line with organic farming principles. Nevertheless, diverse contaminants in waste streams create doubts among organic farmers about potential risks for soil health. Here, we gather the current knowledge on contaminant levels in waste streams and recycled nutrient sources, and discuss associated risks. For potentially toxic elements (PTEs), the input of zinc (Zn) and copper (Cu) from mineral feed supplements remains of concern, while concentrations of PTEs in many waste streams have decreased substantially in Europe. The same applies to organic contaminants, although new chemical groups such as flame retardants are of emerging concern and globally contamination levels differ strongly. Compared to inorganic fertilizers, application of organic fertilizers derived from human or animal feces is associated with an increased risk for environmental dissemination of antibiotic resistance. The risk depends on the quality of the organic fertilizers, which varies between geographical regions, but farmland application of sewage sludge appears to be a safe practice as shown by some studies (e.g. from Sweden). Microplastic concentrations in agricultural soils show a wide spread and our understanding of its toxicity is limited, hampering a sound risk assessment. Methods for assessing public health risks for organic contaminants must include emerging contaminants and potential interactions of multiple compounds. Evidence from long-term field experiments suggests that soils may be more resilient and capable to degrade or stabilize pollutants than often assumed. In view of the need to source nutrients for expanding areas under organic farming, we discuss inputs originating from conventional farms vs. non-agricultural (i.e. societal) inputs. Closing nutrient cycles between agriculture and society is feasible in many cases, without being compromised by contaminants, and should be enhanced, aided by improved source control, waste treatment and sound risk assessments.