Stigma and judgment by health workers toward people living with HIV (PLHIV) and key populations can undermine the uptake of HIV services. In 2014, we recruited health workers delivering HIV services from 21 urban communities in South Africa and Zambia participating in the first year of the HPTN 071 (PopART) cluster-randomized trial. We analyzed self-reported levels of stigma and judgment toward (1) PLHIV, (2) women who sell sex, (3) men who have sex with men (MSM), and (4) young women who become pregnant before marriage. Using logistic regression, we compared responses between three health worker cadres and explored risk factors for stigmatizing attitudes. Highest levels of stigma and judgment were in relation to women who sell sex and MSM, especially in Zambia. Heath workers did not generally think that clients should be denied services, although this was reported slightly more commonly by community health workers. Higher education levels were associated with lower judgmental beliefs, whereas higher perceptions of coworker stigmatizing behaviors toward PLHIV and each key population were associated with holding judgmental beliefs. Training experience was not associated with judgmental attitudes for any of the key populations. Our findings confirm a high prevalence of judgmental attitudes toward key population groups but lower levels in relation to PLHIV, among all cadres of health workers in both countries. Planning and implementing targeted stigma reduction interventions within health settings are critical to meet the needs of vulnerable populations that face more stigmatizing attitudes from health workers.