Human T cell lymphotropic virus-1 (HTLV-1) primarily infects CD4+ T cells, causing inflammatory disorders or a T cell malignancy in 5% to 10% of carriers. The cytotoxic T lymphocyte (CTL) response is a key factor that controls the viral load and thus the risk of disease. The ability to detect the viral protein Tax in primary cells has made it possible to estimate the rate at which Tax-expressing infected cells are eliminated by CTLs in persistently infected people. However, most HTLV-1-infected cells are Tax–at a given time, and their immunophenotype is poorly defined. Here, we aimed to identify a cell-surface molecule expressed by both Tax+ and Tax–HTLV-1-infected cells and use it to analyse the CTL response in fresh peripheral blood mononuclear cells. Cell adhesion molecule 1 (CADM1/TSLC1) was the best single marker of HTLV-1 infection, identifying HTLV-1-infected cells with greater sensitivity and specificity than CD25, CCR4 or ICAM-1. CADM1+CD4+ T cells carried a median of 65% of proviral copies in peripheral blood. In a cohort of 23 individuals, we quantified the rate of CTL-mediated killing of Tax+ and Tax−CADM1+ cells. We show that CADM1 expression is associated with enhanced susceptibility of infected cells to CTL lysis: despite the immunodominance of Tax in the CTL response, Tax+CADM1– cells were inefficiently lysed by CTLs. Upregulation of the CADM1 ligand CRTAM on CD8+ T cells correlated with efficient lysis of infected cells. Tax–CADM1+ cells were lysed at a very low rate by autologous CTLs, however, were efficiently killed when loaded with exogenous peptide antigen. High expression of CADM1 on most HTLV-1-infected cells in the face of enhanced CTL counterselection implies that CADM1 confers a strong benefit on the virus.