SARS-CoV-2 variants have emerged that escape neutralization and potentially impact vaccine efficacy. T cell responses play a role in protection from reinfection and severe disease, but the potential for spike mutations to affect T cell immunity is incompletely understood. We assessed neutralizing antibody and T cell responses in 44 South African COVID-19 patients infected either with the Beta variant (dominant from November 2020 to May 2021) or infected prior to its emergence (first wave, Wuhan strain), to provide an overall measure of immune evasion. We show that robust spike-specific CD4 and CD8 T cell responses were detectable in Beta-infected patients, similar to first wave patients. Using peptides spanning the Beta-mutated regions, we identified CD4 T cell responses targeting the wild type peptides in 12/22 first wave patients, all of whom failed to recognize corresponding Beta-mutated peptides. However, responses to mutated regions formed only a small proportion (15.7%) of the overall CD4 response, and few patients (3/44) mounted CD8 responses that targeted the mutated regions. Among the spike epitopes tested, we identified three epitopes containing the D215, L18, or D80 residues that were specifically recognized by CD4 T cells, and their mutated versions were associated with a loss of response. This study shows that in spite of loss of recognition of immunogenic CD4 epitopes, CD4 and CD8 T cell responses to Beta are preserved overall. These observations may explain why several vaccines have retained the ability to protect against severe COVID-19 even with substantial loss of neutralizing antibody activity against Beta.