The brain is the most metabolically active organ in the body, which increases its metabolic activity, and thus oxygen consumption, with increasing cognitive demand. This motivates us to question whether increased cognitive workload may be measurable through changes in blood oxygen saturation. To this end, we explore the feasibility of cognitive workload tracking based on in-ear SpO2 measurements, which are known to be both robust and exhibit minimal delay. We consider cognitive workload assessment based on an N-back task with randomised order. It is shown that the 2-back and 3-back tasks (high cognitive workload) yield either the lowest median absolute SpO2 or largest median decrease in SpO2 in all of the subjects, indicating a measurable and statistically significant decrease in blood oxygen in response to increased cognitive workload. This makes it possible to classify the four N-back task categories, over 5 second epochs, with a mean accuracy of 90.6%, using features derived from in-ear pulse oximetry, including SpO2, pulse rate and respiration rate. These findings suggest that in-ear SpO2 measurements provide sufficient information for the reliable classification of cognitive workload over short time windows, which promises a new avenue for real time cognitive workload tracking.