Semantic surgical scene segmentation is crucial for accurately identifying and delineating different tissue types during surgery, enhancing
outcomes and reducing complications. Hyperspectral imaging provides detailed information beyond visible color filters, offering an enhanced
view of tissue characteristics. Combined with machine learning, it supports critical tumor resection decisions. Traditional augmentations fail
to effectively train machine learning models on illumination and sensor sensitivity variations. Learning to handle these variations is crucial
to enable models to better generalize, ultimately enhancing their reliability in deployment. In this paper we introduce Calibration-Jitter, a
spectral augmentation technique that leverages hyperspectral calibration variations to improve predictive performance. Evaluated on scene
segmentation on a neurosurgical dataset, Calibration-Jitter achieved a F1-score of 74.35% with SegFormer, surpassing the previous best of
70.2%. This advancement addresses limitations of traditional augmentations, improving hyperspectral imaging segmentation performance.