The topological semimetal β-Ag2Se features a Kramers-Weyl node at the origin in momentum space and a quadruplet of spinless Weyl nodes, which are annihilated by spin-orbit coupling. We show that single-crystalline β-Ag2Se manifests giant Shubnikov-de Haas oscillations in the longitudinal magnetoresistance, which stem from a small electron pocket that can be driven beyond the quantum limit by a field less than 9 T. This small electron pocket is a remainder of the spin-orbit annihilated Weyl nodes and thus encloses a Berry-phase structure. Moreover, we observed a negative longitudinal magnetoresistance when the magnetic field is beyond the quantum limit. Our experimental findings are complemented by thorough theoretical band-structure analyses of this Kramers-Weyl semimetal candidate, including first-principles calculations and an effective k·p model.