A search for resonances in the mass range 12–70 GeV produced in association with a b quark jet and a second jet, and decaying to a muon pair, is reported. The analysis is based on data from proton-proton collisions at center-of-mass energies of 8 and 13 TeV, collected with the CMS detector at the LHC and corresponding to integrated luminosities of 19.7 and 35.9 fb−1, respectively. The search is carried out in two mutually exclusive event categories. Events in the first category are required to have a b quark jet in the central region (|η| ≤ 2.4) and at least one jet in the forward region (|η| > 2.4). Events in the second category are required to have two jets in the central region, at least one of which is identified as a b quark jet, no jets in the forward region, and low missing transverse momentum. An excess of events above the background near a dimuon mass of 28 GeV is observed in the 8 TeV data, corresponding to local significances of 4.2 and 2.9 standard deviations for the first and second event categories, respectively. A similar analysis conducted with the 13 TeV data results in a mild excess over the background in the first event category corresponding to a local significance of 2.0 standard deviations, while the second category results in a 1.4 standard deviation deficit. The fiducial cross section measurements and 95% confidence level upper limits on those for a resonance consistent with the 8 TeV excess are provided at both collision energies.