The plasmonic nanolens was proposed as a deterministic method to achieve high field enhancements and hence enable single molecule photonic devices, but experimental results have failed to live up to these expectations, and recent theoretical works have brought its long-assumed advantages into doubt. To explore the limits of cascade field enhancements we consider possible quantum solutions ('going small'), and using phononic materials at longer wavelengths ('going large'). We find that entering the quantum plasmonic limit, to enhance the size ratio between constituent nanoparticles, is not a fruitful strategy as the increased electron-surface scattering decreases the field enhancements by over an order of magnitude. Using larger nanoparticles is limited in metals by retardation but using localised surface phonon polaritons, which can be excited in polar dielectrics, is an effective strategy due to the lower energy phonon frequency and high quality factor. We compare the nanolens against the more usual dimer configuration and find that the superior geometry depends crucially on the material used, with noble metal nanolenses unlikely to offer better performance to equivalent dimers. In contrast, SiC nanolenses can offer a larger maximum field enhancement, up to 104, compared to the corresponding dimer configuration, suggesting that future endeavours in constructing nanolenses should be based on polar dielectrics. This could have wide-ranging implications for IR/THz surface-assisted spectroscopies.