We report contributions to cosmic infrared background (CIB) intensities originating from known galaxies and their faint companions at submillimeter wavelengths. Using the publicly available UltraVISTA catalog and maps at 250, 350, and 500 μm from the Herschel Multi-tiered Extragalactic Survey, we perform a novel measurement that exploits the fact that uncataloged sources may bias stacked flux densities—particularly if the resolution of the image is poor—and intentionally smooth the images before stacking and summing intensities. By smoothing the maps we are capturing the contribution of faint (undetected in ${K}_{S}\sim 23.4$) sources that are physically associated, or correlated, with the detected sources. We find that the cumulative CIB increases with increased smoothing, reaching 9.82 ± 0.78, 5.77 ± 0.43 and $2.32\pm 0.19\;{\mathrm{nWm}}^{-2}\;{\mathrm{sr}}^{-1}$ at 250, 350, and 500 μm at $300\;\mathrm{arcsec}$ FWHM. This corresponds to a fraction of the fiducial CIB of 0.94 ± 0.23, 1.07 ± 0.31, and 0.97 ± 0.26 at 250, 350, and 500 μm, where the uncertainties are dominated by those of the absolute CIB. We then propose, with a simple model combining parametric descriptions for stacked flux densities and stellar mass functions, that emission from galaxies with log($M/{M}_{\odot })\gt 8.5$ can account for most of the measured total intensities and argue against contributions from extended, diffuse emission. Finally, we discuss prospects for future survey instruments to improve the estimates of the absolute CIB levels, and observe any potentially remaining emission at $z\gt 4$.