This paper proposes a novel nested-loop optimization approach which utilizes the maximum tolerable model accuracy loss as a hyperparameter to improve DNNs compression for hardware accelerators. This process includes local inner-loop optimization and global outer-loop optimization with bottom-up feedback. Our multi-level approach encompasses optimization tasks distributed across different computational spaces, such as software and hardware (High-Level Synthesis, HLS). As an example of an optimization task, we introduce and detail the mixed-precision Quantization Heuristic Search (QHS), which adjusts numerical representations, reducing hardware complexity while maintaining accuracy within user-defined tolerances. This approach offers a new perspective for model compression, leading to efficient and effective DNN hardware accelerators.