Phase modulation is a useful property in designing optical systems. Diffractive optical elements, with their phase modulation characteristics, can effectively complement refractive systems to balance structural complexity and imaging performance, and are suitable for long-wave infrared systems under size constraints. This work presents a phase retrieval-based design method for binary diffractive surfaces in hybrid refractive-diffractive systems. The approach derives the required phase distribution from wavefront error calculations and ray propagation analysis, thereby avoiding local optima of the traditional optimization process and enabling direct mapping between design and fabrication. A practical hybrid system for airborne thermal radiation detection is designed, fabricated, and tested. Compared to a traditional refractive system (f/1.2), the proposed hybrid design achieves an f-number of 0.8 without increasing system complexity and enhances transmitted energy and detection range by 2.25 and 1.5 times, respectively. Experimental results validate the method’s effectiveness in improving system performance. Hopefully, the proposed framework can provide theoretical insight for the development of compact, high-performance infrared optical systems