Thin overlays have become a primary pavement maintenance measure in recent years. Research on thin overlays primarily focuses on optimizing material composition and applying cost-effective overlay types. The influence of commonly used raw materials on the service performance of thin overlays remains insufficiently understood. Therefore, this study compared the service performance of thin overlays prepared with coarse aggregates (basalt, limestone, steel slag, and finely separated reclaimed asphalt pavement (RAP)). Specifically, two typical gradations, stone matrix asphalt (SMA-10) and asphalt concrete (AC-10C), were first selected. Then, the 2.36–4.75 mm, 4.75–9.5 mm, and 9.5–13.2 mm fractions were completely replaced with the four target coarse aggregates on an equal-volume basis. Finally, the service performance of the resulting specimens was tested and evaluated. The results showed that steel slag exhibited significant advantages in high-temperature stability, shear resistance, and skid resistance. It was identified as the optimal coarse aggregate. The SMA-10 overlay demonstrated superior overall performance compared to the AC-10C, and should be preferred when the budget allows. Finely separated RAP can be used in thin overlays, but it exhibited the poorest overall performance, with its freeze–thaw resistance approaching the lower specification limit. This indicates a need for further field validation.