To investigate the damage mechanism of the wide and narrow joints of CRTS Ⅱ slab ballastless tracks and identify damage, a field investigation was conducted on a passenger-dedicated line in the East China region, followed by long-term monitoring of track slab temperature and joint displacement. Based on the DBSCAN clustering algorithm, temperature data representing energy input and displacement data representing energy output during high-temperature seasons were screened and abnormal feature data were extracted. By incorporating the standard deviation, which represents data dispersion, to analyze the energy accumulation and sudden release process within the track slabs, a comprehensive computational model combining data-driven and physical mechanisms was established. A novel method for identifying wide and narrow joints damage based on abnormal feature extraction was proposed. The results show that, during the period with prolonged high temperature in summer, there is a long-term inconsistency between the temperature of the track slab and the displacement of the joints. The established physical model supplemented eight additional abnormal characteristic data for damage identification analysis of wide and narrow joints. In addition, prolonged high temperatures lead to energy accumulation and sudden release within the track slabs, with more pronounced energy changes observed at wide joint locations, thereby causing significant damage to the wide and narrow joints.