The inversion of track irregularity using vehicle vibration responses serves as a vital method for track condition detection and a key part in achieving intelligent operation and maintenance of railway vehicles. To this end, this paper takes a railway vehicle operating at 160 km/h as an example, establishing three dynamic models for the vehicle system: lateral, vertical, and lateral-vertical coupled. The track irregularity inversion equation is derived within the state-space framework of the vehicle system. An inversion process for track irregularity based on the classical Kalman filter (KF) and adaptive Kalman filter (AKF) algorithms is presented. Finally, a detailed investigation is conducted into the influence patterns of the Kalman filter algorithms, vehicle dynamics models,and observation schemes on the inversion results of lateral and vertical track irregularities. The results indicate: Compared to single lateral or vertical models, the lateral-vertical coupled model yields the best inversion results under the KF algorithm, demonstrating its superior capability in simulating the lateral and vertical motion behaviors of the vehicle. The AKF algorithm performs better in single lateral and vertical models but fails to leverage its adaptive parameter-tuning advantages in the lateral-vertical coupled model. This suggests that adaptive strategies may not guarantee convergence to optimal solutions for complex high-dimensional coupled models, whereas simpler low-dimensional models benefit more from adaptive inversion. The observation scheme significantly impacts track irregularity inversion results. Specifically, relying solely on vibration acceleration measurements proves inadequate for effective inversion. Therefore, effective vibration responses shall be supplemented in combination with the actual situation.and observation schemes on the inversion results of lateral and vertical track irregularities. The results indicate: Compared to single lateral or vertical models, the lateral-vertical coupled model yields the best inversion results under the KF algorithm, demonstrating its superior capability in simulating the lateral and vertical motion behaviors of the vehicle. The AKF algorithm performs better in single lateral and vertical models but fails to leverage its adaptive parameter-tuning advantages in the lateral-vertical coupled model. This suggests that adaptive strategies may not guarantee convergence to optimal solutions for complex high-dimensional coupled models, whereas simpler low-dimensional models benefit more from adaptive inversion. The observation scheme significantly impacts track irregularity inversion results. Specifically, relying solely on vibration acceleration measurements proves inadequate for effective inversion. Therefore, effective vibration responses shall be supplemented in combination with the actual situation.