Background The association between sarcopenia and cardiometabolic index (CMI) among the Chinese elderly adults remains unclear. The present study aimed to examine the association based on a national survey. Methods Data from China Health and Retirement Longitudinal Study (CHARLS, wave 2011 and 2015) were employed. All subjects met the inclusion criteria were classified by the median of CMI. Four logistic regression models were established to explore the cross-sectional and longitudinal associations between CMI and sarcopenia. Subgroups analysis, tendency analysis, etc. were conducted to testify the robustness of the association. The non-linear relationship was explored using restricted cubic spline (RCS). Results 5,016 participants were enrolled in this study. The ORs of the longitudinal analysis in the models were 0.42(0.34,0.53), 0.37(0.29,0.46), 0.40(0.31,0.51) and 0.42(0.33,0.53)), respectively. Similar results were also found after propensity score matching (PSM), inverse probability weighting (IPTW) and cross-sectional analysis. Moreover, a non-linear relationship was found between CMI and sarcopenia in the total participants (both p for overall and p for non-linear < 0.001). Conclusion A U-shaped relationship between CMI and sarcopenia risk was found in the middle aged and elderly Chinese, with both deficient and excessive CMI levels showing detrimental effects. Maintaining a moderate CMI significantly diminishes the risk of sarcopenia.