Studying the effects of site factors, soil nutrient factors and their interactions on the growth of average dominant trees, and construct site index models with site, soil nutrient and their interactions, respectively, can provide insights for site quality evaluation of Chinese fir plantation. Based on the analytical data of average dominant trees in 40 plantation plots of Chinese fir in Hunan Province, quantitative method I, random forest algorithm, K-means clustering and nonlinear mixed effects regression analysis were used to determine the site and soil nutrient factors that significantly affected the dominant height growth. The optimal basic model was screened and a nonlinear hybrid site index model containing site, soil nutrients and their interaction effects was constructed.Three evaluation indexes, AIC, BIC and R², were used to evaluate the fitting effect of the model and derive the optimal site index model. The results showed that altitude, slope, soil type, organic matter, total nitrogen and total potassium had significant effects on mean height of dominant trees. The significance order of site factors was altitude > slope > soil type, and the importance order of soil nutrient factors was organic matter > total nitrogen > total potassium. Among the seven models, the optimal basic model was Gompertz (R²=0.6876, MAE=6.6922, RMSE=2.7448). The accuracy of the mixed model with site, soil nutrients and their interaction type effects was improved to 0.7827, 0.7765 and 0.8400, respectively. Based on the standard clustering with accuracy ≥ 95%, the accuracy of the mixed model with site type group, soil nutrient type group and site-soil nutrient interaction type group was improved by 13.05%, 12.52% and 21.42%, respectively, compared with the basic model, and compared with the mixed model with all type effects, AIC and BIC were lower. Therefore, the site, soil nutrients and their interactions all have significant effects on the average dominant tree growth. The fitting effect of the mixed model with interaction effects is better than that of the single site and soil nutrient effects. The simulation of the growth law of the average dominant tree based on the multiform site index curve of the site-soil nutrient interaction type group is more accurate. Finally, the optimal site index model with the random effect of site-soil nutrient interaction type group was established H_jSSNMTG=a_jSSNMTG×exp(-b×exp(-c×T_jSSNMTG))+ε_jSSNMTG, which could be used for site quality evaluation of Chinese fir plantation under complex site conditions in Hunan Province.