As pivotal components of belowground communities, soil fungi and bacteria are important indices for evaluating the ecological benefits of the Grain for Green program in China. In this study, we used a high-throughput sequencing approach to compare the succession of soil fungal and bacterial communities in a typical chronosequence of abandoned agricultural lands in Yunnan Province, southwest China. The results showed that bacterial diversity decreased significantly and then increased gradually after the soil was abandoned, while fungal diversity showed no significant change. However, when plant communities changed from herb to woodland via the shrub stage during reforestation, the proportion of Sordariomycetes decreased significantly from 30% to approximately 10% and the proportion of Agaricomycetes increased from 5% to more than 20%; there was no obvious change in the bacterial community composition along the chronosequence. Clustering analysis showed that the successional pattern of the fungi community was more consistent with plant communities than bacterial along successional stages. Both fungi and bacteria communities in undisturbed natural forest soil differed significantly from those in other stages, indicating that anthropogenic disturbance of soil microbial communities has long-term effects. These results revealed the succession characteristics of soil fungal and bacterial communities during reforestation in typical abandoned soil in Yunnan Province and provide essential data support for the comprehensive evaluation of the ecological benefits of the Grain for Green project in Southwest China.