Understanding the contributions of rare and common species to plant diversity in forest communities helps clarify species coexistence mechanisms and provides a foundation for forest conservation and management. However, the contributions of rare and common species to species and phylogenetic diversity in forest communities are still not well understood. The purpose of this study is to analyze the impact of rare and common species on taxonomic diversity and species-area relationships in a karst primary forest and a secondary forest, and to investigate the relative contributions of rare and common species to species and phylogenetic diversity, as well as their relationship with the spatial scale of the plots. We established forest plots of 2-ha each in both primary and secondary evergreen-deciduous broad-leaved mixed forests in Guizhou Province. We analyzed the contributions of rare and common woody plants to the species and phylogenetic diversity patterns at three spatial scales (10 m × 10 m, 20 m × 20 m, and 50 m × 50 m) within these plots. Conclusions are as follows: (1) The woody plant richness (278 species) and the number of rare species (182 species) in primary forests were significantly higher than those in secondary forests (66 species and 32 species), and the species composition differed markedly between the two forests. (2) Species-area relationship analysis showed that, in primary forests, the trend of species increase was similar whether rare species or common species were removed, with an initial sharp increase followed by stabilization. In secondary forests, the removal of rare species also led to an initial rapid increase in species number, which then leveled off, but the removal of common species did not result in a significant increase in species number with increasing sampling area. (3) The removal of common species had a significant impact on species and phylogenetic diversity in both primary and secondary forests across all three spatial scales. However, the removal of rare species primarily affected species and phylogenetic diversity in primary forests at the 50 m × 50 m scale, while it had no significant impact on species and phylogenetic diversity in secondary forests at any of the three scales. Our findings demonstrate that rare species make significant contributions to species and phylogenetic diversity in high-species-richness forests primarily at larger spatial scales. In contrast, common species play important roles in shaping species and phylogenetic diversity across different spatial scales, regardless of the community’s species richness. Moreover, these findings enhance our ability to conserve biodiversity and manage karst forests sustainably.