Understanding the overall patterns of spatial variation in species diversity and how these patterns are affected by species distribution is a key issue in species diversity studies, which facilitates biodiversity conservation and management. Alpha (α), beta (β), and gamma (γ) species richness and the Shannon diversity index of Quercus wutaishanica communities were analyzed by additive partitioning across four different nested spatial scales, i.e., α1 scale (within subplots), β1 scale (among subplots), β2 scale (among plots), and β3 scale (among sites), in the mid Loess Plateau, northwestern China. We also investigated the relative importance of rare (< 5% of total frequency) and common species (> 25% of total frequency) as contributors to overall plant species diversity by comparing the degree of species diversity variation before and after the removal of these species. The results showed that: (1) species diversity of all three layers (i.e., herb, shrub, and tree) displayed similar patterns at all spatial scales. Species richness increased with increasing spatial scales, and β richness at the site scale contributed the most (β3 > 50%); (2) although the Shannon diversity displayed an opposite trend to the species richness patterns and decreased with increasing spatial scales, the greatest contribution to γ diversity was from α diversity at the finest scale (α1 > 45%); (3) the effects of common species on species richness decreased with the increasing spatial scales, and mainly acted at both α1 and β1 scales. The decrease of species richness caused by the removal of rare species increased with increasing spatial scales, and the main effects acted at both the β2 and β3 scales; (4) Removal of rare species had no obvious effect upon Shannon diversity at all scales, and the curves before and after removal were almost identical. By contrast, removal of common species influenced Shannon diversity considerably at all scales. After the removal of the common species, the Shannon diversity of the herb layer increased at all spatial scales, whereas the shrub and tree layers decreased at the α1 scale, but increased at the other scales. Our results indicated significant scale-dependent contributions of rare and common species to species diversity in plant communities. Rare species had a higher contribution to the overall richness patterns at the broad scales (β2 and β3). Meanwhile, common species drive both species richness at finer scales (α1 and β1) and species evenness patterns at broad scales (β2 and β3). Our study revealed an important role of spatial scales in biodiversity conservation and management. Furthermore, additive partitioning is a simple and effective tool to unravel changes and sources in diversity over spatial scales in plant communities.