The construction mechanisms of plant communities are a hot topic in the field of ecology. However, the question of how communities are constructed has not yet been adequately explained. Whether these mechanisms are driven by niche theory, neutral theory, or a combination of the two has been confirmed in a variety of research studies; most of them concentrated on the study of tropical rain forests. So we looket at grassland communities on China's Loess Plateau as example, hoping to provide some grounds for answering this question in this paper. Three different grass habitats were selected: land which had been abandoned for five years, north-facing slopes and south-facing slopes. The Mantel test analysis was applied to test the correlation between similarity in community composition, geographic distance and environment distance. The result of our research suggested that geographic distance has a significant effect on the community composition (P<0.01); there is a correlation between environment distance and community composition (P<0.01), but the function of environment distance is not as good as geographic distance. In order to further explore the influence, PCNM analysis was used to test for fitness: the percentage of community assembly can be explained in terms of the geographic distance and environmental distance. The results indicate the distribution of plants are highly variable and more strongly related to geographical distance rather than to environmental differences. Together, geographical distance and environmental distance explained 79.3% of the variation in floristic similarity. Pure geographic distance or pure environmental distance alone explained 33.8% and 14.2% of the variation in floristic similarity, respectively. The neutral theory plays a strong role in the species assemblages within ecological communities, but the function of the niche can't be ignored.