Grassland ecosystem is one of the most important terrestrial ecosystems in the world which may impose an impact upon global environmental change. The dynamics of a grassland ecosystem can be affected by disturbance in terms of promoting community succession and changing community structure. Among the disturbing factors, grazing is considered one of the key factors which affect both above-and below-ground processes in the ecosystem. One of the most prominent consequences of overgrazing is the transformation of grassland to shrub-land. In order to protect grassland ecosystem and restore vegetation productivity, a number of measures have been put into practice, among which fencing an area to exclude livestock is often considered to be a simple but effective way. The aim of the present study was to explore the response of the Caragana microphylla-dominated community to fencing and to predict when the degraded community would restore to its normal status after grazing exclusion. The study site was located in Xilingol degraded grassland of Inner Mongolia, China. Long-time grazing resulted in the expansion of C. microphylla populations in this site, and we set different treatments since 2003. Animals were excluded by fencing in the plot of grazing exclusion (EG), while the grazing intensity was controlled at 3 sheep/hm² in the plot of grazing (G). The two plots were adjacent and had similar environment until 2003. We investigated the growth and physiological traits of C. microphylla and the basic characteristics of the herbaceous communities, and compared the temporal changes in plot EG after 5-8 years of fencing and the spatial differences between these two plots after eight years of different utilization regimes. We analyzed the response of the traits of C. microphylla and the composition of herbaceous community to grazing and fencing, and also the relationship between the cover of C. microphylla and the ‘Importance Value’ of different functional groups. (1) The coverage of C. microphylla decreased in the plot EG during the four years while the morphological traits showed no significant variation, indicating that C. microphylla population declined after grazing exclusion. The decline of C. microphylla reflected a decrease of population coverage instead of the growth status of individual modules. Moreover, leaf nitrogen and phosphorus content, especially the latter, decreased after grazing exclusion, which might be associated with the decline of C. microphylla population. (2) Herbaceous community composition showed significant change after eight years of grazing exclusion. Annuals disappeared from the community, while the Important Value of perennial bunch grasses increased significantly in the plot EG both in temporal and spatial comparison. The change of community structure indicated that grazing exclusion had a positive effect on the restoration of C. microphylla-dominated grassland. (3) The coverage of C. microphylla decreased with the increasing Importance Value of perennial bunch grasses, indicating that there existed a competition between C. microphylla and the perennial bunch grasses. The decline of C. microphylla population might be caused by the expansion of well-grown perennial bunch grasses. It is suggested that grazing exclusion impose an effect on the restoration of C. microphylla-dominated community, but re-establishment of the typical steppe with climax community needs at least eight years; in other words, the restoration of C. microphylla-dominated grassland to its original status is a long-term process.