The influence of grazing activities on the population of small rodents is a hot issue in grassland ecosystem research. The determination of trophic niche relationship between sheep and small rodents plays an important role on the management of grassland ecosystems. In present study, microscopic analysis of feces was applied to analyze the diet composition of Brandt's voles and sheep, whose fecal samples were collected from the Animal Ecology Research Station located in the Maodeng pasture (Xilinhot, Inner Mongolia, China) from June to September, 2012. The changes in diet selection and trophic niches of the two herbivores would be determined to reveal the patterns by which the population of Brandt's vole was affected by grazing activities, and the relationship of competition and coexistence between these two herbivores from the perspective of food utilization. Our results showed that a total of 23 plant species were found in the fence, with Cleistogenes squarrosa, Stipa krylovii, Leymus chinensis, and Dysphania aristata being the dominant plants. Brandt's vole and sheep fed on 10 and 8 plant species, respectively. Gramineous plants were the main forage plants for both animals. Brandt's vole and sheep had their favorite plant species which varied in different seasons; except for that in July, there was a significantly positive correlation between dietary diversity of animals and plant diversity within the fences, suggesting an effect of changes in available plant resources on animals' diet selections. The trophic niche overlap between Brandt's vole and sheep was higher than 0.9 except for that in August (0.691), indicating that there was fierce food competition between the two. In addition, the grazing activities significantly reduced the aboveground biomass of plants from the Gramineae family, leading to a decrease in the quantity of food for Brandt's vole; and grazing activities significantly increased the number of plant species favored by Brandt's voles and the breadth of its trophic niche (except for September), leading to a decrease in the quality of food for Brandt's vole. These results indicated that high trophic niche overlap could cause fierce food competition between animals; grazing could affect the diet selection of Brandt's voles by influencing changes of plant resources in the fences, and grazing activities may have a negative effect on the population of Brandt's vole through reducing the quality and quantity of its food. These findings suggest that moderate grazing activities not only increase the utilization of pastures but also reduce the population of small rodents and their damage.