The present study investigated two Gramineae species growing in both rhizosheath and non-rhizosheath soils on the southern margin of the Junggar Basin. The Biolog-ECO microplate method was used to study the microbial community functional diversity of these species. The results showed that the organic matter, total nitrogen, available nitrogen, and available potassium in the rhizosheath of the two plant species were higher than in the non-rhizosheath soil. The average well color development (AWCD), Shannon diversity index, evenness index, Simpson index, and richness index were all higher in the rhizosheath soil than in the non-rhizosheath soil; the main carbon sources of microbes in the rhizosheath of the two plant species were carboxylic acids, polymers, and carbohydrates. The main carbon sources of microbes in the rhizosheath of Achnatherum splendens (Trin.) Nevski were carboxylic acids, carbohydrates, polymers, and amino acids. The main carbon sources of microbes in the rhizosheath of Stipagrostis pennata were carbohydrates, amino acids, and polymers. Significant correlations with the AWCD were found for the Shannon diversity index, Evenness index, available potassium, and total nitrogen. There were positive correlations with AWCD for all physical-chemical soil properties, except for total potassium. Overall, the rhizosheath enhanced the ability to alleviate negative outside influences by improving microorganism survival and increasing the microbial community functional diversity.