Terrestrial ecosystems are playing important roles in global carbon cycling. However, the information is still limited with regard to the semi-arid sandland or desert area, compared with the thorough studies on forest and grassland. We here estimated the biomass carbon storage, net primary production (NPP) and rain use efficiency (RUE) of Hunshandake Sandland, a semi-arid sandy region in Inner Mongolia covered with vegetation of Siberian elm (Ulmus pumila L.) sparse forest grassland. Five main habitats, i.e. fixed dunes, semi-fixed dunes, shifting dunes, lowland, and wetland, were compared to analyze the patterns of carbon storage and NPP distribution. The average biomass (9.19 Mg C ha^-1) and NPP (4.79 Mg C ha^-1 yr^-1) of the sparse forest grassland were respectively 82% and 54% higher than the mean level of the surrounding temperate grassland. Governed by the same climate, sparse forest grassland ecosystem had RUE almost twice that of surrounding grassland. The ratio of below to aboveground biomass was 3.5: 1 in the sandland, indicating that most of the vegetational carbon was stored in belowground pool. Although trees were functionally critical in maintaining the integrity of sparse forest grassland, they accounted for only 10.6% and 1.2% of the biomass and NPP, respectively. The sparse forest grassland in Hunshandake Sandland should be recognized as a temperate savanna ecosystem which is distinctively different from typical temperate grassland in the same region as evidenced by the higher NPP and vegetation carbon storage. Well designed management and restoration efforts can potentially sustain ecosystem services in both forage production and carbon sequestration.