Using field data collected in six major temperate grasslands on the Inner Mongolian Plateau and the Tibetan Plateau (Qinghai and Tibet), we analyzed spatial variation in surface soil sulfur and associated environmental constraints. The concentration of sulfur in surface soil (i.e., 0-10 cm depth) was significantly higher in the Tibetan grasslands (430.8 mg/kg) than in the Inner Mongolian grasslands (181.4 mg/kg). The surface soil sulfur was mainly in organic form, and there was relatively little inorganic sulfur at the sampling sites that can be absorbed by plants. The percentages of inorganic sulfur at the sampling sites in the Tibetan and Inner Mongolian grasslands were 24.5% and 14.7%, respectively. The concentrations of surface soil sulfur were significantly positively related to surface soil carbon and nitrogen and negatively related to soil pH. The C/S and N/S ratios were low in all of the study sites, indicating that sulfur could be the major nutrient constraint in vegetation productivity in the temperate grasslands of China. In the Inner Mongolian grasslands, the concentrations of total sulfur, solvent sulfur, and adsorbed sulfur were significantly positively related to annual precipitation and negatively related to annual mean temperature. However, in the Tibetan grasslands, most sulfur types were not associated with climatic factors, except for solvent sulfur, which was significantly positively related to annual precipitation.