Soil water is an important element of "green water" that constitutes the physical linkage between vegetation and hydrological system. Water retention function represents the process and ability of retaining water in the ecosystem and is an essential ecosystem service in the mountainous ecosystems. Due to the soil heterogeneity, vegetation and topography, water retention function is variable in space and time. This study aimed to characterize the differences of water retention function of three typical vegetation types in the Pailugou catchment, Qilian Mountain, northwest China. The abilities of different vegetation types are evaluated by the water conservation function index, which portrayed as the integrated effect of soil properties, soil moisture, precipitation and temperature during the growing season. The results indicated that 1) the value of soil organic matter, sand and silt content, field capacity, saturation moisture capacity and porosity were considerably higher in the shrubland and Qinhai spruce (Picea crassifolia) forest than that in the grassland. By contrast, the value of the bulk density and clay content in the grassland was higher in comparison to that of shrubland and forestland; 2) The cumulative water infiltration of the root zone in the Qinhai spruce forest was higher than that in the shrubland and grassland, and the rate of soil water loss at the grassland reached the highest level among the different vegetation types; 3) Soil moisture in the Qinhai spruce forest and shrubland was higher than that in the grassland during the whole growing season, and the water conservation function index were mostly larger than 1. In conclusion, the Qinhai spruce forest had a higher soil water holding capacity in comparison to the shrubland and grassland. Thus, the results of this study can provide scientific guidelines for the management and sustainable development of the mountainous ecosystems in the semi-arid region of China.