Alfalfa (Medicago sativa L.) is an important artificial herb species planted widely in the Loess Plateau during the ‘Grain to Green’ large-scale revegetation program that been in operation since 1999. It plays an important ecological role in controlling soil erosion, increasing plant carbon sequestration, and improving ecological conditions. However, the growth of alfalfa grassland is unsustainable and shows signs of growth degradation after about 6 years in the Loess Plateau because of its high growth rate and deep root systems. The current understanding of the physiological mechanisms of the growth degradation of alfalfa grassland in the Loess Plateau is still limited. In this study, we collected samples of the main roots of alfalfa and a natural control, the herbaceous species Chinese cinquefoil (Potentilla chinensis) along a precipitation gradient (280 mm, 400 mm, 500 mm, and 550 mm) in the Loess Plateau, then analyzed the variations in radial incremental growth (growth rings) and xylem size, as well as the climatic factors that correlate with these characteristics, of the main roots of the two perennial forb species. The results indicated that the annual ring widths of alfalfa and Chinese cinquefoil both decreased noticeably with increasing age, and that the growth reduction was more pronounced at sites with lower precipitation. This shows that the perennial forbs in the Loess Plateau were confronted with deteriorating environmental conditions with increasing age, and the recent trend of a warmer and drier climate is further aggravating water deficits for these forb species. The vessel sizes of plants in the alfalfa grasslands along the precipitation gradient all declined monotonically with increasing age, and the vessel reduction was more obvious at sites with lower precipitation. With the gradually intensifying water deficits with increasing age, the vessel diameter of alfalfa decreased significantly and could not satisfy the self-growing water demand, and serious growth degradation had occurred in recent years. If the alfalfa grassland was located at a dry site with lower precipitation, the time of growth degradation occurred earlier. In contrast, the vessel size of the main roots of Chinese cinquefoil kept consistently increasing with increasing age, and the amplitude of the increase in vessel size at the relatively dry site was comparatively large. This indicated that Chinese cinquefoil could enlarge its vessel size to enhance water-use efficiency, then satisfy its increasing water demand with increasing age. The concurrently accelerated water acquisition ability of Chinese cinquefoil with increasing age also indicated that it can adapt to the semi-arid climatic conditions of the Loess Plateau, and maintain a state of sustainable growth development.