The effect of vegetation on the processes of runoff and sediment production on slopes varies with vegetation types and their vertical structural components. However, there are few quantitative analysis on how these factors affect runoff and sediment yield processes on the slope. In this study, based on field runoff plots and artificial simulated rainfall experiments, the effects of three different typical shrubs (Hippophae rhamnoides, Caragana korshinskii, and Medicago sativa) and their different vertical structural components (leaf, stem, litter, and root) on runoff and sediment yield processes in the loess hilly region were studied. The experiment was carried out in Anjiagou catchment of Dingxi, Gansu Province and the runoff and sediment reduction of these three shrubs were apparent. Compared with the bare slope, runoff and sediment reduction ratios of three shrub and grass vegetations were from 32.49%-44.86% and from 72.99%-80.63%, and the flow rate reduction was about 29.17%-45.83%, respectively, indicating the better sediment reduction effects. Amongst three shrub and grass vegetations, M. sativa has the best flow reduction benefit, which was about 44.86%, and C. korshinskii has the best sediment reduction benefit, which was 80.63%. The runoff reduction benefits of the three types of vegetation differed are significantly at different runoff generation periods, and gradually decreased in the early, middle and late runoff periods; while the sediment reduction benefits had no obvious changes in different runoff periods. The relative contribution of different components of the vegetation vertical structure to the benefits of flow reduction, and sediment reduction was closely related to its morphological characteristics and its spatial distribution. The contribution rate of the above-ground part to the flow reduction benefit and deceleration benefit was 75.42% and 68.38% on average, respectively, while the relative contributions of different vegetation stems, leaves and litter have some differences. Compared with the Aboveground part of vegetation, the root system played a greater role in reducing sediment, with a relative contribution of 78.44%. The more complete the vertical structure of vegetation, the more significant the benefits of vegetation reduction and sediment reduction. This paper provides an important scientific basis and theoretical guidance for soil and water conservation, plant restoration and construction in the loess hilly areas.