Soil erosion is one of the main challenges to sustainable development that human beings face. Vegetation reconstruction can play an important role in controlling soil erosion and reducing sediment yield. Given that soil and water resources are becoming increasingly scarce, optimizing the vegetation distribution pattern would undoubtedly be helpful in resolving such problems, as well as in environmental improvement of ecologically fragile regions such as the Loess Plateau. Hence, there is an urgent need to elucidate the effects of vegetation distribution pattern on soil erosion and sediment yield. Keeping this in mind, this study has summarized and reviewed the progress in research on both the slope and the watershed scale on the following two issues: the characterization of vegetation patterns in soil erosion and sediment yield research, and the response of soil erosion and sediment yield to the changes in vegetation patterns. The following problems were identified following the analysis: 1) vegetation pattern metrics that can characterize the effects of soil erosion and sediment yield are still lacking; 2) the coupled effects of vegetation patterns and landforms on soil erosion and sediment yield are rarely studied; and 3) the intuitive response of soil erosion and sediment yield to the vegetation pattern has not been established. In order to promote relevant research, studies on the combined effects of vegetation patterns and landforms on soil erosion and sediment yield should be emphasized in the future, and distributed physical models with high practical accuracy should be used to obtain data on soil erosion and sediment yield, which would reflect the impact of vegetation patterns. In addition, GIS spatial analysis technology should be adopted to design more typical vegetation patterns and study the corresponding soil erosion and sediment yield, which could enrich the basic information available and reduce the presence of not relevant or inaccurate information.