China is one of the countries with the largest distribution area, the most types and the most obvious vulnerability of ecologically fragile areas in the world. As the main human activity area in the ecologically fragile area, the rural settlements have become an important source of interference that leads to the deterioration of the ecological environment. In this study, taking Xichang City, Sichuan Province as an example, the habitat quality model, sediment delivery ratio model, and nutrient delivery ratio model were used to evaluate the impact of layout optimization of rural settlements on the regional habitat quality, soil output and N-P output. The results indicate that the average score of habitat quality increased by 22.4%, soil output decreased by 4.6%, and N-P output decreased by 13.1% and 8.2% after layout optimization of rural settlements in the Xichang City. Therefore, layout optimization of rural settlements is beneficial to the improvement of the ecosystem service function in the Xichang City. After the layout optimization of rural settlements, the reduction in the number and area of rural settlements reduced the probability of forest fires, land reclamation and other vegetation destruction events, which become the main reason for the improvement of habitat quality. The decline in the area of rural settlements and farmland, and the increase in the area of woodland are the main reasons for the decline in soil output and N-P output. The decrease in the number of rural settlements will gradually reduce the cost of waste recycling and sewage treatment in rural areas, thereby further reducing the output of N-P.