Agricultural landscape is an important source of human welfare, and an important area for biodiversity conservation. The middle and lower reaches of the Yellow River is among the most important agricultural areas in China. With the continuing expansion of agricultural land and the intensification of land-use, biodiversity of agricultural landscapes in this region continues to decline. Therefore, in this area, it is necessary to understand the difference in plant community composition and species diversity among non-agricultural habitats, and the effect of agricultural landscape heterogeneity and community structure on plant species diversity at different scales in the agricultural landscape. To address the issues mentioned above, we set up experimental plots in a representative region in Gongyi City of Henan province. A total of 910 quadrats (including 210 in secondary shrub habitat, 60 in secondary forest, 295 in artificial forest, and 345 in farmland edge) were investigated by using the Braun-Blanquet method. We found that:(1) the species richness and diversity of secondary shrub habitats were significantly higher than those in the other non-agricultural habitats. The farmland edge has more species than we expected. (2) Beta diversity (β₂) was the main component of species richness in all habitats at the landscape scale. The contributions of alpha diversity at the quadrat and plot scale were small in all habitats, whereas that in two relatively more natural habitats (secondary shrub and secondary forest) were higher than that in relatively more artificial habitats (artificial forest and farmland edge). (3) The effect of landscape heterogeneity on species diversity was mainly evident at the quadrat scales (α_quadrat), whereas beta and gamma diversity are not very closely related to landscape heterogeneity indices. However, height and coverage of shrub and herb layers, which can represent the habitat characteristics, might provide better predictions of plant species diversity. Overall, dynamics of landscape pattern would affect the local ecological factors more directly, unlike the characteristic indices of common community, such as Shannon diversity, evenness, and Simpson index. In summary, it is believed that at small scales, such as quadrat and plot scales, the effect of local environmental factors on plant community would be stronger and more direct than that at the landscape scale. However, the changes of regional landscape pattern, which can be indicated by the changes of local ecological factors, would potentially affect the plant community. At large scales, the dynamics of regional landscape pattern affect plant species richness and diversity by influencing landscape composition and configuration, which can become evident as changes of beta diversity. In general, the present study was expected to provide a better understanding of the status and contributions of plant diversity of non-agricultural habitats in the study area, and was important for explaining the effects of landscape pattern dynamics on plant community and the underlying mechanisms. Besides, our findings might provide a theoretical basis for the conservation and maintenance of biodiversity and ecosystem services, and might help in developing a sustainable agro-landscape for the study area in the future.