The remnant of karst mountainous cities is a vital hub for biodiversity resources and a crucial zone for its conservation and utilization. Understanding the plant diversity characteristics of remnant mountains of different scales holds immense significance for the ecological restoration and protection of plant diversity in these karst mountainous cities. This study focused on remnant mountains of varying sizes in the central urban area of Guiyang, a karst mountainous city, as the research objects, and analyzed the characteristics of plant diversity in urban remnant mountains as well as the relationship between plant diversity and mountain scale using one-way analysis of variance, the least significant difference method (LSD), and univariate linear regression. The results showed that: (1) a total of 342 plant species belonging to 102 families and 257 genera were recorded across the nine mountains studied. Specifically, 255, 107, and 171 plant species were found in large, medium, and small mountains, respectively. Within these mountains, the α diversity of plants revealed that the herb layer and the shrub layer exhibited greater diversity compared to the tree layer. (2) The α and β diversity indices of plant communities were highest in the large mountain and lowest in the medium mountain, indicating that the scale of remnant mountains has a notable impact on plant diversity. (3) The α diversity index of plant communities in the remnant mountain was positively correlated with the mountain area. However, when the remnant mountain area was less than 10 hm2, the α diversity index of mountain plant communities was negatively correlated with the mountain area. Thus, in the context of urban construction and biodiversity protection in karst mountainous cities, it is imperative to minimize the fragmentation and encroachment of large remnant mountains, while also strengthening the protection of plant diversity in existing small mountains. The findings of this study could provide theoretical support and a scientific foundation for the conservation and planning of plant diversity in mountainous cities.