Rare species are an important group for species coexistence and diversity maintenance in marine ecosystems, but the regulatory mechanisms of rare species on species diversity and functional diversity of fish communities are still unclear. Taking the fish community in Sanmen Bay as the research object, this study identified rare species in the community based on geographical range, habitat specificity, and population size, and used self-sampling technology to simulate the random loss of rare species under natural conditions, thereby exploring the impact of rare species loss on species diversity and functional diversity. The results indicated that the loss of rare species has led to a significant decrease in species richness and functional richness; the species evenness had lower stability in summer and higher stability in other seasons. The functional evenness had higher stability in winter and lower stability in other seasons, which depended on whether the species or functional traits occupied a special position in the geographical range or trait space; the Simpson index and Shannon-Wiener index showed a downward trend, but the former had a smaller range of numerical changes than the latter, which was closely related to community richness and evenness. The functional divergence showed a decreasing trend in summer and remained relatively stable in the other three seasons, which was related to the number of species located at the edge of the trait space. Overall, the loss of rare species had an impact on the stability of species diversity and functional diversity, and the correlation between species diversity and functional diversity was strong.