The key ecological function area is the core area of global ecosystem protection and sustainable development of human beings. As the main body of ecological protection in key ecological functional areas, farmers' well-being is directly related to the realization of the main functions of key ecological functional areas. Scientifically assessing the relative poverty level of farmers in the key ecological functional areas, accurately identifying the relative poor population and revealing their influencing factors are crucial to promote the coordinated development of livelihoods and ecology in key ecological functional areas. Therefore, this paper takes Longnan Mountain Area, which is located in the important water conservation area and biodiversity ecological function area of Qinba Mountain, as the case area. Based on the sustainable livelihood analysis framework, this paper uses household survey data to systematically identify relatively poor farmers by building a relative poverty measurement index system, and uses a hierarchical linear model to explore the key influencing factors of relative poverty on the family-village dual scale. The results show that:(1) the relative poverty level of rural households in Longnan mountain area is different. Among which, the relative poverty level of rural households in mountain area, the subsidized rural households and high family support rural households is higher. Relatively poor farmers account for 20.39% of the surveyed households, and are mainly concentrated in alpine areas, high family support and subsidized farmers. (2) The household attribute of farmers is the decisive factor that causes farmers to fall into relative poverty. The endogenous driving force for poverty alleviation and the improvement of livelihood diversity are the key factors to alleviate relative poverty, while the increase of medical and educational burdens will cause farmers to fall into relative poverty. (3) Village characteristics magnify the impact of family characteristics on the relative poverty level, and village natural conditions, socio-economic conditions, and the external contacts, etc. strengthen the impact on the relative poverty level of farmers by affecting family characteristics.