It is facing severe livelihood and ecological conflicts in the key ecological functional areas. Scientific assessment of the relative poverty level and accurate identification of the relatively poor population are crucial to promoting the coordinated development of livelihood and ecology in the key ecological functional areas. In view of this, based on the framework of sustainable livelihood analysis, this paper uses farmer survey data to systematically identify relative poor households by constructing the relative poverty measurement index system, and uses hierarchical linear model to explore the key influencing factors of relative poverty at the "family-village" dual scale. The results show that :(1) the relative poverty degree of farmers in longnan mountainous area is differentiated, among which, the relative poverty degree of farmers in high mountainous area, subsidized farmers and farmers with high family dependency are higher. Relatively poor households accounted for 20.39% of the surveyed households, and were mainly concentrated in high mountainous areas, households with low educational level, high family support and subsidized households. (2) The household attributes of farmers are the decisive factors leading to relative poverty, the endogenous motivation of poverty alleviation and the improvement of livelihood diversity are the key factors alleviating relative poverty, while the increase of medical burden and education burden will lead to relative poverty of farmers. (3) Village characteristics magnify the impact of family characteristics on the relative poverty degree, and the natural conditions, social and economic conditions and external contacts of villages strengthen the impact on the relative poverty degree of farmers through influencing family characteristics.