The Loess Plateau is a vast potential organic carbon pool and is also an ecologically fragile region, exhibiting sensitivity to climate change. Various types of droughts, stemming from the imbalance between water supply and demand, will have a profound and far-reaching impact on the steady enhancement of ecosystem carbon sinks. However, currently, research on the impacts of various types of droughts on the annual gross primary productivity (AGPP) in the Loess Plateau is still in its infancy. In this study, AGPP, soil moisture (SM), and climatic factors were used to analyze the temporal dynamics of AGPP. Additionally, the interannual variations of AGPP in relation to SM, aridity index (AI), and vapor pressure deficit (VPD) were explored for the period from 2004 to 2020 in the Loess Plateau. The results demonstrate that over the past 20 years, the overall growth rate of AGPP has reached 38.30%, and the AGPP in the major vegetation restoration area has increased by 56.45%. The temporal trends of AI and SM exhibited a remarkable inclination to ascend, whereas the temporal trends of VPD did not display any significant directional alterations. The temporal correlation pattern between AGPP and SM, AI, as well as VPD suggested that soil drought preponderantly governed the interannual variation of AGPP, while the influence of atmospheric drought remained relatively inconspicuous in the Loess Plateau. The research outcomes will supply a theoretical and practical foundation for the construction of the regional ecological environment and the enhancement of the ecological carbon sink.