Biological nitrogen fixation is a fundamental part of ecosystem functioning. Anthropogenic nitrogen deposition and climate change may, however, limit the competitive advantage of nitrogen-fixing plants, leading to reduced relative diversity of nitrogen-fixing plants. Yet, assessments of changes of nitrogen-fixing plant long-term community diversity are rare. Here, we examine temporal trends in the diversity of nitrogen-fixing plants and their relationships with anthropogenic nitrogen deposition while accounting for changes in temperature and aridity. We used forest-floor vegetation resurveys of temperate forests in Europe and the United States spanning multiple decades. Nitrogen-fixer richness declined as nitrogen deposition increased over time but did not respond to changes in climate. Phylogenetic diversity also declined, as distinct lineages of N-fixers were lost between surveys, but the “winners” and “losers” among nitrogen-fixing lineages varied among study sites, suggesting that losses are context dependent. Anthropogenic nitrogen deposition reduces nitrogen-fixing plant diversity in ways that may strongly affect natural nitrogen fixation.