Climate change is projected to exacerbate food insecurity in sub-Saharan Africa (SSA) by reducing crop yields and soil fertility. Many climate change impact studies in SSA have overlooked long-term effects of soil fertility on crop yield. We evaluated maize yields under different scenarios of soil fertility (using soil organic carbon as a proxy) and climate change (considering changes in temperature, rainfall, and CO2) at four sites in SSA. Using an ensemble of 15 calibrated soil-crop models, we found a strong consensus that, without fertilization, soil fertility declines over time, impacting maize yields more strongly than changes in temperature, rainfall, or CO2. The model ensemble indicated that when accounting for soil fertility changes, the yield benefits of combined application of organic and mineral inputs increase over time, even under climate change. These findings highlight the importance of considering long-term change in soil fertility when assessing impacts of climate change and integrated nutrient management on crop production in SSA.