In the context of subsistence farming in Madagascar, upland rice producers have limited access to mineral fertilizers and yields remain very low. Genetic improvement of yield through the nitrogen use efficiency (NUE) component is an avenue to be explored in breeding programs. A recent GWAS study carried out on an upland rice diversity panel allowed to detect genomic regions involved in NUE variability, nevertheless these regions explained only a moderate part of the total genetic diversity observed in the panel. We investigated the potential of genomic prediction for NUE in order to optimize our upland rice breeding program for this trait. We evaluated the predictive ability of genomic prediction using two validation experiments. The first consisted of a standard cross-validation with 5-fold subdivision of the diversity panel (DP) and the second consisted of an independent experiment involving a breeding population (BP) derived from the DP. The DP was structured into five genetic clusters of different sizes and with some degree of admixture, while the BP was composed of two main clusters. The best prediction ability for NUE was obtained in crossvalidation within the DP. The predictive ability in the independent validation experiment was weak (r = 0.25), about three times less than those obtained in the cross-validation. The low kinship between DP and BP, different genetic structures and slightly different LD patterns probably explains the low predictive ability of across populations genomic prediction. Practical implications for our rice breeding program are discussed.