The high concentration of arsenic (As) in rice grains, in a large proportion of the rice growing areas, is a critical issue. This study explores the feasibility of conventional (QTL-based) marker-assisted selection and genomic selection to improve the ability of rice to prevent As uptake and accumulation in the edible grains. A japonica diversity panel (RP) of 228 accessions phenotyped for As concentration in the flag leaf (FL-As) and in the dehulled grain (CG-As), and genotyped at 22,370 SNP loci, was used to map QTLs by association analysis (GWAS) and to train genomic prediction models. Similar phenotypic and genotypic data from 95 advanced breeding lines (VP) with japonica genetic backgrounds, was used to validate related QTLs mapped in the RP through GWAS and to evaluate the predictive ability of across populations (RP-VP) genomic estimate of breeding value (GEBV) for As exclusion. Several QTLs for FL-As and CG-As with a low-medium individual effect were detected in the RP, of which some colocalized with known QTLs and candidate genes. However, less than 10% of those QTLs could be validated in the VP without loosening colocalization parameters. Conversely, the average predictive ability of across populations GEBV was rather high, 0.43 for FL-As and 0.48 for CG-As, ensuring genetic gains per time unit close to phenotypic selection. The implications of the limited robustness of the GWAS results and the rather high predictive ability of genomic prediction are discussed for breeding rice for significantly low arsenic uptake and accumulation in the edible grains.