Since optical remote sensing images are useless in cloudy conditions, a possible alternative is to resort to synthetic aperture radar (SAR) images. However, many conventional techniques for Earth monitoring applications require specific spectral features which are defined only for multispectral data. For this reason, in this work we propose to estimate missing spectral features through data fusion and deep learning, exploiting both temporal and cross-sensor dependencies on Sentinel-1 and Sentinel-2 time-series. The proposed approach, validated focusing on the estimation of the normalized difference vegetation index (NDVI), shows very interesting results with a large performance gain over the linear regression approach according to several accuracy indicators.