With the advent of the Sentinel-2 mission and its high revisit frequency, high-resolution time series of optical images, the use of satellite image time series for automatic land cover mapping has fostered. However, one of the main limitations related to this kind of imagery is the presence of clouds, which often hinders its descriptive potential by reducing the actual temporal resolution. Although some common practices exist to enable their use in land cover processing chains, the majority of them aims at reconstructing the time series upstream to the classification task, hence introducing a heavy, error-prone pre-processing step. With the aim of exploiting the capacity of deep learning networks to adaptively combine tasks, in this preliminary study we propose an end-to-end framework that simultaneously perform cloud removal and classification of a Sentinel-2 image time series for the downstream task of land cover mapping. The proposed framework is evaluated over an agricultural area in Northern Benin. Our first results show comparable performances with respect to using state-of-the-art gap filling pre-processing on Sentinel-2 time series, hence motivating further exploration.