Pyrolysis is a promising way to upgrade large amounts of residues from olive oil processing into charcoal. Pyrolysis of the stone and pulp fractions needed to be investigated before conclusions could be drawn. We subjected the olive stone fraction, the pulp fraction, and a mixture of the two to dynamic pyrolysis and isothermal pyrolysis at 360°C. We characterized the charcoals resulting from isothermal pyrolysis at 360°C for different durations in terms of the fixed-carbon content (FCC), carbon content (CC), and high heating value (HHV). We found that charcoal yield from the pulp was higher than that from the stones, which were 38.1% and 32.9%, respectively, after pyrolysis for 360 min. This seemingly unexpected result was due to the high contents of ash (6.22%) and extractives (13%) in the pulp, which remained completely and partially undecomposed, respectively, in the charcoals and are accounted for when calculating yields. However, charcoals obtained from the stones were of higher quality than charcoals from the pulp, with lower ash content and higher FCC, CC, and HHV. In particular, the FCC, CC, and HHV after pyrolysis for 360 min were 73.2%, 74.4%, and 30.2 MJ/kg for the stones and only 61.8%, 63.2%, and 25.9 MJ/kg for the pulp, respectively. Depending on the required quality of the final charcoal, our results help decide whether to pyrolyse the entire olive residues or only one of the two fractions, more likely the stones.