How pathogens coevolve with and adapt to their hosts are critical to understanding how host jumps and/or acquisition of novel traits can lead to new disease emergences. The Xanthomonas genus includes Gram-negative plant-pathogenic bacteria that collectively infect a broad range of crops and wild plant species. However, individual Xanthomonas strains usually cause disease on only a few plant species and are highly adapted to their hosts, making them pertinent models to study host specificity. This review summarizes our current understanding of the molecular basis of host specificity in the Xanthomonas genus, with a particular focus on the ecology, physiology, and pathogenicity of the bacterium. Despite our limited understanding of the basis of host specificity, type III effectors, microbe-associated molecular patterns, lipopolysaccharides, transcriptional regulators, and chemotactic sensors emerge as key determinants for shaping host specificity.