Matrix population models are widely used to predict population dynamics, but when applied to species-rich ecosystems with many rare species, the small population sample sizes hinder a good fit of species-specific models. This issue can be overcome by assigning species to groups to increase the size of the calibration data sets. However, the species classification is often disconnected from the matrix modelling and from the estimation of matrix parameters, thus bringing species groups that may not be optimal with respect to the predicted community dynamics. We proposed here a method that jointly classified species into groups and fit the matrix models in an integrated way. The model was a special case of mixture with unknown number of components and was cast in a Bayesian framework. An MCMC algorithm was developed to infer the unknown parameters: the number of groups, the group of each species and the dynamics parameters. We applied the method to simulated data and showed that the algorithm efficiently recovered the model parameters. We applied the method to a data set from a tropical rain forest in French Guiana. The mixture matrix model classified tree species into well-differentiated groups with clear ecological interpretations. It also accurately predicted the forest dynamics over the 16-year observation period. Our model and algorithm can straightforwardly be adapted to any type of matrix model, using the life cycle diagram. It can be used as an unsupervised classification technique to group species with similar population dynamics.