Participatory modeling is increasingly recognised as an effective way to assist collective decision-making processes in the domain of natural resource management. This paper introduces a framework for evaluating projects that have adopted a participatory modeling approach. This framework - known as the 'Protocol of Canberra' - was developed through a collaboration between French and Australian researchers engaged in participatory modeling and evaluation research. The framework seeks to assess the extent to which different participatory modeling practices reinforce or divert from the theoretical assumptions they are built upon. The paper discusses the application of the framework in three case-studies, two from Australia and one from the Pacific island of the Republic of Kiribati. The paper concludes with some comments for future use of the framework in a range of participatory modeling contexts, including fostering consideration of why and how different methodological approaches are used to achieve project aims and to build a collective vision amongst diverse stakeholders.