Kwame Gyekye seeks to address the complex question of political legitimacy particularly on the African continent. He argues that the justification for political legitimacy need not necessarily depend on the economic performance of any given regime. For him, justification for legitimacy merely lies in whether all correct processes and procedures were properly followed in the assumption of power. He is of the view that military coups should not be tolerated as they lack legitimacy although they might have justification usurping power. He also argues that popular uprisings may have the justification to assume power but should subject themselves to a plebiscite to have legitimacy. In this paper I seek to argue that Gyekye's distinction between legitimacy and justification of exercise of political power is unsustainable. In contrast to Gyekye I seek to argue for a more plausible account of legitimacy that takes the substantive requirement much more seriously. I do this by showing the importance of the function of institutional checks on power in traditional African societies and seek to argue for the urgent need of such institutionalised checks on power in post-colonial Africa.