What ought beneficiaries of injustice to do with the privileges unjustly conferred upon them? This article examines how those who have been privileged as a consequence of injustice can best contribute to struggles for justice. In particular, I ask whether we ought to renounce privileges which have been unjustly conferred, or whether it may be better to use such privileges in ways that help bring about justice. The article engages in particular with feminist literature on the topic of privilege, building on arguments provided in this literature to argue that in many cases the best contribution the privileged can make to struggles for justice, is to use unjustly conferred privileges in a way that ultimately undermines the unjust systems and structures that conferred them. I tentatively outline some ways in which the privileged can develop the sensibilities which will allow them to use their privilege in this way.