Senior emergency management personnel face more extreme events and more complex challenges than their predecessors and these challenges will increase in the future. One of the key challenges that senior emergency managers face is the evaluation of operational performance in the context of increasing scrutiny from media, legal and political spheres. To investigate this issue we collected survey data from 33 senior emergency management leaders who operate at the strategic level (above the local IMT) as part of a broader survey examining the challenges of strategic emergency management (funded by the Bushfire CRC, Owen et al., 2013). Participants reported concerns that operational performance is currently judged by external sources (such as the media) in an often post-hoc and arbitrary manner and is dependent on whether or not what happened in the end was perceived as a good outcome. Reliance on the outcome of a complex event is problematic because there is not an absolute correlation between the process of managing an emergency and the outcome. Bad outcomes can occur despite good operational processes and good outcomes can occur despite bad operational processes. For example, all the best processes might have been in place and performed well but the outcome was bad because of unexpected climactic conditions, such as a wind change. This paper outlines the views of senior leaders in emergency management regarding what needs to be taken into account when measuring operational performance and shares critical insights into the indicators of effective performance from the perspective of different stakeholders.