Unexpected events during an evaluation all too often send evaluators into crisis mode. This insightful book provides a systematic framework for diagnosing, anticipating, accommodating, and reining in costs of evaluation surprises. The result is evaluation that is better from a methodological point of view, and more responsive to stakeholders. Jonathan A. Morell identifies the types of surprises that arise at different stages of a program's life cycle and that may affect different aspects of the evaluation, from stakeholder relationships to data quality, methodology, funding, deadlines, information use, and program outcomes. His analysis draws on 18 concise cases from well-known researchers in a variety of evaluation settings. Morell offers guidelines for responding effectively to surprises and for determining the risks and benefits of potential solutions.

1. From Firefighting to Systematic Action Adding "Surprise" to the Mix Historical Roots: Evaluation, Planning, and System Behavior From Explaining Surprise to Dealing with It Development Path of This Book Guiding Principles How to Read This Book In Sum 2. Structure of the Unexpected Where Does Surprise Come From? Beyond Simple Distinctions In Sum 3. Placing Surprise in the Evaluation Landscape When Is the Probability of Surprise High? When Is Surprise Disruptive to Evaluation? In Sum 4. Minimizing Foreseeable Surprise Theory: Using Explanatory Power and Simplified Relationships Exploiting Past Experience: Capitalizing on What We Already Know Limiting Time Frames to Minimize the Opportunity for Surprise In Sum 5. Shifting from Advance Planning to Early Detection Leading Indicators System-Based Logic Modeling In Sum 6. Agile Evaluation Data Agile Methodology Retooling Program Theory Agility and Stakeholder Needs In Sum 7. How Much Is Too Much?: Appreciating Trade-Offs and Managing the Balance A Framework for Appreciating Design Trade-Offs Maximizing Choice, Minimizing Risk Evaluation Design In Sum 8. Applying the Examples to Categories of Cases: The Life Cycle View "Unintended Consequences": Unity across Programs and Their Evaluations Interpreting Cases through a Life Cycle Perspective In Sum 9. Applying the Examples to Categories of Cases: The Social/Organizational View Navigating through the Cases Placement of Cases on the Social/Organizational Map Categorizations Derived from the Data In Sum 10. Lessons fr