Background Learning about the impact of public health policy presents significant challenges for evaluators. These include the nebulous and organic nature of interventions ensuing from policy directives, the tension between long-term goals and short-term interventions, the appropriateness of establishing control groups, and the problems of providing an economic perspective. An example of contemporary policy that has recently been subject to evaluation is the first phase of the innovative Scottish strategy for suicide prevention (Choose Life). Discussion and summary This paper discusses how challenges, such as those above, were made manifest within this programme. After a brief summary of the overarching approach taken to evaluating the first phase of Choose Life, this paper then offers a set of recommendations for policymakers and evaluators on how learning from a second phase might be augmented. These recommendations are likely to have general resonance across a range of policy evaluations as they move from early planning and implementation to more mature phases.