public sector

An Introduction to Evaluation Part I: The Formative/Summative Dualism

Scriven’s (1980) formative and summative dichotomy is arguably the most accepted and renowned evaluation typology. One could suppose that the formative/summative dichotomy is therefore fundamental to understanding evaluation. In this post, the concepts of formative and summative evaluation will be explored.

The label, formative evaluation, refers to an evaluation approach that focuses on evaluation improving a programme, essentially allowing an evaluator an interactive (rather than independent) role (Herman et al, 1987). Reporting occurs throughout an evaluation (rather than at the end or at a certain point) meaning closer and sustained contact with a programme (Clarke, 1999). Summative evaluation focuses on formally reporting findings at a certain point in time.

Prescott et al (2002) emphasise that the two approaches are separate and distinct processes. According to Patton (1996), the formative/summative dichotomy, ‘captures the entire array of evaluation purposes’, as it suggests that, ‘anything that is not formative is summative’.

There is much criticism of the summative/formative dichotomy in modern evaluation: ‘the world of evaluation has grown larger than the boundaries of formative and summative evaluation’ (Patton, 1996). Debate exists over which of these approaches is most vital, Cronbach (1966) suggests that formative evaluation is more important that summative evaluation, whereas Scriven (1967) notes strengths of the summative approach. Patton (1996) also suggests that, ‘formative evaluation rests in the shadow of summative’.

At the simplest level, the dichotomy offers a labelling system to categorise evaluation. Indeed, various labels are applied to evaluation (Scriven’s dichotomy included).  McKie (2003) finds that evaluation labels and typologies are excluding to those who are not familiar with the terms (for example stakeholders within the programme being evaluated). Further, Ussher and Earl (2010) suggest that the terms summative and formative can be confusing, however, they do note some value in applying labels: ‘the identification, definition and consistent use of specific labels are useful for developing understanding and communicating with others’.

Where evaluation is dual-level, that is, an evaluation is required at both national and local levels (Allen and Black, 2006), both summative and formative types could still be utilised. A formative evaluation approach could be applied to local-level evaluation with the increased likelihood of the evaluation being used to generate improvements (according to Herman et al, 1987). A summative evaluation type might occur at a national-level evaluation with the likely audience being policymakers, funders and the public according to Herman et al (1987).

This post has explored formative and summative evaluation (in the context of public programmes). I will save some of the more specific debates of evaluation’s purpose , for instance, of evaluator independence (should an evaluator be assisting a programme to improve through formative evaluation? etc.) for another post.

Part II of the Introduction to Evaluation series (available soon) will consider the history of evaluation.

Advertisements

8 Ways for Public Sector Organisations to Make Efficiency Savings

Public spending has taken a number of severe hits in the last few years, prompting several phases of efficiency activity. But, after recruitment freezes, restructures and process reviews where should an organisation go next? 

1. Consider how shared services could work for you

Whether this be shared services or a joint management arrangement such as a shared executive.

2. Consider how you can make money rather than just saving money

One council is selling advertising on employee payslips. Another makes money from Google Adsense on its website. Be innovative.

3. Engage employees in the change process

Invite the participation of your employees regardless of level. All too often organisations are engaging mainly middle managers but those that are most familiar with inefficiency of process may be at lower levels. Even if this just means the good old staff suggestions box.

4. Look to your leaders to deliver a narrative which supports the change

Change is difficult at the best of times, leaders who are not driving your vision forward won’t drive your employees forward. Ensure that your leaders deliver a convincing narrative for change (see CIPD Leading Culture Change PDF)

5. Reconsider purchasing options

Organisations in Yorkshire save a great deal through group purchasing with the Yorkshire Purchasing Organisation for instance.

6. Consider lean business models and how these can be applied to your organisation

The service waste model for instance might encourage you to consider unnecessary duplication, delay, movement, communications and inventory processes. Are there unnecessary movements in your processes? Are there duplicate forms? Have payments been reviewed to ensure there are no duplications?

7. Consider how your university can provide consultancy or students to help give a fresh perspective

You could give students vital placement experience, or a case study to consider and respond too. Academic expertise could save a great deal on consultancy too.

8. Learn from organisations in a similar position

Work with business change teams in other organisations and learn from one another – and encourage your managers to do the same. When I talk with the police, fire service or councils that I am working with they are all interested in the same solutions, they all voice the same issues – you will have more in common than you think.

Did you know? “Over 90% of managers in the public sector had been involved in a major organisational restructuring” (http://blogs.lse.ac.uk/politicsandpolicy/archives/35109)