Month: August 2013

What is Research?

I had an interesting conversation today about what research is and is not.

  • Is research turning practice into theory?
  • Is it turning theory into practice and observing the effects?
  • Is it anything that involves measuring something else?
  • Is academic research a different type of research to research?

Now, I am aware of what books/REF criteria might say research is, but am interested to know how you would define it?

“An August 2013 account of what research is”…. 


Power and Evaluation

It could be argued that research on power and evaluation is still relatively underdeveloped – for instance, Pawson and Tilley (1997, p.20) highlight a ‘failure to appreciate the asymmetries of power’ within the evaluation context. Yet, many of the issues in evaluation (utilisation of, access) appear to arise from issues relating to power. For this reason, I briefly want to consider Yukl’s, French and Raven’s, and Greene and Elfrers’ theories of power and apply this to the evaluation context. 


Burton and Thakur (1995, p.354) define power as ‘the ability or the potential ability of a person or a group to influence another person or group’. In this way there are two observable parties the the influenced, and the influencers. French and Raven concur, stating that ‘the phenomena of power and influence involve a dyadic relation between two agents’ (2001, p.61). Abma and Widdershoven’s definition also suggests multiple parties in a power dynamic:

Every social relationship involves power. Power refers to the possibility of letting someone do something he or she would not do otherwise. Power is thus relational, and parties in a power relationship are tied to each other by mutual dependency (Abma and Widdershoven, 2008, p.212).

In the context of the evaluation relationship, the primary parties are the evaluator and evaluated (initiative, project, authority), however, other parties also exist (for example the community or beneficiaries).

Various origins and forms of power are presented within available literature. French and Raven (1959 in Moorhead and Griffin, 1989, p.358) identify five power bases from which power can originate: ‘legitimate’; ‘coercive’; ‘reward’; ‘expert’ and ‘referent’. Similarly, Greene and Elfrers’ (1999, p.178) boast analogous categories but extend to include ‘connection’ and ‘information’ power. Yukl (1998) offers three origins of power; broadly correlating with those proposed by French and Raven, and Green and Elfrers, these are ‘position power’, ‘political power’ and ‘personal power’. The power bases typified by French and Raven, Greene and Elfrer, and Yukl are applicable to the relationships and interactions occurring through evaluation. The table below provides a summary of these power typologies and their relevance to evaluation.

Characteristic Power type Evaluation influence
Authority, delegated, formal, within organisation Legitimate Power (Green and Elfrers, French and Raven)Position Power (Yukl)Political Power (Yukl) An evaluator can be given authority to exert power over others perhaps by directing resource. Politically evaluator knowledge and findings means that their position is well regarded.
Control sanctions Coercive Power (Green and Elfrers, French and Raven)Position Power (Yukl) An evaluation function can instigate the most punishing sanction if used for governance purposes. Evaluation findings can ultimately prevent the continuation of a programme.
Control reward Reward Power (Green and Elfrers, French and Raven)Position Power (Yukl) The reporting of positive results may lead to rewards for the programme concerned.
Specialist skills/knowledge Expert Power (Green and Elfrers, French and Raven)Information Power (Green and Elfrers)Personal Power (Yukl) The skills possessed and learned throughout the evaluation experience provide an evaluator with information or expert types of power.
Social need to be liked to influence others, personality  Referent Power (Green and Elfrers, French and Raven) This may present difficulties for an evaluator with issues of independency clashing with personal needs.
Networks – both internal and external Connection power (Green and Elfrers)Political Power (Yukl) Evaluators should become well connected within the programme setting maintaining good relationships with members of these networks. There are many power consequences if these are maintained, or oppositely neglected.

Position Power and Evaluation

Yukl’s notion of ‘position’ power refers to the authority that is delegated to stakeholders within an organisation, mirroring French and Raven’s (1959), and Greene and Elfrers (1999) concepts of ‘legitimate’, ‘reward’ and ‘coercive’ power. Position power gives the possessor (in this case evaluator) formal authority, status and control, over operations and other stakeholders, to a specified extent.

In the case of an evaluator, there could be said to be position power, with their work being capable of influencing the continuation of a programme. This position power gives the evaluator formal authority to act on behalf of other stakeholders according to Clarke (1999, p.26), who asserts ‘evaluators have a moral responsibility to act as advocates for powerless stakeholder groups’.

There is also a risk that position power is wrongly exerted, creating friction and conflict amongst stakeholders (Weiss, 1972; Gordon, 1991; Davies, 1999; Sell et al, 2004; Abma and Widdershoven, 2008). Formalising the power structure, including where an evaluator sits among this may be worthwhile; as such position power may also refer to a role within a political structure (Ridgeway, 1991; Sell et al, 2004):

Within an organisation the allocation of status assists in operational and power structures. The formalisation of power through such a structure/status has been found to assist the authority of power players to be accepted (Sell et al, 2004, p.47).

Evaluators should not necessarily be seen as the subordinates of managers, commissioners or other stakeholders, although and here lies a potential power struggle which needs to be managed for a healthy stakeholder relationship. Poor clarity of the role and purpose of evaluator only adds to this issue: ‘it is useful to realize that clients may have specific ideas about what evaluation means and how an evaluation is supposed to be conducted’ (Stecher and Davis, 1988, p.22):

The key question for any local evaluator remain the same, how does one engage with such networks in which issues of power and difference often remain unacknowledged (Diamond, 2005, p.179).

Understandably, there is opportunity for an evaluator and another delegated power-player to disagree. An imbalance of ‘knowledge’ and power between funders, initiatives and participants may cause complications (McKie, 2003, p.321; Huberman, 1990; Lincoln, 1994). Clarke (1999, p.15) remedies this by suggesting that ‘it is essential at the outset that an evaluator obtains a clear understanding as to what the client requires from the evaluation and develops the evaluation research design accordingly’. The type of evaluation being conducted influences greatly the importance of power in the evaluator/stakeholder relationship.

Reward Power and Evaluation

Coercive and Reward power allows the holder to impose sanctions or entice subordinates with rewards. Coercion in a power context has been discussed by a host of exchange and political theorists (Lasswell and Kaplan, 1950; French and Raven, 1959; Molm, 1997; Sell et al, 2004). Generally (in the UK), bureaucratic mechanisms such as processes, procedures or legislation limit the allocation of rewards and sanctions (Burton and Thakur, 1995). The perceived legitimacy of coercive power may also affect its degree of acceptance, if it is felt that power has been exerted unfairly then conflict may occur.

A sense of forcing or getting another to do a certain task, in a certain way or for certain purposes is fraught within evaluation politics (Frohock, 1974; Mohan and Sullivan, 2006). It may be an evaluator attempting to have their findings used within the policy decision-making process, or programme staff trying to get evaluators to report favourably on the programme in question.

Expert Power 

Expert power considers the specialist knowledge or skills of the holder giving them power and control over others. Evaluators may seen to have expert power, as experts in evaluation research. Burton and Thakur (1995, p.357) also explain that a compromise between the expert and managers can sometimes be overcome by ‘power sharing’, ‘when a manager is aware that subordinates possess significant experience, it is common for the manager to allow these subordinates the exercise of considerable power’. 

But you forgot Referent and Connection Power! 

Referent power and connection power relate to relationships, networks and social needs. These are interesting areas and bring into play aspects of evaluator independence. Whilst I have researched a great deal on these and written about them I would like to conduct more research to look at them in greater depth – watch this space. 

8 Ways for Public Sector Organisations to Make Efficiency Savings

Public spending has taken a number of severe hits in the last few years, prompting several phases of efficiency activity. But, after recruitment freezes, restructures and process reviews where should an organisation go next? 

1. Consider how shared services could work for you

Whether this be shared services or a joint management arrangement such as a shared executive.

2. Consider how you can make money rather than just saving money

One council is selling advertising on employee payslips. Another makes money from Google Adsense on its website. Be innovative.

3. Engage employees in the change process

Invite the participation of your employees regardless of level. All too often organisations are engaging mainly middle managers but those that are most familiar with inefficiency of process may be at lower levels. Even if this just means the good old staff suggestions box.

4. Look to your leaders to deliver a narrative which supports the change

Change is difficult at the best of times, leaders who are not driving your vision forward won’t drive your employees forward. Ensure that your leaders deliver a convincing narrative for change (see CIPD Leading Culture Change PDF)

5. Reconsider purchasing options

Organisations in Yorkshire save a great deal through group purchasing with the Yorkshire Purchasing Organisation for instance.

6. Consider lean business models and how these can be applied to your organisation

The service waste model for instance might encourage you to consider unnecessary duplication, delay, movement, communications and inventory processes. Are there unnecessary movements in your processes? Are there duplicate forms? Have payments been reviewed to ensure there are no duplications?

7. Consider how your university can provide consultancy or students to help give a fresh perspective

You could give students vital placement experience, or a case study to consider and respond too. Academic expertise could save a great deal on consultancy too.

8. Learn from organisations in a similar position

Work with business change teams in other organisations and learn from one another – and encourage your managers to do the same. When I talk with the police, fire service or councils that I am working with they are all interested in the same solutions, they all voice the same issues – you will have more in common than you think.

Did you know? “Over 90% of managers in the public sector had been involved in a major organisational restructuring” (

Academics as evaluation experts

I have been pondering what makes an evaluation expert? It comes back a little to the age old debate over academics versus practitioners and has been sparked by some evaluation tenders that I have been looking at.

With a PhD that focused on evaluation practice, and experience of conducting and devising methodologies for several evaluations, I could be considered an evaluation expert. Yet, is this to say that a consultant with dozens of evaluations completed is any more, or less, of an expert?

  • Academics bring: research excellence, experience, knowledge from theory (likely to be more so than consultants), a concern for quality to protect the university reputation, extra resource through student researchers, other commitments to their institution, academics are well-linked and can often access to experts in other areas
  • Consultants bring: research knowledge, evaluation experience (perhaps more so than academics), other commitments to their other clients, business acumen.

Finding your evaluation expert may mean looking to a university, or it may not.

For anyone requiring an evaluation I would suggest:

  • Do look to consider the benefits of numerous types of evaluators (self employed consultants, larger private consultancies, universities) and discuss with them their research approach to ensure that it fits your needs;
  • Do ask for references or examples of work previously conducted. This might not just be evaluation research, there are some very strong academics and researchers who may not be experienced in evaluation, but who can use research principles effectively;
  • Do try and involve your local university for advice or when producing a tender for the research work (not necessarily the evaluation itself), universities are strong in research, and evaluation is research (I am happy to work with organisations to ensure that a strong evaluation brief is created);
  • Do consider alternative options, perhaps discuss how a student could conduct your evaluation. A PhD studentship might be one solution, meaning that you sponsor a student to complete their PhD study, and in return you gain a research-savvy individual who will conduct your evaluation research over a period to suit you (around 3 years).