International Evaluation Conference

The video is now available from my guest role at the Polish Agency for Economic Development’s International Evaluation Conference.

I also wrote a summary of my position on the future of evaluation which can be found in an earlier blog post here: XII International Evaluation Conference 

Key parts are:

57:28 Erosion of Experts, Ethnography and Evaluation
59:54 Rethinking how we present data
1:00:00 This isn’t evaluator bashing – evaluators do a really good job!
1:00:01 We need a systems approach to evaluation
1:01:41 UKES’ Voluntary Peer Review scheme
1:02:52 Using Evaluation
1:04 The Evidence Base
1:14:40 Should we quit EB approaches
1:16:45-1:20 Being strategic in evaluation (whilst not quite fulfilling the question!)
1:38:36 Conclusion

Awesome (Free) Visual Idea Tools

Ok, so the awesome bit is subjective! I have just returned from a predominantly practitioner-based conference, where as an academic, I was impressed with the powerful visuals in most of the presentations. Gone seem the days where ‘visuals’ constitute a relevant photograph flourishing a crammed PowerPoint slide. The infographics, process diagrams, and cartoons present at the conference were impactful, conveyed some really complex information, and yet were not produced by multi-million pound organisations with teams of graphics experts – many were charities and government departments.

This prompted me to explore a little more this visual idea domain. The post doesn’t intend to exhaust all options, or ‘taste test’ all for the best in class. What it does do is give an overview of some of the options out there to encourage a bit of creative toe-dipping.

1. Canva

Anyone who knows me will know how impressed I have been with Canva (canva.com) and their mission of ’empowering the world to design’ – my students are using it for posters, my sister is using it for her wedding invites, my partner is using it for social media marketing – I am a major fan! Canva have many templates (termed ‘layouts’) and there are many free ones to get started. There are many preloaded images and users can upload their own too (paid images are only $1). Very easy to use and has apps (I use iPhone and iPad apps – useful on the go) but needs internet connection to use.

 

Support Guide DProf.jpgUntitled design.jpg

2. Storyboard That 

Storyboard That (storyboardthat.com) allows users to create comicstrips. There are lots of other storyboard creators out there too which look great and perhaps look a bit more effective than what I have managed to produce on Storyboard That, but it is relatively intuitive which is a big thing for me.

Snip20170512_3

‘Storyboard That’ Example

Storyboards are appealing to explain or add humour and these tools seem to be trying to market a lot to educators. Storyboard That refer to their offering as ‘digital storytelling’. The example adjacent was my first ever dabble with this tool and took about 10 minutes to create. However, as with many of the tools out there, the number of projects and features are restricted unless you pay for premium access.

3. Piktochart

Piktochart (https://piktochart.com) is similar to Canva and relatively easy to use. For me, their infographic and A4 reports (in ‘printables”) are the better of their current offering.  I prefer Piktochart over Canva for presenting numbers since you can add data into tables and maps. At the time of writing there is limited colour choices on some of the templates (but a coming soon message so think they are onto this).

Of course, the focus here has been on free tools only. Upgrades to premium membership might be worthwhile if you are going to use such tools regularly (Venngage is great for infographics but at $19 a month I can’t justify it).

What do you use?

How do you like to convey complex information?

Is there a place for more visual means?

 

 

 

 

 

 

XII International Evaluation Conference 

It was a great pleasure to speak at the opening of the XII International Evaluation Conference by invitation of the Polish Ministry of Economic Development, and Polish Agency for Enterprise Development yesterday (21st June 2017).

I had been asked to speak about the future of evaluation and evidence based policy and a summary of what I discussed is outlined below. 

Challenges facing evaluators at the current time

The Death of Experts. During the U.K. ‘Brexit’ referendum campaigning there were several examples of expertise being undermined by both the general public and some politicians (e.g. Michael Gove:”people in this country have had enough of experts”). At the same time we are seeing evidence, knowledge and expertise shared via internet blogs and websites; and arguably being utilised at a greater rate than other formal evidence mechanisms (evaluation reports, academic publication) – despite no guarantee of their quality. Evaluators have long struggled for legitimacy and this erosion of the expert role complicates this challenge further. 

What does the erosion of expertise mean for evaluation evidence and its future use?
How should evaluators respond to the erosion of expertise and further challenges to their legitimacy ? 

Changing consumption of data. Big data is trending right now and evaluation is following suit (Lou Davina-Stouffs of Nesta UK shared an example in Wales, UK, where data is being scraped to look at indications of SME development/improvement). Evaluators and policy makers may need to approach this data with caution. Similarly, we are seeing strength in story telling – rich qualitative methodologies may support this to be captured. This is not about polarising qualitative (ethnography, interviews) and quantitative (big data) approaches – but evaluators need to move with these changing methodologies and embrace technology to do this. We are seeing the use of audio-visual means of presentation as a valuable means to disseminate evaluation findings with better consumption than written evaluation reports (being anecdotally reported). This twitter post demonstrates this point:  

Source: @evaluationmaven

 

Mariana Hristcheva (Director-General of Urban and Regional Policy, European Commission) referred to this point later in the conference, and noted that evaluation studies of European Union funded cohesion policy initiatives are now beginning to be produced in video format.

Anyone can evaluate.  Many evaluation practitioners are engaged in evaluation societies, networks and professional development throughout the world, engaging in debate about how to advance evaluation practice; yet many are not (there is no data available to demonstrate just how many evaluators are engaged/disengaged with such practice development to support any such proportion to be known). I am preaching to the converted as it is those who are disengaged and absent from practice development (professional development or methodological) who we ought to worry about. We know the quality of evaluation varies vastly and so this lack of oversight/governance is unhelpful. 

Evaluation societies such as the UK Evaluation Society (UKES), American Evaluation Association (AEA) and European Evaluation Society (EES) are engaging in work to professionalise evaluation and we can recognise capability frameworks, guidelines for practice and, more recently, voluntary peer review scheme pilots (EES, UKES). The European Commission are also taking a more proactive role to enhance their evaluation studies offering summer schools in evaluation practice.

Wolfgang Meyer, later at the conference, highlighted the absence of renowned European evaluation scholars and theorists. US scholars from the 1980s still dominate our academic contributions and this is unhelpful to the education of evaluators and evaluation capacity-building in Europe.

Providing a consistent evaluation experience across the industry remains a challenge.

Should we abandon evidence based policy?

The evidence based policy approach continues to be challenged across numerous sectors. No, I do not think that this should be abandoned – to do so would be to abandon the very notion of evidence, knowledge and its transformative potential in policy development. However, the very fact it is being challenged might prompt us to craft a new narrative that sits behind it, and to explore further the issues we face in practicing it. 

The recent work of Newman, Cherney and Head (2017) detailed the result of a study among over 2000 Australian public servants and found that almost 60% used e-databases to search for academic abstracts, articles and reports, and just over 60% had used academic work in reports over the preceding 12 months. This suggests that the evidence base is being consulted. Although worryingly, 75% of the same respondents didn’t feel they had the expertise to apply the results – one possible avenue for improvement of the use of evidence. 

Redefining EBP is unlikely to support us to overcome the barriers to its effectiveness, pushing a clear agenda to engage all parties to systematically address the challenges that face it might help. Our struggles as evaluators to transform policy need closer inspection. We are likely to struggle to prevent evidence being manipulated for political gain but it is likely we can support the issues of cultural difference between evaluators and the consumers of evidence, seen for instance in long-standing remarks about the presentation of evaluation findings. Evaluators alone cannot solve this, it needs to be a systems approach involving policy makers, public servants, funders if relevant, and evaluators.

To abandon EBP would be to abandon faith in knowledge, learning and improvement (Wond, 2017)

But we can’t afford to wait for data can we? 

This timeliness challenge is never going to cease, the two are going to struggle to synchronize but they can be mitigated to some extent. I can’t imagine a time when society will stop to ponder the evaluation reports of the previous initiative before proceeding to the next and I do wonder whether evaluation has to step back and consider a longer term role for itself instead. Seeing evaluation as a longer term game may actually be helpful, for instance we can reflect on the way evaluation is funded (short term not supporting us to establish longer term impact) and whether this fits.

 

Writing up Qualitative Research

I have hit a bit of a wall recently after a rather scathing reviewer comment. It has led me to revisit how I approach purely qualitative work.

Whilst re-educating myself a little on best practice in this area I have stumbled upon this:

http://www.parint.org/isajewebsite/bookimages/isaje_2nd_edition_chapter6.pdf

It provides some useful points about what, and what not to do. All things I know but that in practice have possibly slipped a bit. In particularly one of the papers recommended has refreshed my thoughts on approaching the write up of my data…

http://journals.sagepub.com/doi/abs/10.1177/002204260703700404?legid=spjod%3B37%2F4%2F805&patientinform-links=yes&

How have you found publishing qualitative work, and what are your tips?

Age discrimination… of the young variety.

I had a really interesting discussion with a fellow researcher today about age. She had remarked that my age had featured in a discussion she had recently had with someone else. In someways I should feel proud that the context in which this had arose was in an inspirational sense – that the other person had challenged their own barriers relating to their age after reflecting on my successes. Whilst a lot of discussion on age has focused on discrimination against older people/employees, the age discussion we had today was underpinned by my young age (and some might say looks!)

Over the years, my relatively young age has emerged frequently in my career story and I have had to learn to deal with that, progress with confidence in my own ability, and in a way learn to secretly enjoy surprising others. Although, I will admit it has stolen the focus of a few mentoring opportunities in the past, ‘how did you deal with people perceiving you as too young?‘ I asked one mentor (Doh! So much more valuable things I would ask now!).

Personally, I don’t see myself as particularly young (32), or see that this should be a barrier. If I consider the ages that I achieved certain milestones, then:

  • At the age of 16 I had set up, sought and won full sponsorship for and was running the first ladies team for my hometown;
  • At the age of 26 I gained my PhD;
  • At the age of 28, I had written and made a Unit of Assessment submission to the Research Excellence Framework that punched above its weight (I am sure there will be other sub-30 year old REF coordinators but I am yet to meet one – hello if this is you!);
  • At the age of 31, I became a Head of Research for a University department (College of Business, Law and Social Sciences).
  • At the age of 32, I can count at least 30 noteworthy instance where age has overtly featured in work-related discussions/decisions/reactions.

I do still occasionally have to show ‘I.D.’ when a bottle of wine is in order!

I still don’t consider that I have been ‘successful’. My journal output productivity has given way to my passion for fixing any rogue process that I possibly can, and being a ‘yes’ person doesn’t help (my gym kit is gathering dust too).

So, why am I telling you all this? 

  1. Because ‘being young’ can be an issue in the workplace, yet I don’t believe is being discussed enough or challenged.
  2. Because if I can inspire just one person to overcome their insecurities at being ‘younger’ than their colleagues, or being ‘too young’ to apply for that job then that is good enough for me.
  3. Because I believe our younger workforce have a lot to offer beyond the entry-level roles they often find themselves in (pop them in the boardroom for 30 minutes and see what happens!)
  4. Finally, I think some of the tales (below) I have of my experiences might entertain or educate.

Tales through my Career

The Job Interview

Towards the end of my first degree I began with the usual activity of job-seeking. In one particular interview, for a HR role for a national construction retailer, I was met with an unusual remark, ‘as you will have noticed we have quite a lot of older ladies working in this office and you are young, I am concerned that you won’t settle because you are younger than them obviously’. In the politest way possible, I explained my past experiences of working with ‘older’ colleagues, that this had been healthy and productive. In case it was a trick (it was for a HR role), I then went onto comment jovially that this, ‘of course, shouldn’t influence a recruitment decision‘. I didn’t get the job.

The Meeting

Arriving for a meeting in a cafe with someone I had never met but who I was to work on a project with, I spotted the gentleman I was meeting (the product of a quick Google search). He was looking out for ‘Dr Wond’ and looked past me several times. I began to introduce myself, ‘Hi, ****’, I began. He cut me short, ‘Oh, can Dr Wond not make it?’ . He assumed I was an assistant, remarked about not expecting ‘someone so young’ and we continued our meeting.

The ‘Give it a Few Years’ discussion 

Developing a challenging programme and delivering it with some great examples of business engagement globally didn’t come particularly easy. I had to fight central process embedded throughout the organisation, change it, and eventually amassed several ‘wins’. When a more senior central opportunity arose, I was excited. It meant managing the central processes, and I invited the Director of that particular department for coffee to discuss whether he would welcome my application. That hour was painful! From the outset he described a more mature, male being the ideal candidate: ‘I know you could do the job, but I had a more mature professor in mind, a guy who could impress with lots of papers and a great C.V….give it a few years’. The Director later debriefed my manager on our discussion, explicitly stating I was too young.

The Director later approached me to second into this position when his first choice let him down.

The Last Word

I identified a huge quality issue that went against internal procedures (and that many universities had fallen foul of during their reviews), on attempting to discuss this with the colleague to whom it concerned she was quite defensive. Her final words as I attempted to share these concerns? ‘I am older and have been at this game longer than you’.

Working with others we addressed this quality gap. There was nothing accurate or correct in what my colleague was stating, it was a way to move past the conversation.

Lessons

So what lessons can I share with anyone finding themselves in this position?

Lesson 1: Picking the Right Fight

At times, I wish I had fought a little more against the assumptions that others had. They have impacted on the doors that opened and the doors that closed (in job and opportunity terms). Yet, I am not entirely sure what this will have done, perhaps their mind was already made? Looking back, I wish I had challenged discrimination against the young since I didn’t have much to lose, spoken out a little perhaps and made a fuss (for the benefit of the next person that interviewer meets!).

Lesson 2: Let this motivate you

Show them what you are made of – perform your best, achieve, succeed! I was driven to address my experience gaps so much more by every ‘age-related oversight’. What those discriminating against me had actually meant to say was probably actually, ‘perhaps due to age you are inexperienced and therefore not up to this’. I made it my mission to get this experience. I analysed the job descriptions of my bosses and my bosses’ bosses. I demanded extra responsibility (line management, budgets, projects) and where I couldn’t get it I sought it in other ways – through volunteer opportunities with community organisations for instance.

Lesson 3: Not much has changed over the past decade

Despite being 10 years older, age still ‘crops up’ now (when will the world look on me as old/less young?!). This suggests to me that either (a) I have failed to age; (b) society has not changed, more specifically workplaces; (c) not enough action has happened to address this barrier.

Lesson 4: Don’t let age stop you being great!

As I have already mentioned, I have let this get in the way at times, I have worn the problem and let it weigh me down. It didn’t help. It did not resolve the biases others have, it didn’t help me conduct my role with confidence. OK, it did provide me with lesson 2 above…but please, don’t let it negatively affect you.

Lesson 5: When you get there don’t feel guilty.

I have made sacrifices to get to where I am. I rarely turn-off my computer before midnight, I have two children but have only managed several weeks maternity leave for both – so when someone comments that I am ‘too young‘, as though I have been privileged, given a boost, or anything else that suggests that this is odd, then it is a bit of an insult. So my final lesson is not to feel apologetic or guilty about where you work hard to get to – because if you want to get there you will.

Call for Papers

I am chairing the following conference in August/September of this year with the SRR Network. The conference website is currently being developed but please find the Call for Papers below. Feel free to share within your networks.

16th International Conference on Corporate Social Responsibility and 7th Organisational Governance Conference

Responsibility and Governance: The Twin Pillars of Sustainability

Buxton, University of Derby, UK

30 August – 1 September 2017

Call for Papers

For the 2017 conference we return to the UK. This year the conference will be hosted by the University of Derby at their Buxton campus in the historic spa town of Buxton, Derbyshire – the highest town in England. The conference will be organised jointly by the University of Derby and the Social Responsibility Research Network.

There is considerable evidence that the field of social responsibility is changing and maturing – and increasingly becoming synonymous with the field of governance. This can be seen from the issues which are of concern to people currently researching in the field. The concept of CSR has gradually spilled over to the other fields of inquiry so much so that today we can speak about the inclusion of social responsibility in any type of human activity (business, politics, justice, etc.). At the same time governance failures have been shown to be at the root of many economic and social problems besetting corporations and other organisation. In fact increasingly the terms social responsibility, corporate responsibility, sustainability and governance have become intertwined and are often treated as aspects of the same issue.

With this in mind we have decided to adopt a theme of sustainability and to consider responsibility and governance as its twin pillars for our forthcoming conference – which of course includes all the other terms within the theme. This raises the questions of what do we mean by sustainability, how do we achieve this and what is the relationship between responsibility, governance and sustainability. These are questions which we will raise and address during this conference.

This conference is designed to act as a forum for the debate and analysis of contemporary issues in this broad area. It is intended to attract people from a wide variety of disciplines and geographic regions for an exchange of views.

The conference is intended to be interdisciplinary and welcomes contributions from anyone who has a perspective on this important issue. Papers are welcome on any topic related to this broad issue and suggested themes for papers include:

  • Defining sustainable development
  • The Triple Bottom Line and its critics
  • Environmental performance and auditing
  • Ethics and corporate behaviour
  • Globalisation and corporate activity
  • Governmental influences on corporate behaviour
  • Protests concerning corporate activity
  • Regulation of corporate social behaviour
  • Sustainability and marketing
  • The role of accounting in corporate accountability
  • The role of corporate governance
  • Governance and regulation
  • National vs. supra-national governance
  • Free markets and governance
  • Multi-national Accountability
  • Sustainability and free markets
  • Social entrepreneurship
  • Socially responsible business activities
  • Globalisation and sustainable development
  • Regulation of corporate social and environmental behaviour
  • Governmental influences on sustainable development
  • Sanctions for non-compliance in a global market
  • Regulating the regulators
  • Organisational governance in the public / NGO sector
  • Audit and organisational governance
  • Models of organisational governance

 

Offers to run workshops, symposia, poster sessions, themed tracks or alternative events are especially welcome. Please contact either Shahla (shahlaseifi@yahoo.com) or David (davideacrowther@aol.com) with suggestions.

Although preference will be given to full papers, abstracts of 200-500 words will also be considered. All papers and abstracts should be sent by 1st June 2017 by email to davideacrowther@aol.com. No more than 2 papers will be accepted from any author.

This year we will also be running a doctoral stream within the conference. In this stream presenters will get extra time for presentation and discussion and will receive personalised feedback from an expert in their area. Initially an abstract should be submitted stating that it is for the doctoral stream.

Poster presentation are also accepted. Again an abstract is required in the normal way.

Final versions of accepted papers will be required by 1st August 2017.

Conference fees – to be announced later

 

Please check if you need a visa for your travel to the UK. To get an official invitation for the conference please contact the conference chair.

We will publish proceedings but we encourage all delegates to further seek to get their paper published in an appropriate outlet. Advice will be given during the conference. It is also expected that a book of selected papers will be published. Full details concerning other publishing opportunities for the papers presented at the conference will be provided during the conference.

Venue of the Conference

The conference will be held at the Buxton campus of the University of Derby.

The conference fee will include meals and conference materials. There is plenty of good hotel accommodation nearby to suit all budgets and details will be given with registration details.

An optional sightseeing tour will be arranged for after the conference and details will be announced later.

Full and updated details can be found at the conference website:

We look forward to welcoming you to Buxton in 2017 for what promises to be an exciting conference.

 

Dr Tracey Wond                                                                        Professor David Crowther

Conference Chair                                                                      Chair of Scientific Committee

University of Derby                                                                   President, SRRNet

UK                                                                                           www.socialresponsibility.biz

email: t.wond@derby.ac.uk                                                        davideacrowther@aol.com