Action, Innovation, Reflection – CES 2017 Day 1

For those who have read my posts from previous conferences, you wouldn’t be surprised that I get a lot out of these events – hearing great ideas, learning about new tools, connecting with friends and colleagues, and re-igniting the passion for the field I’m working in. Less than two days into the Canadian Evaluation Society’s 2017 conference, all of those positives are definitely here: if anything, it’s all heightened this year. There’s a feeling that my professional interests and skills are strongly aligning with those of my colleagues and the field as a whole, and that altogether there’s increased emphasis on what difference we as evaluators can make not just in our practice but for our communities and society as a whole.

I’m still thinking over everything I’ve seen and heard so far, from the great workshop on Developmental Evaluation led by Nora Murphy, Kate McKegg, and Nan Wehipeihana to a bumper-crop of thought-provoking Ignite presentations. This morning’s keynote speakers, Indigenous evaluation consultant Kim van der Woerd, co-founder of the Moosehide Campaign Paul Lacerte, and Elder Roberta Price, really helped set the narrative for this conference and our field in general – that we all need to work towards reconciliation, and as Lacerte said, we are at a precious moment in the history of our country (and the world!) to face forward towards a shared future. Outside of the formal program, I’m incredibly grateful to both connect with like-minded conspirators and co-create with them. As an example, I’m writing this post at the lunch table with someone I’ve known for several years now as well as someone I’ve just met: we all connected via Twitter to make this time to write and reflect happen, and I won’t be surprised if other ideas and initiatives arise from these planned and spontaneous gatherings in the days to come.

If you’re wanting to see some specifics about ideas and concepts, check out Twitter – I’ve been live-tweeting pretty regularly and the conference hashtag (#EvalC17) has seen lots of activity and discussion. 

As a final thought, a tweet that summarizes my feelings so far:

“My take-away from #EvalC17 so far: we as evaluators do not work in a vacuum – we must attend to history and culture including our own”

Evaluate As If Nobody’s Watching

Two quick announcements before today’s post. First, as I mentioned last week, I’m heading off to the annual conference of the Canadian Evaluation Society, taking place April 30 – May 3 in Vancouver. I’m looking forward to connecting with friends and colleagues during that time, so if you’ll be in attendance, look me up! I’m hoping to write some blog posts while out there and will likely be active on Twitter. Important to note – while I will have email access, I may be delayed in responding to emails, both during the conference and the following week (up to and including May 10) as I’ll be taking some vacation time in the area following CES.

The second announcement is more of testing the waters. Vu Le, non-profit unicorn extraordinaire and the man behind the thought-provoking site Nonprofits with Balls1, recently suggested Thursday May 25 as a day for non-profit agencies and funders to meet (jump to the end of the post) – not with any formal agenda, but simply to connect as individuals and chat. I’m thinking to arrange this kind of meetup in Saskatoon – any of my readers interested? Drop me a line if you this is something you’d like to see.

(As a side note, I had drafted up the post below before learning about this event – chalk it up to coincidence and/or fate that it’s related to funders and evaluation!)

Evaluation and accountability: for many non-profit organizations, these two concepts are inextricably linked. Granting bodies, in providing funds for programs and services, expect that their resources are used to create positive change, whether it’s improved test scores for children, stabilized living situations for those in crisis, or fewer people living in poverty. As part of the application process, most funders ask would-be recipients to provide the goals and objectives of their program, along with outlining how they will demonstrate whether these goals are being met. Successful applicants are then required to report on these metrics to show that the funds were used effectively.

Don’t get me wrong – accountability is an important function for evaluation, both for the funders and the organization themselves. After all, why spend money and staff time on a program or service that is not creating its intended effect? At the same time, evaluation can provide multiple benefits for non-profits, including through means that wouldn’t prompt comparisons with a visit from financial auditors.

Uncertain about how evaluation can be useful beyond satisfying funder requirements? Follow along for a quick thought experiment. Take the phrase “dance as if nobody was watching” and apply the general principle – namely “Evaluate as if no funder or key external stakeholder would see the results” (not as succinct as the original, but you get the idea). What if we evaluated for a purpose beyond accountability? What would the process look like, in what ways would it differ from previous evaluations completed solely for a funder, and how would we use it?

Continue reading

CES 2017: Evaluation and Change

I find it interesting that the two major evaluation conferences I attend are usually scheduled in spring and fall (respectively), the seasons of change1. At its heart, the field is about change: both in terms of understanding and cataloguing what has transpired as the result of an intervention or program, but also supporting new developments by providing feedback and ideas for further improvements and refinement. There’s one more reason why change is central to evaluation, though – our work is not immune to it either. We have to be aware that our contexts and those we partner with also face change regularly, requiring us evaluators to identify and understand what’s happening around us and respond in kind.

The two sessions I’m presenting at the upcoming Canadian Evaluation Society (CES) conference in Vancouver both touch on this idea of being aware of and responsive to our contexts2. Kirstie Gibson, a former placement student with Strong Roots Consulting, and I will be presenting on research that we conducted with local non-profits that examined their perspectives on evaluation. My other session, with Craig Moore of South Shore Evaluation in Nova Scotia, is a thematic breakfast roundtable that invites other consultants from smaller cities and regions in Canada (loosely defined) to chat about the benefits and challenges related to being “the only one in town” when it comes to evaluation consulting. Two pretty different topics, but I think there’s a common thread around the importance of communication and collaboration to successfully navigate situations that are much more complex than the safe examples we practice with in workshops or classroom settings.

With CES approaching in less than two weeks, I’m looking forward to the opportunity to connect with colleagues from across Canada and even other continents (I’m attending a workshop that includes presenters from New Zealand!). More importantly, these events always provide an opportunity to expand my thinking, identify new approaches and methods that I can incorporate into my practice, and most importantly, remind me that when change happens, there’s a community there to help me make sense of it. For my friends and colleagues who will be in attendance, I’ll see you soon in Vancouver! Everyone else, keep an eye on this blog and the Strong Roots Twitter account the week of April 30 as I share insights and ideas from the conference.

  1. Yes, I understand that there are only so many months in the year to hold a conference, that traveling during the winter sucks and many people would rather be on vacation in summer than sitting in a convention centre somewhere – it’s a metaphor, just let me roll with it! 
  2. I’d like to pretend that this was intentional – perhaps subconsciously? 

Sharing and Connecting: Innovation at CES 2017

It’s most definitely fall in Saskatoon – nights dip below freezing, the trees have shed most of their leaves, and it’s only a matter of time before the snow hits1. Another sign of fall? Upcoming due dates to submit presentations for conferences, including the Canadian Evaluation Society’s 2017 conference in Vancouver.

Earlier this year, I wrote about the conference and its theme of Facing Forward: Innovation, Action, and Reflection. Today, I’d like to focus on Innovation – but instead of looking at innovation in evaluation, let’s talk innovation in conferences. My experiences at evaluation conferences (both in Canada and south of the border) have been overwhelmingly positive: at the same time, I’m aware that these events can be criticized for not being the most effective use of time and resources. Indeed, there are numerous ways today that we can learn from others without having to leave the comfort of home – including blogs such as this one! As a result, it’s a legitimate question to ask whether it’s worth spending hundreds of dollars (if not thousands, when you account for travel and accommodations) to make the trip.

One of the key benefits I have taken away from conferences is the opportunity to connect with others walking a similar journey. These events give me the chance not just to learn from others, but to share my own experiences and have a real dialogue where ideas can arise from the space between. True, we can have this two-way communication through words or video, but there’s something about being in the same room together or perhaps pitching ideas over a meal (or pint) that contributes to the generation of innovative ideas.

Although most of these connections are informal, the conference organizers can set things up to encourage such dialogue. For CES 2017, the team behind the conference have provided some interesting presentation types that I’m excited to both see in person and also participate in as a presenter. One type in particular, called “Consultation and Collaboration”, is a highly interactive format that asks attendees to work in groups in response to a problem, issue, or topic brought forward by the session organizers. Giselle Patrick, Carolyn Camman, and I have submitted a proposal under this format on the topic of evaluation capacity building (ECB), with a focus on how ECB can be conceptualized and put into practice for different types of organizations. We look forward to bringing some unique engagement activities to our session and seeing what comes out of the conversation!

There are several other interesting presentation types this year, including Storytelling, Thematic Breakfast Roundtables (I’m thinking to host one on consulting in small and mid-sized cities), and Lightning Round Tables that sound like a mix of speed networking and information sharing. And of course, there’s my perennial favourite, Ignite presentations, where your slide deck automatically advances through 20 slides, 15 seconds a slide, for a total of 5 minutes. I’ve found it a fun challenge to craft a coherent presentation to fit within these limitations, and likewise have enjoyed seeing what others do with this format!

One activity that I’d love to see at CES conferences is based on an idea from our American colleagues, specifically the Community Psychology Topic Interest Group (CP TIG) of the American Evaluation Association (AEA)2. At the annual AEA conference, this TIG hosts an event called “Walk the Talk”, where conference attendees can sign up for a tour of a local community agency or group to learn more about their work, see how they use evaluation, and provide ideas and suggestions for the agency to move forward in accomplishing their mission. Vancouver, like communities across this country, is home to numerous agencies of all stripes and sizes doing amazing work, and I think it would be worthwhile to go beyond the conference venue to learn more and contribute back to these organizations.

So suffice to say, I’m looking forward to this conference, both as a participant and contributor – the hard part is deciding what ideas I want to present on! Fortunately, I still have another week and a half to finalize and submit my proposal(s). Over to you now: What are you planning to present on, if you’re going? What are some innovative presentation formats that you’ve seen at other conferences? Share in the comments below or on Twitter, or drop me a line!

  1. That is, until the snow hits again: Saskatoon saw an extended preview of winter at the beginning of October.
  2. Apologies for the acronym soup in this post!

The Next Best Thing

Tomorrow, Apple will lift the veil on its newest product, most likely a new iPhone1. If past behaviour is any indicator, there will be much discussion and many questions asked in the immediate aftermath: is this a good product, should people buy it, is Apple being forward-thinking or foolish in removing the headphone jack (if the rumours are to believed), and so on. One topic that likely won’t come up, though, is what the company will do with the profits it realizes from the new phone and its other products.

What we don’t ask of Apple or other for-profit companies stands in notable contrast to the questions and restrictions imposed on non-profit organizations when they receive funding. I won’t belabour the general point here, as others have written amazing pieces dismantling these harmful ideas: as one example, Vu Le provided a hilarious-because-it’s-absurdly-true example of trying to run a bakery if we brought to bear the overhead and sustainability myths that non-profits routinely face. More specific to my focus on evaluation and research, Ann Rosenfield (hat tip to Joe Travers) pointed out that donors should actually want to pay for a charity to collect and analyze data about their programs, spend time understanding local trends, and develop and test new approaches to fulfilling their mission. Although these dollars may not go directly towards the front-line program or service, they ensure that the organization continues to operate effectively and is prepared for what the future will bring.

When buying any kind of gadget from a for-profit company, it’s understood that the amount we spend will be used for more than just the product itself. In fact, I think we would be surprised and maybe even angry if companies like Apple didn’t spend at least some money on improving their products and developing new ideas. In the non-profit world, our R&D work (which includes evaluation for me) is often seen as an afterthought, not just by donors and funders but by those working in the field. Sure, we may never see our work featured and feted like a new technological marvel: instead, though, we can develop new ways of creating positive change that save lives, dismantle systems of oppression, and generally improve quality of life for individuals and communities around the world. To reach that potential, though, our sector as a whole needs to invest the time and resources necessary to improve what we’re already doing, understand what’s coming over the horizon, and develop our own “next best thing”.

  1. And possible a new Apple Watch, depending on which prognosticators you want to believe.

Facing Forward

From previous blog posts (see here, here, here, and here), you can tell that I’m a fan of expanding my skills and learning in evaluation through conferences. Attending these events, hosted by the Canadian Evaluation Society and the American Evaluation Association, has also connected me into a diverse community of practitioners, academics, consultants, students, grantmakers, government officials, and more who are brought together by a common interest in the field of evaluation. Given this blend of learning, sharing, and connecting, I try to make it out to these conferences whenever circumstances allow.

The next Canadian Evaluation Society conference will be taking place in Vancouver from April 30 to May 3, and I’m excited for a couple of reasons – not least because Vancouver is a beautiful city to visit, relatively close1 to my home base of Saskatoon, and home to several evaluation colleagues and friends. What really piqued my interest is an initiative for the conference that’s being led by one of those colleagues and friends, Sarah Farina of Broadleaf Consulting. Evaluators are being encouraged to blog on the conference theme (Facing Forward: Innovation, Action and Reflection) in the months leading up to the conference. It’s a neat way to start the conversation well in advance of the meeting itself, and hopefully have it continue afterwards!

Evaluation’s Present and Future

Over the next 8 months, I’ll be delving more deeply into each component of the conference theme: today’s post will take more of an overarching view.

Innovation, Action, Reflection. My first thought on seeing these three words is that I wish that the organizing group hadn’t chosen this theme, as I would be tempted to use it as a tagline for my business! Unpacking the ideas a bit further, I think it reflects where evaluation is today and our current trajectory. Our field has had the reputation (with some historical basis) of acting as arbiters of a program’s fate by rendering simplistic thumbs-up/thumbs-down judgements, and thus raining on the parades of those who are trying to make a difference in the world. While evaluation continues to play a role in promoting accountability, to say that’s our only focus does the field a great disservice.

Based on my experience, the core element of evaluation is using applied social research skills to help those seeking to create change learn more about the work they do and how they can improve their efforts. Evaluators help organizations understand their successes (and failures), provide feedback and suggestions for change, and work alongside developing projects or initiatives to provide real-time data and insights. Yes, there will always be times where we have to pass judgement and say that this program did or did not work as intended to achieve its desired outcomes: that being said, we have both the opportunity and the responsibility to ask how those assessments can support action, encourage reflection, and ultimately promote innovation to tackle the complex issues we see affecting our communities and the world.

As noted above, my plan is to blog about each of those three aspects (Innovation, Action, Reflection) over the next several months leading up to the conference. For more information about the conference, check out

  1. Well, close by Canadian standards, at least. 

Learning Circles

Professional Development on the Prairies

This summer will mark four years for me in Saskatoon, and while there are many things I love about this city1, there are a few small challenges. One specifically related to my work is the relative isolation: the nearest city of comparable size is the provincial capital of Regina, which is a two hour drive away: for larger centres, it’s five hours to Edmonton, seven to Calgary, and eight to Winnipeg. There have been many times where I’ve learned about interesting professional development opportunities in those places or further afield: unfortunately, the time and cost associated with travel can be huge barriers to attending. Although technology has helped fill the gap somewhat, such as through webinars, those methods also have their downsides. Interaction with other participants and the presenters tends to be minimal or non-existent, and it’s all too easy to skip a webinar or attempt to multi-task during it (hint: that strategy doesn’t work, at least not for me!).

Yesterday, I learned about an interesting professional development model that could provide the best of both worlds. Shelley Borys and Natalie Kishchuk presented a webinar hosted by the Canadian Evaluation Society on the concept of learning circles. In their use of this model, a small group of participants (5-7 typically) would meet regularly through one-hour teleconferences: through a cycle of six such sessions, they would discuss different evaluation topics and methods, chosen based on the interest of the group members. Each session would have one of the members act as leader who would be responsible for facilitating the discussion and also selecting the materials and resources for everyone to review in advance. Although the group used teleconference as the means for conversation, online tools such as Dropbox and a shared spreadsheet helped disseminate resources and select discussion topics, respectively.

Evaluate Thyself

Beyond finding this idea interesting, there’s another reason why I’m sharing this example. As you’d expect from a group of evaluators, they evaluated how well this method worked as an educational tool through a brief survey of participants. The outcomes were positive, though with an unexpected result when they delved a bit deeper into the findings. As one would reasonably assume, participants brought different levels of knowledge on different topics: one person may have a high learning need for, say, developmental evaluation, but would only need a refresher on empowerment approaches. However, some participants would initially rate their knowledge on a certain topic as low, but after participating in the discussion would realize they knew more on the subject than they thought. In these cases, dubbed “Comforted” learners by the evaluators, participants would gain confidence in the knowledge that they already possessed (usually from their own personal experience), and also gain support and validation in their skills from the other group members.

These and other findings, such as the importance of balancing similarity and diversity among the group members, highlights the importance of looking beyond surface-level question such as “Did the program work?” or “Were participants satisfied?”. The knowledge gained from examining the reasons for success and underlying processes could help identify who would most benefit from the program: for example, the finding around Comforted learners could indicate that such circles could support early- to mid-career evaluators who have developed a skill base but may be uncertain in their own knowledge. Alternatively, if a group doesn’t work out, the evaluation findings could help determine why: perhaps the composition was too uniform and thus lacked diversity of knowledge and opinion, or the trust among group members wasn’t sufficient to allow people to admit uncertainty or lack of knowledge. Rather than throw out the learning circles model with the bath water, changes could be made to improve the odds of success for the future.

Let’s Start!

Based on what was shared in the webinar, I’m really excited about the prospect of being involved in a similar group. I already belong to a small online group of Canadian evaluators, and I think this approach could be a promising one to introduce some regular discussion. A learning circle approach held in-person could potentially be fruitful here in Saskatoon, though perhaps on a broader topic such as community-based research. If you’re interested in working with me to start up a learning circle in either (or both) directions, or would like to share your experiences with this type of mutual learning, feel free to share in the comment section below, drop me a line through the Contact page or find me on Twitter.

  1. Especially now that we’re fully into spring and patio season is just around the corner! 

Learning Experiences

Student practicums or placements can be a great opportunity for non-profit organizations to provide a student with real-world experience while benefiting from their knowledge and skills. Most placements are required for the completion of a graduate or professional degree; however, some upper-year undergraduate courses include this form of experimental learning as well. Depending on the nature of the placement and the course or degree requirement, the student will be required to complete a set number of hours for the organization, although they may also need to complete specific project or reporting requirements in order to receive credit.

Here in Saskatoon, we’re fortunate to have multiple opportunities to engage with students through programs at the University of Saskatchewan and Sask Polytech, as well as students living here who are enrolled in programs at the University of Regina (notably social work students) and other institutions. Working with local non-profits, it’s pretty rare to encounter one that does not have previous experience with a placement student: in particular, I don’t know if I have worked with an organization that doesn’t have a report penned by a Masters of Public Health student somewhere in their file drawers!

Today I wanted to quickly draw attention to two placement programs that may fly under the radar, plus a learning opportunity that I'll be offering through Strong Roots starting tomorrow!

Applied Social Psychology

As the name suggests, the Applied Social Psychology focuses on the use of social psychology research methods to address social issues1. Part of their coursework includes learning about evaluation approaches and techniques. In fact, the program regularly participates in the Canadian Evaluation Society’s Student Evaluation Case Competition, with the university’s teams winning in 2004, 2011, and 2015, and being selected as finalists in 2002 and 2010. Clearly, they know something about evaluation!

Graduate students in both the Masters and PhD programs are required to complete two placements, one over the summer and one during the school year, with external organizations. I’ve been fortunate to have Kirstian Gibson, who’s in her second year of the Masters, working with me since the fall. In the past several months, she has taken lead on a pro bono evaluation project with a local non-profit, provided support on some ongoing client project, and also co-initiated a research project on how non-profit staff and leaders in Saskatoon view evaluation. It’s been great working with a student who has a strong grasp of evaluation and social research methods (she was part of the winning case competition team in 2015), and I look forward to working with another student from the same program this coming summer.

For more information about the placement, the best person to contact is Dr. Karen Lawson.

Governance Leadership Development Practicum

The for-profit and non-profit worlds don't always see eye-to-eye, but in the case of the Governance Leadership Development Practicum at U of S' Edwards School of Business, it's a great match. This fourth year undergraduate course combines an intense seminar focusing on non-profit governance issues with a practicum experience that places students with local non-profit boards over the course of two semester. I've been involved with this program for the last two years now as a non-profit board mentor, and it's been a great experience with both students. The course provides them with a wealth of knowledge about board governance issues, and the students’ business backgrounds have provided new ideas and insights for the board moving forward.

For more information about this opportunity (applications for the fall should be opening later this spring), contact Dr. Chelsea Willness.

Numbers and Stories office hours

On the topic of learning and sharing skills, I’m happy to announce that Strong Roots Consulting is partnering with The Stand Community Organizing Centre here in Saskatoon to host a regular ”Numbers and Stories" office hour! Any non-profit is welcome to visit me at The Stand (615 Main Street, just east of Broadway: same building as Turning the Tide bookstore) on Wednesdays from 1-3pm for free advice and assistance on topics related to evaluation and community-based research. Some of the help I can provide includes giving feedback on a survey, deciphering what a grant application is asking for in the evaluation section, or brainstorming ideas on how to best engage with your organization’s members.

The first office hour happens tomorrow afternoon – hope to see you there!

These three opportunities are just scratching the surface for different initiatives that support non-profits. Share your experiences and recommendations for placement programs or other educational opportunities in the comments field below, or continue the conversation on Twitter!

  1. The field is very similar in focus to my own disciplinary background of Community Psychology. ↩︎

AEA Conference – Connections

Right now, I have a bunch of ideas floating in my writing list from this conference. New tools, new insights, new ideas for programs or approaches that could be useful back home in Saskatoon. I think I could write up one of those topics each week between now and the end of the year and perhaps get through them all, though I still have a half-day left to add to that list.

I’m not going into any of those takeaways right now, because one thing keeps coming up in my mind: the importance of connections at this conference. Over the past few days, the laws of probability have been strained around how often I’d run into colleagues and friends, including one person who (we managed to figure out) I had met at a conference 8 years ago. I’ve had conversations with people from around the world and even ran into a few non-Canadians who know where Saskatoon is. I’ve had the opportunity to learn from and be inspired by other independent consultants, and hopefully provide them with some ideas in return. And last but not least, I’ve had the opportunity to laugh and share some inside evaluator humour, including through the magic of Chris Lysy’s cartooning.

Over the coming days, I hope to produce a more content-focused roundup of the conference, but in the meantime, to everyone I’ve had the chance to discuss and learn from and share with, thank you.

AEA Conference – A Call

Given that this is my third such conference, I should no longer be surprised that Eval 2015 is an amazing, overwhelming, encouraging, and ultimately empowering event! Both of my workshops were great learning opportunities leaving me with lots of ideas to bring back home to Saskatoon, I’ve run into several friends and colleagues already, and I managed to push past the butterflies in my stomach to successfully deliver my Ignite presentation. Seriously, give that format (or its cousin, Pecha Kucha) a try sometime – a set number of slides that change automatically, it’s a rush to present!
AEA Opening Plenary

The theme of this year’s conference is Exemplary Evaluations in a Multicultural World, and if the number of flags on stage at the opening ceremony is any indication, it’s definitely a global audience. I was proud, though perhaps not too surprised, to see that Canadians make up the largest non-American contingent at 139. And yet we don’t have our own badge ribbon? Next conference I should bring some maple leaf stickers.

I was heartened to see the opening ceremony beginning with a traditional welcome from a local Indigenous group. There’s a growing recognition in Canada and apparently the US as well of at least acknowledging the history of the land and people where we are meeting, and I hope this becomes the norm for events generally.

If there’s one take away from my experiences here thus far, it’s that we have the opportunity and the call to up our game as evaluators. Our field is gaining international recognition and legitimacy (2015 is the UN-recognized International Year of Evaluation), our skill sets are increasingly viewed as important and even necessary in creating sustainable social change, and at the same time we’re recognizing that we’re working in increasingly complicated and even complex situations. Our toolkit is growing to incorporate new ways of presenting data, engaging stakeholders, and demonstrating impact. We’re called to be exemplary.

Yes, it can be overwhelming and even a bit anxiety-provoking – who am I to take on these tasks and responsibilities? At the same time, it’s exhilarating to be here at this place and time, and to have this opportunity to learn and grow. I look forward to the rest of the conference, and more importantly, what I can and will do in the weeks and months to come to create a better world.