Tuesday Seeds for Thought – Home

Last week saw me in the Big Smoke for a family event. It was great to reconnect with relatives and friends, but as always with travel, it’s nice to come back home. This week’s Seeds (loosely) follows that theme of home, whether it falls under developments in my geographic neighbourhood or what constitutes a professional identity “home”.

From Last Week

I had hoped to get last week’s Seeds written and posted before I left, but that didn’t happen. However, I did submit a conference proposal with my friend and colleague Chi Yan Lam. If accepted, we’ll be co-hosting a “Birds of a Feather” gathering at this fall’s American Evaluation Association conference to bring together people interested in developmental evaluation. We’re not calling it a Community of Practice ( … yet) but hopefully we can help connect practitioners who may be otherwise isolated in promoting this approach.

Around the Web

  • I have considered the non-profit sector to be my home for some time, even though I’m technically on the for-profit side now (with a social purpose, but still a business). Gordon Brown provides his take on What You Need To Know About Working For A Non-Profit – I don’t agree with everything he wrote, but it’s a good overview.
  • During last week’s trip I had a conversation with my sister-in-law about experimental designs, and the next day came across this Chris Lysy cartoon set on the closely-related Randomized Controlled Trial. Given that I “grew up”, academically speaking, in traditional research psychology, I have seen how experimental designs can be elevated as the best (if not only) way to conduct research: however, my understanding of what constitutes knowledge and ways to understand the world has expanded since then. That being said, feel free to call me out if I ever call someone a “randomista”.
  • Helping people understand research and evaluation is often about finding metaphors that resonate. Charles Gasper at the Evaluation Evangelist uses the analogy of picking your March Madness bracket: I’m not a basketball fan, but hey, if it helps you understand evaluation, I’m all for it!

Around Toon Town

  • Leadership Saskatoon is holding a Lunch and Learn session tomorrow (Wednesday) on the topic of innovation in organizations. The session takes place at the Saskatchewan Abilities Council site on Kilburn starting at 12:05 and participants are asked to RSVP.
  • Leadership Saskatoon is also accepting nominations for their Community Leadership award (due April 1) and applications for next year’s leadership development program (due June 10, but as there’s usually a wait list so best to submit ASAP!) – more information on their website.
  • Next Wednesday (April 2), catch the Fuze Conference on marketing and communication, which includes some great speakers on community engagement.
  • Shameless plug: the Saskatoon Community Band (I play trombone in their wind ensemble) is welcoming the arrival of spring with a concert next Wednesday at 7:30 at the Broadway Theatre. Tickets are $10 ($5 for students) – contact me if you’re in Saskatoon and interested!

Feeling at home with this format, or ready to pack up and move on? Either way, let me know through comments below, on Twitter, or the standard contact means.

Seeds for Thought: Volunteer Retention

Volunteers are an important piece of the puzzle for any non-profit organization. Whether they’re contributing to programs and special events, helping out with fundraising and outreach, or providing guidance and leadership as members of the board, good volunteers are indispensable. As these individuals are giving their time and effort without compensation (at least of the financial kind), organizations are increasingly recognizing that they can’t take these superstars for granted.

Along those lines, this week’s entry in the Seeds for Thought category is a case study from the Stanford Social Innovation Review on volunteer retention for Girls Scouts of Northern California (GSNorCal). Like many other nonprofits focused on youth development, GSNorCal relies heavily on volunteers and as a result already uses many best practices in orientation, training, and recognition: however, broader changes within and outside of the organization has made it difficult to keep volunteers returning. In response, the organization hired a consultant, TCC Group, to “mine its data and pinpoint ways to keep volunteers engaged”. Through a survey of 1,371 current and past volunteers and follow-up focus groups, TCC Group identified factors that predicted volunteer retention and suggested improvements to GSNorCal’s practices.

This example demonstrates the value in using multiple sources of information, in this case quantitative data from a large survey, qualitative insights from small groups of volunteers, and general principles from scholarly research on the topic. If you don’t have the resources that GSNorCal (or even if you do) and want to learn more about your volunteers, what can you do?

  • Start by counting. How many volunteers do you currently have, how long have they been volunteering, how many new volunteers have come onboard recently and how many have left? How many hours are they contributing? Are there differences in these numbers based on demographic factors or what tasks they’re doing for your organization?
  • Use some simple questionnaires with both current and former volunteers. I could spend a full post or three on what a volunteer questionnaire could look like, but at the very least it should include questions around overall satisfaction, support from the organization (or lack thereof) and what keeps them volunteering and what makes them leave. Just remember to use a mix of question types and watch out for potentially misleading numbers.
  • Take a participatory approach. Include volunteers in the discussion, both long-time contributors and those who are new or in a temporary position, such as through a World Cafe: as a bonus, this approach can help improve retention by demonstrating to volunteers that their opinion is valued by the organization. Another idea – have a staff member step into the shoes of a volunteer for a shift to get a firsthand perspective!
  • Partner with organizations that can provide a broader view. Many cities have a volunteer centre (either standalone or part of a larger organization like the United Way) or a professional association of volunteer administrators such as PAVRO liaisons in Ontario that can link you with resources on volunteering and keep you in the loop about new developments in the field. Volunteerism is also becoming increasingly recognized as a topic of scholarly research, so look into partnerships with universities: programs related to community development, organizational studies, public policy, and even business are good starting points.
  • A bit of self-interest here: consultants can help! If resources are tight, use consulting expertise for specific tasks that may be impractical to do in-house, such as analyzing complex statistical data or acting as a neutral party to collect feedback (current and even former volunteers may be hesitant to provide criticism directly to staff). Volunteer management, especially as it relates to research and evaluation, is one of Strong Roots’ strengths, so drop us a line if you want to have a chat about how to learn more about your volunteers!

Question: What are some strategies that you have seen successfully used to engage volunteers and improve retention?

Seeds for Thought: Five and Change

This week’s seed features Chi Yan Lam, a friend and colleague who is completing his PhD in Education at Queen’s University. We share an interest in developmental and collaborative approaches to evaluation, though as you can see from his about page, he comes at it more from the academic and theoretical side.

In support of writing his dissertation, Chi recently relaunched his personal site as a process journal to “chronicle and archive [his] emerging thinking and serendipitous discoveries around evaluation and design”. A recent post brings up the idea of the Stanford $5 challenge, where students in the Technology Ventures program are asked to use $5 and two hours of time to make a profit. Those most successful didn’t end up using the money: that resource all too often turned out to be a trap, too little to turn into anything with taking a huge risk like buying a lottery ticket or hitting the slot machines.

This example really resonates with my experiences in the nonprofit field. The first question that’s usually raised after generating a new idea for a program or service is where will the money and resources come from: in response, many organizations will “gamble” staff time and resources on preparing a grant application. If the gamble doesn’t pay off, the idea is dead in the water, morale drops, and staff are discouraged from coming up with innovative solutions in the future.

Instead of focusing immediately on what we need for success, oftentimes we need to take a step back as Chi suggests and first determine the need for a program (or to borrow from the business world, whether the “market” is there), and then whether our theory of change (the steps from here to there) matches our plan of action. These two steps can help identify faulty assumptions or leaps of logic in your plan, but more importantly, they force you to question if there is a better path to success. For example, is it possible for the program to take advantage of existing in-house resources such as a spare room and some dedicated volunteers, or draw on connections with community partners such as a university community service-learning project? A successful program will at some point need dedicated resources, just as a successful business venture will need capital to go to scale: however, if an idea can show some initial successes on $5 and two hours of time, it’s an easier argument to make that investing more time and money will be worthwhile.

(A quick shameless self-promotion here – my approach to supporting project development takes a similar approach, working with organizations to better understand the need and context, clarify how the program will work, and identify potential resources. If you’re at this stage of a program design and not sure how to proceed, drop me a line!)

Question: Think about a cause or issue you’re passionate about – what would you do to start creating change with $5 and two hours?

Seeds for Thought: Misleading Numbers

Earlier this year, I wrote a post on simply counting as an easy way to start evaluating a program or initiative. Although this approach can provide some good insights, numbers can easily mislead based on the manner of collection or when viewed in isolation from the broader context, as this week’s two seeds for thought (both from the Harvard Business Review blog) demonstrate.

First up, Peter Kriss provided the example of a hotel chain that revamped their guest feedback website to make it easier to access on mobile devices. In contrast to expectations, overall satisfaction with the hotel took a nosedive in response, which on the surface didn’t make much sense: why would a better survey make people dislike their stay? The answer was that improving accessibility to the survey led to more responses from less-engaged guests, and since their impressions tended toward the midpoint (i.e. neither great nor poor), the addition of these datapoints led to a lower average. The lesson here? Whenever you change how participants interact with or give feedback to your program or organization, be prepared for unexpected results!

A time-honoured method of assessing impact involves taking a baseline measurement of participants before an intervention (“pre”) followed by a similar (if not identical) measurement afterwards (“post”): if there’s a difference in scores, you could claim with some degree of certainty that the program made a difference. Now I could probably write several posts about the pitfalls of pre-post measurement, but Ron Shaich’s article on growth illuminated one that I probably would have missed. In the cutthroat restaurant industry, Shaich discovered, you can’t assume zero growth as a baseline: because of the strong competition, a restaurant that does nothing will naturally lose customers.

Adapting this example to the non-profit world, imagine a service or program that aims to improve participants’ lives in some way (e.g. physical health, food security, access to housing) with a corresponding pre-post measurement. If the program is working in a community or with a population facing multiple challenges, the “norm” might be decline rather than stability: in the absence of the service, things may well get worse rather than stay the same. The good news in this scenario is that a lack of pre-post change therefore might not be a complete failure, but program planners may need to set their sights higher to create a positive change.

The general takeaway from these examples is that you shouldn’t blindly trust the numbers or read too much into what they could mean: instead, take some time to look at other factors that could explain what you’re seeing. Got examples of your own when the numbers were confusing or misleading? Share them below!

Seeds for Thought: The Case for Strategic Plans

Ever have the experience where you read a blog post or article online and immediately say “That captures my thoughts perfectly! Why didn’t I write that?”. This week the honour goes to my friend and colleague Natalie Brown Kivell of Common Thread Consulting, specifically her post making the case for the humble strategic plan. Contrasting the view of strategic planning as a quick fix or a pointless activity done solely to please funders, Natalie provides five solid reasons for why planning should be part of your organization’s lifecycle and how both the process and the final product can provide tangible benefits.

For me, the key takeaway from Natalie’s post is that the strategic plan should be co-created through a participatory process, working with your organization’s diverse stakeholders. That term, “stakeholders”, gets thrown around a lot in the social change sector to the point of cliche, but think for a minute about the root of the term. Everybody – program participants, staff, volunteers, leadership, partnering agencies, funders, the broader community – has a stake in the success of the organization and its efforts. Every stakeholder stands to benefit in some way, but they also risk the loss of time, energy, resources, or even hope if things don’t work out. Stakeholders also commit in some way to the project, whether in terms of providing tangible resources, participating in good faith, or by providing indirect support such as a community playing host to a social service. With these diverse groups all having a stake in the non-profit’s success, it makes sense for all of them to be involved right from the beginning in the planning process.

At the end of this article Natalie mentioned that she’ll be writing some future posts on strategic planning, so be sure to keep an eye on her site!

Seeds for Thought: Negative Results

Whether you are evaluator of a program or someone associated with the initiative being evaluted (the evaluatee?), it’s probably safe to say that everyone hopes for good results: proof that all the planning, effort, and resources that went into the program made a difference. Sadly, that doesn’t always happen, leaving the evaluator to figure out how to present the information accurately and constructively.

Susan Lilley recently compiled a ten-point list (PDF) on this topic, based on discussion from the American Evaluation Association’s listserv (Hat tip to the Better Evaluation blog which provides some more context on the discussion and some commentary on the points). To my eye, all the points are great – in particular, #4 (“Build in time for course correction”) and #8 (“Present results in terms of lessons learned”) provide a great rationale for a developmental evaluation approach that understands from the get-go that some components of any given project will need to be tweaked or changed wholesale in response to changing circumstances. What I really appreciate about this list, however, is the very first point – “Use a participatory approach from the start”. Engaging stakeholders and working as a partner with clients are more than “feel-good” tactics: they help create a sense of ownership of the results and builds that crucial relationship that allows for sharing both good and bad news, as well as having a frank discussion as to what the results mean for future work.

What tip do you think is most crucial for sharing negative results? If you have been on the giving or receiving end of bad evaluation news, what helped turn the episode into something constructive? Share below!

Seeds for Thought: Noble Arsonist

If you follow Strong Roots’ Twitter feed, you may have noticed that I attended the MoSo conference last week here in Saskatoon. It was my first time attending this particular conference but hopefully it won’t be my last, with some great sessions and resources that were relevant to me as both a social entrepreneur and someone connected with the non-profit sector. I live-tweeted from most of the sessions and keynotes that I sat in on, so if you want to see what I took away check out my personal account around June 13-14 for ideas and insights.

Out of all the presentations, it was the very last one I caught that social change agents would probably find the most interesting. Theodora Lamb from Vancouver-based Capulet Communications specializes in social marketing for non-profits: her talk, “The Noble Arsonist: Tips for stoking fires and igniting movements for NGOs (and companies that care)” provided some great seeds for thought on the use of social media, particularly around creating “remarkable” campaigns. In contrast to the regular “heartbeat” relationship with a core audience that keeps them engaged and informed, remarkable campaigns are something out of the ordinary that makes your group stand out and get people talking who normally wouldn’t be interested.

Lamb provided several ideas and examples of how to run a successful remarkable campaign, all of which is covered in her free ebook, The Noble Arsonist. One insight that stuck with me was the benefit of using humour. Given that most (if not all) issues that non-profits and like-minded folk take on are quite serious, humour may seem inappropriate: however, “serious” can easily tip over into “negative” and discourage people from getting involved, whereas some light humour can facilitate conversation and foster broader awareness. This approach is also useful for criticizing a popular brand or figure while avoiding an us-vs-them battle: check out the e-book’s description of Greenpeace UK’s Dark Side campaign against Volkswagon as a prime example.

Over to you now – how does your organization initiative use social media? What worked well and what fizzled? Have you been involved with any remarkable campaigns, either as an organizer or supporter?

Seeds for Thought: Scale

Today’s Seed for Thought comes from the Stanford Social Innovation Review’s blog, a site that covers (as the name suggests) social innovation and related conepts like philanthropy, social entrepreneurship, and nonprofit organizational development. In the latter category, an article today provided a five-question checklist for nonprofits to assess their readiness to scale and increase their impact. The second question asked whether your program model has been tested: according to a survey of American nonprofits, “only 39 percent of nonprofits that are scaling or intending to scale have evaluated the impact of their work”. To me, that’s a surprising result – in my mind, before growing a program or initiative you should take some time to make sure it’s actually achieving the results that you think it is!

Although I’m glad that evaluation is included in the list, I think there’s a danger that evaluation and research is relegated to a one-time “check it off the list” task. In scaling a program to new communities or populations, organizations are bound to run into unexpected challenges. Elements and approaches that were beneficial in the initial program may be less useful or even detrimental in new situations. One example from my own experience was with an educational support program that had its roots in a dense urban core and was being scaled to other smaller cities. The new site I was involved with was very different from the original program site in terms of geography, history, and demographics: for example, the original site was very ethnically diverse, while the families in the catchment area for the new site were primarily white and had lived in Canada for multiple generations. As a result, our new site did not have to do much work around English as a Second/Additional Language, but we did face unique challenges such as around parent and family engagement. Collecting and analyzing information about our neighbourhood, both government sources like the Census and on-the-ground knowledge from teachers, service providers, and community members, helped us to understand the context and respond appropriately.

Funds, resources, and organizational practices are important elements to consider when scaling up: at the same time, nonprofits need the capacity to recognize the changes that come with growth and adapt accordingly. One tool that can be helpful in this case is developmental evaluation, which as recognized in Michael Quinn Patton’s handbook on the subject, can help organizations identify effective principles from earlier work and determine when it’s better to adapt to local conditions rather than adhere to acontextual “best practices”. By integrating relevant and timely data collection and sense-making into the process, developmental evaluation can help nonprofits learn more about the new situations they are entering, avoid potential pitfalls, and successfully scale.

What else would you add to the checklist?

Seeds for Thought: Attributing Laughs

“I don’t care about the details, just tell me what impact you had!” The impact of interventions that deal with complex social issues often can’t be boiled down to a simple yay/nay vote, especially when examining longer-term outcomes: unfortunately, explaining that fact to stakeholders and funders without seeming evasive can be difficult.

Chris Lysy over at Fresh Spectrum has penned five humourous illustrations on this difficult topic, which I’m keeping in my back pocket the next time I have a discussion with anyone about attributing impact. I find the idea of a “logic model repair shop” (#3) to be hilarious (and I’ve seen models that look that complex!), and the image of someone asking a whole community to stop the good work they’re doing to avoid messing up his or her impact assessment raises a good point that nothing we do happens in isolation. That being said, I think my favourite is the quote from John Mayne: “We need to accept the fact that what we are doing is measuring with the aim of reducing the uncertainty about the contribution made, not proving the contribution made.”

What’s your favourite out of the five?

Seeds for Thought: Big Data

During the span of a week, I come across lots of interesting stories, resources, and sites online that may be of interest to those in the non-profit-sector. In line with my approach of connecting people with resources and sharing information, I’m thinking about starting a weekly feature to highlight some of those links – consider this the pilot edition!

This week, I’m highlighting a trio of posts from the Harvard Business Review’s Blog Network, a site I recently started following. Although the focus is primarily on for-profit organizations, I’ve already seen content on social enterprises, philanthropy, and international development, as well as resources and trends that would be equally applicable on the non-profit side.

All three articles below relate to managing and using data, particularly “Big Data”. The term recognizes that collectively we are producing and storing exponentially-greater amounts of data in recent years than at any other point in human history – the first article cites research that 90% of data currently in existence was created in the past two years! This explosion in information can help grow our understanding of practically every facet of life, but there are challenges in analyzing and interpretating these giant data sources as well as limits to how much we can learn from them.

  • Jeff Bladt and Bob Filbin’s article title says it all – A Data Scientist’s Real Job: Storytelling. It’s similar to a truism I learned from a great professor during my undergraduate education, that all research projects have to tell a story: we start at some point of knowledge, we run an experiment or collect some information, and we learn something as a result. Tables of numbers and statistical tests are essential tools, but by themselves they do not advance our knowledge. As Bladt and Filbin put it, “Data gives you the what, but humans know the why“.
  • Presenting data in an accurate, easily-comprehensible visual form has become a field in its own right. If you’re not sure where to start in sharing information, Nancy Durante gives a simple suggestion: When Presenting Your Data, Get to the Point Fast. Check her post for some good tips on how to help your audience focus on the key numbers (hint: tables of numbers and pie charts are not in the cards!).
  • Finally, Kate Crawford explores The Hidden Biases in Big Data. Even databases with millions of records may not cover the full spectrum of a phenomenon: Crawford gives the example of the 20 million tweets generated during Hurricane Sandy, the majority of which came from tech-connected Manhattan compared to harder-hit neighbourhoods. Her prescription? “Take a page from social scientists”: pay attention to where the data comes from, examine your cognitive biases in interpreting the data, and utilize a diverse range of methods including qualitative approaches like interviews to complement the quanatitative data findings.

If you have any thoughts or additional links to share on this topic, I’d love to see them! You can use the comments field below or find me on Twitter. Also, any feedback or suggestions on this approach of weekly annotated links would be greatly appreciated.