We know that it’s one thing to deliver waste education and it’s quite another thing to design a behaviour change intervention, gather data throughout its delivery and evaluate its effectiveness after the fact. This is what we set out to do for National Recycling Week 2020.
We ran a behaviour change experiment campaign to demonstrate the value of measurement and data and reviewing the effectiveness of the campaign to gather insights to shape future behaviour change efforts.
The behaviour change campaign.
We developed our campaign based on the Planet Ark APCO CheckIt! Before You Chuck It. program that is designed to get Australian consumers checking for the Australasian Recycling Label on packaging and following it’s instructions. The program aims to reduce yellow bin contamination and the amount of recyclables going to landfill through better source separation behaviours in the home.
In order to use the ARL and accurately source separate according to the guidance, we identified precursor knowledge and behaviours that we focused our campaign around promoting:
- Ability to correctly identify the ARL and it’s three unique labels
- Ability to follow the conditionally recyclable guidance
- Establishment of a soft plastics recycling bin in the home
- Establishment of a system for dropping off soft plastics for recycling
- Remembering to check for the ARL before placing a packaging item in the bin.
What we did.
During the week prior to NRW we promoted the upcoming behaviour change experiment primarily through Faceboook and Instagram and also through our monthly Resource Hub newsletter. We encouraged our audience to sign up for our NRW e-newsletter and complete our pre experiment (baseline) survey with the chance to win a prize. We also published and promoted a blog explaining the purpose of the experiment.
During NRW we delivered a digital campaign primarily through Instagram, Facebook and an opt-in NRW e-newsletter. We created content building upon the existing Planet Ark CheckIt! toolkit resources and added our Resource Hub touch. Our core campaign included three posts (1 x boosted), three 30sec CheckIt! animations, five 1-3min explainer videos, one 20min Facebook Live Q&A and one call-to-action CheckIt! Tuesday competition.
In the week following NRW, we promoted the winners of our two prizes, invited our audience to complete a post experiment survey (with the reward of free bin posters) and hinted about our data findings.
In addition to our digital campaign, we wanted to get some real data on whether the campaign had an impact on actual waste outcomes – mixed recycling contamination levels and the amount of recyclables in the waste bin. We engaged two households who vounteered to follow the campaign and let us check out their trash before and after NRW.
We collected qualitative and quantitaive data through three main channels:
- Digital engagement data for our e-newsletter, Instagram, Facebook and LinkedIn campaign activities.
- Online surveys to give us insight into perceptions and self-reported behaviours before and after the campaign.
- Waste assessment data from two volunteer households.
We reviewed both the qualitative and quantitative data to understand:
- How engaging the campaign was and how much participation we created
- Perceptions of our audience towards wasting less and recycling
- The effectiveness of different types of content
- Impact on targeted behaviours
- Impact on perceptions about recycling
- Impact on knowledge about the ARL
Our top 5 discoveries.
- Qualitative and quantitaive data are useful in unique ways.
When it comes to humans and our behaviours, there are so many assumptions around the numbers because peoples behaviours are influenced and driven by so many complex factors. We found the quantitative data brought to light more questions as we came up with more and more theories about what the data indicated about our audience.
The quantitative data was excellent for validating (and invalidating) our assumptions. We checked our hypotheses about, (1) what types of content were effective at creating engagement (socials and email data), (2) existing and changed on waste behaviours and perceptions (self-reported survey data), and (3) how changed behaviours would impact what ended up in which bin (households audit).
It was the qualitative data that provided the most illuminating insights with a richness and depth that wasn’t available from the numbers and statistics. It was the conversations and responses we had with people during NRW about the campaign and the ARL (both in person and online) that revealed where people’s knowledge gaps and perceptions actually were with regards to source separation of waste and recycling in the home. It was also through sorting the waste and recycling of two Brisbane households that we could see what waste people are generating and how they sort, separate and manage it at home.
The insights gleaned from qualitative data provided great lessons to guide future behaviour change initiative design, and the greatest lessons came from the comparison of both – both the community engagement comments and conversation, alongside the measurement of respondent perceptions and behaviours.
All data is valuable – from the smallest comment to the most detailed survey response – and we should ensure that we design in a process for capturing, analysing and presenting this data effectively in our behaviour change programs to guide adjustment and focus – whether than data relates to clear numbers/checks or words/feedback.
- Our audience was a waste conscious audience.
Our quantitative survey data confirmed that the audience we engaged in the campaign was indeed more waste conscious than the average Australian. As expected, our audience was already predisposed to be interested in the kind of information that we were presenting about recycling. This was reflected in the comments and feedback received where people were asking for more detail about items recyclability and the why behind the ARL guidance. We also saw this reflected in the high engagement levels on our longer (1-3min) explainer videos and our 20min Live Q&A on Facebook. Participants clearly wanted to learn!
From the perspective of ‘solving Australia’s waste and recycling issues’ it’s worth noting that it’s often the less waste conscious audience that are the cause of contamination issues and challenges for our waste and recycling industry. Thus, it’s important to acknowledge that this type of campaign is not necessarily going to reach the not waste conscious audience because of it’s opt-in nature. We attracted people who already think wasting less and recycling well is important.
- Recycling is confusing and the campaign didn’t significantly reduce confusion.
Our hypotheses that recycling is confusing and even waste conscious people are unsure about which bin to choose was shown to be accurate with most people being unsure about whether an item can go in the mixed recycling bin some of the time in our baseline survey. While, the ARL claims to take the confusion out of recycling, in the post campaign survey we didn’t see any significant reduction in respondents confusion.
Therefore, it is important that we, as an industry, continue to work on reducing confusion around recycling and providing good and sufficiently detailed information for the waste conscious community.
Following on from this, we utilised the data we gathered to confirm which items and behaviours participants were confused about and which were they confident about. Responses to specific content – in particular the Grace’s ‘explainer videos’ – indicated that our predictions about the topics, behaviours and items causing confusion were accurate and we got great engagement with these videos. We also now know that having a real person, presenting an item and talking through the “what and why” of recyclability provides more clarity for a waste conscious audience and is more likely to create the desired behaviour than a generic animated video.
- There is still some misunderstanding about recycling labelling.
While all respondents to our post survey followed our CheckIt! campaign, 50% of these correctly identified the ARL. 25% of respondents incorrectly identified other logos as the ARL – specifically the plastic resin identification code label and the 10c containers for change label. This shows that despite following our CheckIt! campaign some respondents were still not clear on which labels are the ARL. This may explain why people still feel confused despite looking for the ARL. Compared to these other recycling logos, the ARL is the only one that provides detailed guidance on what can be placed in Australian household mixed recycling bins. Only the ARL goes into component detail and takes into account the actual technical requirements of Australia’s recycling systems.
- The ARL isn’t the perfect tool that we thought it was and can’t be completely relied on to remove recycling confusion.
We were excited to promote the logo as the place to go for accurate and clear recycling information upon which people could rely on and trust. We had hoped an assumed that the ARL could be relied upon as a point of truth for recyclability that our audience could use not only to tell them which bin to put it in but also use the detailed information to form an understanding of how a different materials and items need to be treated to ensure their recyclability.
It was thus frustrating to find that, while the ARL is a world leading labelling system, I found a number of instances where the ARL on an item contradicted our standard recycling guidance or surprised or perplexed ME as a waste educator with a solid understanding of waste and recycling processes.
Here are some examples:
- Soft plastic recyclable items displaying the Not Recyclable label. This is likely due to these items being assessed for the ARL prior to the introduction of the Return to Store / Store Drop Off conditionally recyclable label.
- Identical packaging items across different brands, such as plastic mesh produce bags, showing different ARL guidance.
- Inconsistent messaging about tops and lids on bottles.
This is absolutely not to say that the ARL is wrong or poorly designed. We endorse the ARL as a helpful tool to create better bin behaviours and source separation by the average Australian. However, for the waste conscious Australian, the ARL should not be relied upon as the single source of truth for recyclability. For this audience, they still will benefit from an understanding of waste and recycling processes (the why behind the label) so that they can make more informed decisions about every item they go to throw away. This was clearly indicated to us through the high levels of engagement we had on the explainer videos and Live Q&A with Grace which dove into these details.
The key takeaway here is that the CheckIt! behaviour focus was less value for our audience than the discussion and learnings generated through the explanations of the Conditionally Recyclable labels. It’s like our audience was saying to us “Okay we know how to read and follow the Recyclable and the Not Recyclable labels. What we really want to know is the why behind the Conditionally Recyclable labels.”
The CheckIt! campaign and the ARL is great for the average Australian and our audience isn’t the average Australian. Knowing this about our audience indicates that next time, we would get strong engagement through targeting higher level waste conscious behaviours such as soft plastics recycling, home composting and utilising community drop off schemes for specific items like medicines and mobile phones.
For the delivery of a more comprehensive CheckIt! campaign over a longer timeframe, targeted at a broader (and less waste conscious) Australian consumer audience, we would offer the following tips:
- Get clear on your behaviours. Before you start designing your campaign identify: 1) what new behaviours you want to create, 2) who needs to do them, and 3) where they need to do them. Consider precursor behaviours. Within the complex wicked world of improving waste outcomes this will help you focus and increase your likelihood of getting the outcome you’re looking for.
- Establish measures of success and data collection methodologies. Data is important so ensure you have a way to establish if you’ve created the desired behaviours, waste outcomes or even perception shifts. Decide what kind of data will give you most value, find the easiest way to collect it and have a plan for how you’re going to analyse it. Maybe you bring in someone with great data analytics skills like we did for this!
- Choose your content wisely. Reliance on external content was very helpful and allowed us to build upon quality content rather than ‘reinventing the wheel’. However, assessing the relevance of external content for your particular audience, target behaviours and waste and recycling systems is important. Identify any content that needs expansion, exclusion or clarification before you roll it out. We don’t want to be adding to the recycling confusion!
- Include plenty of ‘explainer videos’. Use ‘real people’ to explain key scenarios, what to do and why this is helpful to the recycling process. Real world examples had improved interaction with measured change in specific behaviours.
- Include the ARL content (animations and graphics). These increased brand and ARL logo awareness and are well designed and very engaging.
- Consider utilising competitions and free giveaways. Free stuff generates ‘reach’ which was helpful to get people paying attention to the campaign particularly early on. Significantly more people saw our campaign because of free bin posters and a competition for a free zero waste container pack – we got over 30 times the audience for giveaway content!
- Track engagement during the program and adjust. We lost 40% of audience along the journey. Identifying the reasons behind audience reduction are important, and on a longer campaign we would target this information early in the program and adjust to retain and reengage our audience.