Today, with the help of a few guest experts, we’re responding to a subscriber participation dilemma. It’s an issue that has been put forward by several subscribers to our e-newsletter the Discussion as their ‘biggest community engagement challenge’.
Today’s challenge
Representative views: how to ensure you’re hearing from people that are proportional to and reflective of the wider community or stakeholders you’re trying to engage with.
It’s a great topic to tackle. Representativeness is an important factor in engagement. A representative process is more likely to include diverse voices and ideas, build trust in and support for the decision, and result in an outcome that addresses the needs of the community affected.
There are ways you can help to increase the representativeness of a process. For example, you can include and encourage diverse perspectives –we recently wrote about this in our blog. However, the most effective way to achieve a descriptively representative spread of participants is to use stratified random selection.
As facilitators, we regularly work with randomly selected groups. However, this type of recruitment is a specialised field that is managed independently. To assist with transparency and building trust, participants are not known to facilitators or the host organisation until the group walks into the room for the first session.
So, we’ve interviewed some of the best in the business for you. Kyle Redman - Project Manager, the newDemocracy Foundation and Nivek Thompson – Director, Deliberately Engaging have kindly offered up their expert knowledge. Delve in and discover fascinating insights into how it works, the benefits that can be achieved, and some of the key factors involved in selection success.
Why invest in a random, stratified selection process? How does it make the process more representative?
Kyle:
newDemocracy (nDF) have found that relate-able everyday people standing behind tough decisions improves public trust. Stratified random selection allows you to get beyond the loudest and most articulate voices on either side of a tough issue. Hearing from people who are unfamiliar or otherwise previously uninformed about an issue can introduce fresh perspectives and get beyond the binary nature of right and wrong to find a common ground solution.
Nivek:
This selection method helps to bring a more diverse group into the room. Diverse participants bring a range of backgrounds, life experiences and perspectives with them and that gives you a rich conversation, new ways of thinking, creative ideas and enhanced outcomes. This is a result of a number of factors including:
stratification, which helps to target people you wouldn’t usually hear from,
people are paid a nominal amount to cover their costs, which reduces socioeconomic barriers to participating (like childcare and travel),
people who have never been actively involved in civic issues before say yes because they were directly invited to take part, and feel they can truly add value or have an impact.
How representative can a group really be when selected via this method?
Kyle:
This all depends on what type of representation the process is trying to capture. We prioritise a form of descriptive representation that focuses on diversity of people in the room: the test is to see people from all walks of life without visible skews (i.e. to a certain occupation or age group).
In terms of how ‘representative’ the selection process can be – this relies on the quality of the pool you are drawing from and the precision of your stratification criteria. A large, initial sample (e.g. using all addresses in an area, or the electoral roll) makes it easier to be precise in your criteria because it dilutes any possible skew. The less people you are randomly drawing from the broader your criteria should be. (See also Kyle’s response to the next question below for an explanation of stratification criteria.)
It is important to realise that within a mini-public process, representation is not limited to the small group of participants in the room. The information sources that feed into the process ought to seek out discursive diversity to ensure the group hears from a range of differing view-points throughout the process. Ultimately, it is important to think of a mini-public process as a whole rather than focusing on the selection process when it comes to types of representation in deliberative democracy.
Nivek:
The selected group won’t be perfectly statistically representative of the wider community - they will be descriptively representative - and much more representative of the wider public than a self-selected group of participants.
Other voices can and usually are also represented in the process. There is an important place for technical experts, key stakeholders, those that have an active interest in the issue and others that you would normally hear from in these processes. These contributors need to be engaged through other methods, and this input should be captured. It generally feeds into the deliberative discussion by the randomly selected group.
Explain ‘stratification’ measures: How are they chosen and is there a reason behind the number or type of filters you select?
Kyle:
Stratification is a balancing act. We balance how precise the stratification measures are with the level of possible ‘self-selection bias’ in each stratification pool. For example, if we have a narrow (very specific) selection criteria, and only 5 people in that demographic register interest, our ultimate selection is skewed towards those 5 people. Alternatively, if 25 people in that demographic register their interest, each of those people are 5 times less likely to be selected – therefore we reduce the skew towards their self-selection, improve the randomness and reduce bias in the selection process.
Strong, broad criteria such as gender, age and geographic location get a larger pool of diverse people. Over the years, we have learnt that other filters such as tenancy (own/rent) or level of education are good broad indicators of socio-economic status that can be used to avoid asking for specifics around a person’s income.
The more personal the questions on the recruitment form, the less likely someone is to complete it. Which means we must strike a balance between rich (precise, specific) stratification measures and a deep pool to choose from.
Nivek:
Stratification is about putting ‘filters’ on your selection process. These filters are often measures such as age, gender, geographic location, and owner occupier v. tenant. Randomly selected households or individuals are sent an invitation to register their interest, and then the final group is selected from the pool of people who register. They are selected to match (as closely as possible) the public they are representing. The percentages the group needs to meet are generally measured against demographic data from the Census.
What are the key success factors when conducting a stratified random selection process?
Kyle:
we use Australia Post databases of all addresses to ensure minimal bias,
we do not put names on the invitation – this ensures we capture renters/tenants and decreases risk of having recipients send invites back via 'return to sender'.
the question being asked is stated in plain English,
we clearly express the level of authority (level of influence) the group will have over the decision,
we make it clear participants will be paid,
we determine our stratification criteria in a way that balances the precision of the descriptive representation with our ability to accurately recruit within a specific sample,
we send out a lot of invitations to minimise the inherent self-selection skew within the selection pool.
Nivek:
Some crucial aspects include:
How many people register their interest after receiving the initial, random invitation. The bigger the pool of people who register, the more diverse the pool of people to randomly sample from, and the easier it is to meet stratification targets.
How attractive, accessible and easy the opportunity appears to invitees. There’s important thinking around communication, invitations, and the how people are informed about the issue and decision to be made.
Does the host organisation play a role in the selection process?
Kyle:
newDemocracy entirely manages the selection and recruitment process. This means we set the stratification criteria, design and post invitations, and do the online RSVP and random selection process.
Our interest is in proving a point regarding the value of including randomly-selected everyday people, we would lose that by ‘tricking up’ a recruitment to satisfy a government. Our philanthropic support means we do not have a billing pressure, while our bi-partisan backing from former Premiers Geoff Gallop and Nick Greiner assuages concerns that the process/recruitment is geared to help a given side of politics.
The organisation should be separate from the selection and recruitment process because of the fundamental need for independence. Any perception that the organisation was a part of picking and choosing who was a part of the process would contaminate trust in the process, even if done purely and with honest intentions.
Nivek:
The host organisation identifies the relevant demographics and other representation goals to be used for the stratified random sample. Sometimes the topic being considered supports additional representation goals such whether the person is employed or studying where the topic relates to workforce participation. However the host organisation does not have a say in who gets selected.
What else should we keep in mind when it comes to hearing diverse voices and representative viewpoints?
Kyle:
Including broad stakeholder input early in the process ensure strong view-points have direct controlled, input into the process.
It is also important to remember that the community will not always have strongly held views on a topic until they become reasonably informed – some might even change their mind. The strength of a mini-public is in the demonstrated difference between someone’s 5-10 minute opinion and their 40-50 hour judgment.
Nivek:
I often hear organisations say they are worried that no one will register to get involved in such an in-depth process. Of course, not everyone who is invited responds. However, it is astounding how many people, who you would never otherwise have heard from, register their interest.
There are different variations and models of random selection – e.g. mix of random selection and call out for volunteers via area of interest, or a mix of randomly selected panellists mixed with targeted stakeholders. This recruitment method can be weaved into different type of processes, and while they generally are associated with a deliberative process, they can also be used in other engagement approaches.
Finally, the people selected via this type of process aren’t activitists or professional representatives and so they approach the process with a fresh outlook. Professional design and facilitation which recognises the differences between this type of process and a public meeting is important.
FURTHER READING
Nivek has provided the following links to episodes of her podcast Real Democracy Now! that delve deeper into this topic:
Hear from participants in deliberative process from around the world
What are deliberative mini-publics?
Nivek also published ‘Participatory budgeting – the Australian way’ in the Journal of Public Deliberation. The article includes insights from research and anecdotal evidence from talking to randomly selected participants and others who were part of the Canada Bay Citizens’ Panel.
Dr. Lyn Carson, Research Director, the newDemocracy Foundation, has prepared a research and development note: Sample Size for Mini-Publics that draws lessons from newDemocracy’s experiences operating various citizens’ juries in Australia including, the South Australia Nuclear Fuel Cycle Citizens’ Jury, and the Australian Citizens’ Parliament.