Delib's 3 quick start articles on data analysis in Citizen Space
Tips for creating a report
After having prepared well and done some quality analysis of your responses, it’s now time to report them to your top brass and the general public! For more on how to do this, look at the below and have a read of our blog about producing a great report here.
Who is your audience?
Two of the most important rules when you are preparing a report on your data (and is valid for lots of other tasks) is:
1. Who will be reading the report.
2. What the information will be used for.
If you are making two reports – one for an external audience and one for an internal group – you will probably want to give them slightly different content and tailor the presentation accordingly.
- An audience of lay members of the public may prefer a short, highly visualised report that summarises the key findings. They may also expect some information in the report about what the next steps for your organisation will be.
- The policy team in your organisation who will potentially be acting on the results of your consultation will probably want to see much more detail, some interpretation that builds on the basic findings of the consultation, and will be less concerned about presentation
- Senior members of your organisation will probably want to see a short summary of high-level findings, as you might produce for the public - but may prefer it to be presented in a simple and informative way, rather than infographics and engaging visualisations.
These are, of course, not hard and fast rules - you know your audience much better than we do!
There’s always a temptation to stick to the traditional formats to present a report of your data. We’d encourage you to be creative. Would an alternative method of reporting back be more applicable? This doesn’t only apply to the visual presentation of your report. Particularly if your consultation gathered a lot of qualitative data, there is potential to be creative in terms of conveying the ‘story’ that the data tells.
If you want to be more creative - why does the information produced need to be fed back in a written report? Why not produce a video of the key outcomes, or a presentation. Alternatively a live-discussion, debate or web-based report may be useful. Choose the best method of presentation based on the most applicable context for you.
Hans Rosling is a statistician who makes data come alive in his engaging presentations using infographics and interactive graphs - his 20 min TED talk is fascinating and excellent food for thought.
Be careful of context
When you are reporting, it’s important to be aware of the context of the data you have gathered. A common complaint when the government, politicians, retailers and the media misuse statistics is that they report statistical findings stripped of their context.
- A survey (carried out by a shampoo manufacturer) asks 1000 mothers in the UK about their level of satisfaction with a children’s shampoo produced by the same company. They ask ‘what difference has using our product made to your child’s hair?’, with the response options ‘it has improved a lot’, ‘it has improved somewhat’, or ‘no change’. 600 of the women polled reply; of those, 450 respond with one of the first two options. This leads to the company branding their product with:
‘Shampoo X: recommended by 75% of British mums!’
There is some truth in this statement - but few would contest that it does not tell the whole story of the data that has been gathered.
There are a few things to be aware of when reporting, to make sure you are painting an accurate picture
Sample size: Make sure you are clear when using percentages that you refer to ‘X % of respondents’ rather than ‘% of members of the public’. If your sample is particularly small (eg. out of a target audience of 5000 people, only 50 answer your question), that should definitely be noted when you are reporting. Instead, use numbers rather than percentages when you report your findings. For example, if you received 25 questionnaires back, say that ‘20 students felt that...’ rather than ‘80% of students felt that...’
Representation: If you are running a survey on a planning issue that affects a whole district, but you only get responses from a few streets, you may have a lot of responses, but they are not quite representative! You should seek to avoid this happening when you design and market your survey - but if it does, you should be clear about it in your reporting.
Response rate: Remember that in the case of optional questions - or those required questions with a 'no comment' option - respondents failing to answer does not mean they void their responses! When a question gives the option of yes, no, or don't know, they don't know's are just as important to work into your analysis.
Built-in features for publishing
If you want to report on the results of your consultation to the public, you can use Citizen Space’s response publishing feature – this is a feature of Citizen Space available on request, it’s not automatically turned on as it's not needed by all users, so please let us know if you’d like to use it.
Response publishing simply allows you to make responses visible to other users, so it’s not entirely accurate to call it analysis – but if you simply want to make the data you have gathered available to the public, it is a good way of doing that. You will need to have a consent question in your survey asking if respondents mind having their data published, and you’ll also be able to moderate responses before they get published, and so remove parts of quantitative responses that it might not be appropriate to publish.
Results Publication/We Asked, You Said, We did
Citizen Space has a built in feature to publish results. This is a convenient way of posting the results of the consultation, and any relevant analysis, on your consultation’s Citizen Space page. This article explains how to do it – you can enter text/images, embed links and upload files, which could include, for example, a pdf showing some analysis.
The ‘We Asked, You Said, We Did’ feature is another way of doing this – it gives you a template in which to enter text in a way that focuses on the three steps in the consultation cycle that the title refers to – the question, the public response, and the ensuing action.
PDF Summary report
Citizen Space can also automatically generate a PDF summary report for you at any point during the consultation. This report only contains the ‘raw data’ – so won’t really include any analysis. It will contain only quantitative answers; no responses to qualitative questions except for how many responses were received for the question will be included. However, it can be a useful first step in terms of getting a summary, (funnily enough!), and for sharing that around your organisation.
How to present your findings
Make information easy to read
There are a lot of ways to visualise your data, and it can make things a lot more interesting than looking at rows of text and numbers. There's an old saying that "a picture is worth a thousand words" and it could equally be said that "a picture is worth a thousand numbers". Media organisations like the BBC and the Guardian are very good at turning data into visually interesting representations - we would recommend having a look at both of their websites for some inspiration.
Whilst it may be beyond your capability to create custom infographics, Excel has enough functionality and customisability to turn a chart of values into an interesting bar chart, line graph or pie chart - and often these are the most straightforward tools in any case. You should remember, however, to always name your source on a graphic - and also, if possible, make the raw data available to your reader by linking to your original Excel/Word file.
There are some very useful tutorials online, such as this one: annkemery.com/dataviz-design-process.
No matter which type of report you use, always remember that information can be more powerfully displayed in a graphic format verses a text or tabular representation. Often, trends and patterns are more obvious and recommendations more effective when presented visually. Ideally, when making a comparison of one or more groups of respondents, it is best to show a chart of each groups responses side-by-side. This side-by-side comparison allows your audience to quickly see the differences you are highlighting and can lead to more support for your conclusions.
When you are putting data in tables, make sure they are simple and easy to read.
As we mentioned earlier in the section on context - look at how many responses you have! If you only have a small number, (for example, under twenty), it may not be worth spending a lot of time trying to convert this data into exciting visualisations. A pie chart that shows a 33% split between three options is not very useful if there were only three responses. In these cases may be better to refer specifically to the number of responses.
Labelling charts clearly, including counts of responses, the base number and the number of missing responses will help ensure the information is understandable - frequency tables can be effective at displaying information in this way. These tables show the possible responses, the total number of respondents for each part, and the percentages of respondents who selected each answer.
Give your report a clear structure
Choose how to break your report down. Producing a report on a per-question basis can be a simple way of presenting all of the information.
Present the key findings and then make an overall recommendation which is clear. A rough guide on how to structure your consultation might be:
- Make recommendations and include key info - perhaps producing some key best-practice standards in your organisation.
- Put a summary of what the report says on the front page or equivalent.
- Use the main themes as sub-headings.
- Finish with conclusions, and restate your recommendations.
- Have a methods section - this might be best located at the end. Did you just use an online survey or were focus groups also conducted? It’s also worth mentioning any events or engagement exercises here. Why were these methods used in this context?
- Appendices - you can include supporting data and references here.