The scientist versus the spin doctor

This session introduces students to a statistical report. This learning could be carried out as part of your literacy programme.

1.

Introduce the students to a questionable statistical report. You may have your own resources to draw upon, but you could also use this report on water quality.

2.

Have the students answer the following:

• Who is responsible for preparing this report?
• Who might use this report?
• What are the main findings of this report?

3.

Call the students back together and discuss their impressions of the report and responses to the three questions posed. Emphasise the responsibility of the writers of the report in relaying accurate information and how damaging misleading information might be.

4.

Play the podcast of the interview from Radio NZ where a scientist questions the validity of the report, from 0:00 to 2:48. While listening to these first few minutes, note the words:

• most
• time period
• stable

5.

Ask the students to explain what the scientist's concerns were. Draw attention to concerns over the terms most (when actually only 50 out of 400 waterways or 12% were involved), the time period (ten years was considered too short a time period to draw valid conclusions from), and stability (this was argued to be an incorrect conclusion).

6.

Set the students, in their groups, to the task of being editors. They must rewrite three key sentences within the first two sections of the report to change the tone of the report and pay heed to the scientist’s concerns. They may need to listen to the whole podcast, read through more of the report, and follow the links to do full justice to this task.

7.

Call the students back to report back on their editing.

8.

Have the students summarise the key aspects covered in this discussion and critique a statistical report. The key aspects should include data collection, purpose, target audience, and fairness or representativeness.

In this session, show the students a report where the proportions given in the report are either misleading or ambiguous. You may have your own resources to draw upon, or, in keeping with the waterways context in the example for the previous session, you may choose to use the example below. It concerns the nitrogen levels in tested waterways.

1.

Give the students this editorial piece from the New Zealand Herald to read.

2.

Ask the students to say what the main message(s) of the report were.

3.

Re-read the following extract from the report: "This rate of conversion will continue to have a telling impact on our waterways. Based on the projections above, nitrogen loads in New Zealand rivers would increase by 6 percent by 2020. Canterbury alone has already seen a 27 percent increase in nitrogen loads between 1996 and 2008, and given the conversion rates above, we will see a further 15 percent rise by 2020. Many of our rivers are already unsafe for swimming in.”

4.

Ask the students why they think nitrogen levels might be monitored. Make a summary of the key points in this context. (An indicator of pollution is leaching from farmland; increases may be due to increased stock numbers, particularly dairy stock; weather conditions such as flooding affect readings of levels.) It is important for students to have an understanding of the context in order to evaluate and/or construct a statistical report.

5.

Discuss who might be the target audience of this article (e.g., environmentalists, dairy farmers, river and lake users such as kayakers, swimmers, fishermen, etc.).

• Considering the viewpoint of the readers, might the quoted percentage increase in nitrogen levels in this article be useful knowledge or possibly damaging to their interests?

6.

Arrange the students into small groups and ask them to find further information on the percentage nitrogen levels of NZ rivers. Some useful sites to begin their search are:

• Statistics New Zealand
• Wikipedia
• Niwa 

7.

Bring the groups back together and pool the combined efforts of their research to list percentage increases in nitrogen levels by region and date. Look also at the graph of nitrogen levels (Figure 4b) on the Statistics New Zealand link, which shows a peak in 2006 and then a drop. Use this to make calculations of the percentage increase (shown on the graph) from 1996–2006 and 1996–2007. Discuss the difference between these two values and which should be chosen. Discuss seasonal variation, the possibilities of differing weather patterns affecting readings, and show how a trend line could be drawn through the graph. (As an extension, this could be used to discuss how predictions can be made.)

8.

Get the students to prepare three "press statement" sentences on the percentage increase of nitrogen levels in New Zealand rivers. One each for The Green Party, Fish and Game NZ, and Federated Farmers.

9.

Compile a display, or have the students read out to share, their press statements.

A group of secondary school students in Dunedin, as part of Healthy Harbour Watchers, have been measuring water quality data, including nitrogen (listed as NNN) levels. This study will form the basis for the final three sessions of this unit. The Healthy Harbour Watchers data can be found at Wikipedia | Water Pollution in New Zealand.

1.

Arrange the students into groups and show them the Healthy Harbour Watchers results page on their website. Using the interactive map, change the data display to NNN and assign one harbour location per group of students to note the past readings by changing the date that the data is displayed. Have each group record the data for their location and calculate all the relevant statistics. This should include time series graphs and calculations of range, mean, median, and mode.

2.

Call the groups back and discuss the relative merits of the different statistics.

3.

Give each of the groups the task of choosing appropriate statistics and data displays to write a report for the Healthy Harbour Group with the aim of highlighting concerns with the nitrogen levels in the harbour. They may need to research what is considered acceptable. They might look at long-term trends, or they might highlight the times in the past when the levels were higher. Ask the students to summarise the key (negative) trends and/or statistics in their data. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of the choices, statements, and conclusions the students are making while they plan and compile their report.

4.

Call the groups back to present their key findings.

5.

Discuss whether the levels and the long-term trends of nitrogen levels in the harbour were of as much concern as they have just presented, or were they able to distort the picture they gave by careful choice of what they presented?

1.

Bring the class reports from session three together and discuss the scenario of a journalist being given the task of reporting on the Healthy Harbour Watchers’ findings. Remind the students that the journalist has a responsibility to report in a fair and unbiased manner. The article is to be titled "Is it safe to play in our harbour?"

2.

The students should remain working in groups but may need to undertake further research to complete their report. Encourage the groups to take a stand of yes or no if their research suggests this is the case, but be sure to back this stand up with citations, data displays, and statistics. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of the choices, statements, and conclusions the students are making while they plan and compile their report.

3.

Bring the class back together to share, compare, and contrast their reports.

4.

These reports will be used for session five, so try to find some way of making them available to the whole class. This could be a wall display, a wiki, or a photocopied booklet.

1.

Bring the journalists’ articles together from Session 4 and discuss with the class:

• the validity of a statistical report with reference to: 
  • sample size
  • proportions stated
  • central tendency
  • the presentation of data
  • data collection
• the voice of a statistical report

2.

Give the students the task of each writing "a letter to the editor" disputing the report on the basis of at least one aspect of the statistics quoted by the journalist. The writer of this letter is to be from the water quality office of the regional council. This activity should take some time and give the teacher plenty of opportunity to discuss the validity of the choices, statements, and conclusions the students are making while they plan and compile their report.

3.

Bring the class together to share and discuss their letters to the editor.

• Which voice was the most responsible? The students, the journalist, and the water quality office?
• How can you trust the statistics presented in a report?
• Can you choose statistics to say just what you want, or is there only a little wiggle room?
• How can you be sure of the real story?

4.

Summarise their ideas over what is a responsible/fair/representative use of statistics with regard to each of the following terms: data gathering, sample size, population, proportion, percentage, mean, median, mode, and time period. Try to link in with clear guidelines, such as those published in the explanatory notes for this unit.