Measuring the Science Media Centre’s impact

By Georgia Nixon, Shaun Hendy, and Dacia Herbulock — Thanks to Te Pūnaha Matatini for initiating this collaboration.

The Science Media Centre has been connecting scientists and the media since 2008, either directly via responses to media queries, or using Expert Reactions, which provide commentary on breaking news, and online Q&A/Briefings, where journalists can ask questions of experts on a particular topic. Since 2012, the SMC has also offered Science Media Savvy courses, which provide an intensive 2-day introduction for scientists to working with the media. We added another media training option in 2015, Savvy Express, where researchers at conferences can get a 15-minute one-on-one coaching session on how to speak effectively on camera.

Last year, we thought it was time to try to measure the impact of these initiatives on the presence of scientists in the media.

With the help of Shaun Hendy and Georgia Nixon from Te Pūnaha Matatini, we analysed the set of 2545 researchers who had interacted with the Science Media Centre via a Q&A Briefing, an Expert Reaction, or by attendance at one of the 2-day Science Media Savvy courses. We web-scraped appearances of these scientists in a selection of online mainstream media channels (New Zealand Herald, Stuff, Newshub) from 2009 onwards. Around 2000 researchers of these have appeared in a Q&A Briefing or Expert Reactions, while around 1000 of these have attended a Science Media Savvy course.

We began by looking at the time-based cross-correlation between interactions the researchers have with the SMC and their appearance in the media. This reveals the distribution of time intervals between an SMC interaction and an appearance in the media for the researchers in our data set. The cross-correlations, which are illustrated in the figures below (Figure 1), show that when scientists are quoted in a Q&A or Expert Reaction online article, their mainstream media presence over that month is significantly increased. This boost is not sustained however – an appearance in an Expert Reaction or Q&A does not significantly increase a researchers presence in the media beyond a month. The frequency of appearances both before and after the SMC interaction is about the same.

Figure 1: Temporal cross-correlations between interactions the researchers have with the SMC and their appearance in the media:
(a) for Expert Reactions 
(left) or (b) for Q&A Briefings (right). The time difference is measured in number of days.

This is not entirely unexpected. Expert reactions – as it says on the label – are reactions to breaking news and are published as a list of comments by a range of researchers. Journalists will often quote these published comments directly without making contact with the researcher. Similarly, Q&A Briefings cover topical issues in more depth but again, journalists can quote researchers without conducting an interview. Once the issue ceases to be topical, the results suggest that these one-way engagements tend not to result in further media appearances for the researchers.

In contrast, Science Media Savvy workshops seem to result in sustained media appearances that persist over a number of years. Figure 2 shows a similar cross-correlation for scientists and researchers who attended a full Savvy workshop. Interestingly, media appearances for these researchers seem to be increasing before the Savvy course: this may reflect the career profile of researchers who take the course, which tends to be skewed towards early career researchers, or it could be that increasing contact with the media motivates researchers to take the course. Either way, researchers experience an immediate and relatively sustained increase in their media presence following the course.

Figure 2: Temporal cross-correlations between researchers who take the 2-day SMC Science Media Savvy course and their appearances in mainstream media channels. The relative time is measured in days.

The outcome illustrated in Figure 2 is consistent with the goals of the Savvy course, which is to “upskill researchers in science communication and improve engagement with media and the wider public”. It is worth noting that taking a Savvy course also brings researchers’ expertise to the attention of the Science Media Centre staff and may mean that researchers are more likely to be selected for Expert Reactions and Q&A Briefings subsequently. Thus the long term persistence of media presence for Savvy participants may be partly attributable to other SMC activities. Another caveat worth noting is that Figures 1 and 2 don’t necessarily demonstrate causality: journalists may have sought out these scientists for comment anyway. Nonetheless, if we hadn’t observed these effects, it would be grounds for concern!  

Finally, we constructed a network diagram to illustrate the range of topics covered by the SMC and picked up by these media channels. Each small circle (node) represents a media appearance of a SMC researcher, where nodes are coloured according to the area of expertise of that scientist (as recorded by the SMC). Small nodes represent appearances before SMC interactions, while large nodes represent appearances after a SMC interaction. Open circles indicate cross-disciplinary researchers, that bridge multiple subject areas. To create the network, appearances are linked by the shared expertise of researchers. Perhaps not surprisingly, health and the environment have tended to dominate the Science Media Centre’s activities.

Figure 3: Science Media Centre-linked researchers from a range of disciplines been quoted in mainstream media channels. The distribution indicates the frequency from which different disciplines are quoted and their relationship to each other.