Large tracking study: moving Ofcom's telephone survey to online and face-to-face

Ofcom has just released its 2016 Public Service Broadcasting Annual Report (PSB). Previously, the interviewing for the research was done via telephone.  However in 2015 we used our Convertr™ technique to help Ofcom move the methodology to a mixture of online and face-to-face fieldwork.

This enabled them to:

1. Reduce the cost of the fieldwork.

2. Ensure the results are properly representative of the views of UK adults, and will continue to be so for future waves.

3. Continue to allow results to be tracked and compared to those from previous years (despite the change in methodology).

The PSB tracker is a high profile study, and findings are reported widely in both the research world and the national press.  It was, therefore, vital that if Ofcom was to change the method, that it could be confident that the results would continue to be highly robust and accurate, and stand up to scrutiny.

Ofcom is required by the 2003 Communications Act to assess public service broadcasters (the BBC, Channel 4, ITV1, Five and S4C).  To provide data for this, Ofcom started running its Public Service Broadcasting tracker in 2005, and BVA BDRC started working on it in 2011.  It involves several thousand interviews each year, with very detailed quotas, ensuring it provides a highly robust and representative sample of UK adults.  Data from the tracker are ‘Official Statistics’ and subject to a high level of scrutiny, including from government, industry, the media and the general public.

However in 2015 it was felt that telephone interviewing was no longer the most appropriate method to collect data for this project.  This was due to:

1. Non-response figures - many people no longer answer the phone to numbers they don’t recognise
2. Representivity - only 85% of homes have a landline telephone
3. Data quality - it is a long and complex interview to conduct by telephone
4. Cost-effectiveness - as a public body, Ofcom needs to provide (and to be seen to be providing) value for money, so it wanted to explore alternative methods

But given the fact it was a tracking survey - with many years of previous data - a big challenge for the project was to retain the ability to compare results with previous years, despite a change in the method.

A further challenge was to ensure that the data collection method continued to provide robust and representative results.

Face-to-face is still generally regarded as the most representative data collection method (for nat-rep samples, at least).  However, it costs significantly more than online interviewing.

A purely online approach would have been the most cost-effective, but this approach also suffers from issues of representativity (perhaps even more so than telephone), as I’ve mentioned previously.

Therefore we suggested a mixed-method approach with 25% of interviews conducted F2F and 75% online, providing significant cost savings.  By using Convertr™ we were able to ensure that the mixed method delivered nationally representative results that would meet the rigorous project objectives, as well as stand up to robust external scrutiny.

We then adapted our Convertr™ technique and applied it to the historical PSB data, which meant that it was possible to recalibrate the previously reported results to what would have been achieved via the Tracker’s new method.  This means the historical results could be reported, the time series continued, and comparisons with existing (and future) results could continue to be made over time.

If this all sound rather exciting, then you’ll probably want to find out more.  So if you’re interested in seeing how Convertr™ could reduce the cost of your research, move a tracking survey to a new method, or ensure that your online fieldwork is delivering representative results, please free to get in touch.