Schoolboy error by ONS in web scraping trial

On September 1st, ONS published the results of a trial to scape food prices daily from the internet (see discussion here). These resulted in headlines such as: “costs rocketing”; “cost of basic items has risen by 8% in last year”; and “spaghetti up 20% in a year”. Yesterday, ONS put out a correction saying that they had got the scale inverted and that food prices had actually fallen by 3% a year – a figure now almost the same as the -2% decline in prices seen in CPI.

Pete Comley, of the website Inflation Matters, said: “I pointed out this error to the ONS in early June after they published some initial findings on the web data trial. It looked so obviously wrong to me. Despite this, ONS went on to publish more data with the error in September and have only now admitted to the mistake.”

He continued: “This miscalculation comes after other major revisions to the results between June and September. ONS’s initial analysis created headlines of inflation being potentially over-estimated versus CPI. In September they created the opposite headlines. Such mistakes undermine the public confidence with other pricing statistics such as CPI and RPI. There should be better quality control procedures in place at the ONS. They should at least apply the old sense check of: if it is interesting, it is probably wrong.”

Comley was concerned what the long-term impact of this error might be: “The collection of prices online could have a lot of benefits not just to the ONS but to the public by providing more timely and detailed price indices. It would be a shame if this opportunity for development of our price indices is wasted due to basic errors in a trial.”

The trial referred to involved ONS collecting prices of 35 foods and drinks automatically from Tesco, Sainsburys and Waitrose websites between June 2014 and June 2015. Some 6500 prices were checked every day and a daily chained-linked price index created and compared with changes in average unit prices paid (of the items that could be checked each week). The error occurred in the chain-linking causing price declines to be stated as rises, and vice versa. In addition, ONS also appear to have made some other revisions to their chain-linked data.

A summary of some of the key data published in September (and reported widely) and the latest revisions are:

Original chain-linked inflation est. Revised chain-linked inflation est. Unit price change inflation est.
All food and non-alcoholic drinks +8% -3% -3%
Dry spaghetti/pasta +19% -10% +1%
Red wine bottle +14% -11% -4%
Cola drink 2 litre +14% -2% 0%
Cheddar per kg +14% -10% -6%
Fresh orange juice +9% -4% -1%
Plain biscuits 2-300g +8% -6% -1%
New potatoes per kg +7% -6% -2%
Semi-skimmed milk +2% -2% -2%
Tea bags -4% +4% +3%

 

Link to the full ONS corrected report: http://www.ons.gov.uk/ons/rel/cpi/consumer-price-indices/research-indices-using-web-scraped-price-data/index.html

Link to a copy of the original ONS report can be found at: http://inflationmatters.com/is-inflation-really-8-ons-web-scraping-trial-problems/

1 comment on “Schoolboy error by ONS in web scraping trial”

  1. Pingback: This is Money coverage of ONS data error • Inflation Matters

Leave a Reply