Understanding skill of seasonal outlooks

Model accuracy (also known as model confidence or model skill) is a measure of how well the model has previously performed at that time of year. One way that the Department of Primary Industries and Regional Development (DPIRD) and the Bureau of Meteorology (BoM) measure the accuracy of their climate models is by comparing how often the real outcomes matched the forecast. This measurement of accuracy is known as per cent consistent (PC), and has been tested for DPIRD's Statistical Seasonal Forecast (SSF) over the period 1950-2020 and BoM's Australian Community Climate Earth-System Simulator – Seasonal (ACCESS–S) seasonal outlook model for the period from 1990-2012.

The per cent consistent skill score is calculated by allocating the forecasted rainfall into two categories: above; or below median. The per cent consistent is the percentage of forecasts consistent with the observed rainfall. If the PC is less than 50%, the forecast is not useful, as it is no better than guessing. If the PC is greater than 65%, the forecast can correctly predict a wetter or drier season in at least two out of every three years. In general, three month outlooks are more skilful than forecasts for individual months.

Percent Consistent skill of the SSF at forecasting June to August rainfall using data up to and including May. Skill is 50 to 100 percent consistent.

Percent Consistent skill of the Bureau of Meteorology’s outlook for June to August.

It is important to consider the skill of a model before using it to make farming decisions. Below is a table showing the per cent consistent skill of DPIRD's SSF and BoM's ACCESS-S2 for the SWLD, over the year. There are a range of values because the skill varies across the region in any season. The SSF has the most skill from May to September and ACCESS-S2 has the most skill from April to July. The poorest skill for the SSF model is from September to November, and ACCESS-S2 poorest skill is from January to March. Use the outlooks with caution at these times. Maps of model skill for each season are available on DPIRD and BoM websites.