Understanding skill of seasonal outlooks

Page last updated: Tuesday, 4 February 2020 - 2:04pm

Many seasonal outlooks are available throughout the year for use in the South West Land Division (SWLD) whcih includes the Western Australian grainbelt. This page decribes the percent consistent (or past accuracy) of two models used to forecast grainbelt seasonal rainfall and how skill of the two models differs throughout the year.

Model accuracy (also known as model confidence or model skill) is a measure of how well the model has performed at that time of year in the past. One way that the Department of Primary Industries and Regional Development (DPIRD) and the Bureau of Meteorology measures the accuracy of their climate models is by comparing how often the real outcomes matched the forecast. This measurement of accuracy is known as percent consistent (PC), and has been tested for DPIRD's Statistical Seasonal Forecast (SSF) over the period 1950-2017 and the Bureau's Australian Community Climate Earth-System Simulator – Seasonal (ACCESS–S) seasonal outlook model over the period from 1990-2012.

The percent consistent skill score is calculated by allocating the forecasted rainfall into two categories: above or below median. The percent consistent is the percentage of forecasts consistent with the observed rainfall. If the PC is less than 50% then the forecast is not useful, as it is no better than guessing. If the PC is greater than 65% then the forecast can correctly predict a wetter or drier season in at least two out of every three years. In general, three month outlooks are more skilful than forecasts for individual months.

Percent Consistent skill of the SSF at forecasting June to August rainfall using data up to and including May. Skill is 50 to 100 percent consistent
Percent Consistent skill of the SSF at forecasting June to August rainfall using data up to and including May. Skill is 50 to 100 percent consistent.
Percent Consistent skill of the Bureau of Meteorology’s outlook for January to March rainfall.Predictive skill for the January to March period is 45-75% consistent, which is considered poor to good.
Percent Consistent skill of the Bureau of Meteorology’s outlook for January to March rainfall.Predictive skill for the January to March period.

It is always important to consider the skill of a model before using it in farming decisions. Below is a table showing the percent consistent skill of DPIRD's SSF and BoM's ACCESS-S for the SWLD over the year. There are a range of values because the skill varies across the region in any season. The SSF has the most skill from May to September and ACCESS-S has the most skill from April to July. The poorest skill for the SSF model is from September to November and ACCESS-S poorest skill is from January to March. Use the outlooks with caution at these times. Maps of model skill for each season are available on DPIRD and BoM websites.

Forecast period SSF skill (%) ACCESS-S skill (%)
January to March 50-75 >45-65
Februar to April 50-60 45-65
March to May (autumn) 50-75 45-65
April to June 50-70 55-75
May to July 50-75 55-75
June to August (winter) 50-75 50-65
July to September 50-75 55-75
August to October 50-60 45-65
September to November (spring) 50-70 45-65
October to December 50-70 >45-65
November to January 50-60 45-65
December to February (summer) 50-60 45-65

Contact information

Author

Meredith Guthrie