Model accuracy (also known as model confidence or model skill) is a measure of how well the model has performed at that time of year in the past. One way that the Department of Primary Industries and Regional Development (DPIRD) and the Bureau of Meteorology measures the accuracy of their climate models is by comparing how often the real outcomes matched the forecast. This measurement of accuracy is known as percent consistent (PC), and has been tested for DPIRD's Statistical Seasonal Forecast (SSF) over the period 1950-2017 and the Bureau's Australian Community Climate Earth-System Simulator – Seasonal (ACCESS–S) seasonal outlook model over the period from 1990-2012.
The percent consistent skill score is calculated by allocating the forecasted rainfall into two categories: above or below median. The percent consistent is the percentage of forecasts consistent with the observed rainfall. If the PC is less than 50% then the forecast is not useful, as it is no better than guessing. If the PC is greater than 65% then the forecast can correctly predict a wetter or drier season in at least two out of every three years. In general, three month outlooks are more skilful than forecasts for individual months.
It is always important to consider the skill of a model before using it in farming decisions. Below is a table showing the percent consistent skill of DPIRD's SSF and BoM's ACCESS-S for the SWLD over the year. There are a range of values because the skill varies across the region in any season. The SSF has the most skill from May to September and ACCESS-S has the most skill from April to July. The poorest skill for the SSF model is from September to November and ACCESS-S poorest skill is from January to March. Use the outlooks with caution at these times. Maps of model skill for each season are available on DPIRD and BoM websites.
|Forecast period||SSF skill (%)||ACCESS-S skill (%)|
|January to March||50-75||>45-65|
|Februar to April||50-60||45-65|
|March to May (autumn)||50-75||45-65|
|April to June||50-70||55-75|
|May to July||50-75||55-75|
|June to August (winter)||50-75||50-65|
|July to September||50-75||55-75|
|August to October||50-60||45-65|
|September to November (spring)||50-70||45-65|
|October to December||50-70||>45-65|
|November to January||50-60||45-65|
|December to February (summer)||50-60||45-65|