
The EROC Method
An Extended Procedure for Implementing the Relative Operating Characteristic Graphical Method
Fredrick H. M. Semazzi, and Roberto J. Mera. Journal of Applied Meteorology and Climatology, Semptember 2006.
Full text 
Abstract
The functional relationship between the relative operating characteristic (ROC) and the
economic value (EV) graphical methods have been exploited to develop a hybrid procedure called
the extended ROC (EROC) method. The EROC retains the appealing simplicity of the traditional
ROC method and the ability of the EV method to provide evaluation of the performance of
an ensemble climate prediction system (EPS) for a hypothetical end user defined by the costloss ratio (mu = C/L).
An inequality defining the lower and upper theoretical bounds of mu has been derived.
Outside these limits, the EPS yields no added benefits for end user mu relative to the use
of climatological persistence as an alternative prediction system. In the traditional
ROC graphical method, the ROC skill (ROCS) is often expressed in terms of the area between
the ROC graph and the diagonal baseline passing through the origin with slope m = 1.
Thus, ROCS = 2A1, where A is the area under the ROC graph. In the proposed EROC approach,
a more general procedure is recommended based on the construction of userspecific
baselines that do not necessarily pass through the origin and, in general, have slope m not
equal to 1. The skill of a particular EPS computed from the EROC method is proportional
to the corresponding estimated value based on the EV graphical method. Therefore,
the EROC geometry conveys the same basic information as the EV method. The Semazzi & Mera
skill score (SMSS) is proposed as a convenient and compact way of expressing the combined
verification based on the ROC and EV methods. The ROCS estimate is a special case of the SMSS.
The nearhorizontal traillike geometry sometimes exhibited by EV graphs is also examined.
It is shown to occur when either the hitrate or falsealarm term dominates in the formula
for EV, unlike the more typical situation in which both terms are comparable in magnitude.
Presentation at the Sixth Annual Climate Prediction Applications Science Workshop (CPASW), March 2008, Chapel Hill, NC, USA
Sample slides:
ApplicationEfforts currently underway for the UCAR Africa Initiative will seek to use an ensemble of large scale models to provide operational forecasts for the Africa initiative. The NCSU Climate Modeling Laboratory will also explore the use of ensemble forecasting by downscaling each of the models in operational mode using WRF. Additionally, we will explore the relative skill and economic value of the ensemble prediction forecast using the recently published EROC method. Currently underway is a WRF climate mode ensemble using Final Analyses (FNL) initial conditions and varying physics in the model to simulate multiple model ensemble. Realtime, initialconditions varying forecasts are also provided in a biweekly timescale during the meningitis season. Click on the icon on right to access the forecasts. 