US forecast models have been pretty terrible during Hurricane Irma

The Rocketry Forum

Help Support The Rocketry Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

Winston

Lorenzo von Matterhorn
Joined
Jan 31, 2009
Messages
9,560
Reaction score
1,748
My question - why isn't this a cooperative international effort with open source, shared code and coding effort? Or is it already? My guess at the answer if it isn't - job protection within large national government bureaucracies.

US forecast models have been pretty terrible during Hurricane Irma
NOAA's best weather model seems to be getting worse with hurricanes, not better.

https://arstechnica.com/science/201...e-been-pretty-terrible-during-hurricane-irma/

We have written a fair amount at Ars recently about the superiority of the European forecast model, suggesting to readers that they focus on the ensemble runs of this system to get a good handle on track forecasts for Hurricane Irma. Then we checked out some of the preliminary data on model performance during this major hurricane, and it was truly eye-opening.

An independent expert on global forecast models, Ryan Maue, said the NOAA office responsible for developing US computer models, the National Centers For Environmental Prediction, is understaffed and has less funding than the European forecasting center, which is based in the United Kingdom. America, he said, is getting what it pays for.

"NOAA and the National Weather Service are stretched a mile and an inch deep in some places for all of the responsibilities that they have," said Maue, a research meteorologist at the Cato Institute. "If we want to focus on having the best weather forecast in the world, we should focus on having the best weather forecast."

[I'm not sure I buy the overworked excuse. Why not just use the European model code? - W]

Brian Tang, an atmospheric scientist at the University of Albany, tabulates data on "mean absolute error" [MAE] for the location of a storm's center at a given time and where it was forecast to be at that time. Hurricane Irma has been a thing for about a week now, so we have started to get a decent sample size—at least 10 model runs—to assess performance.


Legend:

AVNO: US Global Forecast System, or GFS. The premiere US global model

CMC: Canadian global model

UKM: UK Met Office global model

ECMWF: European global model

NGX: US Navy global model

HMON: New, experiment US hurricane model

HWRF: Operational US hurricane model

TVCN: Consensus model, essentially an average of the global models

OFCL: Official National Hurricane Center forecast

mae.png
 
One of our local atmospheric sciences professors writes a lot about this on his blog. The most recent was here: https://cliffmass.blogspot.com/2017/01/make-us-numerical-weather-prediction.html He has some links to prior posts as well. There are a couple of additional posts that might be interesting, although they border on political.

https://cliffmass.blogspot.com/2017/03/might-bipartisan-actions-revolutionize.html
https://cliffmass.blogspot.com/2017/05/president-trumps-budget-plan-weakens-us.html

If you think the current forecasts aren't good enough, read the budget one and call your congresscritter.
 
One of our local atmospheric sciences professors writes a lot about this on his blog. The most recent was here: https://cliffmass.blogspot.com/2017/01/make-us-numerical-weather-prediction.html He has some links to prior posts as well. There are a couple of additional posts that might be interesting, although they border on political.

https://cliffmass.blogspot.com/2017/03/might-bipartisan-actions-revolutionize.html
https://cliffmass.blogspot.com/2017/05/president-trumps-budget-plan-weakens-us.html

If you think the current forecasts aren't good enough, read the budget one and call your congresscritter.
Thanks!
 
It's at best guesswork. All they can do is gather data then go outside and spit into the wind. All we can do is run like hell when we know one is coming. Time to visit relatives in Colorado...


edit: I have family in Chrystal River, I hope they are running before they get run over. They are just south of Pensacola, directly in the projected path....
 
One of our local atmospheric sciences professors writes a lot about this on his blog. The most recent was here: https://cliffmass.blogspot.com/2017/01/make-us-numerical-weather-prediction.html He has some links to prior posts as well. There are a couple of additional posts that might be interesting, although they border on political.

https://cliffmass.blogspot.com/2017/03/might-bipartisan-actions-revolutionize.html
https://cliffmass.blogspot.com/2017/05/president-trumps-budget-plan-weakens-us.html

If you think the current forecasts aren't good enough, read the budget one and call your congresscritter.
Just a few excerpts from the outstanding page you provided. EXACTLY as I suspected - NOT due to "inadequate funding" as whined about in the article excerpt I provided in my original post but due to to the usual crushing, inefficient, disorganized, turf protecting government bureaucracies.

https://cliffmass.blogspot.com/2017/01/make-us-numerical-weather-prediction.html

For seasonal forecasts (out to 9 months), the NWS is running a seasonal climate model (the CFS) , even though it has no demonstrable skill over statistical methods (I am doing research on this right now). But there seems little impetus to pull the plug on this non-productive effort or make the radical changes necessary to make seasonal or subseasonal (weeks to a few months) better.

The U.S. is profoundly lagging in other areas of numerical weather prediction as well. Numerous workshops, reports, and meetings have called for the National Weather Service to run a high-resolution ensemble (many forecasts) over the U.S. to aid forecasting of thunderstorms and smaller scale features. They have not done so, while the research community (NCAR) has an experimental operational system in place. Nor has the NWS been effective in developing and applying improved model physics (such as the microphysics of clouds) and its statistical post-processing (improving the forecasts with statistics) is based on simple 1960s approaches.

There are many other examples of U.S. numerical weather prediction falling behind the inherent capabilities and needs of the nation, but the above examples suffice. And keep in mind that the U.S. spends more on numerical prediction than anyone else and has the biggest weather research community in the world.

I have studied this question for years, published peer-reviewed articles on the subject, and have served (and am now serving on) advisory committees to the U.S. government and other entities. I believe I know where the skeletons in the closets are found. And I am at the point in my career that I can afford to take some risks in the hope of promoting critically needed change. So here it goes.

It would be hard to think of an organizational structure that is more problematic than the one used by NOAA/NWS/US government NWP development and research. The byzantine organizational structure of NOAA and the NWS are found below for reference.

But first a lot of acronyms. Operational numerical weather prediction (NWP) is found in the Environmental Modeling Center (EMC) of the NCEP (National Centers for Environmental Prediction), which is part of the National Weather Service (NWS), which is part of the National Oceanographic and Atmospheric Administration (NOAA).

Amazingly, folks with the main responsibility for developing the models are NOT in the NWS, which is responsible for operational NWP, but in NOAA (at the ESRL and GFDL labs). So the folks who run the weather forecasting models (NCEP EMC) don't have control of the scientists developing the models.

And this has led to a LOT of problems. For example, the NOAA ESRL folks spent a lot of resources on models that never went operational, antagonism between the model developers and the operational people has been rampant (although better recently), and other issues.

But it is worse than that. The decisions on future model use is decided upon by ANOTHER group in the NWS outside of NCEP, called OSTP (Office of Science and Technology Policy). And the development of the statistical post-processing of model is not done by the modelers or in the same organization doing operational NWP (NCEP's EMS), but another group in OSTP (called MDL). It is surprising ANYTHING is accomplished in this crazy quilt bureaucracy.

But it is worse than that. The U.S. Navy runs its own independent global and regional weather prediction effort at FNMOC (Fleet Numerical Meteorology and Oceanographic Center), based on science/technology developed by the Navy's own research arm (MRL Monterey). A mainly duplicative effort, except for a greater emphasis on the oceans. And all this is done, even though the Navy's global model's verification scores are consistently inferior to the National Weather Services (see above for proof, FNO is the Navy). I am amazed Congress hasn't noticed this.

Can it get even worse and wasteful? You bet. The US Air Force runs ANOTHER NWP center and they decided to use a foreign model (the UKMET office unified model), rejecting the previous use of the NWS model (GFS) and aregional scale WRF model developed by the academic community (NCAR, the National Center for Atmospheric Research). This decision was so silly, that anonymous researchers in the US Air Force (AFWA, Air Force Weather Agency, now Detachment 7), made a very funny video about it, suggesting that even Hitler would have been upset about the wacky decision. So the US Air Force thought the NWS global model was so poor, they had to run a foreign model. Just stunning.

In short, huge amounts of U.S. resources have been wasted in funding duplicate national weather prediction entities, with all being suboptimal for state-of-the-science weather prediction. Combined and organized in a rational way, they could be the best in the world. Separated in this separate efforts, they are seriously lagging behind.
 
Back
Top