Introduction

In this document I will explore how to create the first part of the evaluation system I proposed. The working title of this is the “Forecast-Hour Evaluation.” The idea here is that we are looking at the performance of the model by looking at how it performed with different start times (using the most recent 00-hr forecast as input).

## Warning in rgl.init(initValue, onlyNULL): RGL: unable to open X11 display
## Warning: 'rgl.init' failed, running with 'rgl.useNULL = TRUE'.

Read Model and Observation Data

Read WRF Data

For this evaluation system we need to look at three different output folders. Here we use the folders named, forecast_day_minus_0, forecast_day_minus_1, forecast_day_minus_2. The contents of each of these folders will be similar: wrfout files for 86 forecast hours and time-series data for different locations of interest. Here we will first read the forecast data.

Read the OBS Data

Now we will read the observation data from the ASOS stations. The script that downloads the data is in ./obs_station_day_minus_0/dl_ny_asos.py. The lines for the dates to download need to be changed before running it. Once the files are download, the lines below reads the data and adds column names.

Unit Conversion

Model and observation data do not share the same units for the same variable. For temperature, WRF is in Kelvin and ASOS is in degreesF. For winds, WRF is in m/s and ASOS is in knots. The formulas used to convert the numbers to a common system is shown here. For temperature I will use Kelvin, and m/s for wind speeds.

Combined Data Frames

Now we have one data frame for all the observations, and three (3) data frames of the WRF data (one data frame per forecast init time). The lines below provide a visual of the data frames.

##             Date.Time year mon day hour min     sec Temperature Mixing.Ratio
## 1 2020-07-20 00:00:05 2020   7  20    0   0  5.0004    298.9888      0.01875
## 2 2020-07-20 00:00:10 2020   7  20    0   0 10.0008    299.1123      0.01875
## 3 2020-07-20 00:00:15 2020   7  20    0   0 15.0012    299.2151      0.01875
## 4 2020-07-20 00:00:20 2020   7  20    0   0 20.0016    299.3042      0.01875
## 5 2020-07-20 00:00:24 2020   7  20    0   0 24.9984    299.3829      0.01874
## 6 2020-07-20 00:00:29 2020   7  20    0   0 29.9988    299.4472      0.01874
##     U_WIND  V_WIND Wind.Speed Wind.Direction Station
## 1  0.11679 6.50548   6.506528       181.0285     JFK
## 2  0.02975 6.10297   6.103043       180.2793     JFK
## 3 -0.02853 5.82294   5.823010       179.7193     JFK
## 4 -0.07052 5.60088   5.601324       179.2786     JFK
## 5 -0.09859 5.41937   5.420267       178.9578     JFK
## 6 -0.11801 5.27576   5.277080       178.7186     JFK
##             Date.Time year mon day hour min     sec Temperature Mixing.Ratio
## 1 2020-07-19 00:00:05 2020   7  19    0   0  5.0004    298.5450      0.01776
## 2 2020-07-19 00:00:10 2020   7  19    0   0 10.0008    298.6396      0.01776
## 3 2020-07-19 00:00:15 2020   7  19    0   0 15.0012    298.7230      0.01776
## 4 2020-07-19 00:00:20 2020   7  19    0   0 20.0016    298.7982      0.01776
## 5 2020-07-19 00:00:24 2020   7  19    0   0 24.9984    298.8667      0.01776
## 6 2020-07-19 00:00:29 2020   7  19    0   0 29.9988    298.9297      0.01775
##    U_WIND  V_WIND Wind.Speed Wind.Direction Station
## 1 0.40536 3.72305   3.745052       186.2138     JFK
## 2 0.36550 3.56774   3.586413       185.8493     JFK
## 3 0.33498 3.45794   3.474127       185.5331     JFK
## 4 0.30943 3.35717   3.371400       185.2661     JFK
## 5 0.28847 3.26732   3.280030       185.0455     JFK
## 6 0.27264 3.19378   3.205396       184.8793     JFK
##             Date.Time year mon day hour min     sec Temperature Mixing.Ratio
## 1 2020-07-18 00:00:05 2020   7  18    0   0  5.0004    298.2605      0.01863
## 2 2020-07-18 00:00:10 2020   7  18    0   0 10.0008    298.3093      0.01863
## 3 2020-07-18 00:00:15 2020   7  18    0   0 15.0012    298.3527      0.01863
## 4 2020-07-18 00:00:20 2020   7  18    0   0 20.0016    298.3923      0.01863
## 5 2020-07-18 00:00:24 2020   7  18    0   0 24.9984    298.4288      0.01862
## 6 2020-07-18 00:00:29 2020   7  18    0   0 29.9988    298.4628      0.01862
##     U_WIND  V_WIND Wind.Speed Wind.Direction Station
## 1 -0.04684 3.14331   3.143659       179.1463     JFK
## 2 -0.06773 3.03175   3.032506       178.7202     JFK
## 3 -0.08092 2.94604   2.947151       178.4266     JFK
## 4 -0.09300 2.87122   2.872726       178.1448     JFK
## 5 -0.10738 2.80724   2.809293       177.8094     JFK
## 6 -0.10912 2.74670   2.748867       177.7250     JFK
##   Station           Date.Time Temperature Relative.Humidity Wind.Direction
## 1     JFK 2020-07-19 00:00:00         NaN               NaN            200
## 2     JFK 2020-07-19 00:05:00         NaN               NaN            190
## 3     JFK 2020-07-19 00:10:00         NaN               NaN            180
## 4     JFK 2020-07-19 00:15:00         NaN               NaN            160
## 5     JFK 2020-07-19 00:20:00         NaN               NaN            160
## 6     JFK 2020-07-19 00:25:00         NaN               NaN            170
##   Wind.Speed year mon day hour min sec
## 1   1.028807 2020   7  19    0   0   0
## 2   3.600823 2020   7  19    0   5   0
## 3   2.057613 2020   7  19    0  10   0
## 4   2.057613 2020   7  19    0  15   0
## 5   2.057613 2020   7  19    0  20   0
## 6   2.057613 2020   7  19    0  25   0

Locations for Plots

Match Times for Model and Observations

Select Day of Interest

Time-matching is performed using a routine that can be found in Analysis01-Time_Matching_Problem.Rmd. The time matching will be done per variable. For the Forecast-Hour Evaluation product, we will focus on the temperature, wind speed and wind direction variables. Also, now that we have read all the TS data and ASOS data, we need to extract the day of interest, or doi for the time-series.

Note that for this product the “day of interest” will always be the UTC date of the day before.

We now have filtered data frames for the observations and model data for the day of interest.

Next, we will select only the temperature data for comparing the model and observations. This needs to be done on a per station basis. Note that we use the function drop_na() to drop rows which contain NaN or NA data. Since each variable is measured at different intervals, not all variables will have data available at every time step in the ASOS data. The functions may be too sensitive to missing data and thus we take care to remvove it here from the observations, after we have isolated a particular variable.

Temperature Time-Matching

Location: JFK

Location: LGA

Location: NYC

Wind Speed Time-Matching

Location: JFK

Location: LGA

Location: NYC

Wind Direction Time-Matching

Location: JFK

Location: LGA

Location: NYC

Forecast Hour Evaluation for JFK

For the temperature data I will use Bias, RMSE and MAE for the comparison statistics

JFK - WRF 2-m Temperature (K) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 0.679 1.254 0.953
2 WRF D-1 -0.058 2.011 1.825
3 WRF D-2 -1.287 2.444 2.163
JFK - WRF 10-m Wind Speed (m/s) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 -0.729 1.640 1.342
2 WRF D-1 -0.522 1.558 1.269
3 WRF D-2 -1.393 2.299 1.792
JFK - WRF 10-m Wind Direction (degN) Performance
Forecast.Init RMSE MAE
1 WRF D-0 25.697 18.496
2 WRF D-1 30.855 20.660
3 WRF D-2 55.347 36.302

Forecast Hour Evaluation for LGA

LGA - WRF 2-m Temperature (K) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 0.106 1.787 1.638
2 WRF D-1 -1.472 3.319 2.805
3 WRF D-2 -1.855 3.567 3.018
LGA - WRF 10-m Wind Speed (m/s) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 -1.647 2.247 1.877
2 WRF D-1 -1.469 2.157 1.792
3 WRF D-2 -1.656 2.318 1.956
LGA - WRF 10-m Wind Direction (degN) Performance
Forecast.Init RMSE MAE
1 WRF D-0 38.543 28.399
2 WRF D-1 44.069 35.789
3 WRF D-2 51.217 39.638

Forecast Hour Evaluation for NYC

NYC - WRF 2-m Temperature (K) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 2.518 2.854 2.533
2 WRF D-1 1.193 2.536 2.125
3 WRF D-2 0.545 2.950 2.657
NYC - WRF 10-m Wind Speed (m/s) Performance
Forecast.Init BIAS RMSE MAE
1 WRF D-0 -0.878 1.357 1.199
2 WRF D-1 -0.910 1.379 1.244
3 WRF D-2 -0.962 1.400 1.265
NYC - WRF 10-m Wind Direction (degN) Performance
Forecast.Init RMSE MAE
1 WRF D-0 75.215 60.291
2 WRF D-1 79.199 64.095
3 WRF D-2 77.587 55.932

uWRF and ASOS Time-Series Visualization

Temperature

## Warning: Removed 280 rows containing missing values (geom_point).
## Warning: Removed 280 rows containing missing values (geom_path).

Wind Speed

Wind Direction