EMS reports "File Missing" for locally downloaded ERA files

Questions and other topics related to UEMS 18.
Post Reply
michaelwx
Posts: 10
Joined: Mon Nov 18, 2019 5:29 am

EMS reports "File Missing" for locally downloaded ERA files

Post by michaelwx » Tue Mar 31, 2020 3:17 am

This one has me truly stumped. When running ems_prep --dset era20c --date 20020202 --domains 2,3 --cycle 06 --length 24 --analysis UEMS repeatedly throws an error: "File Missing" then lists the missing files.

I used "download_rda_ds626.0_era20c.pl 200202" and everything went fine. I used "process_rda_ds626.0.pl" and once again, everything went fine. I triple-checked the resulting file permissions, the subdirectory where the files are stored, era20c_gribinfo.conf making sure that files are consistently named and re-read both the User Manual (at least 4 times) along with reading "What You Need to Know." I even removed those files not targeted with the analysis.

Same result: Files missing. The files are in ~/uems/runs/<domain_name>/grib but nothing I have tried seems to enable UEMS to find the files that are visible with ls -al.

Any suggestions?

Michael
Michael Allen
Industrial Weather

michaelwx
Posts: 10
Joined: Mon Nov 18, 2019 5:29 am

Re: New Question Topic

Post by michaelwx » Wed Apr 08, 2020 1:40 am

Well solved it, sort of...

I used narr, attempted to run a 24 hour simulation, then a 48-hour simulation. Same issue: couldn't find the files in the /grib directory even though I could see them and all configuration scripts were triple-checked.

Then, on a hunch I removed two of the nests, using the largest one. Voila. No more issues. Alas, no matter what time step I set it continued to crash. I ended up with a timestep of 480 seconds (!!!), and simulation worked. Then I tried everything divisible by 6 and 3. Nada.

So, who knows? I'm still attempting to find the solution.

Michael
Michael Allen
Industrial Weather

pattim
Posts: 199
Joined: Sun Jun 24, 2012 8:42 pm
Location: Los Angeles, CA, USA

Re: EMS reports "File Missing" for locally downloaded ERA files

Post by pattim » Wed May 06, 2020 8:47 pm

Is it possible that you're running into a switchover of BC data times from 3-hourly to 6-hourly, etc.? Several of the datasets do this. Also, check the specific dataset - it might have changed on the server without the **gribinfo.conf files being updated...

Code: Select all

/OSS/Data_drive/users/00_UEMS/uems/runs/local_05> ems_prep --dslist
  Starting UEMS Program ems_prep (V19.8.1) on linux-lhkc at Wed May  6 20:45:46 2020 UTC
    Initialization datasets currently supported for use with the UEMS:
        I.  Analysis Datasets

              gfssoil       sub-surface (soil) fields from the 0.25 degree global GFS - FOR USE WITH THE --lsm OPTION ONLY
              gfsanl        GFS Analysis Dataset Global 1 degree - 6hourly - Must use with --analysis flag
              fnl           GFS Final Analysis Dataset - 6hourly - Must use with --analysis flag
              ruchyb        RUC 13.5km resolution CONUS 130 grid - Hybrid coordinate (B grid) - Hourly
              dummy         A Dummy template to be used to populate parameters when WRF intermediate files are used

            Analysis Personal Tile Datasets

              rucpt         RUC 13.5km resolution CONUS 130 grid - Hybrid coordinate (B grid) - Hourly

       II.  Forecast Datasets

              rap           RAP 13.5km resolution CONUS 130 grid - Hybrid coordinate (B grid) - Hourly
              arw           WRF ARW Output processed into GRIB 2 by UEMS UPP - Your data the way you like it!
              fv3p25        FV3 0.25 degree dataset on pressure surfaces - hourly through 120 hours
              gfsp50        GFS 0.5 degree dataset on pressure surfaces - 3hourly out through 180 hours
              gfs           GFS 0.5 degree dataset on pressure surfaces - 3hourly out through 180 hours
              locdset       GFS 0.25 degree dataset on pressure surfaces - 3hourly out through 180 hours
              raphyb        RAP 13.5km resolution CONUS 130 grid - Hybrid coordinate (B grid) - Hourly
              gefs          GFS 0.5 degree dataset on pressure surfaces (Post 2015 update!) - 3hourly out through 180 hours
              namca         NAM Model 0.108 deg Carribean & Central America Domain (25mb) - 3-hourly through hr 60
              namak         NAM Model 11.25km Alaska grid - Isobaric coordinate (25mb) - 3hourly through hr 60
              namnest       NAM 3km nested domain - isobaric coordinate (Very Large!) -[b] Hourly through 36 hours, 3-hourly through 60 hours[/b]
              cfspgb        Climate Forecast System (CFS) Version 2 - Global, 1.0 degree
              hrrr          The full High Resolution Rapid Refresh (HRRR) data on pressure surfaces - 3km, hourly out through 15 hours
              nam           NAM Model 12.2 km 218 grid - Isobaric coordinates, [b]hourly through hour 36 then 3-hourly through 84 hours[/b]
              gfsp25        GFS 0.25 degree dataset on pressure surfaces - hourly through 120 hours

            Forecast Personal Tile Datasets

              rappt         RAP 13.5km resolution CONUS 130 grid - Hybrid coordinate (B grid) - Hourly
              gfsp50pt      Personal tile version of the GFS 0.50 degree dataset on pressure surfaces - 3-hourly through 180 hours
              namakpt       Person tile version of the NAM Model 11.25km Alaska grid - Isobaric coordinate (25mb) - 3hourly through hr 60
              gfsp25pt      Personal tile version of the GFS 0.25 degree dataset on pressure surfaces - 3-hourly through 120 hours
              nampt         NAM Model 12km personal tile - Isobaric coordinate hourly through hour 36 then 3-hourly through 84 hours
              namcapt       NAM Model 0.108 deg Carribean & Central America Domain (25mb) - 3hourly through hr 60
              hrrrpt        Personal tile version of the full High Resolution Rapid Refresh (HRRR) data on pressure surfaces - 3km, hourly out through 15 hours
              gfspt         Personal tile version of the GFS 0.50 degree dataset on pressure surfaces - 3-hourly through 180 hours
              namaknestpt   NAM Model 3km Alaska nest grid - Isobaric coordinate (Very Large!) - Hourly through 36 hours, 3-hourly through 60 hours
              namnestpt     NAM 3km nested domain - isobaric coordinate (Very Large!) - Hourly through 36 hours, 3-hourly through 60 hours


      III.  Historical Datasets

              jra55         Japanese 55-year Reanalysis - 1.25 degree analysis data on isobaric surfaces in GRIB 1
              cfsrv2        Climate Forecast System Reanalysis (CFSR) Version 2 - Global, 0.5 degree (4/2011 to present)
              era5          ECMWF ERA5 reanalysis dataset (0.28 degree) - Isobaric coordinates in GRIB 1 & 2 (you decide)
              era20c        ECMWF ERA20 reanalysis dataset (1.125 degree) - Isobaric coordinates in GRIB 1 & 2 (you decide)
              cfsrv1        Climate Forecast System Reanalysis (CFSR) Version 1 - Global, 0.5 degree (1/1979 to 3/2011)
              nnrp          NCEP/NCAR Reanalysis Project 1948 through 2004 dataset 2.5 degree - Isobaric coordinate
              era-i         ECMWF ERA Interim analysis dataset (0.7 degree) - Isobaric coordinates in GRIB 1 or 2
              narr          North American Regional Reanalysis A GRIBs (NARR; 32km 221 grid) GRIB 1 (1/1979 to current)

            Historical Personal Tile Datasets

              cfsrpt        Climate Forecast System Reanalysis (CFSR) Versions 1 & 2 - Global, 0.5 degree (1/1979 to 3/2011, 3/2011 to present)
              narrpt        North American Regional Reanalysis A GRIBs (NARR; 32km 221 grid) GRIB 1 (1/1979 to current)
It sounds, however, like it's a Perl code bug if all the files are actually there, and you might have to go into the perl code. Did you try the --debug parameter for ems_prep? I've only used it for problems with metgrid, I believe, and that's all it may be set up to handle.

Post Reply