[FEATURE REQUEST] Print the names of all files that are being read by extensions

[yantosca]

Overview

This is related to geoschem/geos-chem#106 and geoschem/geos-chem-cloud#25. Right now, not all files read by HEMCO (esp. by extensions) are printed out. Having a printout of all files that are being read will facilitate our efforts to:

1. Implement a GEOS-Chem and HEMCO dryrun feature,
2. Auto-generate scripts that can be used to download only the files that are read by GEOS-Chem (e.g. from the AWS S3 bucket to an EBS).

I will be working on this feature in the GEOS-Chem repo but we will have to also make the equivalent changes in this standalone HEMCO repo.

[jimmielin]

Hi Bob, thanks for opening this. I was planning to achieve feature 1) through directly tapping into hcoio_read_std_mod.f90 so that we can get a list of the files/fields read at a very low level. I was wondering if there are any instances (i.e. in extensions) where I/O is not done through the HCOIO module? Either way making logs more informative in the extensions can be very helpful. Just wanted to know if there are exceptions to the general way of doing IO so we are not missing any files. Thanks!

[yantosca]

Hi Haipeng! I think most base emisisons write the name of the file with "HEMCO: Opening ...". I think for some extensions where direct reads happen (e.g. PARANOX look up table, etc) those file names don't get printed. (Also a few places in GC like FAST-JX and UCX input files). I'll try to flag all the instances of HEMCO I/O not thru hcoio_write_std_mod for you.

[yantosca]

Here are the HEMCO extension modules that read files directly, as opposed to hcoio_read_std_mod.F90. These files are mostly look-up tables as opposed to gridded data.

hcox_lightnox_mod.F90:1130:  OPEN( IU_FILE, FILE=TRIM( FILENAME ), STATUS='OLD', IOSTAT=IOS )
hcox_paranox_mod.F90:1883:   OPEN ( fID, FILE=TRIM(FILENAME), FORM="FORMATTED", IOSTAT=IOS )
hcox_paranox_mod.F90:2086:   OPEN ( fID, FILE=TRIM(FILENAME), ACTION="WRITE", FORM="FORMATTED", IOSTAT=IOS )
hcox_volcano_mod.F90:547:    OPEN ( LUN, FILE=TRIM(ThisFile), STATUS='OLD', IOSTAT=IOS )

[yantosca]

Also note: The above extensions do write the file names being opened to the HEMCO.log file, e.g.

READ_LUT_NCFILE: Reading /home/ubuntu/ExtData/HEMCO/PARANOX/v2015-02/ship_plume_lut_02ms.nc
READ_LUT_NCFILE: Reading /home/ubuntu/ExtData/HEMCO/PARANOX/v2015-02/ship_plume_lut_06ms.nc
READ_LUT_NCFILE: Reading /home/ubuntu/ExtData/HEMCO/PARANOX/v2015-02/ship_plume_lut_10ms.nc
READ_LUT_NCFILE: Reading /home/ubuntu/ExtData/HEMCO/PARANOX/v2015-02/ship_plume_lut_14ms.nc
READ_LUT_NCFILE: Reading /home/ubuntu/ExtData/HEMCO/PARANOX/v2015-02/ship_plume_lut_18ms.nc
     - INIT_LIGHTNOX: Reading /home/ubuntu/ExtData/HEMCO/LIGHTNOX/v2014-07/light_dist.ott2010.dat
AeroCom: reading /home/ubuntu/ExtData/HEMCO/VOLCANO/v2019-08/2016/07/so2_volcanic_emissions_Carns.20160701.rc

We have a minor modification to the GCPy routine core.extract_pathnames_from_log to make the search for path names case-insensitive.

[yantosca]

I think this issue can be closed. As far as I can tell, HEMCO extensions do write all of the file paths to the HEMCO.log file.

Also see: geoschem/geos-chem#106. We had to add more file path printout for GEOS-Chem specific routines in 12.6.1. But we did not have to modify any HEMCO routines.