Upgrade ABM3 to use ActivitySim v1.3.4

[JoeJimFlood]

Proposed changes

This pull request upgrades the resident, crossborder, visitor, and airport access models to work with ActivitySim version 1.3.2. This version includes the BayDAG enhancements along with more a more memory-efficient skim format. It will also put us in an easier position to incorporate Sharrow in the near future.

Summary of changes:

• Conversion of skims from omx to omxz format, reducing size and time to read, including adding step for conversion after assignment and skimming
• Removing superfluous settings from configuration files as those are no longer allowed
  • max_blend_distance in network LOS settings
  • CHOICE_COL_NAME in tour mode choice settings
• Removed nonexistent fields from TOURS_MERGED_CHOOSER_COLUMNS in the trip mode choice settings for the airport access models
• Changing the filename in the logging configuration to reflect a change in how that is set
• Added compute_settings to tour_od choice settings in crossborder model to prevent origin_destination column from being removed from the tours table in internal memory
• Specified tour mode choice in the airport access models as a nested logit model as a workaround to ActivitySim Issue #908
• Added explicit stating of resident trip scheduling logic version
• Renamed COLS_TO_INCLUDE_IN_ALTERNATIVES_TABLE to COLS_TO_INCLUDE_IN_ALTS_TABLE in vehicle type choice settings
• Specified that the trip_veh_age field in the write trip matrices preprocessor needs to be an integer
• Added empty shadow pricing yaml file to prevent a crash as the existence of such a file is required
• Made changes to extension scripts to reflect Orca removal (this mostly involved copying files from sandag-abm3-example)
• Added model settings objects as ActivitySim 1.3 now stores settings in those and not as dictionaries this mostly involved copying files from sandag-abm3-example)
• Updated code in the scenario manager using ruamel as that has changed with its latest version
• Updated createPMSAomx.py to reflect updates in Geopandas
• Updated batch files to activate new asim_132 environment instead of asim_baydag

Impact

There should be minimal impact on the model results, though during the full run there were some increases in runtime observed. Two base year runs were conducted, one using the BayDAG version of ActivitySim (Scenario 239) and the other using version 1.3.2 (Scenario 249). When switching to ActivitySim 1.3.2 there was a 139,296 increase in loaded network VMT, or a 0.18% increase. The differences in most key metrics were negligible for the two runs, which are shown under further comments.

The test did show an increase in runtime, with the base run taking 43 hours and the run with ActivitySim 1.3.2 taking 47 hours in total. The chart below shows how the difference was in global iteration 3 between the two runs. All of the ActivitySim models took longer with v1.3.2, but some of this was due to the skim memory manager present in the BayDAG version of the code not being in v1.3.2. Iteration 3 took a total of 11 hours and 24 minutes with the BayDAG code and 12 hours and 34 minutes with ActivitySim v1.3.2.

Types of changes

What types of changes does your code introduce to ABM?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

How has this been tested?

Please describe the tests that you ran to verify your changes.

• Each of the resident, crossborder, visitor, and airport access models were tested
• A full base year 2022 run was completed with the updated code and configs

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

Further comments

VMT by mode

VMT by functional class

Resident model mode share

Transit boardings by mode

[dhensle]

Hi Joe, looks pretty good, just a couple of minor suggested edits.

[dhensle]

I think we should be able to delete this whole setting -- we added functionality in phase 9B to remove unnecessary columns automatically.

[JoeJimFlood]

Was that functionality incorporated into release 1.3.2?

[dhensle]

It was added as part of this PR into ActivitySim: ActivitySim/activitysim#833

[dhensle]

Same here, shouldn't need this anymore.

[dhensle]

Suggest adding a comment to this file pointing to the open ActivitySim issue on needing this yaml.

[dhensle]

I think we should set the defaults to None, aka no iterations.

[dhensle]

Suggest adding some inline documentation here as to what this wring call does so future folks have more context.

[dhensle]

@JoeJimFlood Do you have the ActivitySim timing log file output between the BayDAG and the new v1.3 code? I am curious where the differences in runtime are showing up.

[JoeJimFlood]

@dhensle I've attached a comparison of the timing logs, but the run using asim 1.3.2 was shorter despite the overall time for the resident model being longer. I wonder if there's some additional overhead not included in the timing log?
timing_log_comparison.csv

[dhensle]

Yeah the timing log is encouraging. baydag version takes 4.1 hours compared to 3.3 hours for v1.3. There is some additional overhead that doesn't make it into the ActivitySim timing log file, but the overall log file should have an accurate total run time. Can you please take a look at the run time in activitysim.log file for both of these? If the results are pretty consistent to the runtimes in the timing log, the runtime increase must be in another non-resident model step. Otherwise, we should probably create an ActivitySim issue for further investigation.

Interesting that mandatory tour scheduling decreased by a significant amount but non-mandatory scheduling increased. Not sure why that would be...

[JoeJimFlood]

It looks like 4:28 for the BayDAG code and 4:49 for v1.3.2, so that's more consistent with our full runtime summary.

[JoeJimFlood]

I just realized that I need to update src\asim\scrips\environment.yml so I'm going to do that.

[JoeJimFlood]

I updated it but there are some issues that need to be resolved before we can confirm it works.

[JoeJimFlood]

I added TOURS_MERGED_CHOOSER_COLUMNS back to the airport trip mode choice configs because their absence was causing trip mode choice to crash here.