New feature: downloadRates() - a function to download and convert rate files from HMD, HFD, UN

[tomthe]

I started to work on a new function: I pre-compute rate files in socsim format from input data that I download from the UN World Population Prospects . In the future, I would like to add data from HMD and HFD, of course.

I think this would be a good API:

retrieveRates(folder, countrycode, yearStart, yearEnd, source, supfile)

folder - directory where the rates should be saved (maybe save them to a new subdirectory?)
countrycode -iso2code
yearStart - 1950 by default
yearEnd - 2100 by default
source - "UN" or "HFDHMD" or "combined" .... what do you think?
supfile - name of a supfile that uses the rates. None if you don't want to create one. By default: countrycode_yearStart_yearEnd_source.sup

---

How does it work?

I looked into whether this function could download the data directly from the databases (HMD, UN...), but that would be complicated for this reason:

• Access is restricted:
  • The UN has an API, but limited by API-keys. I requested one and havn't heard back in a few days.
  • HFD has no API? You need a login and manually download the data.
  • HMD - same as HFD

On the other hand, all 3 organizations distribute the data with a Creative Commons license. We can download, modify and share the data, as long as we do appropriate attribution. I will mention the attribution in a print statement and in the rate files as comments.

I use Python to precompute rate files for all years and countries.
I will put them on a webserver.
retrieveRates() will download the correct rate files and save them to the correct place. It will also create a sup file that does a simulation. Users can use that sup file or not. Or they can copy some parts of it.

What do you think?
I will update this issue in the next few days.

[tomthe]

This is the code to

• Download rates from UN, HMD, HFD, etc. (only UN yet)
• convert them into socsim rate files
• save them in a structured directory as zip files. One zip per country and event (death, birth)

https://github.com/tomthe/retrieveVitalRates_for_rsocsim_API

I uploaded the generated files to a MPIDR webserver. Here is an example:

https://user.demogr.mpg.de/theile/socsimratesAPI/v1/UN/de/socsim_fert_DE_rates.zip

mortality rates and other countries are also available there.

[tomthe]

If you see any flaws in my plan or if you have any other suggestions, please tell me.