Improve reporting (phase 2)

[gidden]

FYI @khaeru I tried rebasing #144 on the new master, but got a weird test error. If I merge (fastforward), I get no such issue. If you want to give the rebasing a try, please feel free.

This is built on #144, text is copied for reference below:

This will implement the second phase of the reporting update. Supersedes #126 which was from a branch in a fork; all can commit to this branch. See there for more comments & discussion.

See also:

• Improve reporting (phase 2) message_ix#206 for corresponding changes to message_ix.
• iiasa/message_data#41 for corresponding changes to the MESSAGE global model.
• Improve reporting features message_ix#149 master issue.

Progress
Address requirements from wiki:

• A1. Provide equal support for any type of model that can be built on the ixmp.
• A2. Support use of exogenous data.
• A3. Support basic operations:
  • A3i. Weighted sums using model or exogenous weights.
  • A3ii. Disaggregation using model or exogenous shares.
  • A3iii. Interpolation: linear, exponential, etc. deferred to after 0.2
• A4. Support operations using quantities at different levels of aggregation.
• A5. Support user-defined operations.
• A6. Duplicate or clone existing operations for multiple other sets of inputs or outputs.
• A7. Support renaming of quantities using multiple name mappings:
  • A7i. …given programmatically.
  • A7ii. …loaded from file. deferred to after 0.2
• A8. Compute only a subset of quantities from a larger/complete report.
  • A8i. …for testing/development use-cases.
  • A8ii. …for high-performance use-cases.
  • A8iii. …using command-line arguments or other straightforward inputs to specify the subset.
• A9. Handle units for quantities.
  • Both quantities with provided units, and quantities without units.
  • Derive units for operations such as division and multiplication.
• A10. Provide automated description of how quantities are computed, as text (console output) or image.
• A11. Callable through retixmp (tests require Test rixmp using 'testthat' #135)

PR checklist:

• Tests added (ongoing)
• Documentation added — preview here.
• Description in RELEASE_NOTES.md added

Miscellaneous:

• Choose target release: 0.3 or 1.0?
• Decide: this adds dependencies on dask[array] (thus toolz), graphviz (for dask.visualize), pint, and xarray. Should these be non-optional requirements, or extras (ixmp[reporting])?
• Per @gidden comment in superseded PR, provide user-friendly exception handling when bad units are encountered.
• as discussed 2019-06-19, use ixmp_source 0.2 features to defer loading of item elements from database until data_for_quantity() is needed; keys_for_quantity() only loads metadata.
• as discussed 2019-06-19, add set_filters() to filter item data ex ante, i.e. before other computations take place, by using existing functionality in data_for_quantity().

[khaeru]

The errors (I found them here in the Travis build log) occur because dot is not available. This is one of the graphviz executables, and is used by dask.visualize() to draw the computation graph. I think the clue might be further up in the log:

Installing Homebrew Packages
10.11s$ rvm $brew_ruby do brew bundle --verbose --global
/usr/local/bin/brew tap homebrew/bundle
==> Tapping homebrew/bundle
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-bundle'...
remote: Enumerating objects: 81, done.
remote: Counting objects: 100% (81/81), done.
remote: Compressing objects: 100% (74/74), done.
remote: Total 81 (delta 7), reused 24 (delta 4), pack-reused 0
Unpacking objects: 100% (81/81), done.
Tapped 0 formulae (172 files, 244KB)
Error: Your Homebrew is outdated. Please run `brew update`.
Error: Kernel.exit

These lines in .travis.yaml are supposed to cause that package to be installed on OS X:

addons:
  homebrew:
    packages:
    - graphviz

…but it looks like they never complete. No clue why this is occurring—maybe a one-time glitch on Travis' workers? Will dig more…

[gidden]

I also got an error on the aggregation test locally (on the rebase effort, not the merge effort)

[gidden]

restarting CI for failed job to see if that helps..

[gidden]

went ahead and cherry-picked the commit in #149 here to keep moving forward

[khaeru]

As of now this branch is feature-complete for the portions we intend to include in ixmp 0.2/message_ix 1.2.

Further commits should only be made in order to help complete iiasa/message_ix#206.

[gidden]

Should we add a warning in the docs that reporting is only py3 compatible?