Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve reporting (phase 2) #144

Closed
wants to merge 85 commits into from
Closed

Improve reporting (phase 2) #144

wants to merge 85 commits into from

Conversation

khaeru
Copy link
Member

@khaeru khaeru commented Jun 13, 2019 •
edited
Loading

Superseded by #150; text below may be out of date.

This will implement the second phase of the reporting update. Supersedes #126 which was from a branch in a fork; all can commit to this branch. See there for more comments & discussion.

See also:

Progress
Address requirements from wiki:

  • A1. Provide equal support for any type of model that can be built on the ixmp.
  • A2. Support use of exogenous data.
  • A3. Support basic operations:
    • A3i. Weighted sums using model or exogenous weights.
    • A3ii. Disaggregation using model or exogenous shares.
    • A3iii. Interpolation: linear, exponential, etc.
  • A4. Support operations using quantities at different levels of aggregation.
  • A5. Support user-defined operations.
  • A6. Duplicate or clone existing operations for multiple other sets of inputs or outputs.
  • A7. Support renaming of quantities using multiple name mappings:
    • A7i. …given programmatically.
    • A7ii. …loaded from file.
  • A8. Compute only a subset of quantities from a larger/complete report.
    • A8i. …for testing/development use-cases.
    • A8ii. …for high-performance use-cases.
    • A8iii. …using command-line arguments or other straightforward inputs to specify the subset.
  • A9. Handle units for quantities.
    • Both quantities with provided units, and quantities without units.
    • Derive units for operations such as division and multiplication.
  • A10. Provide automated description of how quantities are computed, as text (console output) or image.
  • A11. Callable through retixmp (tests require Test rixmp using 'testthat' #135)

PR checklist:

  • Tests added (ongoing)
  • Documentation added — preview here.
  • Description in RELEASE_NOTES.md added

Miscellaneous:

  • Choose target release: 0.3 or 1.0?
  • Decide: this adds dependencies on dask[array] (thus toolz), graphviz (for dask.visualize), pint, and xarray. Should these be non-optional requirements, or extras (ixmp[reporting])?
  • Per @gidden comment in superseded PR, provide user-friendly exception handling when bad units are encountered.

khaeru added 30 commits May 31, 2019 09:03
@khaeru
add stub class for dask-based reporting
5a70135
@khaeru
add inital methods for Key, Reporting
42a475b
@khaeru
move reporting module to a subtree
128ac81
@khaeru
split reporting module to submodules
67153b7
@khaeru
make Reporter a top-level name
7392ec3
@khaeru
more implementation notes
04e60f3
@khaeru
test Reporter.from_scenario() on Dantzig problem
2c0f187
@khaeru
improve Key, Key.aggregates()
7f3b9d9
@khaeru
keep xarray attributes in calculations
ec12490
@khaeru
test (dis)aggregation on Dantzig
0124e3e
@khaeru
add computation TODO
96097c3
@khaeru
add & test exogenous data input/file output
d4b281f
@khaeru
improve docstrings
bf0713a
@khaeru
retrieve equations and variables in Reporter.from_scenario()
1bcf207
@khaeru
retrieve sets in Reporter.from_scenario()
a15debd 
- use partial() in Key.aggregates() to encode string arguments.
@khaeru
move quantity_as_xr to reporting.utils
4e9342d
@khaeru
implement Reporter.read_config(); parse csv
d5a8b5e
@khaeru
refresh TODOs
bc44c52
@khaeru
use find_packages() in setup.py to install ixmp.reporting
ebd54c1
@khaeru
add dask to requirements
0fc18b5
@khaeru
add xarray to requirements
9b96c27
@khaeru
appease stickler
c99eb8e
@khaeru
fix tests after rebase
6d89e6e
@khaeru
disable py27 builds on Travis and Appveyor
344b0ac
@khaeru
add Reporter.visualize(); improve documentation
52314f2
@khaeru
repeat sets of computations for different nodes (req. A6)
abffd6e
@khaeru
allow reporting from command-line (req. A8(iii))
3d5015e
@khaeru
install dask[array] to force toolz installation on CI?
a13889c
@khaeru
try to resolve conflict/typo in cli.py
b66da6b
@khaeru
accept master typo in cli.py :(
bb5659c
@khaeru khaeru mentioned this pull request Jun 13, 2019
Closed
27 tasks
@khaeru khaeru self-assigned this Jun 13, 2019
@khaeru khaeru added this to the 0.2 milestone Jun 13, 2019
@khaeru khaeru added the enh New features & functionality label Jun 13, 2019
@khaeru khaeru mentioned this pull request Jun 13, 2019
Closed
24 tasks
@gidden gidden mentioned this pull request Jun 13, 2019
Closed
5 tasks
@khaeru khaeru mentioned this pull request Jun 13, 2019
Merged
15 tasks
khaeru and others added 3 commits June 13, 2019 22:33
@khaeru
Add AttrSeries.to_series() no-op
96d98cd
@khaeru
Update comments re: Quantity
d74c493
@gidden @khaeru
Add Quantity-agnostic assertions to ixmp.testing (#147)
4a8b4b3 
* first test for quantity equals

* add test for attrs property

* fix failing test where attrs provided in constructor

* update test_reporting.py to use ixmp.testing and remove Quantity shim

* rm extraneous test

* make some tests more general

* stickler

* add check_attrs option

* add test for check_attrs=False
@gidden gidden mentioned this pull request Jun 18, 2019
Merged
29 tasks
@khaeru khaeru added the reporting Post-processing of model results label Jun 18, 2019
@gidden
Copy link
Member

gidden commented Jun 19, 2019

closing in favor of #150

@gidden gidden closed this Jun 19, 2019
@khaeru khaeru deleted the feature/reporting2 branch April 26, 2020 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@khaeru khaeru

Labels
enh New features & functionality reporting Post-processing of model results
Projects
None yet
Milestone
0.2
Development

Successfully merging this pull request may close these issues.
2 participants
@khaeru @gidden