Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/reporting2 qty testing #147

Merged
merged 9 commits into from
Jun 17, 2019
Merged

Feature/reporting2 qty testing #147

merged 9 commits into from
Jun 17, 2019

Conversation

gidden
Copy link
Member

@gidden gidden commented Jun 14, 2019

@khaeru this extends ixmp.testing to include assert_qty_equal and assert_qty_allclose. some outstanding questions in TODOs. Shim removed, tests pass.

stickler-ci
stickler-ci reviewed Jun 14, 2019
View reviewed changes
gidden
gidden commented Jun 14, 2019
View reviewed changes
ixmp/reporting/utils.py
if 'attrs' in kwargs:
# Use provided attrs
attrs = kwargs.pop('attrs')
elif hasattr(args[0], 'attrs'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is the exact right way to do things, but I had a test fail because attrs was being passed in the constructor with args[0] was an xr.DataArray.

@gidden gidden mentioned this pull request Jun 14, 2019
Merged
@khaeru khaeru merged commit 4a8b4b3 into iiasa:feature/reporting2 Jun 17, 2019
khaeru pushed a commit that referenced this pull request Jun 19, 2019
@gidden @khaeru
Add Quantity-agnostic assertions to ixmp.testing (#147)
c210d5e 
* first test for quantity equals

* add test for attrs property

* fix failing test where attrs provided in constructor

* update test_reporting.py to use ixmp.testing and remove Quantity shim

* rm extraneous test

* make some tests more general

* stickler

* add check_attrs option

* add test for check_attrs=False
khaeru pushed a commit that referenced this pull request Jun 21, 2019
@gidden @khaeru
Improve reporting (phase 2) (#150)
c611c45 
- Add ixmp.reporting submodule meeting requirements from
  https://github.com/iiasa/message_ix/wiki/Reporting#requirements
  - Classes Reporter, Key, 
  - Submodules computations, describe, utils (AttrSeries, Quantity, etc.)
  - CLI using click.
  - Hide code/imports under py27.
  - Implementation notes.
- doc: add documentation of reporting
- ixmp: make Reporter a top-level name
- ixmp.testing: add Quantity-agnostic assertions
- tests:
  - test ixmp.reporting thoroughly
  - skip tests on py27
  - add --testr command-line option/mark
- Packaging:
  - add requirements: click, dask, graphviz, pint, PyYAML, xarray
  - use find_packages() in setup.py to install ixmp.reporting
  - update .gitignore
- CI:
  - install requirements, including graphviz with 'Chocolatey' on AppVeyor
  - set 'python: 3' for Stickler to avoid mistaken E999; ignore W503 which is
    contra PEP8

Full list of commits:

- add stub class for dask-based reporting
- add inital methods for Key, Reporting
- move reporting module to a subtree
- split reporting module to submodules
- make Reporter a top-level name
- more implementation notes
- test Reporter.from_scenario() on Dantzig problem
- improve Key, Key.aggregates()
- keep xarray attributes in calculations
- test (dis)aggregation on Dantzig
- add computation TODO
- add & test exogenous data input/file output
- improve docstrings
- retrieve equations and variables in Reporter.from_scenario()
- retrieve sets in Reporter.from_scenario()
  - use partial() in Key.aggregates() to encode string arguments.
- move quantity_as_xr to reporting.utils
- implement Reporter.read_config(); parse csv
- refresh TODOs
- use find_packages() in setup.py to install ixmp.reporting
- add dask to requirements
- add xarray to requirements
- appease stickler
- fix tests after rebase
- disable py27 builds on Travis and Appveyor
- add Reporter.visualize(); improve documentation
- repeat sets of computations for different nodes (req. A6)
- allow reporting from command-line (req. A8(iii))
- install dask[array] to force toolz installation on CI?
- try to resolve conflict/typo in cli.py
- accept master typo in cli.py :(
- add graphviz to requirements
- install graphviz on CI infra; add PyYAML to requirements
- convert str to Path for subprocess in test_reporting_cli
- same as previous
- install graphviz with 'Chocolatey' on AppVeyor
- Add Reporter.describe() and tests
- Set 'python: 3' for Stickler to avoid mistaken E999
- Fix recursion in Reporter.describe()
- Add weighted aggregation
- Simplify reporting CLI using click
- Pass/derive units for common operations
- Also handle units in reporting.utils.quantity_as_xr; write CSV & XLSX
- Fix reporting CLI test
- Add reporting.utils.rename_dims and .clean_units() for MESSAGEix
- Update gitignore
- Delay load; create reporting.describe
- New computation aggregate2 for grouping
- New 'aggregate' and 'sum' semantics
- Add Reporter.full_key convenience method
- Appease stickler
- Add pint to requirements for CI
- Add click to requirements for CI
- Stickler: ignore W503, which is against PEP8
- hex(id(...)) is not repr() on Windows
- Upper-case previous
- Fix Windows hex(id(...))
- Simplify sum()
- Update documentation
- Make git ignore .coverage
- Simplify keys_for_quantity() to avoid expensive data load
- Use dask.optimization.cull()
- Add Reporter.__contains__
- Fix xarray alignment in computations.product
- Add Keys.product()
- Add Reporter.add_product()
- Import 'make_dantzig' in test_reporting.py
- Place test_reporter_visualize result in a tmp_path
- Handle more configuration options for units
- Handle rename_dims through configure()
- Update test_report_size
- Add more checks for sparseness
- Add warning message in docs
- Fix bug in data_for_quantity logging
- Debug again…
- Disambiguate Reporter from message_ix Reporter for docs
- Handle size == 0 in sparseness check
- first light - dantzig test passes
- trying aggregate
- testing
- Expand AttrSeries/Quantity features
- Add AttrSeries.to_series() no-op
- Update comments re: Quantity
- Add Quantity-agnostic assertions to ixmp.testing (#147)
- first test for quantity equals
- add test for attrs property
- fix failing test where attrs provided in constructor
- update test_reporting.py to use ixmp.testing and remove Quantity shim
- rm extraneous test
- make some tests more general
- stickler
- add check_attrs option
- add test for check_attrs=False
- drop values in computations
- try updating brew before install
- explicitly install
- Move Travis trickery to ci/travis-before_install.sh
- Move reporting docs
- Remove an abortive attempt at reporting.computations.rebase
  - It was picked up somehow in the rebase.
- Remove utf-8 character in reporting/__init__.py (py2 compat)
- Re-add py2.7 tests to Travis; skip for reporting
- Fix previous
- Add test for rudimentary rixmp usage of ixmp.reporting
- Remove more utf-8 characters for py2
- add testr cli
- Support load=False in Scenario._item()
- Use _item(..., load=False) in keys_for_quantity()
- py2 compat: reorder args to Reporter.product()
- Add filters arg to data_for_quantity()
- py2 compat: don't use * in unpacking assignment
- Replace utf-8 character in ixmp.reporting.utils
- Add test_reporting_filters
- Add Reporter.filter()
- Configure reporting filters via configuration file or key
- Fix expected results of Reporter.describe() with filters
- Raise user-friendly exceptions from pint exceptions
- More py2 compat in reporting.utils
- Rename Reporter.filter → Reporter.set_filters
- More flexible test for log records
- py2 compat in report.utils._parse_units
- Further py2 compat
- Change signature of Reporter.add_product()
- Further py2 compat
- py2 compat: require xarray < 0.12
- Further hide ixmp.reporting on py2
- Further hide ixmp.reporting in ixmp.testing, for py2 compat
- Remove utf-8 characters in test_reporting.py
- py2 compat in test_reporting.py
- py2 compat in test_reporting.py 2
- py2 compat in test_reporting.py 3
- py2 compat in test_reporting.py 4
- Skip rixmp tests of ixmp.Reporting on py2
- Final tweaks to documentation
- Avoid importing xarray in ixmp.testing for Appveyor / Python 2.7
- Delay importing xarray on py2 in test_reporting.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stickler-ci stickler-ci stickler-ci left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gidden @stickler-ci @khaeru