Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Comparer internal data structure as xarray Dataset #172

Merged
merged 38 commits into from
Mar 24, 2023
Merged

Comparer internal data structure as xarray Dataset #172

merged 38 commits into from
Mar 24, 2023

Conversation

jsmariegaard
Copy link
Member

@jsmariegaard jsmariegaard commented Feb 24, 2023 •
edited
Loading

The purpose of this PR is to replace the internal data structure of the Comparer. It should have a single data attribute: an xarray Dataset. This dataset should completely define the Comparer allowing for easier filtering and persistence to disk.

@jsmariegaard
simplify, remove unused
2170e87
@jsmariegaard
fix problems with n_variables; rename index to time; rename to raw_mo…
eb2066a 
…d_data
@jsmariegaard
SingleObsComparer does not have n_variables
fe11b1d
@jsmariegaard
remove instance attributes
53ab03c
@jsmariegaard
n_points instead of len(comparer.data)
c5d4a6c
@jsmariegaard
n_points instead of len(data)
65570dc
@jsmariegaard
residual hist to new test
660ab02
@jsmariegaard
dropna not possible on dataset
d6d8b3b
@jsmariegaard
sel instead of loc
98fa2a8
@jsmariegaard
no index property and remove @python3_7_or_above
f757326
@jsmariegaard
change .data to xr.Dataset, introduce .time, change add_model_data to…
2be200c 
… parse_modeldata
@jsmariegaard
fix remove bias
b0eeca1
@jsmariegaard
x and y properties; from_compared_data
11c7dd3
@jsmariegaard
first version of sel method
50a53dd
@jsmariegaard
skip init if from compared data
f0c915b
@jsmariegaard
don't refer to observation
3e4603d
@jsmariegaard
remove observation from comparer
a1862f1
@jsmariegaard
remove observation object from comparer - move some obs fields to dat…
0977a61 
…a.attrs
@jsmariegaard
don't refer variables by id, only name
c645a5c
@jsmariegaard
avoid using set when creating unique list, as it may change order
ce6edd4
@jsmariegaard
mod_names
7ba8d14
@jsmariegaard
remove attributes from ComparerCollection, reorganize
a1c1dad
@jsmariegaard
re-organization
956309f
@jsmariegaard
sel method for cc
ebf69e1
@jsmariegaard
remove df argument from all methods, replace sel_df with sel, fix pro…
12ca488 
…blems in sel()
@jsmariegaard
split mean_skill into new test
98004a8
@jsmariegaard
split test in two, temporary disable testing of mean_skill_points()
dbcebd9
@jsmariegaard
fix mean_skill
85c6f3f
@jsmariegaard
remove sel_df and references
94e856f
@jsmariegaard
Merge branch 'main' into Comparer-data-xarray
f101004
@jsmariegaard jsmariegaard marked this pull request as ready for review March 24, 2023 07:43
ecomodeller
ecomodeller approved these changes Mar 24, 2023
View reviewed changes
Copy link
Member

@ecomodeller ecomodeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in this PR ✅

But we are not done, there is still some untested functionality and optimal handling of empty collections to be considered🤔

@jsmariegaard jsmariegaard merged commit 70328ad into main Mar 24, 2023
@jsmariegaard jsmariegaard deleted the Comparer-data-xarray branch March 24, 2023 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ecomodeller ecomodeller ecomodeller approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jsmariegaard @ecomodeller