Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dask to run tasks #1714

Closed
wants to merge 31 commits into from
+292 −70
Closed

Use dask to run tasks #1714

Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
63b032a
Use dask to run preprocessing tasks (and disable other tasks for now)
bouweandela Sep 2, 2022
4792193
Improve use of dask client as suggested by @zklaus
bouweandela Sep 13, 2022
11b12f1
Restore sequential and parallel task run
bouweandela Sep 13, 2022
6faa8f6
Add missing compute argument to save function for multimodel statisti…
bouweandela Sep 23, 2022
49e3589
Add distributed as a dependency
bouweandela Sep 23, 2022
54edb73
Restore previous API to fix tests
bouweandela Sep 23, 2022
021b3e0
Add xarray as a dependency
bouweandela Sep 23, 2022
53aa585
Merge branch 'main' into dask-distributed
bouweandela Sep 23, 2022
d5cf4f6
Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di…
bouweandela Oct 14, 2022
0f5bda8
Support configuring dask
bouweandela Oct 14, 2022
52af816
Add a suffix to output_directory if it exists instead of stopping
bouweandela Oct 16, 2022
823c731
Fix tests
bouweandela Oct 18, 2022
6f5a6bf
single call to compute
fnattino Nov 7, 2022
b455bbb
Only start cluster if necessary and fix filename-future mapping
bouweandela Nov 10, 2022
05d69f7
Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di…
bouweandela Nov 21, 2022
be991f8
Use iris (https://github.com/SciTools/iris/pull/5031) for saving
bouweandela Nov 21, 2022
93b6da1
Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di…
bouweandela Nov 22, 2022
37bb757
Point to iris branch
bouweandela Dec 5, 2022
2b60264
Merge branch 'main' into dask-distributed
bouweandela Dec 5, 2022
66bdf2e
Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di…
bouweandela Mar 30, 2023
065b8a4
Work in progress
bouweandela Apr 4, 2023
f47d6da
Update branch name
bouweandela Apr 4, 2023
6e25901
Remove type hint
bouweandela Apr 4, 2023
e176745
Get iris from main branch
bouweandela Apr 21, 2023
d6787fa
Merge branch 'main' into dask-distributed
bouweandela Apr 21, 2023
222c7e5
Add default scheduler
bouweandela Apr 28, 2023
b760abb
Use release candidate
bouweandela May 11, 2023
6b35638
Try to install iris from PyPI
bouweandela May 11, 2023
cf79d20
Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di…
bouweandela May 12, 2023
4348953
Update dependencies
bouweandela May 19, 2023
6bf6328
Remove pip install of ESMValTool_sample_data
bouweandela May 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into dask-distributed
@bouweandela
bouweandela authored Apr 21, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
commit d6787fa5a6ee614a5a3bff5130f39db72270d0b3

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.