Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gymnasium DDP (Sebulba) #26

Merged
merged 18 commits into from
Feb 15, 2025
Merged

Gymnasium DDP (Sebulba) #26

merged 18 commits into from
Feb 15, 2025

Conversation

garymm
Copy link
Owner

@garymm garymm commented Feb 15, 2025

not needed for now

garymm added 16 commits February 12, 2025 18:27
@garymm
get rid of AgentState.inference
e91ec6b 
not needed for now
@garymm
update note
e1bc7fe
@garymm
Step -> Act
1504be2 
I like step terminology better, but "act" is
consistent with Acme and podracers papers which is
more important than my asthetic preferences.
@garymm
change Agent.act to only take actor state and nets
4c3dd0f 
also update terminology to use actor instead of inference in most places
@garymm
use learner terminology
1fc68f2
@garymm
add jupyter kernel extra
677dbf0
@garymm
update terminlogy in metric key
2eb9ad3
@garymm
gymnax_loop: fix application of pmap
193b2e4
@garymm
gymnasium_loop: sebulba style actor-learner split
57544de 
currently working but only with a single actor and learner
@garymm
work aroud orbax checkpoint flakiness
3ab4513
@garymm
gymnasium: multiple actor devices and 2 threads
acf2429
@garymm
provide actor state pre and post to update experience
d9c8c81
@garymm
make loss not take actor state
83efb38
@garymm
WIP: make optimize not take actor state
231168f 
I think I need another method to reset the actor state before a cycle,
or change the code to just have a max value of t and mod it
@garymm
WIP: gymnasium multi learner
c08ef48
@garymm
finish sharding actor state, add prepare for actor cycle
e535603
@codecov-commenter
Copy link

codecov-commenter commented Feb 15, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 98.12834% with 7 lines in your changes missing coverage. Please review.

Project coverage is 96.74%. Comparing base (8b04c5b) to head (491c473).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
earl/environment_loop/gymnasium_loop.py 97.57% 4 Missing ⚠️
earl/experiments/config.py 81.25% 3 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
+ Coverage   96.44%   96.74%   +0.30%     
==========================================
  Files          12       13       +1     
  Lines        1068     1168     +100     
==========================================
+ Hits         1030     1130     +100     
  Misses         38       38

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@garymm garymm enabled auto-merge (squash) February 15, 2025 01:18
@garymm garymm merged commit 15dbe65 into master Feb 15, 2025
4 checks passed
@garymm garymm deleted the garymm/sebulba-2 branch February 15, 2025 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@garymm @codecov-commenter