You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FP8 training is now supported #2546, but has issues with tensor parallelism which is currently gated. MVP for this feature should include:
Plug-and-play support for enable_fp8_training with setting a tensor_parallel_plan
Compatibility with torch.compile
This issue is to track support and the request. It's not clear to me the scope of what needs to be done to support this. @andrewor14 feel free to comment if there are other requirements for MVP for this feature, or if you want to clarify the scope.
The text was updated successfully, but these errors were encountered:
FP8 training is now supported #2546, but has issues with tensor parallelism which is currently gated. MVP for this feature should include:
enable_fp8_training
with setting atensor_parallel_plan
torch.compile
This issue is to track support and the request. It's not clear to me the scope of what needs to be done to support this. @andrewor14 feel free to comment if there are other requirements for MVP for this feature, or if you want to clarify the scope.
The text was updated successfully, but these errors were encountered: