yul091 / LLM-co-running Public

Notifications You must be signed in to change notification settings
Fork 0
Star 2

Concurrent Training and Serving of Large Language Models on Distributed Systems

2 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
figure		figure
models		models
README.md		README.md
dataset.py		dataset.py
distributed_llm.py		distributed_llm.py
requirements.txt		requirements.txt
utils.py		utils.py

Repository files navigation

LLM-co-running

Concurrent Training and Serving of Large Language Models on Distributed Systems

An illustration of Separate and Mix under concurrent workloads

Separate training and inference

Co-locate training and inference

About

Concurrent Training and Serving of Large Language Models on Distributed Systems

Report repository

Releases

No releases published

Packages

No packages published

Languages

Python 100.0%