Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC]: quant llm from alpindale #8716

Closed
1 task done
flozi00 opened this issue Sep 22, 2024 · 1 comment · May be fixed by #8751
Closed
1 task done

[RFC]: quant llm from alpindale #8716

flozi00 opened this issue Sep 22, 2024 · 1 comment · May be fixed by #8751
Labels

Comments

@flozi00
Copy link
Contributor

flozi00 commented Sep 22, 2024

Motivation.

Higher throughput und memory savings are always cool 😎

I think that could be integrated very easily, what do you think about it's design ?

Proposed Change.

aphrodite-engine/aphrodite-engine@7317765

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@flozi00 flozi00 added the RFC label Sep 22, 2024
@flozi00
Copy link
Contributor Author

flozi00 commented Sep 24, 2024

#8751

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant