Skip to content

[WIP] Support vLLM in transformers hybrid attention implementation #17198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[WIP] Support vLLM in transformers hybrid attention implementation #17198

wants to merge 1 commit into from

Conversation

wuisawesome
Copy link
Contributor

@wuisawesome wuisawesome commented Apr 25, 2025 •
edited by github-actions bot
Loading

No description provided.

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@mergify Mergify
Copy link

mergify bot commented Apr 30, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wuisawesome.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Apr 30, 2025
@wuisawesome wuisawesome force-pushed the alex-transformers-hybrid-attention branch from af87add to c3fd362 Compare May 3, 2025 23:28
@mergify mergify bot removed the needs-rebase label May 3, 2025
@wuisawesome wuisawesome mentioned this pull request May 5, 2025
Merged
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
Merge pull request #2 from character-tech/alex-transformers-hybrid-at…
04752dd 
…tention

transformers hybrid attention implementation vllm-project#17198
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
transformers hybrid attention implementation vllm-project#17198
50b7553 
Handle named parameters

cleanup

actually fix parameter initialization

lint

.

Signed-off-by: Alex <alexwu@character.ai>

.

.

Signed-off-by: Alex <alexwu@character.ai>
@wuisawesome wuisawesome force-pushed the alex-transformers-hybrid-attention branch from 1a626bd to 50b7553 Compare May 5, 2025 19:25
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
transformers hybrid attention implementation vllm-project#17198
f0e5eea 
Signed-off-by: Alex <alexwu@character.ai>
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
Support vLLM in transformers hybrid attention implementation
64fdeaf 
vllm-project vllm vllm-project#17198

Signed-off-by: Alex <alexwu@character.ai>
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
Support vLLM in transformers hybrid attention implementation
babe84a 
vllm-project vllm vllm-project#17198

Signed-off-by: Alex <alexwu@character.ai>
@wuisawesome wuisawesome force-pushed the alex-transformers-hybrid-attention branch from 50b7553 to 792cfb0 Compare May 5, 2025 20:01
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
transformers hybrid attention implementation vllm-project#17198
792cfb0 
Handle named parameters

cleanup

actually fix parameter initialization

lint

.

Signed-off-by: Alex <alexwu@character.ai>

.

.

Signed-off-by: Alex <alexwu@character.ai>

fix comment

Signed-off-by: Alex <alexwu@character.ai>
@wuisawesome
transformers hybrid attention implementation vllm-project#17198
312ab7c 
Handle named parameters

cleanup

actually fix parameter initialization

lint

.

Signed-off-by: Alex <alexwu@character.ai>

.

.

Signed-off-by: Alex <alexwu@character.ai>

fix comment

Signed-off-by: Alex <alexwu@character.ai>

.

Signed-off-by: Alex <alexwu@character.ai>

.

Signed-off-by: Alex <alexwu@character.ai>
@wuisawesome wuisawesome force-pushed the alex-transformers-hybrid-attention branch from c24c55f to 312ab7c Compare May 5, 2025 20:03
wuisawesome added a commit to character-tech/vllm that referenced this pull request May 5, 2025
@wuisawesome
transformers hybrid attention implementation vllm-project#17198
5bc6e99 
Handle named parameters

cleanup

actually fix parameter initialization

lint

.

Signed-off-by: Alex <alexwu@character.ai>

.

.

Signed-off-by: Alex <alexwu@character.ai>

fix comment

Signed-off-by: Alex <alexwu@character.ai>

.

Signed-off-by: Alex <alexwu@character.ai>

.

Signed-off-by: Alex <alexwu@character.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@wuisawesome