Skip to content

Commit fc87ad3

Browse files
tishizakiYuqi Zhang
authored and
Yuqi Zhang
committed
[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (vllm-project#17285)
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com> Signed-off-by: Yuqi Zhang <yuqizhang@google.com>
1 parent d89a110 commit fc87ad3

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

benchmarks/backend_request_func.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,7 @@ async def async_request_deepspeed_mii(
201201
timeout=AIOHTTP_TIMEOUT) as session:
202202

203203
payload = {
204+
"model": request_func_input.model,
204205
"prompt": request_func_input.prompt,
205206
"max_tokens": request_func_input.output_len,
206207
"temperature": 0.01, # deepspeed-mii does not accept 0.0 temp.

0 commit comments

Comments
 (0)