Questions about evaluation results

[yingyibiao]

Thanks for your work!

I'm curious about the evaluation results you reported about Qwen-2.5-Math-7B-Instruct and hkust-nlp/Qwen-2.5-Math-7B-SimpleRL-Zero:

I use your script to evaluate both Qwen-2.5-Math-7B-Instruct and hkust-nlp/Qwen-2.5-Math-7B-SimpleRL-Zero models and get the following results:

The gaps are pretty large such that it will diminish your improvements of Qwen2.5-7B-SimpleRL-Zero if compared to Qwen-2.5-Math-7B-Instruct.

[rabbitjy]

I got similar results

[Zeng-WH]

Could you please provide detailed evaluation parameters so we can see where the problem lies?

[rabbitjy]

Thanks for your response.

I just used the evaluation script as mentioned in readme, and got different results from notion.

---

#Qwen2.5-Math-Instruct Series
PROMPT_TYPE="qwen25-math-cot"

#Qwen2.5-Math-7B-Instruct
export CUDA_VISIBLE_DEVICES="0"
MODEL_NAME_OR_PATH="Qwen/Qwen2.5-Math-7B-Instruct"
OUTPUT_DIR="Qwen2.5-Math-7B-Instruct-Math-Eval"
bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH $OUTPUT_DIR

[yingyibiao]

Thanks for your response.

I just used the evaluation script as mentioned in readme, and got different results from notion.

#Qwen2.5-Math-Instruct Series PROMPT_TYPE="qwen25-math-cot"

#Qwen2.5-Math-7B-Instruct export CUDA_VISIBLE_DEVICES="0" MODEL_NAME_OR_PATH="Qwen/Qwen2.5-Math-7B-Instruct" OUTPUT_DIR="Qwen2.5-Math-7B-Instruct-Math-Eval" bash sh/eval.sh $PROMPT_TYPE $MODEL_NAME_OR_PATH $OUTPUT_DIR

I'm using the same script! @Zeng-WH

[Zhuofeng-Li]

I got similar results @Zeng-WH