adding Context Length Specialization (CCL)

[quic-vjanfaza]

Context-Length-Specialization technique tries to optimize the throughput of large language models (LLMs) on Qualcomm devices when handling very large context lengths. The current Ahead Of Time (AOT) compilation on Qualcomm devices doesn't predict the number of tokens needed, leading to significant throughput drops during the prefilling and the decoding phases. This happens because the system performs attention calculations based on large context length. To address this issue, we introduce Compute Context Length (CCL), an additional ONNX variable that allows for dynamic context-length specialization. By generating tokens using smaller, more manageable context lengths (CCL), we optimize memory reads and attention calculations, thereby improving throughput.

[quic-rishinr]

Could you please run some models on the Qeff mainline and on this PR with CCL enabled and disabled? This will help us better understand the performance impact introduced by the CCL changes.