Adding compilation support for io_encrypt flag #393
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
asmigosw
requested review from
quic-rishinr,
ochougul and
quic-amitraj
as code owners
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
abukhoy
reviewed
May 8, 2025
quic-hemagnih
approved these changes
May 8, 2025
You can get info on all these extra params using: |
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
quic-rishinr
requested changes
May 8, 2025
| @@ -1635,6 +1635,11 @@ def compile( | |||
| **compiler_options, | |||
| ) | |||
|
|
|||
| if compiler_options.get("io_encrypt", None): | |||
There was a problem hiding this comment.
This should happen in the base class. User can pass this flag for any auto model class
mohiso22
pushed a commit
to mohiso22/efficient_transformers
that referenced
this pull request
May 14, 2025
As part of Model-IP I/O encryption feature qaic-exec has added a new command line option: `-io-encrypt` to enable support for decryption/encryption of network I/O. Added support in Qeff to pass this param to qaic-exec when a model needs to be compiled with I/O encryption support. CLI command: `python -m QEfficient.cloud.infer --model_name gpt2 --batch_size 1 --prompt_len 32 --ctx_len 128 --mxfp6 --num_cores 16 --device_group [0] --prompt "My name is" --mos 1 --aic_enable_depth_first --io-encrypt "chacha20"` Note: If the flag is passed via infer CLI, CLI will exit after QPC generation. --------- Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> Signed-off-by: Mohit Soni <quic_mohisoni@quicinc.com>
mohiso22
pushed a commit
to mohiso22/efficient_transformers
that referenced
this pull request
May 14, 2025
As part of Model-IP I/O encryption feature qaic-exec has added a new command line option: `-io-encrypt` to enable support for decryption/encryption of network I/O. Added support in Qeff to pass this param to qaic-exec when a model needs to be compiled with I/O encryption support. CLI command: `python -m QEfficient.cloud.infer --model_name gpt2 --batch_size 1 --prompt_len 32 --ctx_len 128 --mxfp6 --num_cores 16 --device_group [0] --prompt "My name is" --mos 1 --aic_enable_depth_first --io-encrypt "chacha20"` Note: If the flag is passed via infer CLI, CLI will exit after QPC generation. --------- Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> Signed-off-by: Mohit Soni <quic_mohisoni@quicinc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of Model-IP I/O encryption feature qaic-exec has added a new command line option:
-io-encryptto enable support for decryption/encryption of network I/O.Added support in Qeff to pass this param to qaic-exec when a model needs to be compiled with I/O encryption support.
CLI command:
python -m QEfficient.cloud.infer --model_name gpt2 --batch_size 1 --prompt_len 32 --ctx_len 128 --mxfp6 --num_cores 16 --device_group [0] --prompt "My name is" --mos 1 --aic_enable_depth_first --io-encrypt "chacha20"Note: If the flag is passed via infer CLI, CLI will exit after QPC generation.