Classification application for Windows on Snapdragon® with MobileNet-V2 using ONNX runtime.

The app demonstrates how to use the QNN execution provider to accelerate the model using the Snapdragon® Neural Processing Unit (NPU).

• Snapdragon® Platform (e.g. X Elite)
• Windows 11+

• Visual Studio 22
  • Download any variant of Visual Studio here
  • Make sure Desktop development with C++ tools are selected during installation or installed separately later
• QAIRT SDK: Qualcomm AI Runtime SDK (see QNN SDK for older versions)
  • Download and install the latest Qualcomm AI Engine Direct SDK
  • Make libraries from <QNN_SDK>/libs/<target_platform> accessible to app target binary
    • Option 1: add <QNN_SDK>/libs/<target_platform> in $PATH environment variable
    • Option 2: copy libraries from <QNN_SDK>/libs/<target_platform> in same directory as executable
  • e.g. on Windows on Snapdragon®, <QNN_SDK>/libs/aarch64-windows-msvc or <QNN_SDK>/libs/arm64x-windows-msvc should be added in $PATH environment variable.

Download classification MobileNet-V2 ONNX model from AI Hub and place into <project directory>/assets/models/ directory

1. Open Classification.sln

2. Setting up dependencies

   • NuGet packages

     • NuGet packages should automatically restore in Visual Studio during build
     • If packages are not restored automatically, try the following:
       • If prompted by Visual Studio to restore NuGet packages
         • Click on restore to restore all NuGet packages
       • Otherwise,
         • Go to Project -> Manage NuGet packages in Visual studio
         • Install ONNX-Runtime-QNN 1.19.0

   • vcpkg packages

     • Project is configured to work with vcpkg in manifest mode

     • If opencv headers are missing, vcpkg is not setup correctly.

     • Integrate vcpkg with Visual Studio:

       • Go to View -> Terminal in Visual studio
       • Run the following command in terminal

       vcpkg integrate install

3. Build project in Visual Studio

   • It takes around 10 mins to build on X Elite.

Please ensure you have followed Downloading model from AI Hub section and placed mobilenet_v2.onnx into .\assets\models\mobilenet_v2.onnx

Visual studio project is configured with the following command arguments:

--model .\assets\models\mobilenet_v2.onnx --image .\assets\images\keyboard.jpg

You can simply run the app from Visual Studio to run classification on sample image.

.\ARM64\Debug\Classification.exe --model .\assets\models\mobilenet_v2.onnx --image .\assets\images\keyboard.jpg

You can additionally run --help to get more information about all available options:

.\ARM64\Debug\Classification.exe --help

Please refer to QNN EP options that can be provided as --qnn_options to the app.

1. Model input resolution: 224x224
   • If input image is of different shape, it's resized to 224x224
   • You can override model input dimensions if model uses different spatial image dimensions
2. App is built to work with post-processed outputs
   • App processes output logits and produces consumable output as Class Label.
   • If you want to try out any other model than Yolo (with post-processing included in model), please update output handling accordingly.

1. QNN SetupBackend failed:

   QNN SetupBackend failed: Unable to load backend, error: load library failed
   • QNN libraries are not set up correctly and at runtime backend libs were not found.
   • Please refer to Tools and SDK and ensure required libs are either in PATH environment variable or copied into target directory

2. How do I use a model with different input shape than 224x224?

   • Use --model_input_ht / --model_input_wt to model input dimensions.

3. I have a model that does have different post-processing. Can I still use the app?

   • You will have to modify the app and add the necessary post-processing to accommodate that models.

Following section describes how to configure similar project with NuGet and vcpkg from scratch:

1. Start empty Visual Studio project
2. Setup NuGet to install ONNXRuntime QNN Execution provider
   • Go to Project -> Manage NuGet Packages
   • Look up and install the following packages
     • ONNX-Runtime-QNN
3. Set up Visual Studio for vcpkg

   • Enable vcpkg manifest mode

     • Go to Project Setting
     • General -> vcpkg
     • Enable Manifest mode

   • Add OpenCV dependency in vcpkg

     • Run the following commands in Visual Studio powershell:

     vcpkg —new application
     vcpkg add port opencv

     This creates vcpkg.json and adds opencv depedency

4. Now project is setup to work with vcpkg and NuGet package manager