Hugging Face's logo

qualcomm
/
ResNet-Mixed-Convolution

Video Classification
PyTorch
backbone
android
Model card Files
xet
 Community
1
ResNet-Mixed-Convolution
File size: 11,062 Bytes
1cbfac7
 
76be976
1cbfac7
 
 
4b6cff5
1cbfac7
 
 
 
 
7be4d2d
1cbfac7
 
 
7be4d2d
668f3ed
7be4d2d
 
 
 
 
 
 
 
 
 
 
 
f8dadc6
 
 
 
 
7be4d2d
 
 
 
 
 
668f3ed
7be4d2d
 
 
 
 
 
668f3ed
7be4d2d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e105cbd
 
 
 
 
 
 
 
f8dadc6
e105cbd
 
 
 
 
 
 
 
 
 
f8dadc6
 
e105cbd
f8dadc6
 
 
 
 
 
 
 
 
 
 
 
 
 
e105cbd
f8dadc6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e105cbd
 
 
 
 
 
 
 
f8dadc6
e105cbd
 
f8dadc6
1cbfac7
 
a6d3324
 
1cbfac7
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
---
library_name: pytorch
license: other
tags:
- backbone
- android
pipeline_tag: video-classification

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/web-assets/model_demo.png)

# ResNet-Mixed-Convolution: Optimized for Qualcomm Devices

ResNet Mixed Convolutions is a network with a mixture of 2D and 3D convolutions used for video understanding.

This is based on the implementation of ResNet-Mixed-Convolution found [here](https://github.com/pytorch/vision/blob/main/torchvision/models/video/resnet.py).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/resnet_mixed) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

## Getting Started
There are two ways to deploy this model on your device:

### Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.3 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.54.0/resnet_mixed-onnx-float.zip)
| ONNX | w8a16 | Universal | QAIRT 2.42, ONNX Runtime 1.24.3 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.54.0/resnet_mixed-onnx-w8a16.zip)
| QNN_DLC | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.54.0/resnet_mixed-qnn_dlc-float.zip)
| QNN_DLC | w8a16 | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.54.0/resnet_mixed-qnn_dlc-w8a16.zip)
| TFLITE | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.54.0/resnet_mixed-tflite-float.zip)

For more device-specific assets and performance metrics, visit **[ResNet-Mixed-Convolution on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/resnet_mixed)**.


### Option 2: Export with Custom Configurations

Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/resnet_mixed) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for [ResNet-Mixed-Convolution on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/resnet_mixed) for usage instructions.

## Model Details

**Model Type:** Model_use_case.video_classification

**Model Stats:**
- Model checkpoint: Kinetics-400
- Input resolution: 112x112
- Number of parameters: 11.7M
- Model size (float): 44.6 MB
- Model size (w8a16): 11.5 MB

## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.557 ms | 2 - 221 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Elite Mobile | 7.828 ms | 0 - 215 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® X2 Elite | 7.097 ms | 22 - 22 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® X Elite | 13.846 ms | 22 - 22 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® X Elite | 13.846 ms | 22 - 22 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 9.621 ms | 2 - 275 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Qualcomm® QCS8550 (Proxy) | 13.176 ms | 0 - 28 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.828 ms | 0 - 215 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Qualcomm® QCS9075 | 26.23 ms | 2 - 5 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 3.513 ms | 0 - 191 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Elite Mobile | 5.245 ms | 1 - 192 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® X2 Elite | 4.644 ms | 12 - 12 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® X Elite | 9.055 ms | 11 - 11 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® X Elite | 9.055 ms | 11 - 11 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 6.39 ms | 1 - 257 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS6490 | 1727.557 ms | 44 - 62 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 8.531 ms | 1 - 5 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 5.245 ms | 1 - 192 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 878.876 ms | 101 - 109 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCM6690 | 895.757 ms | 106 - 113 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS9075 | 9.035 ms | 1 - 4 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 878.876 ms | 101 - 109 MB | CPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.765 ms | 2 - 232 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 7.629 ms | 0 - 225 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® X Elite | 14.096 ms | 2 - 2 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® X Elite | 14.096 ms | 2 - 2 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 9.536 ms | 2 - 277 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 13.347 ms | 2 - 5 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8775P | 25.188 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8775P | 25.188 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8775P | 25.188 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA7255P | 97.169 ms | 1 - 222 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 27.561 ms | 2 - 239 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8295P | 26.853 ms | 0 - 194 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.629 ms | 0 - 225 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS9075 | 28.015 ms | 2 - 6 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4.033 ms | 1 - 195 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Elite Mobile | 5.36 ms | 1 - 187 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® X2 Elite | 5.407 ms | 1 - 1 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® X Elite | 10.088 ms | 1 - 1 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® X Elite | 10.088 ms | 1 - 1 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Gen 3 Mobile | 6.768 ms | 1 - 244 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS6490 | 37.133 ms | 1 - 3 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS8550 (Proxy) | 9.439 ms | 1 - 3 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8775P | 9.376 ms | 1 - 191 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8775P | 9.376 ms | 1 - 191 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8775P | 9.376 ms | 1 - 191 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA7255P | 30.036 ms | 1 - 189 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 5.36 ms | 1 - 187 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 7 Gen 4 Mobile | 15.942 ms | 1 - 198 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCM6690 | 184.057 ms | 1 - 210 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8295P | 16.133 ms | 0 - 192 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS9075 | 10.383 ms | 1 - 4 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS8450 (Proxy) | 13.172 ms | 1 - 246 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 7 Gen 4 Mobile | 15.942 ms | 1 - 198 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 197.199 ms | 0 - 257 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Elite Mobile | 202.848 ms | 0 - 257 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 221.931 ms | 2 - 305 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 308.873 ms | 0 - 3 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8775P | 312.014 ms | 0 - 250 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8775P | 312.014 ms | 0 - 250 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8775P | 312.014 ms | 0 - 250 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA7255P | 596.879 ms | 0 - 253 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 345.601 ms | 0 - 270 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8295P | 358.731 ms | 0 - 234 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 202.848 ms | 0 - 257 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS9075 | 314.1 ms | 0 - 28 MB | NPU

## License
* The license for the original implementation of ResNet-Mixed-Convolution can be found
  [here](https://github.com/pytorch/vision/blob/main/LICENSE).

## References
* [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://arxiv.org/abs/1711.11248)
* [Source Model Implementation](https://github.com/pytorch/vision/blob/main/torchvision/models/video/resnet.py)

## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).