1 # Performance benchmarks 2 3 This document lists TensorFlow Lite performance benchmarks when running well 4 known models on some Android and iOS devices. 5 6 These performance benchmark numbers were generated with the 7 [Android TFLite benchmark binary](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark) 8 and the [iOS benchmark app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios). 9 10 # Android performance benchmarks 11 12 For Android benchmarks, the CPU affinity is set to use big cores on the device to 13 reduce variance (see [details](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#reducing-variance-between-runs-on-android)). 14 15 It assumes that models were download and unzipped to the 16 `/data/local/tmp/tflite_models` directory. The benchmark binary is built 17 using [these instructions](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#on-android) 18 and assumed in the `/data/local/tmp` directory. 19 20 To run the benchmark: 21 22 ``` 23 adb shell taskset ${CPU_MASK} /data/local/tmp/benchmark_model \ 24 --num_threads=1 \ 25 --graph=/data/local/tmp/tflite_models/${GRAPH} \ 26 --warmup_runs=1 \ 27 --num_runs=50 \ 28 --use_nnapi=false 29 ``` 30 31 Here, `${GRAPH}` is the name of model and `${CPU_MASK}` is the CPU affinity 32 chosen according to the following table: 33 34 Device | CPU_MASK | 35 -------| ---------- 36 Pixel 2 | f0 | 37 Pixel xl | 0c | 38 39 <table> 40 <thead> 41 <tr> 42 <th>Model Name</th> 43 <th>Device </th> 44 <th>Mean inference time</th> 45 </tr> 46 </thead> 47 <tr> 48 <td rowspan = 2> 49 <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a> 50 </td> 51 <td>Pixel 2 </td> 52 <td>123.3 ms</td> 53 </tr> 54 <tr> 55 <td>Pixel XL </td> 56 <td>113.3 ms</td> 57 </tr> 58 <tr> 59 <td rowspan = 2> 60 <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz">Mobilenet_1.0_224 (quant)</a> 61 </td> 62 <td>Pixel 2 </td> 63 <td>65.4 ms</td> 64 </tr> 65 <tr> 66 <td>Pixel XL </td> 67 <td>74.6 ms</td> 68 </tr> 69 <tr> 70 <td rowspan = 2> 71 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a> 72 </td> 73 <td>Pixel 2 </td> 74 <td>273.8 ms</td> 75 </tr> 76 <tr> 77 <td>Pixel XL </td> 78 <td>210.8 ms</td> 79 </tr> 80 <tr> 81 <td rowspan = 2> 82 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a> 83 </td> 84 <td>Pixel 2 </td> 85 <td>234.0 ms</td> 86 </tr> 87 <tr> 88 <td>Pixel XL </td> 89 <td>158.0 ms</td> 90 </tr> 91 <tr> 92 <td rowspan = 2> 93 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a> 94 </td> 95 <td>Pixel 2 </td> 96 <td>2846.0 ms</td> 97 </tr> 98 <tr> 99 <td>Pixel XL </td> 100 <td>1973.0 ms </td> 101 </tr> 102 <tr> 103 <td rowspan = 2> 104 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a> 105 </td> 106 <td>Pixel 2 </td> 107 <td>3180.0 ms</td> 108 </tr> 109 <tr> 110 <td>Pixel XL </td> 111 <td>2262.0 ms</td> 112 </tr> 113 114 </table> 115 116 # iOS benchmarks 117 118 To run iOS benchmarks, the [benchmark 119 app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios) 120 was modified to include the appropriate model and `benchmark_params.json` was 121 modified to set `num_threads` to 1. 122 123 <table> 124 <thead> 125 <tr> 126 <th>Model Name</th> 127 <th>Device </th> 128 <th>Mean inference time</th> 129 </tr> 130 </thead> 131 <tr> 132 <td> 133 <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a> 134 </td> 135 <td>iPhone 8 </td> 136 <td>32.2 ms</td> 137 </tr> 138 <tr> 139 <td> 140 <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz)">Mobilenet_1.0_224 (quant)</a> 141 </td> 142 <td>iPhone 8 </td> 143 <td>24.4 ms</td> 144 </tr> 145 <tr> 146 <td> 147 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a> 148 </td> 149 <td>iPhone 8 </td> 150 <td>60.3 ms</td> 151 </tr> 152 <tr> 153 <td> 154 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a> 155 </td> 156 <td>iPhone 8 </td> 157 <td>44.3</td> 158 </tr> 159 <tr> 160 <td> 161 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a> 162 </td> 163 <td>iPhone 8</td> 164 <td>562.4 ms</td> 165 </tr> 166 <tr> 167 <td> 168 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a> 169 </td> 170 <td>iPhone 8 </td> 171 <td>661.0 ms</td> 172 </tr> 173 </table> 174
