1 <html devsite> 2 <head> 3 <title>Design For Reduced Latency</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 <p> 27 The Android 4.1 release introduced internal framework changes for 28 a <a href="http://en.wikipedia.org/wiki/Low_latency">lower latency</a> 29 audio output path. There were minimal public client API 30 or HAL API changes. This document describes the initial design, 31 which has continued to evolve over time. 32 Having a good understanding of this design should help device OEM and 33 SoC vendors implement the design correctly on their particular devices 34 and chipsets. This article is not intended for application developers. 35 </p> 36 37 <h2 id="trackCreation">Track Creation</h2> 38 39 <p> 40 The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the 41 <code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or 42 <code>AudioTrack::set()</code>. Currently the only clients that do so are: 43 </p> 44 45 <ul> 46 <li>Android native audio based on <a 47 href="https://developer.android.com/ndk/guides/audio/opensl/index.html">OpenSL 48 ES</a> or <a 49 href="https://developer.android.com/ndk/guides/audio/aaudio/aaudio.html">AAudio</a></li> 50 <li><a href="http://developer.android.com/reference/android/media/SoundPool.html">android.media.SoundPool</a></li> 51 <li><a href="http://developer.android.com/reference/android/media/ToneGenerator.html">android.media.ToneGenerator</a></li> 52 </ul> 53 54 <p> 55 The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code> 56 request and may optionally deny the request at client level. If it 57 decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of 58 the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method 59 <code>IAudioFlinger::createTrack()</code>. 60 </p> 61 62 <p> 63 The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may 64 optionally deny the request at server level. It informs the client 65 whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the 66 shared memory control block. 67 </p> 68 69 <p> 70 The factors that impact the decision include: 71 </p> 72 73 <ul> 74 <li>Presence of a fast mixer thread for this output (see below)</li> 75 <li>Track sample rate</li> 76 <li>Presence of a client thread to execute callback handlers for this track</li> 77 <li>Track buffer size</li> 78 <li>Available fast track slots (see below)</li> 79 </ul> 80 81 <p> 82 If the client's request was accepted, it is called a "fast track." 83 Otherwise it's called a "normal track." 84 </p> 85 86 <h2 id="mixerThreads">Mixer Threads</h2> 87 88 <p> 89 At the time AudioFlinger creates a normal mixer thread, it decides 90 whether or not to also create a fast mixer thread. Both the normal 91 mixer and fast mixer are not associated with a particular track, 92 but rather with a set of tracks. There is always a normal mixer 93 thread. The fast mixer thread, if it exists, is subservient to the 94 normal mixer thread and acts under its control. 95 </p> 96 97 <h3 id="fastMixer">Fast mixer</h3> 98 99 <h4>Features</h4> 100 101 <p> 102 The fast mixer thread provides these features: 103 </p> 104 105 <ul> 106 <li>Mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li> 107 <li>Per track attenuation</li> 108 </ul> 109 110 <p> 111 Omitted features: 112 </p> 113 114 <ul> 115 <li>Per track sample rate conversion</li> 116 <li>Per track effects</li> 117 <li>Per mix effects</li> 118 </ul> 119 120 <h4>Period</h4> 121 122 <p> 123 The fast mixer runs periodically, with a recommended period of two 124 to three milliseconds (ms), or a slightly higher period of five ms if needed for scheduling stability. 125 This number was chosen so that, accounting for the complete 126 buffer pipeline, the total latency is on the order of 10 ms. Smaller 127 values are possible but may result in increased power consumption 128 and chance of glitches depending on CPU scheduling predictability. 129 Larger values are possible, up to 20 ms, but result in degraded 130 total latency and so should be avoided. 131 </p> 132 133 <h4>Scheduling</h4> 134 135 <p> 136 The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very 137 little CPU time, but must run often and with low scheduling jitter. 138 <a href="http://en.wikipedia.org/wiki/Jitter">Jitter</a> 139 expresses the variation in cycle time: it is the difference between the 140 actual cycle time versus the expected cycle time. 141 Running too late will result in glitches due to underrun. Running 142 too early will result in glitches due to pulling from a fast track 143 before the track has provided data. 144 </p> 145 146 <h4>Blocking</h4> 147 148 <p> 149 Ideally the fast mixer thread never blocks, other than at HAL 150 <code>write()</code>. Other occurrences of blocking within the fast mixer are 151 considered bugs. In particular, mutexes are avoided. 152 Instead, <a href="http://en.wikipedia.org/wiki/Non-blocking_algorithm">non-blocking algorithms</a> 153 (also known as lock-free algorithms) are used. 154 See <a href="avoiding_pi.html">Avoiding Priority Inversion</a> for more on this topic. 155 </p> 156 157 <h4>Relationship to other components</h4> 158 159 <p> 160 The fast mixer has little direct interaction with clients. In 161 particular, it does not see binder-level operations, but it does 162 access the client's shared memory control block. 163 </p> 164 165 <p> 166 The fast mixer receives commands from the normal mixer via a state queue. 167 </p> 168 169 <p> 170 Other than pulling track data, interaction with clients is via the normal mixer. 171 </p> 172 173 <p> 174 The fast mixer's primary sink is the audio HAL. 175 </p> 176 177 <h3 id="normalMixer">Normal mixer</h3> 178 179 <h4>Features</h4> 180 181 <p> 182 All features are enabled: 183 </p> 184 185 <ul> 186 <li>Up to 32 tracks</li> 187 <li>Per track attenuation</li> 188 <li>Per track sample rate conversion</li> 189 <li>Effects processing</li> 190 </ul> 191 192 <h4>Period</h4> 193 194 <p> 195 The period is computed to be the first integral multiple of the 196 fast mixer period that is >= 20 ms. 197 </p> 198 199 <h4>Scheduling</h4> 200 201 <p> 202 The normal mixer runs at elevated <code>SCHED_OTHER</code> priority. 203 </p> 204 205 <h4>Blocking</h4> 206 207 <p> 208 The normal mixer is permitted to block, and often does so at various 209 mutexes as well as at a blocking pipe to write its sub-mix. 210 </p> 211 212 <h4>Relationship to other components</h4> 213 214 <p> 215 The normal mixer interacts extensively with the outside world, 216 including binder threads, audio policy manager, fast mixer thread, 217 and client tracks. 218 </p> 219 220 <p> 221 The normal mixer's sink is a blocking pipe to the fast mixer's track 0. 222 </p> 223 224 <h2 id="flags">Flags</h2> 225 226 <p> 227 <code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the 228 request will be fulfilled. 229 </p> 230 231 <p> 232 <code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear 233 in server. 234 </p> 235 236 <p> 237 <code>TRACK_FAST</code> is a client -> server concept. 238 </p> 239 240 </body> 241 </html> 242
