1 page.title=Design For Reduced Latency 2 @jd:body 3 4 <!-- 5 Copyright 2013 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <p> 28 The Android 4.1 release introduced internal framework changes for 29 a <a href="http://en.wikipedia.org/wiki/Low_latency">lower latency</a> 30 audio output path. There were minimal public client API 31 or HAL API changes. This document describes the initial design, 32 which has continued to evolve over time. 33 Having a good understanding of this design should help device OEM and 34 SoC vendors implement the design correctly on their particular devices 35 and chipsets. This article is not intended for application developers. 36 </p> 37 38 <h2 id="trackCreation">Track Creation</h2> 39 40 <p> 41 The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the 42 <code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or 43 <code>AudioTrack::set()</code>. Currently the only clients that do so are: 44 </p> 45 46 <ul> 47 <li>Android native audio based on OpenSL ES</li> 48 <li><a href="http://developer.android.com/reference/android/media/SoundPool.html">android.media.SoundPool</a></li> 49 <li><a href="http://developer.android.com/reference/android/media/ToneGenerator.html">android.media.ToneGenerator</a></li> 50 </ul> 51 52 <p> 53 The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code> 54 request and may optionally deny the request at client level. If it 55 decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of 56 the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method 57 <code>IAudioFlinger::createTrack()</code>. 58 </p> 59 60 <p> 61 The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may 62 optionally deny the request at server level. It informs the client 63 whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the 64 shared memory control block. 65 </p> 66 67 <p> 68 The factors that impact the decision include: 69 </p> 70 71 <ul> 72 <li>Presence of a fast mixer thread for this output (see below)</li> 73 <li>Track sample rate</li> 74 <li>Presence of a client thread to execute callback handlers for this track</li> 75 <li>Track buffer size</li> 76 <li>Available fast track slots (see below)</li> 77 </ul> 78 79 <p> 80 If the client's request was accepted, it is called a "fast track." 81 Otherwise it's called a "normal track." 82 </p> 83 84 <h2 id="mixerThreads">Mixer Threads</h2> 85 86 <p> 87 At the time AudioFlinger creates a normal mixer thread, it decides 88 whether or not to also create a fast mixer thread. Both the normal 89 mixer and fast mixer are not associated with a particular track, 90 but rather with a set of tracks. There is always a normal mixer 91 thread. The fast mixer thread, if it exists, is subservient to the 92 normal mixer thread and acts under its control. 93 </p> 94 95 <h3 id="fastMixer">Fast mixer</h3> 96 97 <h4>Features</h4> 98 99 <p> 100 The fast mixer thread provides these features: 101 </p> 102 103 <ul> 104 <li>Mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li> 105 <li>Per track attenuation</li> 106 </ul> 107 108 <p> 109 Omitted features: 110 </p> 111 112 <ul> 113 <li>Per track sample rate conversion</li> 114 <li>Per track effects</li> 115 <li>Per mix effects</li> 116 </ul> 117 118 <h4>Period</h4> 119 120 <p> 121 The fast mixer runs periodically, with a recommended period of two 122 to three milliseconds (ms), or a slightly higher period of five ms if needed for scheduling stability. 123 This number was chosen so that, accounting for the complete 124 buffer pipeline, the total latency is on the order of 10 ms. Smaller 125 values are possible but may result in increased power consumption 126 and chance of glitches depending on CPU scheduling predictability. 127 Larger values are possible, up to 20 ms, but result in degraded 128 total latency and so should be avoided. 129 </p> 130 131 <h4>Scheduling</h4> 132 133 <p> 134 The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very 135 little CPU time, but must run often and with low scheduling jitter. 136 <a href="http://en.wikipedia.org/wiki/Jitter">Jitter</a> 137 expresses the variation in cycle time: it is the difference between the 138 actual cycle time versus the expected cycle time. 139 Running too late will result in glitches due to underrun. Running 140 too early will result in glitches due to pulling from a fast track 141 before the track has provided data. 142 </p> 143 144 <h4>Blocking</h4> 145 146 <p> 147 Ideally the fast mixer thread never blocks, other than at HAL 148 <code>write()</code>. Other occurrences of blocking within the fast mixer are 149 considered bugs. In particular, mutexes are avoided. 150 Instead, <a href="http://en.wikipedia.org/wiki/Non-blocking_algorithm">non-blocking algorithms</a> 151 (also known as lock-free algorithms) are used. 152 See <a href="avoiding_pi.html">Avoiding Priority Inversion</a> for more on this topic. 153 </p> 154 155 <h4>Relationship to other components</h4> 156 157 <p> 158 The fast mixer has little direct interaction with clients. In 159 particular, it does not see binder-level operations, but it does 160 access the client's shared memory control block. 161 </p> 162 163 <p> 164 The fast mixer receives commands from the normal mixer via a state queue. 165 </p> 166 167 <p> 168 Other than pulling track data, interaction with clients is via the normal mixer. 169 </p> 170 171 <p> 172 The fast mixer's primary sink is the audio HAL. 173 </p> 174 175 <h3 id="normalMixer">Normal mixer</h3> 176 177 <h4>Features</h4> 178 179 <p> 180 All features are enabled: 181 </p> 182 183 <ul> 184 <li>Up to 32 tracks</li> 185 <li>Per track attenuation</li> 186 <li>Per track sample rate conversion</li> 187 <li>Effects processing</li> 188 </ul> 189 190 <h4>Period</h4> 191 192 <p> 193 The period is computed to be the first integral multiple of the 194 fast mixer period that is >= 20 ms. 195 </p> 196 197 <h4>Scheduling</h4> 198 199 <p> 200 The normal mixer runs at elevated <code>SCHED_OTHER</code> priority. 201 </p> 202 203 <h4>Blocking</h4> 204 205 <p> 206 The normal mixer is permitted to block, and often does so at various 207 mutexes as well as at a blocking pipe to write its sub-mix. 208 </p> 209 210 <h4>Relationship to other components</h4> 211 212 <p> 213 The normal mixer interacts extensively with the outside world, 214 including binder threads, audio policy manager, fast mixer thread, 215 and client tracks. 216 </p> 217 218 <p> 219 The normal mixer's sink is a blocking pipe to the fast mixer's track 0. 220 </p> 221 222 <h2 id="flags">Flags</h2> 223 224 <p> 225 <code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the 226 request will be fulfilled. 227 </p> 228 229 <p> 230 <code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear 231 in server. 232 </p> 233 234 <p> 235 <code>TRACK_FAST</code> is a client -> server concept. 236 </p> 237
