1 page.title=Design For Reduced Latency 2 @jd:body 3 4 <!-- 5 Copyright 2013 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <p> 28 The Android 4.1 release introduced internal framework changes for 29 a lower latency audio output path. There were no public client API 30 or HAL API changes. This document describes the initial design, 31 which is expected to evolve over time. 32 Having a good understanding of this design should help device OEM and 33 SoC vendors implement the design correctly on their particular devices 34 and chipsets. This article is not intended for application developers. 35 </p> 36 37 <h2 id="trackCreation">Track Creation</h2> 38 39 <p> 40 The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the 41 <code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or 42 <code>AudioTrack::set()</code>. Currently the only clients that do so are: 43 </p> 44 45 <ul> 46 <li>OpenSL ES</li> 47 <li>SoundPool</li> 48 <li>ToneGenerator</li> 49 </ul> 50 51 <p> 52 The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code> 53 request and may optionally deny the request at client level. If it 54 decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of 55 the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method 56 <code>IAudioFlinger::createTrack()</code>. 57 </p> 58 59 <p> 60 The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may 61 optionally deny the request at server level. It informs the client 62 whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the 63 shared memory control block. 64 </p> 65 66 <p> 67 The factors that impact the decision include: 68 </p> 69 70 <ul> 71 <li>presence of a fast mixer thread for this output (see below)</li> 72 <li>track sample rate</li> 73 <li>presence of a client thread to execute callback handlers for this track</li> 74 <li>track buffer size</li> 75 <li>available fast track slots (see below)</li> 76 </ul> 77 78 <p> 79 If the client's request was accepted, it is called a "fast track." 80 Otherwise it's called a "normal track." 81 </p> 82 83 <h2 id="mixerThreads">Mixer Threads</h2> 84 85 <p> 86 At the time AudioFlinger creates a normal mixer thread, it decides 87 whether or not to also create a fast mixer thread. Both the normal 88 mixer and fast mixer are not associated with a particular track, 89 but rather with a set of tracks. There is always a normal mixer 90 thread. The fast mixer thread, if it exists, is subservient to the 91 normal mixer thread and acts under its control. 92 </p> 93 94 <h3 id="fastMixer">Fast mixer</h3> 95 96 <h4>Features</h4> 97 98 <p> 99 The fast mixer thread provides these features: 100 </p> 101 102 <ul> 103 <li>mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li> 104 <li>Per track attenuation</li> 105 </ul> 106 107 <p> 108 Omitted features: 109 </p> 110 111 <ul> 112 <li>Per track sample rate conversion</li> 113 <li>Per track effects</li> 114 <li>Per mix effects</li> 115 </ul> 116 117 <h4>Period</h4> 118 119 <p> 120 The fast mixer runs periodically, with a recommended period of two 121 to three milliseconds (ms), or slightly higher if needed for scheduling stability. 122 This number was chosen so that, accounting for the complete 123 buffer pipeline, the total latency is on the order of 10 ms. Smaller 124 values are possible but may result in increased power consumption 125 and chance of glitches depending on CPU scheduling predictability. 126 Larger values are possible, up to 20 ms, but result in degraded 127 total latency and so should be avoided. 128 </p> 129 130 <h4>Scheduling</h4> 131 132 <p> 133 The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very 134 little CPU time, but must run often and with low scheduling jitter. 135 Running too late will result in glitches due to underrun. Running 136 too early will result in glitches due to pulling from a fast track 137 before the track has provided data. 138 </p> 139 140 <h4>Blocking</h4> 141 142 <p> 143 Ideally the fast mixer thread never blocks, other than at HAL 144 <code>write()</code>. Other occurrences of blocking within the fast mixer are 145 considered bugs. In particular, mutexes are avoided. 146 </p> 147 148 <h4>Relationship to other components</h4> 149 150 <p> 151 The fast mixer has little direct interaction with clients. In 152 particular, it does not see binder-level operations, but it does 153 access the client's shared memory control block. 154 </p> 155 156 <p> 157 The fast mixer receives commands from the normal mixer via a state queue. 158 </p> 159 160 <p> 161 Other than pulling track data, interaction with clients is via the normal mixer. 162 </p> 163 164 <p> 165 The fast mixer's primary sink is the audio HAL. 166 </p> 167 168 <h3 id="normalMixer">Normal mixer</h3> 169 170 <h4>Features</h4> 171 172 <p> 173 All features are enabled: 174 </p> 175 176 <ul> 177 <li>Up to 32 tracks</li> 178 <li>Per track attenuation</li> 179 <li>Per track sample rate conversion</li> 180 <li>Effects processing</li> 181 </ul> 182 183 <h4>Period</h4> 184 185 <p> 186 The period is computed to be the first integral multiple of the 187 fast mixer period that is >= 20 ms. 188 </p> 189 190 <h4>Scheduling</h4> 191 192 <p> 193 The normal mixer runs at elevated <code>SCHED_OTHER</code> priority. 194 </p> 195 196 <h4>Blocking</h4> 197 198 <p> 199 The normal mixer is permitted to block, and often does so at various 200 mutexes as well as at a blocking pipe to write its sub-mix. 201 </p> 202 203 <h4>Relationship to other components</h4> 204 205 <p> 206 The normal mixer interacts extensively with the outside world, 207 including binder threads, audio policy manager, fast mixer thread, 208 and client tracks. 209 </p> 210 211 <p> 212 The normal mixer's sink is a blocking pipe to the fast mixer's track 0. 213 </p> 214 215 <h2 id="flags">Flags</h2> 216 217 <p> 218 <code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the 219 request will be fulfilled. 220 </p> 221 222 <p> 223 <code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear 224 in server. 225 </p> 226 227 <p> 228 <code>TRACK_FAST</code> is a client -> server concept. 229 </p> 230 231
