1 page.title=TV Audio 2 @jd:body 3 4 <!-- 5 Copyright 2015 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <p>The TV Input Framework (TIF) manager works with the audio routing API to support flexible audio 28 path changes. When a System on Chip (SoC) implements the TV hardware abstraction layer (HAL), each 29 TV input (HDMI IN, Tuner, etc.) provides <code>TvInputHardwareInfo</code> that specifies AudioPort information for audio type and address.</p> 30 31 <ul> 32 <li><b>Physical</b> audio input/output devices have a corresponding AudioPort.</li> 33 <li><b>Software</b> audio output/input streams are represented as AudioMixPort (child class of 34 AudioPort).</li> 35 </ul> 36 37 <p>The TIF then uses AudioPort information for the audio routing API.</p> 38 39 <p><img src="images/ape_audio_tv_tif.png" alt="Android TV Input Framework (TIF)" /></p> 40 <p class="img-caption"><strong>Figure 1.</strong> TV Input Framework (TIF)</p> 41 42 <h2 id="requirements">Requirements</h2> 43 44 <p>A SoC must implement the audio HAL with the following audio routing API support:</p> 45 46 <table> 47 <tbody> 48 <tr> 49 <th>Audio Ports</th> 50 <td> 51 <ul> 52 <li>TV Audio Input has a corresponding audio source port implementation.</li> 53 <li>TV Audio Output has a corresponding audio sink port implementation.</li> 54 <li>Can create audio patch between any TV input audio port and any TV output audio port.</li> 55 </ul> 56 </td> 57 </tr> 58 <tr> 59 <th>Default Input</th> 60 <td>AudioRecord (created with DEFAULT input source) must seize <i>virtual null input source</i> for 61 AUDIO_DEVICE_IN_DEFAULT acquisition on Android TV.</td> 62 </tr> 63 <tr> 64 <th>Device Loopback</th> 65 <td>Requires supporting an AUDIO_DEVICE_IN_LOOPBACK input that is a complete mix of all audio output 66 of all the TV output (11Khz, 16bit mono or 48Khz, 16bit mono). Used only for audio capture. 67 </td> 68 </tr> 69 </tbody> 70 </table> 71 72 73 <h2 id="audioDevices">TV audio devices</h2> 74 75 <p>Android supports the following audio devices for TV audio input/output.</p> 76 77 <h4>system/media/audio/include/system/audio.h</h4> 78 79 <p class="note"><strong>Note:</strong> In Android 5.1 and earlier, the path to 80 this file is: <code>system/core/include/system/audio.h</code></p> 81 82 <pre> 83 /* output devices */ 84 AUDIO_DEVICE_OUT_AUX_DIGITAL = 0x400, 85 AUDIO_DEVICE_OUT_HDMI = AUDIO_DEVICE_OUT_AUX_DIGITAL, 86 /* HDMI Audio Return Channel */ 87 AUDIO_DEVICE_OUT_HDMI_ARC = 0x40000, 88 /* S/PDIF out */ 89 AUDIO_DEVICE_OUT_SPDIF = 0x80000, 90 /* input devices */ 91 AUDIO_DEVICE_IN_AUX_DIGITAL = AUDIO_DEVICE_BIT_IN | 0x20, 92 AUDIO_DEVICE_IN_HDMI = AUDIO_DEVICE_IN_AUX_DIGITAL, 93 /* TV tuner input */ 94 AUDIO_DEVICE_IN_TV_TUNER = AUDIO_DEVICE_BIT_IN | 0x4000, 95 /* S/PDIF in */ 96 AUDIO_DEVICE_IN_SPDIF = AUDIO_DEVICE_BIT_IN | 0x10000, 97 AUDIO_DEVICE_IN_LOOPBACK = AUDIO_DEVICE_BIT_IN | 0x40000, 98 </pre> 99 100 101 <h2 id="halExtension">Audio HAL extension</h2> 102 103 <p>The Audio HAL extension for the audio routing API is defined by following:</p> 104 105 <h4>system/media/audio/include/system/audio.h</h4> 106 107 <p class="note"><strong>Note:</strong> In Android 5.1 and earlier, the path to 108 this file is: <code>system/core/include/system/audio.h</code></p> 109 110 <pre> 111 /* audio port configuration structure used to specify a particular configuration of an audio port */ 112 struct audio_port_config { 113 audio_port_handle_t id; /* port unique ID */ 114 audio_port_role_t role; /* sink or source */ 115 audio_port_type_t type; /* device, mix ... */ 116 unsigned int config_mask; /* e.g AUDIO_PORT_CONFIG_ALL */ 117 unsigned int sample_rate; /* sampling rate in Hz */ 118 audio_channel_mask_t channel_mask; /* channel mask if applicable */ 119 audio_format_t format; /* format if applicable */ 120 struct audio_gain_config gain; /* gain to apply if applicable */ 121 union { 122 struct audio_port_config_device_ext device; /* device specific info */ 123 struct audio_port_config_mix_ext mix; /* mix specific info */ 124 struct audio_port_config_session_ext session; /* session specific info */ 125 } ext; 126 }; 127 struct audio_port { 128 audio_port_handle_t id; /* port unique ID */ 129 audio_port_role_t role; /* sink or source */ 130 audio_port_type_t type; /* device, mix ... */ 131 unsigned int num_sample_rates; /* number of sampling rates in following array */ 132 unsigned int sample_rates[AUDIO_PORT_MAX_SAMPLING_RATES]; 133 unsigned int num_channel_masks; /* number of channel masks in following array */ 134 audio_channel_mask_t channel_masks[AUDIO_PORT_MAX_CHANNEL_MASKS]; 135 unsigned int num_formats; /* number of formats in following array */ 136 audio_format_t formats[AUDIO_PORT_MAX_FORMATS]; 137 unsigned int num_gains; /* number of gains in following array */ 138 struct audio_gain gains[AUDIO_PORT_MAX_GAINS]; 139 struct audio_port_config active_config; /* current audio port configuration */ 140 union { 141 struct audio_port_device_ext device; 142 struct audio_port_mix_ext mix; 143 struct audio_port_session_ext session; 144 } ext; 145 }; 146 </pre> 147 148 <h4>hardware/libhardware/include/hardware/audio.h</h4> 149 150 <pre> 151 struct audio_hw_device { 152 : 153 /** 154 * Routing control 155 */ 156 157 /* Creates an audio patch between several source and sink ports. 158 * The handle is allocated by the HAL and should be unique for this 159 * audio HAL module. */ 160 int (*create_audio_patch)(struct audio_hw_device *dev, 161 unsigned int num_sources, 162 const struct audio_port_config *sources, 163 unsigned int num_sinks, 164 const struct audio_port_config *sinks, 165 audio_patch_handle_t *handle); 166 167 /* Release an audio patch */ 168 int (*release_audio_patch)(struct audio_hw_device *dev, 169 audio_patch_handle_t handle); 170 171 /* Fills the list of supported attributes for a given audio port. 172 * As input, "port" contains the information (type, role, address etc...) 173 * needed by the HAL to identify the port. 174 * As output, "port" contains possible attributes (sampling rates, formats, 175 * channel masks, gain controllers...) for this port. 176 */ 177 int (*get_audio_port)(struct audio_hw_device *dev, 178 struct audio_port *port); 179 180 /* Set audio port configuration */ 181 int (*set_audio_port_config)(struct audio_hw_device *dev, 182 const struct audio_port_config *config); 183 </pre> 184 185 <h2 id="testing">Testing DEVICE_IN_LOOPBACK</h2> 186 187 <p>To test DEVICE_IN_LOOPBACK for TV monitoring, use the following testing code. After running the 188 test, the captured audio saves to <code>/sdcard/record_loopback.raw</code>, where you can listen to 189 it using <code>ffmeg</code>.</p> 190 191 <pre> 192 <uses-permission android:name="android.permission.MODIFY_AUDIO_ROUTING" /> 193 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> 194 195 AudioRecord mRecorder; 196 Handler mHandler = new Handler(); 197 int mMinBufferSize = AudioRecord.getMinBufferSize(RECORD_SAMPLING_RATE, 198 AudioFormat.CHANNEL_IN_MONO, 199 AudioFormat.ENCODING_PCM_16BIT);; 200 static final int RECORD_SAMPLING_RATE = 48000; 201 public void doCapture() { 202 mRecorder = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, RECORD_SAMPLING_RATE, 203 AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, mMinBufferSize * 10); 204 AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE); 205 ArrayList<AudioPort> audioPorts = new ArrayList<AudioPort>(); 206 am.listAudioPorts(audioPorts); 207 AudioPortConfig srcPortConfig = null; 208 AudioPortConfig sinkPortConfig = null; 209 for (AudioPort audioPort : audioPorts) { 210 if (srcPortConfig == null 211 && audioPort.role() == AudioPort.ROLE_SOURCE 212 && audioPort instanceof AudioDevicePort) { 213 AudioDevicePort audioDevicePort = (AudioDevicePort) audioPort; 214 if (audioDevicePort.type() == AudioManager.DEVICE_IN_LOOPBACK) { 215 srcPortConfig = audioPort.buildConfig(48000, AudioFormat.CHANNEL_IN_DEFAULT, 216 AudioFormat.ENCODING_DEFAULT, null); 217 Log.d(LOG_TAG, "Found loopback audio source port : " + audioPort); 218 } 219 } 220 else if (sinkPortConfig == null 221 && audioPort.role() == AudioPort.ROLE_SINK 222 && audioPort instanceof AudioMixPort) { 223 sinkPortConfig = audioPort.buildConfig(48000, AudioFormat.CHANNEL_OUT_DEFAULT, 224 AudioFormat.ENCODING_DEFAULT, null); 225 Log.d(LOG_TAG, "Found recorder audio mix port : " + audioPort); 226 } 227 } 228 if (srcPortConfig != null && sinkPortConfig != null) { 229 AudioPatch[] patches = new AudioPatch[] { null }; 230 int status = am.createAudioPatch( 231 patches, 232 new AudioPortConfig[] { srcPortConfig }, 233 new AudioPortConfig[] { sinkPortConfig }); 234 Log.d(LOG_TAG, "Result of createAudioPatch(): " + status); 235 } 236 mRecorder.startRecording(); 237 processAudioData(); 238 mRecorder.stop(); 239 mRecorder.release(); 240 } 241 private void processAudioData() { 242 OutputStream rawFileStream = null; 243 byte data[] = new byte[mMinBufferSize]; 244 try { 245 rawFileStream = new BufferedOutputStream( 246 new FileOutputStream(new File("/sdcard/record_loopback.raw"))); 247 } catch (FileNotFoundException e) { 248 Log.d(LOG_TAG, "Can't open file.", e); 249 } 250 long startTimeMs = System.currentTimeMillis(); 251 while (System.currentTimeMillis() - startTimeMs < 5000) { 252 int nbytes = mRecorder.read(data, 0, mMinBufferSize); 253 if (nbytes <= 0) { 254 continue; 255 } 256 try { 257 rawFileStream.write(data); 258 } catch (IOException e) { 259 Log.e(LOG_TAG, "Error on writing raw file.", e); 260 } 261 } 262 try { 263 rawFileStream.close(); 264 } catch (IOException e) { 265 } 266 Log.d(LOG_TAG, "Exit audio recording."); 267 } 268 </pre> 269 270 <p>Locate the captured audio file in <code>/sdcard/record_loopback.raw</code> and listen to it using 271 <code>ffmeg</code>:</p> 272 273 <pre> 274 adb pull /sdcard/record_loopback.raw 275 ffmpeg -f s16le -ar 48k -ac 1 -i record_loopback.raw record_loopback.wav 276 ffplay record_loopback.wav 277 </pre> 278 279 <h2 id="useCases">Use cases</h2> 280 281 <p>This section includes common use cases for TV audio.</p> 282 283 <h3 id="tvSpeaker">TV tuner with speaker output</h3> 284 285 <p>When a TV tuner becomes active, the audio routing API creates an audio patch between the tuner 286 and the default output (e.g. the speaker). The tuner output does not require decoding, but final 287 audio output is mixed with software output_stream.</p> 288 289 <img src="images/ape_audio_tv_tuner.png" alt="Android TV Tuner Audio Patch" /> 290 <p class="img-caption"> 291 <strong>Figure 2.</strong> Audio Patch for TV tuner with speaker output.</p> 292 293 294 <h3 id="hdmiOut">HDMI OUT during live TV</h3> 295 296 <p>A user is watching live TV then switches to the HDMI audio output (Intent.ACTION_HDMI_AUDIO_PLUG) 297 . The output device of all output_streams changes to the HDMI_OUT port, and the TIF manager changes 298 the sink port of the existing tuner audio patch to the HDMI_OUT port.</p> 299 300 <img src="images/ape_audio_tv_hdmi_tuner.png" alt="Android TV HDMI-OUT Audio Patch" /> 301 <p class="img-caption"> 302 <strong>Figure 3.</strong> Audio Patch for HDMI OUT from live TV.</p> 303
