1 page.title=TV Audio 2 @jd:body 3 4 <!-- 5 Copyright 2014 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <p>The TV Input Framework (TIF) manager works with the audio routing API to support flexible audio 28 path changes. When a System on Chip (SoC) implements the TV hardware abstraction layer (HAL), each 29 TV input (HDMI IN, Tuner, etc.) provides <code>TvInputHardwareInfo</code> that specifies AudioPort information for audio type and address.</p> 30 31 <ul> 32 <li><b>Physical</b> audio input/output devices have a corresponding AudioPort.</li> 33 <li><b>Software</b> audio output/input streams are represented as AudioMixPort (child class of 34 AudioPort).</li> 35 </ul> 36 37 <p>The TIF then uses AudioPort information for the audio routing API.</p> 38 39 <p><img src="audio/images/ape_audio_tv_tif.png" alt="Android TV Input Framework (TIF)" /> 40 <p class="img-caption"><strong>Figure 1.</strong> TV Input Framework (TIF)</p> 41 42 <h2 id="Requirements">Requirements</h2> 43 44 <p>A SoC must implement the audio HAL with the following audio routing API support:</p> 45 46 <table> 47 <tbody> 48 <tr> 49 <th>Audio Ports</th> 50 <td> 51 <ul> 52 <li>TV Audio Input has a corresponding audio source port implementation.</li> 53 <li>TV Audio Output has a corresponding audio sink port implementation.</li> 54 <li>Can create audio patch between any TV input audio port and any TV output audio port.</li> 55 </ul> 56 </td> 57 </tr> 58 <tr> 59 <th>Default Input</th> 60 <td>AudioRecord (created with DEFAULT input source) must seize <i>virtual null input source</i> for 61 AUDIO_DEVICE_IN_DEFAULT acquisition on Android TV.</td> 62 </tr> 63 <tr> 64 <th>Device Loopback</th> 65 <td>Requires supporting an AUDIO_DEVICE_IN_LOOPBACK input that is a complete mix of all audio output 66 of all the TV output (11Khz, 16bit mono or 48Khz, 16bit mono). Used only for audio capture. 67 </td> 68 </tr> 69 </tbody> 70 </table> 71 72 73 <h2 id="Audio Devices">TV audio devices</h2> 74 75 <p>Android supports the following audio devices for TV audio input/output.</p> 76 77 <h4>system/core/include/system/audio.h</h4> 78 79 <pre> 80 /* output devices */ 81 AUDIO_DEVICE_OUT_AUX_DIGITAL = 0x400, 82 AUDIO_DEVICE_OUT_HDMI = AUDIO_DEVICE_OUT_AUX_DIGITAL, 83 /* HDMI Audio Return Channel */ 84 AUDIO_DEVICE_OUT_HDMI_ARC = 0x40000, 85 /* S/PDIF out */ 86 AUDIO_DEVICE_OUT_SPDIF = 0x80000, 87 /* input devices */ 88 AUDIO_DEVICE_IN_AUX_DIGITAL = AUDIO_DEVICE_BIT_IN | 0x20, 89 AUDIO_DEVICE_IN_HDMI = AUDIO_DEVICE_IN_AUX_DIGITAL, 90 /* TV tuner input */ 91 AUDIO_DEVICE_IN_TV_TUNER = AUDIO_DEVICE_BIT_IN | 0x4000, 92 /* S/PDIF in */ 93 AUDIO_DEVICE_IN_SPDIF = AUDIO_DEVICE_BIT_IN | 0x10000, 94 AUDIO_DEVICE_IN_LOOPBACK = AUDIO_DEVICE_BIT_IN | 0x40000, 95 </pre> 96 97 98 <h2 id="HAL extension">Audio HAL extension</h2> 99 100 <p>The Audio HAL extension for the audio routing API is defined by following:</p> 101 102 <h4>system/core/include/system/audio.h</h4> 103 104 <pre> 105 /* audio port configuration structure used to specify a particular configuration of an audio port */ 106 struct audio_port_config { 107 audio_port_handle_t id; /* port unique ID */ 108 audio_port_role_t role; /* sink or source */ 109 audio_port_type_t type; /* device, mix ... */ 110 unsigned int config_mask; /* e.g AUDIO_PORT_CONFIG_ALL */ 111 unsigned int sample_rate; /* sampling rate in Hz */ 112 audio_channel_mask_t channel_mask; /* channel mask if applicable */ 113 audio_format_t format; /* format if applicable */ 114 struct audio_gain_config gain; /* gain to apply if applicable */ 115 union { 116 struct audio_port_config_device_ext device; /* device specific info */ 117 struct audio_port_config_mix_ext mix; /* mix specific info */ 118 struct audio_port_config_session_ext session; /* session specific info */ 119 } ext; 120 }; 121 struct audio_port { 122 audio_port_handle_t id; /* port unique ID */ 123 audio_port_role_t role; /* sink or source */ 124 audio_port_type_t type; /* device, mix ... */ 125 unsigned int num_sample_rates; /* number of sampling rates in following array */ 126 unsigned int sample_rates[AUDIO_PORT_MAX_SAMPLING_RATES]; 127 unsigned int num_channel_masks; /* number of channel masks in following array */ 128 audio_channel_mask_t channel_masks[AUDIO_PORT_MAX_CHANNEL_MASKS]; 129 unsigned int num_formats; /* number of formats in following array */ 130 audio_format_t formats[AUDIO_PORT_MAX_FORMATS]; 131 unsigned int num_gains; /* number of gains in following array */ 132 struct audio_gain gains[AUDIO_PORT_MAX_GAINS]; 133 struct audio_port_config active_config; /* current audio port configuration */ 134 union { 135 struct audio_port_device_ext device; 136 struct audio_port_mix_ext mix; 137 struct audio_port_session_ext session; 138 } ext; 139 }; 140 </pre> 141 142 <h4>hardware/libhardware/include/hardware/audio.h</h4> 143 144 <pre> 145 struct audio_hw_device { 146 : 147 /** 148 * Routing control 149 */ 150 151 /* Creates an audio patch between several source and sink ports. 152 * The handle is allocated by the HAL and should be unique for this 153 * audio HAL module. */ 154 int (*create_audio_patch)(struct audio_hw_device *dev, 155 unsigned int num_sources, 156 const struct audio_port_config *sources, 157 unsigned int num_sinks, 158 const struct audio_port_config *sinks, 159 audio_patch_handle_t *handle); 160 161 /* Release an audio patch */ 162 int (*release_audio_patch)(struct audio_hw_device *dev, 163 audio_patch_handle_t handle); 164 165 /* Fills the list of supported attributes for a given audio port. 166 * As input, "port" contains the information (type, role, address etc...) 167 * needed by the HAL to identify the port. 168 * As output, "port" contains possible attributes (sampling rates, formats, 169 * channel masks, gain controllers...) for this port. 170 */ 171 int (*get_audio_port)(struct audio_hw_device *dev, 172 struct audio_port *port); 173 174 /* Set audio port configuration */ 175 int (*set_audio_port_config)(struct audio_hw_device *dev, 176 const struct audio_port_config *config); 177 </pre> 178 179 <h2 id="Testing">Testing DEVICE_IN_LOOPBACK</h2> 180 181 <p>To test DEVICE_IN_LOOPBACK for TV monitoring, use the following testing code. After running the 182 test, the captured audio saves to <code>/sdcard/record_loopback.raw</code>, where you can listen to 183 it using <code>ffmeg</code>.</p> 184 185 <pre> 186 <uses-permission android:name="android.permission.MODIFY_AUDIO_ROUTING" /> 187 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> 188 189 AudioRecord mRecorder; 190 Handler mHandler = new Handler(); 191 int mMinBufferSize = AudioRecord.getMinBufferSize(RECORD_SAMPLING_RATE, 192 AudioFormat.CHANNEL_IN_MONO, 193 AudioFormat.ENCODING_PCM_16BIT);; 194 static final int RECORD_SAMPLING_RATE = 48000; 195 public void doCapture() { 196 mRecorder = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, RECORD_SAMPLING_RATE, 197 AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, mMinBufferSize * 10); 198 AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE); 199 ArrayList<AudioPort> audioPorts = new ArrayList<AudioPort>(); 200 am.listAudioPorts(audioPorts); 201 AudioPortConfig srcPortConfig = null; 202 AudioPortConfig sinkPortConfig = null; 203 for (AudioPort audioPort : audioPorts) { 204 if (srcPortConfig == null 205 && audioPort.role() == AudioPort.ROLE_SOURCE 206 && audioPort instanceof AudioDevicePort) { 207 AudioDevicePort audioDevicePort = (AudioDevicePort) audioPort; 208 if (audioDevicePort.type() == AudioManager.DEVICE_IN_LOOPBACK) { 209 srcPortConfig = audioPort.buildConfig(48000, AudioFormat.CHANNEL_IN_DEFAULT, 210 AudioFormat.ENCODING_DEFAULT, null); 211 Log.d(LOG_TAG, "Found loopback audio source port : " + audioPort); 212 } 213 } 214 else if (sinkPortConfig == null 215 && audioPort.role() == AudioPort.ROLE_SINK 216 && audioPort instanceof AudioMixPort) { 217 sinkPortConfig = audioPort.buildConfig(48000, AudioFormat.CHANNEL_OUT_DEFAULT, 218 AudioFormat.ENCODING_DEFAULT, null); 219 Log.d(LOG_TAG, "Found recorder audio mix port : " + audioPort); 220 } 221 } 222 if (srcPortConfig != null && sinkPortConfig != null) { 223 AudioPatch[] patches = new AudioPatch[] { null }; 224 int status = am.createAudioPatch( 225 patches, 226 new AudioPortConfig[] { srcPortConfig }, 227 new AudioPortConfig[] { sinkPortConfig }); 228 Log.d(LOG_TAG, "Result of createAudioPatch(): " + status); 229 } 230 mRecorder.startRecording(); 231 processAudioData(); 232 mRecorder.stop(); 233 mRecorder.release(); 234 } 235 private void processAudioData() { 236 OutputStream rawFileStream = null; 237 byte data[] = new byte[mMinBufferSize]; 238 try { 239 rawFileStream = new BufferedOutputStream( 240 new FileOutputStream(new File("/sdcard/record_loopback.raw"))); 241 } catch (FileNotFoundException e) { 242 Log.d(LOG_TAG, "Can't open file.", e); 243 } 244 long startTimeMs = System.currentTimeMillis(); 245 while (System.currentTimeMillis() - startTimeMs < 5000) { 246 int nbytes = mRecorder.read(data, 0, mMinBufferSize); 247 if (nbytes <= 0) { 248 continue; 249 } 250 try { 251 rawFileStream.write(data); 252 } catch (IOException e) { 253 Log.e(LOG_TAG, "Error on writing raw file.", e); 254 } 255 } 256 try { 257 rawFileStream.close(); 258 } catch (IOException e) { 259 } 260 Log.d(LOG_TAG, "Exit audio recording."); 261 } 262 </pre> 263 264 <p>Locate the captured audio file in <code>/sdcard/record_loopback.raw</code> and listen to it using 265 <code>ffmeg</code>:</p> 266 267 <pre> 268 adb pull /sdcard/record_loopback.raw 269 ffmpeg -f s16le -ar 48k -ac 1 -i record_loopback.raw record_loopback.wav 270 ffplay record_loopback.wav 271 </pre> 272 273 <h2 id="Use cases">Use cases</h2> 274 275 <p>This section includes common use cases for TV audio.</p> 276 277 <h3>TV tuner with speaker output</h3> 278 279 <p>When a TV tuner becomes active, the audio routing API creates an audio patch between the tuner 280 and the default output (e.g. the speaker). The tuner output does not require decoding, but final 281 audio output is mixed with software output_stream.</p> 282 283 <p><img src="audio/images/ape_audio_tv_tuner.png" alt="Android TV Tuner Audio Patch" /> 284 <p class="img-caption"> 285 <strong>Figure 2.</strong> Audio Patch for TV tuner with speaker output.</p> 286 287 288 <h3>HDMI OUT during live TV</h3> 289 290 <p>A user is watching live TV then switches to the HDMI audio output (Intent.ACTION_HDMI_AUDIO_PLUG) 291 . The output device of all output_streams changes to the HDMI_OUT port, and the TIF manager changes 292 the sink port of the existing tuner audio patch to the HDMI_OUT port.</p> 293 294 <p><p><img src="audio/images/ape_audio_tv_hdmi_tuner.png" alt="Android TV HDMI-OUT Audio Patch" /> 295 <p class="img-caption"> 296 <strong>Figure 3.</strong> Audio Patch for HDMI OUT from live TV.</p>
