1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 <html> 4 5 <head> 6 <title>OpenMAX AL for Android</title> 7 </head> 8 9 <body> 10 11 <h1>OpenMAX AL for Android</h1> 12 13 This article describes the Android native multimedia APIs based on the 14 Khronos Group OpenMAX AL™ 1.0.1 standard, as of Android API level 14 (Android 15 platform version 4.0) and higher. 16 <p> 17 OpenMAX AL is a companion API to OpenSL ES, but for multimedia (video 18 and audio) rather than audio only. 19 <p> 20 Android 4.0 provides a direct, efficient path for low-level streaming multimedia. The new path is 21 ideal for applications that need to maintain complete control over media data before passing it to 22 the platform for presentation. For example, media applications can now retrieve data from any 23 source, apply proprietary encryption/decryption, and then send the data to the platform for display. 24 <p> 25 Applications can now send processed data to the platform as a multiplexed stream of audio/video 26 content in MPEG-2 transport stream format. The platform de-muxes, decodes, and renders the content. 27 The audio track is rendered to the active audio device, while the video track is rendered to either 28 a Surface or a SurfaceTexture. When rendering to a SurfaceTexture, the application can apply 29 subsequent graphics effects to each frame using OpenGL. 30 <p> 31 OpenMAX AL provides a C language interface that is also callable from C++, and 32 exposes features similar to these Android APIs 33 callable from Java programming language code: 34 <ul> 35 <li><a href="http://developer.android.com/reference/android/media/MediaPlayer.html"> 36 android.media.MediaPlayer</a> 37 </ul> 38 39 As with all of the Android Native Development Kit (NDK), the primary 40 purpose of OpenMAX AL for Android is to facilitate the implementation 41 of shared libraries to be called from Java programming language code via Java Native 42 Interface (JNI). NDK is not intended for writing pure C/C++ 43 applications. 44 45 <p> 46 Note: though based on OpenMAX AL, the Android native multimedia API 47 is <i>not</i> a conforming implementation of either OpenMAX AL 1.0.1 48 profile (media player or media player / recorder). This is because Android does not 49 implement all of the features required by either of the profiles. 50 Any known cases where Android behaves differently than the specification 51 are described in section "Android extensions" below. 52 53 The Android OpenMAX AL implementation has limited features, and is 54 intended primarily for certain performance-sensitive native streaming 55 multimedia applications such as video players. 56 <p> 57 The major feature is the ability to play an MPEG-2 transport stream 58 containing a single program stream made up of one H.264 video elementary 59 stream and one AAC audio elementary stream. The application provides 60 the stream via an Android buffer queue data source, which is based on 61 the OpenSL ES buffer queue concept and Android-specific extensions. 62 <p> 63 The video sink is an <code>ANativeWindow *</code> abstract handle, 64 derived from an <code>android.view.Surface</code> ("surface"). 65 A Surface from <code>SurfaceHolder.getSurface()</code> should be used when displaying 66 an unaltered video within a fixed SurfaceView frame. A Surface from 67 <code>new Surface(SurfaceTexture)</code> allows streaming the decoded 68 video frames to an OpenGL ES 2.0 texture, where the frames can be used 69 as input to a shader algorithm in the Graphics Processing Unit (GPU). 70 Be sure to <code>release()</code> the Surface as soon as possible after 71 calling <code>setSurface</code> or ANativeWindow_fromSurface. 72 <p> 73 The audio sink is always an output mix, a device-independent mixer object 74 similar to that of OpenSL ES. 75 76 <h2>Getting started</h2> 77 78 <h3>Example code</h3> 79 80 <h4>Recommended</h4> 81 82 Supported and tested example code, usable as a model 83 for your own code, is located in NDK folder 84 <code>platforms/android-14/samples/native-media/</code>. 85 86 <h4>Not recommended</h4> 87 88 The OpenMAX AL 1.0.1 specification contains example code in the 89 appendices (see section "References" below for the link to this 90 specification). However, the examples in Appendix D: Sample Code 91 use features 92 not supported by Android. Some examples also contain 93 typographical errors, or use APIs that are likely to change. 94 Proceed with caution in referring to these; 95 though the code may be helpful in understanding the full OpenMAX AL 96 standard, it should not be used as is with Android. 97 98 <h3>Adding OpenMAX AL to your application source code</h3> 99 100 OpenMAX AL is a C API, but is callable from both C and C++ code. 101 <p> 102 Add the following lines to your code: 103 <pre> 104 #include <OMXAL/OpenMAXAL.h> 105 #include <OMXAL/OpenMAXAL_Android.h> 106 </pre> 107 108 <h3>Makefile</h3> 109 110 Modify your Android.mk as follows: 111 <pre> 112 LOCAL_LDLIBS += libOpenMAXAL 113 </pre> 114 115 <h3>Multimedia content</h3> 116 117 The only supported way to supply multimedia content is via an MPEG-2 118 transport stream. 119 <p> 120 Finding or creating useful multimedia content for your application is 121 beyond the scope of this article. 122 <p> 123 Note that it is your responsibility to ensure that you are legally 124 permitted to play the content. 125 126 <h3>Debugging</h3> 127 128 For robustness, we recommend that you examine the <code>XAresult</code> 129 value which is returned by most APIs. Use of <code>assert</code> 130 vs. more advanced error handling logic is a matter of coding style 131 and the particular API; see the Wikipedia article on 132 <a href="http://en.wikipedia.org/wiki/Assertion_(computing)">assert</a> 133 for more information. In the supplied example, we have used <code>assert</code> 134 for "impossible" conditions which would indicate a coding error, and 135 explicit error handling for others which are more likely to occur 136 in production. 137 <p> 138 Many API errors result in a log entry, in addition to the non-zero 139 result code. These log entries provide additional detail which can 140 be especially useful for the more complex APIs such as 141 <code>Engine::CreateMediaPlayer</code>. 142 <p> 143 Use <a href="http://developer.android.com/guide/developing/tools/adb.html"> 144 adb logcat</a>, the 145 <a href="http://developer.android.com/guide/developing/eclipse-adt.html"> 146 Eclipse ADT plugin</a> LogCat pane, or 147 <a href="http://developer.android.com/guide/developing/tools/ddms.html#logcat"> 148 ddms logcat</a> to see the log. 149 150 <h2>Supported features from OpenMAX AL 1.0.1</h2> 151 152 This section summarizes available features. In some 153 cases, there are limitations which are described in the next 154 sub-section. 155 156 <h3>Global entry points</h3> 157 158 Supported global entry points: 159 <ul> 160 <li><code>xaCreateEngine</code> 161 <li><code>xaQueryNumSupportedEngineInterfaces</code> 162 <li><code>xaQuerySupportedEngineInterfaces</code> 163 </ul> 164 165 <h3>Objects and interfaces</h3> 166 167 The following figure indicates objects and interfaces supported by 168 Android's OpenMAX AL implementation. A green cell means the feature 169 is supported. 170 171 <p> 172 <img src="chart3.png" alt="Supported objects and interfaces"> 173 174 <h3>Limitations</h3> 175 176 This section details limitations with respect to the supported 177 objects and interfaces from the previous section. 178 179 <h4>Audio</h4> 180 181 The audio stream type cannot be configured; it is always <code>AudioManager.STREAM_MUSIC</code>. 182 <p> 183 Effects are not supported. 184 185 <h4>Dynamic interface management</h4> 186 187 <code>RemoveInterface</code> and <code>ResumeInterface</code> are not supported. 188 189 <h4>Engine</h4> 190 191 Supported: 192 <ul> 193 <li><code>CreateMediaPlayer</code> 194 </ul> 195 196 Not supported: 197 <ul> 198 <li><code>CreateCameraDevice</code> 199 <li><code>CreateRadioDevice</code> 200 <li><code>CreateLEDDevice</code> 201 <li><code>CreateVibraDevice</code> 202 <li><code>CreateMetadataExtractor</code> 203 <li><code>CreateExtensionObject</code> 204 <li><code>GetImplementationInfo</code> 205 </ul> 206 207 For <code>CreateMediaPlayer</code>, these restrictions apply: 208 <ul> 209 <li>audio sink is an output mix data locator 210 <li>video sink is a native display data locator 211 <li>soundbank, LED array, and vibra sinks must be <code>NULL</code> 212 </ul> 213 214 <h4>MIME data format</h4> 215 216 In the current Android implementation of OpenMAX AL, a media player 217 receives its source data as an MPEG-2 transport stream via a 218 buffer queue. 219 <p> 220 The source data locator must be <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> 221 (see "Android extensions" below). 222 <p> 223 The source data format must be <code>XADataFormat_MIME</code>. 224 Initialize <code>mimeType</code> to <code>XA_ANDROID_MIME_MP2TS</code>, 225 and <code>containerType</code> to <code>XA_CONTAINERTYPE_MPEG_TS</code>. 226 <p> 227 The contained transport stream must have a single program with one H.264 228 video elementary stream and one AAC audio elementary stream. 229 230 <h4>Object</h4> 231 232 <code>Resume</code>, <code>RegisterCallback</code>, 233 <code>AbortAsyncOperation</code>, <code>SetPriority</code>, 234 <code>GetPriority</code>, and <code>SetLossOfControlInterfaces</code> 235 are not supported. 236 237 <h4>StreamInformation</h4> 238 239 Use the StreamInformation interface on a media player object to discover 240 when the video metrics (height/width or aspect ratio) are available or 241 changed, and to then get the sizes. 242 <p> 243 244 Supported: 245 <ul> 246 <li><code>RegisterStreamChangeCallback</code> 247 <li><code>QueryMediaContainerInformation</code> 248 <li><code>QueryStreamInformation</code> 249 <li><code>QueryStreamType</code> 250 </ul> 251 252 Not supported: 253 <ul> 254 <li><code>QueryActiveStreams</code> 255 <li><code>QueryStreamName</code> 256 <li><code>SetActiveStream</code> 257 </ul> 258 259 <h4>VideoDecoderCapabilities</h4> 260 261 This interface on the engine object reports video decoder capabilities 262 without interpretation, exactly as claimed by the underlying OpenMAX IL 263 implementation. 264 <p> 265 These fields in <code>XAVideoCodecDescriptor</code> are filled: 266 <ul> 267 <li><code>codecId</code> 268 <li><code>profileSetting</code> 269 <li><code>levelSetting</code> 270 </ul> 271 The other fields are not filled and should be ignored. 272 <p> 273 Applications should rely on the capabilities documented at 274 <a href="http://developer.android.com/guide/appendix/media-formats.html">Android Supported Media Formats</a>, 275 not the information reported by this interface. 276 277 <h3>Data structures</h3> 278 279 Android API level 14 supports these OpenMAX AL 1.0.1 data structures: 280 <ul> 281 <li>XADataFormat_MIME 282 <li>XADataLocator_NativeDisplay 283 <li>XADataLocator_OutputMix 284 <li>XADataSink 285 <li>XADataSource 286 <li>XAEngineOption 287 <li>XAInterfaceID 288 <li>XAMediaContainerInformation 289 <li>XANativeHandle 290 <li>XA*StreamInformation 291 <li>XAVideoCodecDescriptor 292 </ul> 293 294 <h4>XADataLocator_NativeDisplay</h4> 295 296 The native display data locator is used to specify the video sink: 297 <pre> 298 typedef struct XADataLocator_NativeDisplay_ { 299 XAuint32 locatorType; // XA_DATALOCATOR_NATIVEDISPLAY 300 XANativeHandle hWindow; // ANativeWindow * 301 XANativeHandle hDisplay; // NULL 302 } XADataLocator_NativeDisplay; 303 </pre> 304 305 Set the <code>hWindow</code> field to an 306 <code>ANativeWindow *</code> and set <code>hDisplay</code> to <code>NULL</code>. 307 You can get a <code>ANativeWindow *</code> handle from an <code>android.view.Surface</code>, 308 using this NDK function: 309 <pre> 310 #include <android/native_window_jni.h> 311 312 ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface); 313 </pre> 314 Don't forget to free this handle in your shutdown code with <code>ANativeWindow_release</code>. 315 316 <h3>Platform configuration</h3> 317 318 OpenMAX AL for Android is designed for multi-threaded applications, 319 and is thread-safe. 320 <p> 321 OpenMAX AL for Android supports a single engine per application, and 322 up to 32 objects. Available device memory and CPU may further 323 restrict the usable number of objects. 324 <p> 325 <code>xaCreateEngine</code> recognizes, but ignores, these engine options: 326 <ul> 327 <li><code>XA_ENGINEOPTION_THREADSAFE</code> 328 <li><code>XA_ENGINEOPTION_LOSSOFCONTROL</code> 329 </ul> 330 331 OpenMAX AL and OpenSL ES may be used together in the same application. 332 In this case, there is internally a single shared engine object, 333 and the 32 object limit is shared between OpenMAX AL and OpenSL ES. 334 The application should first create both engines, then use both engines, 335 and finally destroy both engines. The implementation maintains a 336 reference count on the shared engine, so that it is correctly destroyed 337 at the second destroy. 338 339 <h2>Planning for future versions of OpenMAX AL</h2> 340 341 The Android native multimedia APIs at level 14 are based on Khronos 342 Group OpenMAX AL 1.0.1 (see section "References" below). 343 As of the time of this writing, Khronos has recently released 344 a revised version 1.1 of the standard. The revised version 345 includes new features, clarifications, correction of 346 typographical errors, and some incompatibilities. Most of the 347 incompatibilities are relatively minor, or are in areas of OpenMAX AL 348 not supported by Android. However, even a small change 349 can be significant for an application developer, so it is important 350 to prepare for this. 351 <p> 352 The Android team is committed to preserving future API binary 353 compatibility for developers to the extent feasible. It is our 354 intention to continue to support future binary compatibility of the 355 1.0.1-based API, even as we add support for later versions of the 356 standard. An application developed with this version should 357 work on future versions of the Android platform, provided that 358 you follow the guidelines listed in section "Planning for 359 binary compatibility" below. 360 <p> 361 Note that future source compatibility will <i>not</i> be a goal. That is, 362 if you upgrade to a newer version of the NDK, you may need to modify 363 your application source code to conform to the new API. We expect 364 that most such changes will be minor; see details below. 365 366 <h3>Planning for binary compatibility</h3> 367 368 We recommend that your application follow these guidelines, 369 to improve future binary compatibility: 370 <ul> 371 <li> 372 Use only the documented subset of Android-supported features from 373 OpenMAX AL 1.0.1. 374 <li> 375 Do not depend on a particular result code for an unsuccessful 376 operation; be prepared to deal with a different result code. 377 <li> 378 Application callback handlers generally run in a restricted context, 379 and should be written to perform their work quickly and then return 380 as soon as possible. Do not do complex operations within a callback 381 handler. For example, within a buffer queue completion callback, 382 you can enqueue another buffer, but do not create a media player. 383 <li> 384 Callback handlers should be prepared to be called more or less 385 frequently, to receive additional event types, and should ignore 386 event types that they do not recognize. Callbacks that are configured 387 with an event mask of enabled event types should be prepared to be 388 called with multiple event type bits set simultaneously. 389 Use "&" to test for each event bit rather than a switch case. 390 <li> 391 Use prefetch status and callbacks as a general indication of progress, but do 392 not depend on specific hard-coded fill levels or callback sequence. 393 The meaning of the prefetch status fill level, and the behavior for 394 errors that are detected during prefetch, may change. 395 </ul> 396 397 <h3>Planning for source compatibility</h3> 398 399 As mentioned, source code incompatibilities are expected in the next 400 version of OpenMAX AL from Khronos Group. Likely areas of change include: 401 402 <ul> 403 <li>Addition of <code>const</code> to input parameters passed by reference, 404 and to <code>XAchar *</code> struct fields used as input values. 405 This should not require any changes to your code. 406 <li>Substitution of unsigned types for some parameters that are 407 currently signed. You may need to change a parameter type from 408 <code>XAint32</code> to <code>XAuint32</code> or similar, or add a cast. 409 <li>Additional fields in struct types. For output parameters, these 410 new fields can be ignored, but for input parameters the new fields 411 will need to be initialized. Fortunately, these are expected to all 412 be in areas not supported by Android. 413 <li>Interface 414 <a href="http://en.wikipedia.org/wiki/Globally_unique_identifier"> 415 GUIDs</a> will change. Refer to interfaces by symbolic name rather than GUID 416 to avoid a dependency. 417 <li><code>XAchar</code> will change from <code>unsigned char</code> 418 to <code>char</code>. This primarily affects the MIME data format. 419 <li><code>XADataFormat_MIME.mimeType</code> will be renamed to <code>pMimeType</code>. 420 We recommend that you initialize the <code>XADataFormat_MIME</code> 421 data structure using a brace-enclosed comma-separated list of values, 422 rather than by field name, to isolate your code from this change. 423 In the example code we have used this technique. 424 </ul> 425 426 <h2>Android extensions</h2> 427 428 The API for Android extensions is defined in <code>OMXAL/OpenMAXAL_Android.h</code> 429 . 430 Consult that file for details on these extensions. Unless otherwise 431 noted, all interfaces are "explicit". 432 <p> 433 Note that use these extensions will limit your application's 434 portability to other OpenMAX AL implementations. If this is a concern, 435 we advise that you avoid using them, or isolate your use of these 436 with <code>#ifdef</code> etc. 437 <p> 438 The following figure shows which Android-specific interfaces and 439 data locators are available for each object type. 440 441 <p> 442 <img src="chart4.png" alt="Android extensions"> 443 444 <h3>Android buffer queue data locator and interface</h3> 445 446 <h4>Comparison with OpenSL ES buffer queue</h4> 447 448 The Android buffer queue data locator and interface are based on 449 similar concepts from OpenSL ES 1.0.1, with these differences: 450 <ul> 451 <li>Though currently usable with only a media player and MPEG-2 transport 452 stream data, the Android buffer queue API is designed for flexibility 453 so that the API can also apply to other use cases in the future. 454 <li>Commands may be 455 optionally specified by the application at time of <code>Enqueue</code>. 456 Each command consists of an item key and optional item value. 457 Command key/value pairs are carried alongside the corresponding buffer in the queue, 458 and thus are processed in synchrony with the buffer. 459 <li>To enqueue command(s) without associated data, specify 460 a buffer address of NULL and buffer size of zero, along 461 with at least one command. 462 <li>Status may be 463 provided by the implementation during a completion callback. 464 Each status consists of an item key and optional item value. 465 Status key/value pairs are carried alongside 466 the corresponding buffer in the queue, and thus are received by the 467 application in synchrony with the completion callback. 468 <li>The completion callback receives additional parameters: 469 buffer address, buffer maximum data size, buffer actual size consumed (or filled by a future 470 recorder object), and a <code>void *</code> for application. 471 <li>The callback returns a value, which must be <code>XA_RESULT_SUCCESS</code>. 472 </ul> 473 474 The data locator type code is <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> and 475 the associated structure is <code>XADataLocator_AndroidBufferQueue</code>. 476 <p> 477 The interface ID is <code>XA_IID_ANDROIDBUFFERQUEUESOURCE</code>. 478 479 <h4>Usage</h4> 480 481 A typical buffer queue configuration is 8 buffers of 1880 bytes each. 482 <p> 483 The application enqueues filled buffers of data in MPEG-2 transport 484 stream format. The buffer size must be a multiple of 188 bytes, 485 the size of an MPEG-2 transport stream packet. The buffer data must 486 be properly aligned on a packet boundary, and formatted per the MPEG-2 487 Part 1 specification. 488 <p> 489 An application may supply zero or one of these item codes 490 (command key/value pairs) at <code>Enqueue</code>: 491 <dl> 492 <dt>XA_ANDROID_ITEMKEY_EOS</dt> 493 <dd>End of stream. Informs the decode and rendering components that playback is complete. 494 The application must not call <code>Enqueue</code> again. 495 There is no associated value, so <code>itemSize</code> must be zero. 496 There must be no data buffer alongside the EOS command. 497 </dd> 498 <dt>XA_ANDROID_ITEMKEY_DISCONTINUITY</dt> 499 <dd>Discontinuity. This and following buffers have a new presentation time. 500 The new presentation time may be optionally specified as a parameter, 501 expressed in <code>itemData</code> as a 64-bit unsigned integer in units of 90 kHz clock ticks. 502 The <code>itemSize</code> should be either zero or 8. 503 The discontinuity command is intended for seeking to a new point in 504 the stream. The application should flush its internal data, then send 505 the discontinuity command prior to, or alongside of, the first buffer 506 corresponding to the new stream position. 507 The initial packets in the video elementary stream 508 should describe an IDR (Instantaneous Decoding Refresh) frame. 509 Note that the discontinuity 510 command is not intended for stream configuration / format changes; 511 for these use <code>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</code>. 512 </dd> 513 <dt>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</dt> 514 <dd>Format change. This and following buffers have a new format, 515 for example for MBR (Multiple Bit Rate) or resolution switching. 516 </dd> 517 </dl> 518 <p> 519 Upon notification of completion via a registered callback, the now 520 consumed buffer is available for the application to re-fill. 521 <p> 522 The implementation may supply zero or more of these item codes 523 (status key/value pairs) to the callback handler: 524 <dl> 525 <dt>XA_ANDROID_ITEMKEY_BUFFERQUEUEEVENT</dt> 526 <dd>Buffer queue event mask. The <code>itemSize</code> is 4, and <code>itemData</code> contains 527 the bit-wise "or" of zero or more of the <code>XA_ANDROIDBUFFERQUEUEEVENT_*</code> 528 symbols. This event mask explains the reason(s) why the callback handler 529 was called.</dd> 530 </dl> 531 532 <h3>Reporting of extensions</h3> 533 534 <code>Engine::QueryNumSupportedExtensions</code>, 535 <code>Engine::QuerySupportedExtension</code>, 536 <code>Engine::IsExtensionSupported</code> report these extensions: 537 <ul> 538 <li><code>ANDROID_SDK_LEVEL_#</code> 539 where # is the platform API level, 14 or higher 540 </ul> 541 542 <h2>Programming notes</h2> 543 544 These notes supplement the OpenMAX AL 1.0.1 specification, 545 available in the "References" section below. 546 547 <h3>Objects and interface initialization</h3> 548 549 Two aspects of the OpenMAX AL programming model that may be unfamiliar 550 to new developers are the distinction between objects and interfaces, 551 and the initialization sequence. 552 <p> 553 Briefly, an OpenMAX AL object is similar to the object concept 554 in programming languages such as Java and C++, except an OpenMAX AL 555 object is <i>only</i> visible via its associated interfaces. This 556 includes the initial interface for all objects, called 557 <code>XAObjectItf</code>. There is no handle for an object itself, 558 only a handle to the <code>XAObjectItf</code> interface of the object. 559 <p> 560 An OpenMAX AL object is first "created", which returns an 561 <code>XAObjectItf</code>, then "realized". This is similar to the 562 common programming pattern of first constructing an object (which 563 should never fail other than for lack of memory or invalid parameters), 564 and then completing initialization (which may fail due to lack of 565 resources). The realize step gives the implementation a 566 logical place to allocate additional resources if needed. 567 <p> 568 As part of the API to create an object, an application specifies 569 an array of desired interfaces that it plans to acquire later. Note 570 that this array does <i>not</i> automatically acquire the interfaces; 571 it merely indicates a future intention to acquire them. Interfaces 572 are distinguished as "implicit" or "explicit". An explicit interface 573 <i>must</i> be listed in the array if it will be acquired later. 574 An implicit interface need not be listed in the object create array, 575 but there is no harm in listing it there. OpenMAX AL has one more 576 kind of interface called "dynamic", which does not need to be 577 specified in the object create array, and can be added later after 578 the object is created. The Android implementation provides a 579 convenience feature to avoid this complexity; see section "Dynamic 580 interfaces at object creation" above. 581 <p> 582 After the object is created and realized, the application should 583 acquire interfaces for each feature it needs, using 584 <code>GetInterface</code> on the initial <code>XAObjectItf</code>. 585 <p> 586 Finally, the object is available for use via its interfaces, though 587 note that some objects require further setup. 588 Some use cases needs a bit more preparation in 589 order to detect connection errors. See the next section 590 "Media player prefetch" for details. 591 <p> 592 After your application is done with the object, you should explicitly 593 destroy it; see section "Destroy" below. 594 595 <h3>Media player prefetch</h3> 596 597 For a media player, <code>Object::Realize</code> allocates resources 598 but does not connect to the data source (i.e. "prepare") or begin 599 pre-fetching data. These occur once the player state is set to 600 either <code>XA_PLAYSTATE_PAUSED</code> or <code>XA_PLAYSTATE_PLAYING</code>. 601 <p> 602 The prefetch status interface is useful for detecting errors. 603 Register a callback and enable at least the 604 <code>XA_PREFETCHEVENT_FILLLEVELCHANGE</code> and 605 <code>XA_PREFETCHEVENT_STATUSCHANGE</code> events. If both of these 606 events are delivered simultaneously, and 607 <code>PrefetchStatus::GetFillLevel</code> reports a zero level, and 608 <code>PrefetchStatus::GetPrefetchStatus</code> reports 609 <code>XA_PREFETCHSTATUS_UNDERFLOW</code>, then this indicates a 610 non-recoverable error in the data source or in rendering to the video sink. 611 <p> 612 The next version of OpenMAX AL is expected to add more explicit 613 support for handling errors in the data source. However, for future 614 binary compatibility, we intend to continue to support the current 615 method for reporting a non-recoverable error. 616 <p> 617 In summary, a recommended code sequence is: 618 <ul> 619 <li>Engine::CreateMediaPlayer 620 <li>Object:Realize 621 <li>Object::GetInterface for XA_IID_PREFETCHSTATUS 622 <li>PrefetchStatus::SetCallbackEventsMask 623 <li>PrefetchStatus::RegisterCallback 624 <li>Object::GetInterface for XA_IID_PLAY 625 <li>Play::SetPlayState to XA_PLAYSTATE_PAUSED or XA_PLAYSTATE_PLAYING 626 <li>preparation and prefetching occur here; during this time your 627 callback will be called with periodic status updates and 628 error notifications 629 </ul> 630 631 <h3>Destroy</h3> 632 633 Be sure to destroy all objects on exit from your application. Objects 634 should be destroyed in reverse order of their creation, as it is 635 not safe to destroy an object that has any dependent objects. 636 For example, destroy in this order: audio players and recorders, 637 output mix, then finally the engine. 638 <p> 639 OpenMAX AL does not support automatic garbage collection or 640 <a href="http://en.wikipedia.org/wiki/Reference_counting">reference counting</a> 641 of interfaces. After you call <code>Object::Destroy</code>, all extant 642 interfaces derived from the associated object become <i>undefined</i>. 643 <p> 644 The Android OpenMAX AL implementation does not detect the incorrect 645 use of such interfaces. 646 Continuing to use such interfaces after the object is destroyed will 647 cause your application to crash or behave in unpredictable ways. 648 <p> 649 We recommend that you explicitly set both the primary object interface 650 and all associated interfaces to NULL as part of your object 651 destruction sequence, to prevent the accidental misuse of a stale 652 interface handle. 653 654 <h3>Callbacks and threads</h3> 655 656 Callback handlers are generally called <i>synchronously</i> with 657 respect to the event, that is, at the moment and location where the 658 event is detected by the implementation. But this point is 659 <i>asynchronous</i> with respect to the application. Thus you should 660 use a mutex or other synchronization mechanism to control access 661 to any variables shared between the application and the callback 662 handler. In the example code, such as for buffer queues, we have 663 omitted this synchronization in the interest of simplicity. However, 664 proper mutual exclusion would be critical for any production code. 665 <p> 666 Callback handlers are called from internal 667 non-application thread(s) which are not attached to the Dalvik virtual machine and thus 668 are ineligible to use JNI. Because these internal threads are 669 critical to the integrity of the OpenMAX AL implementation, a callback 670 handler should also not block or perform excessive work. Therefore, 671 if your callback handler needs to use JNI or do anything significant 672 (e.g. beyond an <code>Enqueue</code> or something else simple such as the "Get" 673 family), the handler should instead post an event for another thread 674 to process. 675 <p> 676 Note that the converse is safe: a Dalvik application thread which has 677 entered JNI is allowed to directly call OpenMAX AL APIs, including 678 those which block. However, blocking calls are not recommended from 679 the main thread, as they may result in the dreaded "Application Not 680 Responding" (ANR). 681 682 <h3>Performance</h3> 683 684 As OpenMAX AL is a native C API, non-Dalvik application threads which 685 call OpenMAX AL have no Dalvik-related overhead such as garbage 686 collection pauses. However, there is no additional performance 687 benefit to the use of OpenMAX AL other than this. In particular, use 688 of OpenMAX AL does not result in lower audio or video latency, higher scheduling 689 priority, etc. than what the platform generally provides. 690 On the other hand, as the Android platform and specific device 691 implementations continue to evolve, an OpenMAX AL application can 692 expect to benefit from any future system performance improvements. 693 694 <h3>Security and permissions</h3> 695 696 As far as who can do what, security in Android is done at the 697 process level. Java programming language code can't do anything more than native code, nor 698 can native code do anything more than Java programming language code. The only differences 699 between them are what APIs are available that provide functionality 700 that the platform promises to support in the future and across 701 different devices. 702 <p> 703 Applications using OpenMAX AL must request whatever permissions they 704 would need for similar non-native APIs. 705 Applications that play 706 network resources need <code>android.permission.NETWORK</code>. 707 Note that the current Android implementation does not directly access the network, 708 but many applications that play multimedia receive their data via the network. 709 710 <h2>Platform issues</h2> 711 712 This section describes known issues in the initial platform 713 release which supports these APIs. 714 715 <h3>Dynamic interface management</h3> 716 717 <code>DynamicInterfaceManagement::AddInterface</code> does not work. 718 Instead, specify the interface in the array passed to Create. 719 720 <h2>References and resources</h2> 721 722 Android: 723 <ul> 724 <li><a href="http://developer.android.com/resources/index.html"> 725 Android developer resources</a> 726 <li><a href="http://groups.google.com/group/android-developers"> 727 Android developers discussion group</a> 728 <li><a href="http://developer.android.com/sdk/ndk/index.html">Android NDK</a> 729 <li><a href="http://groups.google.com/group/android-ndk"> 730 Android NDK discussion group</a> (for developers of native code, including OpenMAX AL) 731 <li><a href="http://code.google.com/p/android/issues/"> 732 Android open source bug database</a> 733 </ul> 734 735 Khronos Group: 736 <ul> 737 <li><a href="http://www.khronos.org/openmax/al/"> 738 Khronos Group OpenMAX AL Overview</a> 739 <li><a href="http://www.khronos.org/registry/omxal/"> 740 Khronos Group OpenMAX AL 1.0.1 specification</a> 741 <li><a href="http://www.khronos.org/message_boards/viewforum.php?f=30"> 742 Khronos Group public message board for OpenMAX AL</a> 743 (please limit to non-Android questions) 744 </ul> 745 For convenience, we have included a copy of the OpenMAX AL 1.0.1 746 specification with the NDK in 747 <code>docs/openmaxal/OpenMAX_AL_1_0_1_Specification.pdf</code>. 748 749 <p> 750 Miscellaneous: 751 <ul> 752 <li><a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a> 753 <li><a href="http://en.wikipedia.org/wiki/MPEG-2">MPEG-2</a> 754 <li><a href="http://en.wikipedia.org/wiki/MPEG_transport_stream">MPEG-2 transport stream</a> 755 <li><a href="http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC">H.264</a> 756 <li><a href="http://en.wikipedia.org/wiki/Advanced_Audio_Coding">AAC</a> 757 </ul> 758 759 </body> 760 </html> 761
