1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" 2 "http://www.w3.org/TR/html4/loose.dtd"> 3 <html> 4 5 <head> 6 <title>OpenMAX AL for Android</title> 7 </head> 8 9 <body> 10 11 <h1>OpenMAX AL for Android</h1> 12 13 This article describes the Android native multimedia APIs based on the 14 Khronos Group OpenMAX AL™ 1.0.1 standard, as of Android API level 14 (Android 15 platform version 4.0) and higher. 16 <p> 17 OpenMAX AL is a companion API to OpenSL ES, but for multimedia (video 18 and audio) rather than audio only. 19 <p> 20 OpenMAX AL provides a C language interface that is also callable from C++, and 21 exposes features similar to these Android APIs 22 callable from Java programming language code: 23 <ul> 24 <li><a href="http://developer.android.com/reference/android/media/MediaPlayer.html"> 25 android.media.MediaPlayer</a> 26 </ul> 27 28 As with all of the Android Native Development Kit (NDK), the primary 29 purpose of OpenMAX AL for Android is to facilitate the implementation 30 of shared libraries to be called from Java programming language code via Java Native 31 Interface (JNI). NDK is not intended for writing pure C/C++ 32 applications. 33 34 <p> 35 Note: though based on OpenMAX AL, Android native multimedia 36 is <i>not</i> a conforming implementation of either OpenMAX AL 1.0.1 37 profile (media player or media player / recorder). This is because Android does not 38 implement all of the features required by either of the profiles. 39 Any known cases where Android behaves differently than the specification 40 are described in section "Android extensions" below. 41 42 The Android OpenMAX AL implementation has limited features, and is 43 intended primarily for certain performance-sensitive native streaming 44 multimedia applications such as video players. 45 <p> 46 The major feature is the ability to play an MPEG-2 transport stream 47 containing a single program stream made up of one H.264 video elementary 48 stream and one AAC audio elementary stream. The application provides 49 the stream via an Android buffer queue data source, which is based on 50 the OpenSL ES buffer queue concept and Android-specific extensions. 51 <p> 52 The video sink is an <code>ANativeWindow *</code> abstract handle, 53 derived from either an <code>android.view.Surface</code> ("surface") 54 or <code>android.graphics.SurfaceTexture</code> ("surface texture"). 55 A Surface should be used when displaying an unaltered video within a 56 fixed SurfaceView frame. SurfaceTexture allows streaming the decoded 57 video frames to an OpenGL ES 2.0 texture, where the frames can be used 58 as input to a shader algorithm in the Graphics Processing Unit (GPU). 59 <p> 60 The audio sink is always an output mix, a device-independent mixer object 61 similar to that of OpenSL ES. 62 63 <h2>Getting started</h2> 64 65 <h3>Example code</h3> 66 67 <h4>Recommended</h4> 68 69 Supported and tested example code, usable as a model 70 for your own code, is located in NDK folder 71 <code>platforms/android-14/samples/native-media/</code>. 72 73 <h4>Not recommended</h4> 74 75 The OpenMAX AL 1.0.1 specification contains example code in the 76 appendices (see section "References" below for the link to this 77 specification). However, the examples in Appendix D: Sample Code 78 use features 79 not supported by Android. Some examples also contain 80 typographical errors, or use APIs that are likely to change. 81 Proceed with caution in referring to these; 82 though the code may be helpful in understanding the full OpenMAX AL 83 standard, it should not be used as is with Android. 84 85 <h3>Adding OpenMAX AL to your application source code</h3> 86 87 OpenMAX AL is a C API, but is callable from both C and C++ code. 88 <p> 89 Add the following lines to your code: 90 <pre> 91 #include <OMXAL/OpenMAXAL.h> 92 #include <OMXAL/OpenMAXAL_Android.h> 93 </pre> 94 95 <h3>Makefile</h3> 96 97 Modify your Android.mk as follows: 98 <pre> 99 LOCAL_LDLIBS += libOpenMAXAL 100 </pre> 101 102 <h3>Multimedia content</h3> 103 104 The only supported way to supply multimedia content is via an MPEG-2 105 transport stream. 106 <p> 107 Finding or creating useful multimedia content for your application is 108 beyond the scope of this article. 109 <p> 110 Note that it is your responsibility to ensure that you are legally 111 permitted to play the content. 112 113 <h3>Debugging</h3> 114 115 For robustness, we recommend that you examine the <code>XAresult</code> 116 value which is returned by most APIs. Use of <code>assert</code> 117 vs. more advanced error handling logic is a matter of coding style 118 and the particular API; see the Wikipedia article on 119 <a href="http://en.wikipedia.org/wiki/Assertion_(computing)">assert</a> 120 for more information. In the supplied example, we have used <code>assert</code> 121 for "impossible" conditions which would indicate a coding error, and 122 explicit error handling for others which are more likely to occur 123 in production. 124 <p> 125 Many API errors result in a log entry, in addition to the non-zero 126 result code. These log entries provide additional detail which can 127 be especially useful for the more complex APIs such as 128 <code>Engine::CreateMediaPlayer</code>. 129 <p> 130 Use <a href="http://developer.android.com/guide/developing/tools/adb.html"> 131 adb logcat</a>, the 132 <a href="http://developer.android.com/guide/developing/eclipse-adt.html"> 133 Eclipse ADT plugin</a> LogCat pane, or 134 <a href="http://developer.android.com/guide/developing/tools/ddms.html#logcat"> 135 ddms logcat</a> to see the log. 136 137 <h2>Supported features from OpenMAX AL 1.0.1</h2> 138 139 This section summarizes available features. In some 140 cases, there are limitations which are described in the next 141 sub-section. 142 143 <h3>Global entry points</h3> 144 145 Supported global entry points: 146 <ul> 147 <li><code>xaCreateEngine</code> 148 <li><code>xaQueryNumSupportedEngineInterfaces</code> 149 <li><code>xaQuerySupportedEngineInterfaces</code> 150 </ul> 151 152 <h3>Objects and interfaces</h3> 153 154 The following figure indicates objects and interfaces supported by 155 Android's OpenMAX AL implementation. A green cell means the feature 156 is supported. 157 158 <p> 159 <img src="chart3.png" alt="Supported objects and interfaces"> 160 161 <h3>Limitations</h3> 162 163 This section details limitations with respect to the supported 164 objects and interfaces from the previous section. 165 166 <h4>Audio</h4> 167 168 The audio stream type cannot be configured; it is always <code>AudioManager.STREAM_MUSIC</code>. 169 <p> 170 Effects are not supported. 171 172 <h4>Dynamic interface management</h4> 173 174 <code>RemoveInterface</code> and <code>ResumeInterface</code> are not supported. 175 176 <h4>Engine</h4> 177 178 Supported: 179 <ul> 180 <li><code>CreateMediaPlayer</code> 181 </ul> 182 183 Not supported: 184 <ul> 185 <li><code>CreateCameraDevice</code> 186 <li><code>CreateRadioDevice</code> 187 <li><code>CreateLEDDevice</code> 188 <li><code>CreateVibraDevice</code> 189 <li><code>CreateMetadataExtractor</code> 190 <li><code>CreateExtensionObject</code> 191 <li><code>GetImplementationInfo</code> 192 </ul> 193 194 For <code>CreateMediaPlayer</code>, these restrictions apply: 195 <ul> 196 <li>audio sink is an output mix data locator 197 <li>video sink is a native display data locator 198 <li>soundbank, LED array, and vibra sinks must be <code>NULL</code> 199 </ul> 200 201 <h4>MIME data format</h4> 202 203 In the current Android implementation of OpenMAX AL, a media player 204 receives its source data as an MPEG-2 transport stream via a 205 buffer queue. 206 <p> 207 The source data locator must be <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> 208 (see "Android extensions" below). 209 <p> 210 The source data format must be <code>XADataFormat_MIME</code>. 211 Initialize <code>mimeType</code> to <code>XA_ANDROID_MIME_MP2TS</code>, 212 and <code>containerType</code> to <code>XA_CONTAINERTYPE_MPEG_TS</code>. 213 <p> 214 The contained transport stream must have a single program with one H.264 215 video elementary stream and one AAC audio elementary stream. 216 217 <h4>Object</h4> 218 219 <code>Resume</code>, <code>RegisterCallback</code>, 220 <code>AbortAsyncOperation</code>, <code>SetPriority</code>, 221 <code>GetPriority</code>, and <code>SetLossOfControlInterfaces</code> 222 are not supported. 223 224 <h4>StreamInformation</h4> 225 226 Use the StreamInformation interface on a media player object to discover 227 when the video metrics (height/width or aspect ratio) are available or 228 changed, and to then get the sizes. 229 <p> 230 231 Supported: 232 <ul> 233 <li><code>RegisterStreamChangeCallback</code> 234 <li><code>QueryMediaContainerInformation</code> 235 <li><code>QueryStreamInformation</code> 236 <li><code>QueryStreamType</code> 237 </ul> 238 239 Not supported: 240 <ul> 241 <li><code>QueryActiveStreams</code> 242 <li><code>QueryStreamName</code> 243 <li><code>SetActiveStream</code> 244 </ul> 245 246 <h4>VideoDecoderCapabilities</h4> 247 248 This interface on the engine object reports video decoder capabilities 249 without interpretation, exactly as claimed by the underlying OpenMAX IL 250 implementation. 251 <p> 252 These fields in <code>XAVideoCodecDescriptor</code> are filled: 253 <ul> 254 <li><code>codecId</code> 255 <li><code>profileSetting</code> 256 <li><code>levelSetting</code> 257 </ul> 258 The other fields are not filled and should be ignored. 259 260 <h3>Data structures</h3> 261 262 Android API level 14 supports these OpenMAX AL 1.0.1 data structures: 263 <ul> 264 <li>XADataFormat_MIME 265 <li>XADataLocator_NativeDisplay 266 <li>XADataLocator_OutputMix 267 <li>XADataSink 268 <li>XADataSource 269 <li>XAEngineOption 270 <li>XAInterfaceID 271 <li>XAMediaContainerInformation 272 <li>XANativeHandle 273 <li>XA*StreamInformation 274 <li>XAVideoCodecDescriptor 275 </ul> 276 277 <h4>XADataLocator_NativeDisplay</h4> 278 279 The native display data locator is used to specify the video sink: 280 <pre> 281 typedef struct XADataLocator_NativeDisplay_ { 282 XAuint32 locatorType; // XA_DATALOCATOR_NATIVEDISPLAY 283 XANativeHandle hWindow; // ANativeWindow * 284 XANativeHandle hDisplay; // NULL 285 } XADataLocator_NativeDisplay; 286 </pre> 287 288 Set the <code>hWindow</code> field to an 289 <code>ANativeWindow *</code> and set <code>hDisplay</code> to <code>NULL</code>. 290 You can get a <code>ANativeWindow *</code> handle from a Java <code>android.view.Surface</code> or 291 <code>android.graphics.SurfaceTexture</code>, using these NDK functions: 292 <pre> 293 #include <android/native_window_jni.h> 294 295 ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface); 296 ANativeWindow* ANativeWindow_fromSurfaceTexture(JNIEnv* env, jobject surfaceTexture); 297 </pre> 298 Don't forget to free this handle in your shutdown code with <code>ANativeWindow_release</code>. 299 300 <h3>Platform configuration</h3> 301 302 OpenMAX AL for Android is designed for multi-threaded applications, 303 and is thread-safe. 304 <p> 305 OpenMAX AL for Android supports a single engine per application, and 306 up to 32 objects. Available device memory and CPU may further 307 restrict the usable number of objects. 308 <p> 309 <code>xaCreateEngine</code> recognizes, but ignores, these engine options: 310 <ul> 311 <li><code>XA_ENGINEOPTION_THREADSAFE</code> 312 <li><code>XA_ENGINEOPTION_LOSSOFCONTROL</code> 313 </ul> 314 315 OpenMAX AL and OpenSL ES may be used together in the same application. 316 In this case, there is internally a single shared engine object, 317 and the 32 object limit is shared between OpenMAX AL and OpenSL ES. 318 The application should first create both engines, then use both engines, 319 and finally destroy both engines. The implementation maintains a 320 reference count on the shared engine, so that it is correctly destroyed 321 at the second destroy. 322 323 <h2>Planning for future versions of OpenMAX AL</h2> 324 325 The Android native multimedia APIs at level 14 are based on Khronos 326 Group OpenMAX AL 1.0.1 (see section "References" below). 327 As of the time of this writing, Khronos has recently released 328 a revised version 1.1 of the standard. The revised version 329 includes new features, clarifications, correction of 330 typographical errors, and some incompatibilities. Most of the 331 incompatibilities are relatively minor, or are in areas of OpenMAX AL 332 not supported by Android. However, even a small change 333 can be significant for an application developer, so it is important 334 to prepare for this. 335 <p> 336 The Android team is committed to preserving future API binary 337 compatibility for developers to the extent feasible. It is our 338 intention to continue to support future binary compatibility of the 339 1.0.1-based API, even as we add support for later versions of the 340 standard. An application developed with this version should 341 work on future versions of the Android platform, provided that 342 you follow the guidelines listed in section "Planning for 343 binary compatibility" below. 344 <p> 345 Note that future source compatibility will <i>not</i> be a goal. That is, 346 if you upgrade to a newer version of the NDK, you may need to modify 347 your application source code to conform to the new API. We expect 348 that most such changes will be minor; see details below. 349 350 <h3>Planning for binary compatibility</h3> 351 352 We recommend that your application follow these guidelines, 353 to improve future binary compatibility: 354 <ul> 355 <li> 356 Use only the documented subset of Android-supported features from 357 OpenMAX AL 1.0.1. 358 <li> 359 Do not depend on a particular result code for an unsuccessful 360 operation; be prepared to deal with a different result code. 361 <li> 362 Application callback handlers generally run in a restricted context, 363 and should be written to perform their work quickly and then return 364 as soon as possible. Do not do complex operations within a callback 365 handler. For example, within a buffer queue completion callback, 366 you can enqueue another buffer, but do not create a media player. 367 <li> 368 Callback handlers should be prepared to be called more or less 369 frequently, to receive additional event types, and should ignore 370 event types that they do not recognize. Callbacks that are configured 371 with an event mask of enabled event types should be prepared to be 372 called with multiple event type bits set simultaneously. 373 Use "&" to test for each event bit rather than a switch case. 374 <li> 375 Use prefetch status and callbacks as a general indication of progress, but do 376 not depend on specific hard-coded fill levels or callback sequence. 377 The meaning of the prefetch status fill level, and the behavior for 378 errors that are detected during prefetch, may change. 379 </ul> 380 381 <h3>Planning for source compatibility</h3> 382 383 As mentioned, source code incompatibilities are expected in the next 384 version of OpenMAX AL from Khronos Group. Likely areas of change include: 385 386 <ul> 387 <li>Addition of <code>const</code> to input parameters passed by reference, 388 and to <code>XAchar *</code> struct fields used as input values. 389 This should not require any changes to your code. 390 <li>Substitution of unsigned types for some parameters that are 391 currently signed. You may need to change a parameter type from 392 <code>XAint32</code> to <code>XAuint32</code> or similar, or add a cast. 393 <li>Additional fields in struct types. For output parameters, these 394 new fields can be ignored, but for input parameters the new fields 395 will need to be initialized. Fortunately, these are expected to all 396 be in areas not supported by Android. 397 <li>Interface 398 <a href="http://en.wikipedia.org/wiki/Globally_unique_identifier"> 399 GUIDs</a> will change. Refer to interfaces by symbolic name rather than GUID 400 to avoid a dependency. 401 <li><code>XAchar</code> will change from <code>unsigned char</code> 402 to <code>char</code>. This primarily affects the MIME data format. 403 <li><code>XADataFormat_MIME.mimeType</code> will be renamed to <code>pMimeType</code>. 404 We recommend that you initialize the <code>XADataFormat_MIME</code> 405 data structure using a brace-enclosed comma-separated list of values, 406 rather than by field name, to isolate your code from this change. 407 In the example code we have used this technique. 408 </ul> 409 410 <h2>Android extensions</h2> 411 412 The API for Android extensions is defined in <code>OMXAL/OpenMAXAL_Android.h</code> 413 . 414 Consult that file for details on these extensions. Unless otherwise 415 noted, all interfaces are "explicit". 416 <p> 417 Note that use these extensions will limit your application's 418 portability to other OpenMAX AL implementations. If this is a concern, 419 we advise that you avoid using them, or isolate your use of these 420 with <code>#ifdef</code> etc. 421 <p> 422 The following figure shows which Android-specific interfaces and 423 data locators are available for each object type. 424 425 <p> 426 <img src="chart4.png" alt="Android extensions"> 427 428 <h3>Android buffer queue data locator and interface</h3> 429 430 <h4>Comparison with OpenSL ES buffer queue</h4> 431 432 The Android buffer queue data locator and interface are based on 433 similar concepts from OpenSL ES 1.0.1, with these differences: 434 <ul> 435 <li>Though currently usable with only a media player and MPEG-2 transport 436 stream data, the Android buffer queue API is designed for flexibility 437 so that the API can also apply to other use cases in the future. 438 <li>Commands may be 439 optionally specified by the application at time of <code>Enqueue</code>. 440 Each command consists of an item key and optional item value. 441 Command key/value pairs are carried alongside the corresponding buffer in the queue, 442 and thus are processed in synchrony with the buffer. 443 <li>To enqueue command(s) without associated data, specify 444 a buffer address of NULL and buffer size of zero, along 445 with at least one command. 446 <li>Status may be 447 provided by the implementation during a completion callback. 448 Each status consists of an item key and optional item value. 449 Status key/value pairs are carried alongside 450 the corresponding buffer in the queue, and thus are received by the 451 application in synchrony with the completion callback. 452 <li>The completion callback receives additional parameters: 453 buffer address, buffer maximum data size, buffer actual size consumed (or filled by a future 454 recorder object), and a <code>void *</code> for application. 455 <li>The callback returns a value, which must be <code>XA_RESULT_SUCCESS</code>. 456 </ul> 457 458 The data locator type code is <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> and 459 the associated structure is <code>XADataLocator_AndroidBufferQueue</code>. 460 <p> 461 The interface ID is <code>XA_IID_ANDROIDBUFFERQUEUESOURCE</code>. 462 463 <h4>Usage</h4> 464 465 A typical buffer queue configuration is 8 buffers of 1880 bytes each. 466 <p> 467 The application enqueues filled buffers of data in MPEG-2 transport 468 stream format. The buffer size must be a multiple of 188 bytes, 469 the size of an MPEG-2 transport stream packet. The buffer data must 470 be properly aligned on a packet boundary, and formatted per the MPEG-2 471 Part 1 specification. 472 <p> 473 An application may supply zero or one of these item codes 474 (command key/value pairs) at <code>Enqueue</code>: 475 <dl> 476 <dt>XA_ANDROID_ITEMKEY_EOS</dt> 477 <dd>End of stream. Informs the decode and rendering components that playback is complete. 478 The application must not call <code>Enqueue</code> again. 479 There is no associated value, so <code>itemSize</code> must be zero. 480 There must be no data buffer alongside the EOS command. 481 </dd> 482 <dt>XA_ANDROID_ITEMKEY_DISCONTINUITY</dt> 483 <dd>Discontinuity. This and following buffers have a new presentation time. 484 The new presentation time may be optionally specified as a parameter, 485 expressed in <code>itemData</code> as a 64-bit unsigned integer in units of 90 kHz clock ticks. 486 The <code>itemSize</code> should be either zero or 8. 487 The discontinuity command is intended for seeking to a new point in 488 the stream. The application should flush its internal data, then send 489 the discontinuity command prior to, or alongside of, the first buffer 490 corresponding to the new stream position. 491 The initial packets in the video elementary stream 492 should describe an IDR (Instantaneous Decoding Refresh) frame. 493 Note that the discontinuity 494 command is not intended for stream configuration / format changes; 495 for these use <code>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</code>. 496 </dd> 497 <dt>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</dt> 498 <dd>Format change. This and following buffers have a new format, 499 for example for MBR (Multiple Bit Rate) or resolution switching. 500 </dd> 501 </dl> 502 <p> 503 Upon notification of completion via a registered callback, the now 504 consumed buffer is available for the application to re-fill. 505 <p> 506 The implementation may supply zero or more of these item codes 507 (status key/value pairs) to the callback handler: 508 <dl> 509 <dt>XA_ANDROID_ITEMKEY_BUFFERQUEUEEVENT</dt> 510 <dd>Buffer queue event mask. The <code>itemSize</code> is 4, and <code>itemData</code> contains 511 the bit-wise "or" of zero or more of the <code>XA_ANDROIDBUFFERQUEUEEVENT_*</code> 512 symbols. This event mask explains the reason(s) why the callback handler 513 was called.</dd> 514 </dl> 515 516 <h3>Reporting of extensions</h3> 517 518 <code>Engine::QueryNumSupportedExtensions</code>, 519 <code>Engine::QuerySupportedExtension</code>, 520 <code>Engine::IsExtensionSupported</code> report these extensions: 521 <ul> 522 <li><code>ANDROID_SDK_LEVEL_#</code> 523 where # is the platform API level, 14 or higher 524 </ul> 525 526 <h2>Programming notes</h2> 527 528 These notes supplement the OpenMAX AL 1.0.1 specification, 529 available in the "References" section below. 530 531 <h3>Objects and interface initialization</h3> 532 533 Two aspects of the OpenMAX AL programming model that may be unfamiliar 534 to new developers are the distinction between objects and interfaces, 535 and the initialization sequence. 536 <p> 537 Briefly, an OpenMAX AL object is similar to the object concept 538 in programming languages such as Java and C++, except an OpenMAX AL 539 object is <i>only</i> visible via its associated interfaces. This 540 includes the initial interface for all objects, called 541 <code>XAObjectItf</code>. There is no handle for an object itself, 542 only a handle to the <code>XAObjectItf</code> interface of the object. 543 <p> 544 An OpenMAX AL object is first "created", which returns an 545 <code>XAObjectItf</code>, then "realized". This is similar to the 546 common programming pattern of first constructing an object (which 547 should never fail other than for lack of memory or invalid parameters), 548 and then completing initialization (which may fail due to lack of 549 resources). The realize step gives the implementation a 550 logical place to allocate additional resources if needed. 551 <p> 552 As part of the API to create an object, an application specifies 553 an array of desired interfaces that it plans to acquire later. Note 554 that this array does <i>not</i> automatically acquire the interfaces; 555 it merely indicates a future intention to acquire them. Interfaces 556 are distinguished as "implicit" or "explicit". An explicit interface 557 <i>must</i> be listed in the array if it will be acquired later. 558 An implicit interface need not be listed in the object create array, 559 but there is no harm in listing it there. OpenMAX AL has one more 560 kind of interface called "dynamic", which does not need to be 561 specified in the object create array, and can be added later after 562 the object is created. The Android implementation provides a 563 convenience feature to avoid this complexity; see section "Dynamic 564 interfaces at object creation" above. 565 <p> 566 After the object is created and realized, the application should 567 acquire interfaces for each feature it needs, using 568 <code>GetInterface</code> on the initial <code>XAObjectItf</code>. 569 <p> 570 Finally, the object is available for use via its interfaces, though 571 note that some objects require further setup. 572 <p> 573 After your application is done with the object, you should explicitly 574 destroy it; see section "Destroy" below. 575 576 <h3>Destroy</h3> 577 578 Be sure to destroy all objects on exit from your application. Objects 579 should be destroyed in reverse order of their creation, as it is 580 not safe to destroy an object that has any dependent objects. 581 For example, destroy in this order: audio players and recorders, 582 output mix, then finally the engine. 583 <p> 584 OpenMAX AL does not support automatic garbage collection or 585 <a href="http://en.wikipedia.org/wiki/Reference_counting">reference counting</a> 586 of interfaces. After you call <code>Object::Destroy</code>, all extant 587 interfaces derived from the associated object become <i>undefined</i>. 588 <p> 589 The Android OpenMAX AL implementation does not detect the incorrect 590 use of such interfaces. 591 Continuing to use such interfaces after the object is destroyed will 592 cause your application to crash or behave in unpredictable ways. 593 <p> 594 We recommend that you explicitly set both the primary object interface 595 and all associated interfaces to NULL as part of your object 596 destruction sequence, to prevent the accidental misuse of a stale 597 interface handle. 598 599 <h3>Callbacks and threads</h3> 600 601 Callback handlers are generally called <i>synchronously</i> with 602 respect to the event, that is, at the moment and location where the 603 event is detected by the implementation. But this point is 604 <i>asynchronous</i> with respect to the application. Thus you should 605 use a mutex or other synchronization mechanism to control access 606 to any variables shared between the application and the callback 607 handler. In the example code, such as for buffer queues, we have 608 omitted this synchronization in the interest of simplicity. However, 609 proper mutual exclusion would be critical for any production code. 610 <p> 611 Callback handlers are called from internal 612 non-application thread(s) which are not attached to the Dalvik virtual machine and thus 613 are ineligible to use JNI. Because these internal threads are 614 critical to the integrity of the OpenMAX AL implementation, a callback 615 handler should also not block or perform excessive work. Therefore, 616 if your callback handler needs to use JNI or do anything significant 617 (e.g. beyond an <code>Enqueue</code> or something else simple such as the "Get" 618 family), the handler should instead post an event for another thread 619 to process. 620 <p> 621 Note that the converse is safe: a Dalvik application thread which has 622 entered JNI is allowed to directly call OpenMAX AL APIs, including 623 those which block. However, blocking calls are not recommended from 624 the main thread, as they may result in the dreaded "Application Not 625 Responding" (ANR). 626 627 <h3>Performance</h3> 628 629 As OpenMAX AL is a native C API, non-Dalvik application threads which 630 call OpenMAX AL have no Dalvik-related overhead such as garbage 631 collection pauses. However, there is no additional performance 632 benefit to the use of OpenMAX AL other than this. In particular, use 633 of OpenMAX AL does not result in lower audio or video latency, higher scheduling 634 priority, etc. than what the platform generally provides. 635 On the other hand, as the Android platform and specific device 636 implementations continue to evolve, an OpenMAX AL application can 637 expect to benefit from any future system performance improvements. 638 639 <h3>Security and permissions</h3> 640 641 As far as who can do what, security in Android is done at the 642 process level. Java programming language code can't do anything more than native code, nor 643 can native code do anything more than Java code. The only differences 644 between them are what APIs are available that provide functionality 645 that the platform promises to support in the future and across 646 different devices. 647 <p> 648 Applications using OpenMAX AL must request whatever permissions they 649 would need for similar non-native APIs. 650 Applications that play 651 network resources need <code>android.permission.NETWORK</code>. 652 Note that the current Android implementation does not directly access the network, 653 but many applications that play multimedia receive their data via the network. 654 655 <h2>Platform issues</h2> 656 657 This section describes known issues in the initial platform 658 release which supports these APIs. 659 660 <h3>Dynamic interface management</h3> 661 662 <code>DynamicInterfaceManagement::AddInterface</code> does not work. 663 Instead, specify the interface in the array passed to Create. 664 665 <h2>References and resources</h2> 666 667 Google Android: 668 <ul> 669 <li><a href="http://developer.android.com/resources/index.html"> 670 Android developer resources</a> 671 <li><a href="http://groups.google.com/group/android-developers"> 672 Android developers discussion group</a> 673 <li><a href="http://developer.android.com/sdk/ndk/index.html">Android NDK</a> 674 <li><a href="http://groups.google.com/group/android-ndk"> 675 Android NDK discussion group</a> (for developers of native code, including OpenMAX AL) 676 <li><a href="http://code.google.com/p/android/issues/"> 677 Android open source bug database</a> 678 </ul> 679 680 Khronos Group: 681 <ul> 682 <li><a href="http://www.khronos.org/openmax/al/"> 683 Khronos Group OpenMAX AL Overview</a> 684 <li><a href="http://www.khronos.org/registry/omxal/"> 685 Khronos Group OpenMAX AL 1.0.1 specification</a> 686 <li><a href="http://www.khronos.org/message_boards/viewforum.php?f=30"> 687 Khronos Group public message board for OpenMAX AL</a> 688 (please limit to non-Android questions) 689 </ul> 690 For convenience, we have included a copy of the OpenMAX AL 1.0.1 691 specification with the NDK in 692 <code>docs/openmaxal/OpenMAX_AL_1_0_1_Specification.pdf</code>. 693 694 <p> 695 Miscellaneous: 696 <ul> 697 <li><a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a> 698 <li><a href="http://en.wikipedia.org/wiki/MPEG-2">MPEG-2</a> 699 <li><a href="http://en.wikipedia.org/wiki/MPEG_transport_stream">MPEG-2 transport stream</a> 700 <li><a href="http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC">H.264</a> 701 <li><a href="http://en.wikipedia.org/wiki/Advanced_Audio_Coding">AAC</a> 702 </ul> 703 704 </body> 705 </html> 706
