1 page.title=BufferQueue and gralloc 2 @jd:body 3 4 <!-- 5 Copyright 2014 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <p>Understanding the Android graphics system starts behind the scenes with 28 BufferQueue and the gralloc HAL.</p> 29 30 <p>The BufferQueue class is at the heart of everything graphical in Android. Its 31 role is simple: Connect something that generates buffers of graphical data (the 32 <em>producer</em>) to something that accepts the data for display or further 33 processing (the <em>consumer</em>). Nearly everything that moves buffers of 34 graphical data through the system relies on BufferQueue.</p> 35 36 <p>The gralloc memory allocator performs buffer allocations and is 37 implemented through a vendor-specific HAL interface (see 38 <code>hardware/libhardware/include/hardware/gralloc.h</code>). The 39 <code>alloc()</code> function takes expected arguments (width, height, pixel 40 format) as well as a set of usage flags (detailed below).</p> 41 42 <h2 id="BufferQueue">BufferQueue producers and consumers</h2> 43 44 <p>Basic usage is straightforward: The producer requests a free buffer 45 (<code>dequeueBuffer()</code>), specifying a set of characteristics including 46 width, height, pixel format, and usage flags. The producer populates the buffer 47 and returns it to the queue (<code>queueBuffer()</code>). Later, the consumer 48 acquires the buffer (<code>acquireBuffer()</code>) and makes use of the buffer 49 contents. When the consumer is done, it returns the buffer to the queue 50 (<code>releaseBuffer()</code>).</p> 51 52 <p>Recent Android devices support the <em>sync framework</em>, which enables the 53 system to do nifty things when combined with hardware components that can 54 manipulate graphics data asynchronously. For example, a producer can submit a 55 series of OpenGL ES drawing commands and then enqueue the output buffer before 56 rendering completes. The buffer is accompanied by a fence that signals when the 57 contents are ready. A second fence accompanies the buffer when it is returned 58 to the free list, so the consumer can release the buffer while the contents are 59 still in use. This approach improves latency and throughput as the buffers 60 move through the system.</p> 61 62 <p>Some characteristics of the queue, such as the maximum number of buffers it 63 can hold, are determined jointly by the producer and the consumer. However, the 64 BufferQueue is responsible for allocating buffers as it needs them. Buffers are 65 retained unless the characteristics change; for example, if the producer 66 requests buffers with a different size, old buffers are freed and new buffers 67 are allocated on demand.</p> 68 69 <p>Producers and consumers can live in different processes. Currently, the 70 consumer always creates and owns the data structure. In older versions of 71 Android, only the producer side was binderized (i.e. producer could be in a 72 remote process but consumer had to live in the process where the queue was 73 created). Android 4.4 and later releases moved toward a more general 74 implementation.</p> 75 76 <p>Buffer contents are never copied by BufferQueue (moving that much data around 77 would be very inefficient). Instead, buffers are always passed by handle.</p> 78 79 <h2 id="gralloc_HAL">gralloc HAL usage flags</h2> 80 81 <p>The gralloc allocator is not just another way to allocate memory on the 82 native heap; in some situations, the allocated memory may not be cache-coherent 83 or could be totally inaccessible from user space. The nature of the allocation 84 is determined by the usage flags, which include attributes such as:</p> 85 86 <ul> 87 <li>How often the memory will be accessed from software (CPU)</li> 88 <li>How often the memory will be accessed from hardware (GPU)</li> 89 <li>Whether the memory will be used as an OpenGL ES (GLES) texture</li> 90 <li>Whether the memory will be used by a video encoder</li> 91 </ul> 92 93 <p>For example, if your format specifies RGBA 8888 pixels, and you indicate the 94 buffer will be accessed from software (meaning your application will touch 95 pixels directly) then the allocator must create a buffer with 4 bytes per pixel 96 in R-G-B-A order. If instead, you say the buffer will be only accessed from 97 hardware and as a GLES texture, the allocator can do anything the GLES driver 98 wants—BGRA ordering, non-linear swizzled layouts, alternative color 99 formats, etc. Allowing the hardware to use its preferred format can improve 100 performance.</p> 101 102 <p>Some values cannot be combined on certain platforms. For example, the video 103 encoder flag may require YUV pixels, so adding software access and specifying 104 RGBA 8888 would fail.</p> 105 106 <p>The handle returned by the gralloc allocator can be passed between processes 107 through Binder.</p> 108 109 <h2 id=tracking>Tracking BufferQueue with systrace</h2> 110 111 <p>To really understand how graphics buffers move around, use systrace. The 112 system-level graphics code is well instrumented, as is much of the relevant app 113 framework code.</p> 114 115 <p>A full description of how to use systrace effectively would fill a rather 116 long document. Start by enabling the <code>gfx</code>, <code>view</code>, and 117 <code>sched</code> tags. You'll also see BufferQueues in the trace. If you've 118 used systrace before, you've probably seen them but maybe weren't sure what they 119 were. As an example, if you grab a trace while 120 <a href="https://github.com/google/grafika">Grafika's</a> "Play video 121 (SurfaceView)" is running, the row labeled <em>SurfaceView</em> tells you how 122 many buffers were queued up at any given time.</p> 123 124 <p>The value increments while the app is active—triggering the rendering 125 of frames by the MediaCodec decoder—and decrements while SurfaceFlinger is 126 doing work, consuming buffers. When showing video at 30fps, the queue's value 127 varies from 0 to 1 because the ~60fps display can easily keep up with the 128 source. (Notice also that SurfaceFlinger only wakes when there's work to 129 be done, not 60 times per second. The system tries very hard to avoid work and 130 will disable VSYNC entirely if nothing is updating the screen.)</p> 131 132 <p>If you switch to Grafika's "Play video (TextureView)" and grab a new trace, 133 you'll see a row labeled 134 com.android.grafika/com.android.grafika.PlayMovieActivity. This is the main UI 135 layer, which is just another BufferQueue. Because TextureView renders into the 136 UI layer (rather than a separate layer), you'll see all of the video-driven 137 updates here.</p> 138 139 <p>For more information about the systrace tool, refer to <a 140 href="http://developer.android.com/tools/help/systrace.html">Systrace 141 documentation</a>.</p> 142
