1 <html devsite> 2 <head> 3 <title>BufferQueue and gralloc</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 <p>Understanding the Android graphics system starts behind the scenes with 27 BufferQueue and the gralloc HAL.</p> 28 29 <p>The BufferQueue class is at the heart of everything graphical in Android. Its 30 role is simple: Connect something that generates buffers of graphical data (the 31 <em>producer</em>) to something that accepts the data for display or further 32 processing (the <em>consumer</em>). Nearly everything that moves buffers of 33 graphical data through the system relies on BufferQueue.</p> 34 35 <p>The gralloc memory allocator performs buffer allocations and is 36 implemented through a vendor-specific HAL interface (see 37 <code>hardware/libhardware/include/hardware/gralloc.h</code>). The 38 <code>alloc()</code> function takes expected arguments (width, height, pixel 39 format) as well as a set of usage flags (detailed below).</p> 40 41 <h2 id="BufferQueue">BufferQueue producers and consumers</h2> 42 43 <p>Basic usage is straightforward: The producer requests a free buffer 44 (<code>dequeueBuffer()</code>), specifying a set of characteristics including 45 width, height, pixel format, and usage flags. The producer populates the buffer 46 and returns it to the queue (<code>queueBuffer()</code>). Later, the consumer 47 acquires the buffer (<code>acquireBuffer()</code>) and makes use of the buffer 48 contents. When the consumer is done, it returns the buffer to the queue 49 (<code>releaseBuffer()</code>).</p> 50 51 <p>Recent Android devices support the <em>sync framework</em>, which enables the 52 system to do nifty things when combined with hardware components that can 53 manipulate graphics data asynchronously. For example, a producer can submit a 54 series of OpenGL ES drawing commands and then enqueue the output buffer before 55 rendering completes. The buffer is accompanied by a fence that signals when the 56 contents are ready. A second fence accompanies the buffer when it is returned 57 to the free list, so the consumer can release the buffer while the contents are 58 still in use. This approach improves latency and throughput as the buffers 59 move through the system.</p> 60 61 <p>Some characteristics of the queue, such as the maximum number of buffers it 62 can hold, are determined jointly by the producer and the consumer. However, the 63 BufferQueue is responsible for allocating buffers as it needs them. Buffers are 64 retained unless the characteristics change; for example, if the producer 65 requests buffers with a different size, old buffers are freed and new buffers 66 are allocated on demand.</p> 67 68 <p>Producers and consumers can live in different processes. Currently, the 69 consumer always creates and owns the data structure. In older versions of 70 Android, only the producer side was binderized (i.e. producer could be in a 71 remote process but consumer had to live in the process where the queue was 72 created). Android 4.4 and later releases moved toward a more general 73 implementation.</p> 74 75 <p>Buffer contents are never copied by BufferQueue (moving that much data around 76 would be very inefficient). Instead, buffers are always passed by handle.</p> 77 78 <h2 id="gralloc_HAL">gralloc HAL usage flags</h2> 79 80 <p>The gralloc allocator is not just another way to allocate memory on the 81 native heap; in some situations, the allocated memory may not be cache-coherent 82 or could be totally inaccessible from user space. The nature of the allocation 83 is determined by the usage flags, which include attributes such as:</p> 84 85 <ul> 86 <li>How often the memory will be accessed from software (CPU)</li> 87 <li>How often the memory will be accessed from hardware (GPU)</li> 88 <li>Whether the memory will be used as an OpenGL ES (GLES) texture</li> 89 <li>Whether the memory will be used by a video encoder</li> 90 </ul> 91 92 <p>For example, if your format specifies RGBA 8888 pixels, and you indicate the 93 buffer will be accessed from software (meaning your application will touch 94 pixels directly) then the allocator must create a buffer with 4 bytes per pixel 95 in R-G-B-A order. If instead, you say the buffer will be only accessed from 96 hardware and as a GLES texture, the allocator can do anything the GLES driver 97 wants—BGRA ordering, non-linear swizzled layouts, alternative color 98 formats, etc. Allowing the hardware to use its preferred format can improve 99 performance.</p> 100 101 <p>Some values cannot be combined on certain platforms. For example, the video 102 encoder flag may require YUV pixels, so adding software access and specifying 103 RGBA 8888 would fail.</p> 104 105 <p>The handle returned by the gralloc allocator can be passed between processes 106 through Binder.</p> 107 108 <h2 id=tracking>Tracking BufferQueue with systrace</h2> 109 110 <p>To really understand how graphics buffers move around, use systrace. The 111 system-level graphics code is well instrumented, as is much of the relevant app 112 framework code.</p> 113 114 <p>A full description of how to use systrace effectively would fill a rather 115 long document. Start by enabling the <code>gfx</code>, <code>view</code>, and 116 <code>sched</code> tags. You'll also see BufferQueues in the trace. If you've 117 used systrace before, you've probably seen them but maybe weren't sure what they 118 were. As an example, if you grab a trace while 119 <a href="https://github.com/google/grafika">Grafika's</a> "Play video 120 (SurfaceView)" is running, the row labeled <em>SurfaceView</em> tells you how 121 many buffers were queued up at any given time.</p> 122 123 <p>The value increments while the app is active—triggering the rendering 124 of frames by the MediaCodec decoder—and decrements while SurfaceFlinger is 125 doing work, consuming buffers. When showing video at 30fps, the queue's value 126 varies from 0 to 1 because the ~60fps display can easily keep up with the 127 source. (Notice also that SurfaceFlinger only wakes when there's work to 128 be done, not 60 times per second. The system tries very hard to avoid work and 129 will disable VSYNC entirely if nothing is updating the screen.)</p> 130 131 <p>If you switch to Grafika's "Play video (TextureView)" and grab a new trace, 132 you'll see a row labeled 133 com.android.grafika/com.android.grafika.PlayMovieActivity. This is the main UI 134 layer, which is just another BufferQueue. Because TextureView renders into the 135 UI layer (rather than a separate layer), you'll see all of the video-driven 136 updates here.</p> 137 138 <p>For more information about the systrace tool, refer to <a 139 href="https://developer.android.com/studio/profile/systrace-commandline.html">Systrace 140 documentation</a>.</p> 141 142 </body> 143 </html> 144
