1 <html devsite> 2 <head> 3 <title>Implementing graphics</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 27 <p>To implement the Android graphics HAL, review the following requirements, 28 implementation details, and testing advice.</p> 29 30 <h2 id=requirements>Requirements</h2> 31 32 <p>Android graphics support requires the following components:</p> 33 34 <ul> 35 <li>EGL driver</li> 36 <li>OpenGL ES 1.x driver</li> 37 <li>OpenGL ES 2.0 driver</li> 38 <li>OpenGL ES 3.x driver (optional)</li> 39 <li>Vulkan (optional)</li> 40 <li>Gralloc HAL implementation</li> 41 <li>Hardware Composer HAL implementation</li> 42 </ul> 43 44 <h2 id=implementation>Implementation</h2> 45 46 <h3 id=opengl_and_egl_drivers>OpenGL and EGL drivers</h3> 47 48 <p>You must provide drivers for EGL, OpenGL ES 1.x, and OpenGL ES 2.0 (support 49 for OpenGL 3.x is optional). Key considerations include:</p> 50 51 <ul> 52 <li>GL driver must be robust and conformant to OpenGL ES standards.</li> 53 <li>Do not limit the number of GL contexts. Because Android allows apps in 54 the background and tries to keep GL contexts alive, you should not limit the 55 number of contexts in your driver.</li> 56 <li> It is common to have 20-30 active GL contexts at once, so be 57 mindful of the amount of memory allocated for each context.</li> 58 <li>Support the YV12 image format and other YUV image formats that come from 59 other components in the system, such as media codecs or the camera.</li> 60 <li>Support the mandatory extensions: <code>GL_OES_texture_external</code>, 61 <code>EGL_ANDROID_image_native_buffer</code>, and 62 <code>EGL_ANDROID_recordable</code>. In addition, the 63 <code>EGL_ANDROID_framebuffer_target</code> extension is required for 64 Hardware Composer v1.1 and higher.</li> 65 </ul> 66 <p>We highly recommend also supporting <code>EGL_ANDROID_blob_cache</code>, 67 <code>EGL_KHR_fence_sync</code>, <code>EGL_KHR_wait_sync</code>, and <code>EGL_ANDROID_native_fence_sync</code>.</p> 68 69 <p class="note"><strong>Note</strong>: The OpenGL API exposed to app developers 70 differs from the OpenGL implemented on the device. Apps cannot directly access 71 the GL driver layer and must go through the interface provided by the APIs.</p> 72 73 <h3 id=pre-rotation>Pre-rotation</h3> 74 75 <p>Many hardware overlays do not support rotation (and even if they do it costs 76 processing power); the solution is to pre-transform the buffer before it reaches 77 SurfaceFlinger. Android supports a query hint 78 (<code>NATIVE_WINDOW_TRANSFORM_HINT</code>) in <code>ANativeWindow</code> to 79 represent the most likely transform to be applied to the buffer by 80 SurfaceFlinger. GL drivers can use this hint to pre-transform the buffer 81 before it reaches SurfaceFlinger so when the buffer arrives, it is correctly 82 transformed.</p> 83 84 <p>For example, when receiving a hint to rotate 90 degrees, generate and apply a 85 matrix to the buffer to prevent it from running off the end of the page. To save 86 power, do this pre-rotation. For details, see the <code>ANativeWindow</code> 87 interface defined in <code>system/core/include/system/window.h</code>.</p> 88 89 <h3 id=gralloc_hal>Gralloc HAL</h3> 90 91 <p>The graphics memory allocator allocates memory requested by image producers. 92 You can find the interface definition of the HAL at 93 <code>hardware/libhardware/include/hardware/gralloc.h</code>.</p> 94 95 <h3 id=protected_buffers>Protected buffers</h3> 96 97 <p>The gralloc usage flag <code>GRALLOC_USAGE_PROTECTED</code> allows the 98 graphics buffer to be displayed only through a hardware-protected path. These 99 overlay planes are the only way to display DRM content (DRM-protected buffers 100 cannot be accessed by SurfaceFlinger or the OpenGL ES driver).</p> 101 102 <p>DRM-protected video can be presented only on an overlay plane. Video players 103 that support protected content must be implemented with SurfaceView. Software 104 running on unprotected hardware cannot read or write the buffer; 105 hardware-protected paths must appear on the Hardware Composer overlay (i.e., 106 protected videos will disappear from the display if Hardware Composer switches 107 to OpenGL ES composition).</p> 108 109 <p>For details on protected content, see 110 <a href="/devices/drm.html">DRM</a>.</p> 111 112 <h3 id=hardware_composer_hal>Hardware Composer HAL</h3> 113 114 <p>The Hardware Composer HAL (HWC) is used by SurfaceFlinger to composite 115 surfaces to the screen. It abstracts objects such as overlays and 2D blitters 116 and helps offload some work that would normally be done with OpenGL. For details 117 on the HWC, see <a href="/devices/graphics/implement-hwc.html">Hardware 118 Composer HAL</a>.</p> 119 120 <h3 id=vsync>VSYNC</h3> 121 122 <p>VSYNC synchronizes certain events to the refresh cycle of the display. 123 Applications always start drawing on a VSYNC boundary, and SurfaceFlinger always 124 composites on a VSYNC boundary. This eliminates stutters and improves visual 125 performance of graphics. For details on VSYNC, see 126 <a href="/devices/graphics/implement-vsync.html">Implementing 127 VSYNC</a>.</p> 128 129 <h3 id=vulkan>Vulkan</h3> 130 131 <p>Vulkan is a low-overhead, cross-platform API for high-performance 3D graphics. 132 Like OpenGL ES, Vulkan provides tools for creating high-quality, real-time 133 graphics in applications. Vulkan advantages include reductions in CPU overhead 134 and support for the <a href="https://www.khronos.org/spir">SPIR-V Binary 135 Intermediate</a> language. For details on Vulkan, see 136 <a href="/devices/graphics/implement-vulkan.html">Implementing 137 Vulkan</a>.</p> 138 139 <h3 id=virtual_displays>Virtual displays</h3> 140 141 <p>Android added platform support for virtual displays in Hardware Composer v1.3. 142 The virtual display composition is similar to the physical display: Input 143 layers are described in prepare(), SurfaceFlinger conducts GPU composition, and 144 layers and GPU framebuffer are provided to Hardware Composer in set(). For 145 details on virtual displays, see 146 <a href="/devices/graphics/implement-vdisplays.html">Implementing 147 Virtual Displays</a>.</p> 148 149 <h2 id=testing>Testing</h2> 150 151 <p>For benchmarking, use the following flow by phase:</p> 152 153 <ul> 154 <li><em>Specification</em>. When initially specifying the device (such as when 155 using immature drivers), use predefined (fixed) clocks and workloads to 156 measure frames per second (fps) rendered. This gives a clear view of hardware 157 capabilities.</li> 158 <li><em>Development</em>. As drivers mature, use a fixed set of user actions 159 to measure the number of visible stutters (janks) in animations.</li> 160 <li><em>Production</em>. When a device is ready for comparison against 161 competitors, increase the workload until stutters increase. Determine if the 162 current clock settings can keep up with the load. This can help you identify 163 where to slow the clocks and reduce power use.</li> 164 </ul> 165 166 <p>For help deriving device capabilities during the specification phase, use the 167 Flatland tool at <code>platform/frameworks/native/cmds/flatland/</code>. 168 Flatland relies upon fixed clocks and shows the throughput achievable with 169 composition-based workloads. It uses gralloc buffers to simulate multiple window 170 scenarios, filling in the window with GL then measuring the compositing.</p> 171 172 <p class="note"><strong>Note:</strong> Flatland uses the synchronization 173 framework to measure time, so your implementation must support the 174 synchronization framework.</p> 175 176 </body> 177 </html> 178
