1 <html devsite> 2 <head> 3 <title>Graphics architecture</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 27 <p><em>What every developer should know about Surface, SurfaceHolder, 28 EGLSurface, SurfaceView, GLSurfaceView, SurfaceTexture, TextureView, 29 SurfaceFlinger, and Vulkan.</em></p> 30 31 <p>This page describes essential elements of the Android system-level graphics 32 architecture and how they are used by the application framework and multimedia 33 system. The focus is on how buffers of graphical data move through the system. 34 If you've ever wondered why SurfaceView and TextureView behave the way they do, 35 or how Surface and EGLSurface interact, you are in the correct place.</p> 36 37 <p>Some familiarity with Android devices and application development is assumed. 38 You don't need detailed knowledge of the app framework and very few API calls 39 are mentioned, but the material doesn't overlap with other public 40 documentation. The goal is to provide details on the significant events 41 involved in rendering a frame for output to help you make informed choices 42 when designing an application. To achieve this, we work from the bottom up, 43 describing how the UI classes work rather than how they can be used.</p> 44 45 <p>This section includes several pages covering everything from background 46 material to HAL details to use cases. It starts with an explanation of Android 47 graphics buffers, describes the composition and display mechanism, then proceeds 48 to the higher-level mechanisms that supply the compositor with data. We 49 recommend reading pages in the order listed below rather than skipping to a 50 topic that sounds interesting.</p> 51 52 <h2 id=low_level>Low-level components</h2> 53 54 <ul> 55 <li><a href="/devices/graphics/arch-bq-gralloc.html">BufferQueue and 56 gralloc</a>. BufferQueue connects something that generates buffers of graphical 57 data (the <em>producer</em>) to something that accepts the data for display or 58 further processing (the <em>consumer</em>). Buffer allocations are performed 59 through the <em>gralloc</em> memory allocator implemented through a 60 vendor-specific HAL interface.</li> 61 62 <li><a href="/devices/graphics/arch-sf-hwc.html">SurfaceFlinger, 63 Hardware Composer, and virtual displays</a>. SurfaceFlinger accepts buffers of 64 data from multiple sources, composites them, and sends them to the display. The 65 Hardware Composer HAL (HWC) determines the most efficient way to composite 66 buffers with the available hardware, and virtual displays make composited output 67 available within the system (recording the screen or sending the screen over a 68 network).</li> 69 70 <li><a href="/devices/graphics/arch-sh.html">Surface, Canvas, and 71 SurfaceHolder</a>. A Surface produces a buffer queue that is often consumed by 72 SurfaceFlinger. When rendering onto a Surface, the result ends up in a buffer 73 that gets shipped to the consumer. Canvas APIs provide a software implementation 74 (with hardware-acceleration support) for drawing directly on a Surface 75 (low-level alternative to OpenGL ES). Anything having to do with a View involves 76 a SurfaceHolder, whose APIs enable getting and setting Surface parameters such 77 as size and format.</li> 78 79 <li><a href="/devices/graphics/arch-egl-opengl.html">EGLSurface and 80 OpenGL ES</a>. OpenGL ES (GLES) defines a graphics-rendering API designed to be 81 combined with EGL, a library that knows how to create and access windows through 82 the operating system (to draw textured polygons, use GLES calls; to put 83 rendering on the screen, use EGL calls). This page also covers ANativeWindow, 84 the C/C++ equivalent of the Java Surface class used to create an EGL window 85 surface from native code.</li> 86 87 <li><a href="/devices/graphics/arch-vulkan.html">Vulkan</a>. Vulkan is 88 a low-overhead, cross-platform API for high-performance 3D graphics. Like OpenGL 89 ES, Vulkan provides tools for creating high-quality, real-time graphics in 90 applications. Vulkan advantages include reductions in CPU overhead and support 91 for the <a href="https://www.khronos.org/spir">SPIR-V Binary Intermediate</a> 92 language.</li> 93 94 </ul> 95 96 <h2 id=high_level>High-level components</h2> 97 98 <ul> 99 <li><a href="/devices/graphics/arch-sv-glsv.html">SurfaceView and 100 GLSurfaceView</a>. SurfaceView combines a Surface and a View. SurfaceView's View 101 components are composited by SurfaceFlinger (and not the app), enabling 102 rendering from a separate thread/process and isolation from app UI rendering. 103 GLSurfaceView provides helper classes to manage EGL contexts, inter-thread 104 communication, and interaction with the Activity lifecycle (but is not required 105 to use GLES).</li> 106 107 <li><a href="/devices/graphics/arch-st.html">SurfaceTexture</a>. 108 SurfaceTexture combines a Surface and GLES texture to create a BufferQueue for 109 which your app is the consumer. When a producer queues a new buffer, it notifies 110 your app, which in turn releases the previously-held buffer, acquires the new 111 buffer from the queue, and makes EGL calls to make the buffer available to GLES 112 as an external texture. Android 7.0 adds support for secure texture video 113 playback enabling GPU post-processing of protected video content.</li> 114 115 <li><a href="/devices/graphics/arch-tv.html">TextureView</a>. 116 TextureView combines a View with a SurfaceTexture. TextureView wraps a 117 SurfaceTexture and takes responsibility for responding to callbacks and 118 acquiring new buffers. When drawing, TextureView uses the contents of the most 119 recently received buffer as its data source, rendering wherever and however the 120 View state indicates it should. View composition is always performed with GLES, 121 meaning updates to contents may cause other View elements to redraw as well.</li> 122 </ul> 123 124 </body> 125 </html> 126
