The TextureView class introduced in Android 4.0 and is the most complex of the View objects discussed here, combining a View with a SurfaceTexture.
Recall that the SurfaceTexture is a "GL consumer", consuming buffers of graphics data and making them available as textures. TextureView wraps a SurfaceTexture, taking over the responsibility of responding to the callbacks and acquiring new buffers. The arrival of new buffers causes TextureView to issue a View invalidate request. When asked to draw, the TextureView uses the contents of the most recently received buffer as its data source, rendering wherever and however the View state indicates it should.
You can render on a TextureView with GLES just as you would SurfaceView. Just pass the SurfaceTexture to the EGL window creation call. However, doing so exposes a potential problem.
In most of what we've looked at, the BufferQueues have passed buffers between different processes. When rendering to a TextureView with GLES, both producer and consumer are in the same process, and they might even be handled on a single thread. Suppose we submit several buffers in quick succession from the UI thread. The EGL buffer swap call will need to dequeue a buffer from the BufferQueue, and it will stall until one is available. There won't be any available until the consumer acquires one for rendering, but that also happens on the UI thread… so we're stuck.
The solution is to have BufferQueue ensure there is always a buffer available to be dequeued, so the buffer swap never stalls. One way to guarantee this is to have BufferQueue discard the contents of the previously-queued buffer when a new buffer is queued, and to place restrictions on minimum buffer counts and maximum acquired buffer counts. (If your queue has three buffers, and all three buffers are acquired by the consumer, then there's nothing to dequeue and the buffer swap call must hang or fail. So we need to prevent the consumer from acquiring more than two buffers at once.) Dropping buffers is usually undesirable, so it's only enabled in specific situations, such as when the producer and consumer are in the same process.
Because TextureView is a proper citizen of the View hierarchy, it behaves like any other View, and can overlap or be overlapped by other elements. You can perform arbitrary transformations and retrieve the contents as a bitmap with simple API calls.
The main strike against TextureView is the performance of the composition step. With SurfaceView, the content is written to a separate layer that SurfaceFlinger composites, ideally with an overlay. With TextureView, the View composition is always performed with GLES, and updates to its contents may cause other View elements to redraw as well (e.g. if they're positioned on top of the TextureView). After the View rendering completes, the app UI layer must then be composited with other layers by SurfaceFlinger, so you're effectively compositing every visible pixel twice. For a full-screen video player, or any other application that is effectively just UI elements layered on top of video, SurfaceView offers much better performance.
As noted earlier, DRM-protected video can be presented only on an overlay plane. Video players that support protected content must be implemented with SurfaceView.
Grafika includes a pair of video players, one implemented with TextureView, the other with SurfaceView. The video decoding portion, which just sends frames from MediaCodec to a Surface, is the same for both. The most interesting differences between the implementations are the steps required to present the correct aspect ratio.
While SurfaceView requires a custom implementation of FrameLayout, resizing
SurfaceTexture is a simple matter of configuring a transformation matrix with
TextureView#setTransform(). For the former, you're sending new
window position and size values to SurfaceFlinger through WindowManager; for
the latter, you're just rendering it differently.
Otherwise, both implementations follow the same pattern. Once the Surface has been created, playback is enabled. When "play" is hit, a video decoding thread is started, with the Surface as the output target. After that, the app code doesn't have to do anything -- composition and display will either be handled by SurfaceFlinger (for the SurfaceView) or by TextureView.
This activity demonstrates manipulation of the SurfaceTexture inside a TextureView.
The basic structure of this activity is a pair of TextureViews that show two different videos playing side-by-side. To simulate the needs of a videoconferencing app, we want to keep the MediaCodec decoders alive when the activity is paused and resumed for an orientation change. The trick is that you can't change the Surface that a MediaCodec decoder uses without fully reconfiguring it, which is a fairly expensive operation; so we want to keep the Surface alive. The Surface is just a handle to the producer interface in the SurfaceTexture's BufferQueue, and the SurfaceTexture is managed by the TextureView;, so we also need to keep the SurfaceTexture alive. So how do we deal with the TextureView getting torn down?
It just so happens TextureView provides a setSurfaceTexture() call
that does exactly what we want. We obtain references to the SurfaceTextures
from the TextureViews and save them in a static field. When the activity is
shut down, we return "false" from the onSurfaceTextureDestroyed()
callback to prevent destruction of the SurfaceTexture. When the activity is
restarted, we stuff the old SurfaceTexture into the new TextureView. The
TextureView class takes care of creating and destroying the EGL contexts.
Each video decoder is driven from a separate thread. At first glance it might seem like we need EGL contexts local to each thread; but remember the buffers with decoded output are actually being sent from mediaserver to our BufferQueue consumers (the SurfaceTextures). The TextureViews take care of the rendering for us, and they execute on the UI thread.
Implementing this activity with SurfaceView would be a bit harder. We can't just create a pair of SurfaceViews and direct the output to them, because the Surfaces would be destroyed during an orientation change. Besides, that would add two layers, and limitations on the number of available overlays strongly motivate us to keep the number of layers to a minimum. Instead, we'd want to create a pair of SurfaceTextures to receive the output from the video decoders, and then perform the rendering in the app, using GLES to render two textured quads onto the SurfaceView's Surface.