topics/renderscript/graphics.jd

page.title=3D Graphics
parent.title=RenderScript
parent.link=index.html
@jd:body

  <div id="qv-wrapper">
    <div id="qv">
      <h2>In this document</h2>

      <ol>
        <li>
          <a href="#creating-graphics-rs">Creating a Graphics Renderscript</a>
          <ol>
            <li><a href="#creating-native">Creating the native Renderscript file</a></li>
            <li><a href="#creating-entry">Creating the Renderscript entry point class</a></li>
            <li><a href="#creating-view">Creating the surface view</a></li>
            <li><a href="#creating-activity">Creating the activity</a></li>
          </ol>
        </li>
        <li>
          <a href="#drawing">Drawing</a>
          <ol>
            <li><a href="#drawing-rsg">Drawing using the rsgDraw functions</a></li>
            <li><a href="#drawing-mesh">Drawing with a mesh</a></li>
          </ol>
        </li>
        <li>
          <a href="#shaders">Shaders</a>
          <ol>
            <li><a href="#shader-bindings">Shader bindings</a></li>
            <li><a href="#shader-sampler">Defining a sampler</a></li>
          </ol>
        </li>
      </ol>

      <h2>Related Samples</h2>

      <ol>
        <li><a href="{@docRoot}resources/samples/RenderScript/Balls/index.html">Balls</a></li>

        <li><a href="{@docRoot}resources/samples/RenderScript/Fountain/index.html">Fountain</a></li>

        <li><a href="{@docRoot}resources/samples/RenderScript/HelloWorld/index.html">Hello
        World</a></li>

        <li><a
href="{@docRoot}resources/samples/RenderScript/MiscSamples/index.html">Misc Samples</a></li>
      </ol>
    </div>
  </div>

  <p>RenderScript provides a number of graphics APIs for 3D rendering, both at the Android
  framework level as well as at the native level. For instance, the Android framework APIs let you
  create meshes and define shaders to customize the graphical rendering pipeline. The native
  RenderScript graphics APIs lets you draw the actual meshes to render your scene. In general, you
  will need to be familiar with APIs to appropriately render 3D graphics on an Android-powered
  device.</p>

  <h2 id="creating-graphics-rs">Creating a Graphics RenderScript</h2>

  <p>Because of the various layers of code when writing a RenderScript application, it is useful to
  create the following files for a scene that you want to render:</p>

  <ul>
    <li>The native RenderScript <code>.rs</code> file. This file contains the logic to do the
    graphics rendering.</li>

    <li>The RenderScript entry point class that allows your view to interact with the code defined
    in the <code>.rs</code> file. This class contains a RenderScript object(instance of
    <code>ScriptC_<em>renderscript_file</em></code>), which allows your Android framework code to
    call the native RenderScript code. This class also creates the {@link
    android.renderscript.RenderScriptGL} context object, which contains the current rendering state
    of the RenderScript such as programs (vertex and fragment shaders, for example) that you want
    to define and bind to the graphics pipeline. The context object attaches to the RenderScript
    object (instance of <code><em>ScriptC_renderscript_file</em></code>) that does the rendering.
    Our example names this class <code>HelloWorldRS</code>.</li>

    <li>Create a class that extends {@link android.renderscript.RSSurfaceView} to provide a surface
    to render on. If you want to implement callbacks from events inherited from {@link
    android.view.View}, such as {@link android.view.View#onTouchEvent onTouchEvent()} and {@link
    android.view.View#onKeyDown onKeyDown()}, do so in this class as well.</li>

    <li>Create a class that is the main Activity class, like you would with any Android
    application. This class sets your {@link android.renderscript.RSSurfaceView} as the content
    view for this Activity.</li>
  </ul>

  <p>The following sections describe how to implement these three classes by using the HelloWorld
  RenderScript sample that is provided in the SDK as a guide (some code has been modified from its
  original form for simplicity).</p>

  <h3 id="creating-native">Creating the native RenderScript file</h3>

  <p>Your native RenderScript code resides in a <code>.rs</code> file in the
  <code>&lt;project_root&gt;/src/</code> directory. You can also define <code>.rsh</code> header
  files. This code contains the logic to render your graphics and declares all necessary variables
  and pointers. Every graphics <code>.rs</code> file generally contains the following items:</p>

  <ul>
    <li>A pragma (<code>#pragma rs java_package_name(<em>package.name</em>)</code>) that declares
    the package name of the <code>.java</code> reflection of this RenderScript.</li>

    <li>A pragma (<code>#pragma version(1)</code>) that declares the version of RenderScript that
    you are using (1 is the only value for now).</li>

    <li>A <code>#include</code> of the rs_graphics.rsh header file.</li>

    <li>A <code>root()</code> function. This is the main worker function for your RenderScript and
    calls RenderScript graphics APIs to draw meshes to the surface. This function is called every
    time a frame refresh occurs, which is specified as its return value. A <code>0</code> specified
    for the return value says to only render the frame when a property of the scene that you are
    rendering changes. A non-zero positive integer specifies the refresh rate of the frame in
    milliseconds.

      <p class="note"><strong>Note:</strong> The RenderScript runtime makes its best effort to
      refresh the frame at the specified rate. For example, if you are creating a live wallpaper
      and set the return value to 50, the runtime renders the wallpaper at 20fps if it has just
      enough or more resources to do so, and renders as fast as it can if it does not.</p>

      <p>For more
      information on using the RenderScript graphics functions, see the <a href=
      "#drawing">Drawing</a> section.</p>
    </li>

    <li>An <code>init()</code> function. This allows you to do any initialization of your
    RenderScript before the <code>root()</code> function runs, such as initializing variables. This
    function runs once and is called automatically when the RenderScript starts, before anything
    else in your RenderScript. Creating this function is optional.</li>

    <li>Any variables, pointers, and structures that you wish to use in your RenderScript code (can
    be declared in <code>.rsh</code> files if desired)</li>
  </ul>

  <p>The following code shows how the <code>helloworld.rs</code> file is implemented:</p>
  <pre>
#pragma version(1)

// Tell which java package name the reflected files should belong to
#pragma rs java_package_name(com.android.rs.helloworld)

// Built-in header with graphics APIs
#include "rs_graphics.rsh"

// gTouchX and gTouchY are variables that are reflected for use
// by the Android framework API. This RenderScript uses them to be notified of touch events.
int gTouchX;
int gTouchY;

// This is invoked automatically when the script is created and initializes the variables
// in the Android framework layer as well.
void init() {
    gTouchX = 50.0f;
    gTouchY = 50.0f;
}

int root(int launchID) {

    // Clear the background color
    rsgClearColor(0.0f, 0.0f, 0.0f, 0.0f);
    // Tell the runtime what the font color should be
    rsgFontColor(1.0f, 1.0f, 1.0f, 1.0f);
    // Introuduce ourselves to the world by drawing a greeting
    // at the position user touched on the screen
    rsgDrawText("Hello World!", gTouchX, gTouchY);

    // Return value tells RS roughly how often to redraw
    // in this case 20 ms
    return 20;
}
</pre>

  <h3 id="creating-entry">Creating the RenderScript entry point class</h3>

  <p>When you create a RenderScript (<code>.rs</code>) file, it is helpful to create a
  corresponding Android framework class that is an entry point into the <code>.rs</code> file. In
  this entry point class, you create a RenderScript object by instantiating a
  <code>ScriptC_<em>rs_filename</em></code> and binding it to the RenderScript context. The
  RenderScript object is attached to the RenderScript bytecode, which is platform-independent and
  gets compiled on the device when the RenderScript application runs. Both the
  <code>ScriptC_<em>rs_filename</em></code> class and bytecode is generated by the Android build
  tools and is packaged with the <code>.apk</code> file. The bytecode file is located in the
  <code>&lt;project_root&gt;/res/raw/</code> directory and is named <code>rs_filename.bc</code>.
  You refer to the bytecode as a resource (<code>R.raw.<em>rs_filename</em></code>). when creating
  the RenderScript object..</p>

  <p>You then bind the RenderScript object to the RenderScript context, so that the surface view
  knows what code to use to render graphics. The following code shows how the
  <code>HelloWorldRS</code> class is implemented:</p>
  <pre>
package com.android.rs.helloworld;

import android.content.res.Resources;
import android.renderscript.*;

public class HelloWorldRS {
    //context and resources are obtained from RSSurfaceView, which calls init()
    private Resources mRes;
    private RenderScriptGL mRS;

    //Declare the RenderScript object
    private ScriptC_helloworld mScript;

    public HelloWorldRS() {
    }

    /**
     * This provides us with the RenderScript context and resources
     * that allow us to create the RenderScript object
     */
    public void init(RenderScriptGL rs, Resources res) {
        mRS = rs;
        mRes = res;
        initRS();
    }
    /**
     * Calls native RenderScript functions (set_gTouchX and set_gTouchY)
     * through the reflected layer class ScriptC_helloworld to pass in
     * touch point data.
     */
    public void onActionDown(int x, int y) {
        mScript.set_gTouchX(x);
        mScript.set_gTouchY(y);
    }
    /**
     * Binds the RenderScript object to the RenderScript context
     */
    private void initRS() {
        //create the RenderScript object
        mScript = new ScriptC_helloworld(mRS, mRes, R.raw.helloworld);
        //bind the RenderScript object to the RenderScript context
        mRS.bindRootScript(mScript);
    }
}

</pre>

  <h3 id="creating-view">Creating the surface view</h3>

  <p>To create a surface view to render graphics on, create a class that extends {@link
  android.renderscript.RSSurfaceView}. This class also creates a RenderScript context object
  ({@link android.renderscript.RenderScriptGL} and passes it to the Rendscript entry point class to
  bind the two. The following code shows how the <code>HelloWorldView</code> class is
  implemented:</p>
  <pre>
package com.android.rs.helloworld;

import android.renderscript.RSSurfaceView;
import android.renderscript.RenderScriptGL;
import android.content.Context;
import android.view.MotionEvent;

public class HelloWorldView extends RSSurfaceView {
    // RenderScript context
    private RenderScriptGL mRS;
    // RenderScript entry point object that does the rendering
    private HelloWorldRS mRender;

    public HelloWorldView(Context context) {
        super(context);
        initRS();
    }

    private void initRS() {
        if (mRS == null) {
            // Initialize RenderScript with default surface characteristics.
            RenderScriptGL.SurfaceConfig sc = new RenderScriptGL.SurfaceConfig();
            //Create the RenderScript context
            mRS = createRenderScriptGL(sc);
            // Create an instance of the RenderScript entry point class
            mRender = new HelloWorldRS();
            // Call the entry point class to bind it to this context
            mRender.init(mRS, getResources());
        }
    }

    /**
     * Rebind everything when the window becomes attached
     */
    protected void onAttachedToWindow() {
        super.onAttachedToWindow();
        initRS();
    }

    /**
     * Stop rendering when window becomes detached
     */
    protected void onDetachedFromWindow() {
        // Handle the system event and clean up
        mRender = null;
        if (mRS != null) {
            mRS = null;
            destroyRenderScriptGL();
        }
    }

    /**
     * Use callbacks to relay data to RenderScript entry point class
     */
    public boolean onTouchEvent(MotionEvent ev) {
        // Pass touch events from the system to the rendering script
        if (ev.getAction() == MotionEvent.ACTION_DOWN) {
            mRender.onActionDown((int)ev.getX(), (int)ev.getY());
            return true;
        }

        return false;
    }
}

</pre>

  <h3 id="creating-activity">Creating the Activity</h3>

  <p>Applications that use RenderScript still adhere to activity lifecyle, and are part of the same
  view hierarchy as traditional Android applications, which is handled by the Android VM. This
  Activity class sets its view to be the {@link android.renderscript.RSSurfaceView} and handles
  lifecycle callback events appropriately. The following code shows how the <code>HelloWorld</code>
  class is implemented:</p>
  <pre>
public class HelloWorldActivity extends Activity {

    //Custom view to use with RenderScript
    private HelloWorldView view;

    public void onCreate(Bundle icicle) {
        super.onCreate(icicle);
        // Create surface view and set it as the content of our Activity
        mView = new HelloWorldView(this);
        setContentView(view);
    }

    protected void onResume() {
        // Ideally an app should implement onResume() and onPause()
        // to take appropriate action when the activity loses focus
        super.onResume();
        view.resume();
    }

    protected void onPause() {
        // Ideally an app should implement onResume() and onPause()
        // to take appropriate action when the activity loses focus
        super.onPause();
        view.pause();
    }
}
</pre>

  <h2 id="drawing">Drawing</h2>
  <p>The following sections describe how to use the graphics functions to draw with Renderscript.</p>
  <h3 id="drawing-rsg">Drawing using the rsgDraw functions</h3>

  <p>The native RenderScript APIs provide a few convenient functions to easily draw a polygon to
  the screen. You call these in your <code>root()</code> function to have them render to the
  surface view. These functions are available for simple drawing and should not be used for complex
  graphics rendering:</p>

  <ul>
    <li><code>rsgDrawRect()</code>: Sets up a mesh and draws a rectangle to the screen. It uses the
    top left vertex and bottom right vertex of the rectangle to draw.</li>

    <li><code>rsgDrawQuad()</code>: Sets up a mesh and draws a quadrilateral to the screen.</li>

    <li><code>rsgDrawQuadTexCoords()</code>: Sets up a mesh and draws a textured quadrilateral to
    the screen.</li>
  </ul>

  <h3 id="drawing-mesh">Drawing with a mesh</h3>

  <p>When you want to draw complex shapes and textures to the screen, instantiate a {@link
  android.renderscript.Mesh} and draw it to the screen with <code>rsgDrawMesh()</code>. A {@link
  android.renderscript.Mesh} is a collection of allocations that represent vertex data (positions,
  normals, texture coordinates) and index data such as triangles and lines. You can build a Mesh in
  three different ways:</p>

  <ul>
    <li>Build the mesh with the {@link android.renderscript.Mesh.TriangleMeshBuilder} class, which
    allows you to specify a set of vertices and indices for each triangle that you want to draw.
    The downside of doing it this way is there is no way to specify the vertices in your native
    RenderScript code.</li>

    <li>Build the mesh using an {@link android.renderscript.Allocation} or a set of {@link
    android.renderscript.Allocation}s with the {@link android.renderscript.Mesh.AllocationBuilder}
    class. This allows you to build a mesh with vertices already stored in memory, which allows you
    to set the vertices in native or Android code.</li>

    <li>Build the mesh with the {@link android.renderscript.Mesh.Builder} class. This is a
    convenience method for when you know what data types you want to use to build your mesh, but
    don't want to make separate memory allocations like with {@link
    android.renderscript.Mesh.AllocationBuilder}. You can specify the types that you want and this
    mesh builder automatically creates the memory allocations for you.</li>
  </ul>

  <p>To create a mesh using the {@link android.renderscript.Mesh.TriangleMeshBuilder}, you need to
  supply it with a set of vertices and the indices for the vertices that comprise the triangle. For
  example, the following code specifies three vertices, which are added to an internal array,
  indexed in the order they were added. The call to {@link
  android.renderscript.Mesh.TriangleMeshBuilder#addTriangle addTriangle()} draws the triangle with
  vertex 0, 1, and 2 (the vertices are drawn counter-clockwise).</p>
  <pre>
int float2VtxSize = 2;
Mesh.TriangleMeshBuilder triangle = new Mesh.TriangleMeshBuilder(renderscriptGL,
float2VtxSize, Mesh.TriangleMeshBuilder.COLOR);
triangles.addVertex(300.f, 300.f);
triangles.addVertex(150.f, 450.f);
triangles.addVertex(450.f, 450.f);
triangles.addTriangle(0 , 1, 2);
Mesh smP = triangle.create(true);
script.set_mesh(smP);
</pre>

  <p>To draw a mesh using the {@link android.renderscript.Mesh.AllocationBuilder}, you need to
  supply it with one or more allocations that contain the vertex data:</p>
  <pre>
Allocation vertices;

...
Mesh.AllocationBuilder triangle = new Mesh.AllocationBuilder(mRS);
smb.addVertexAllocation(vertices.getAllocation());
smb.addIndexSetType(Mesh.Primitive.TRIANGLE);
Mesh smP = smb.create();
script.set_mesh(smP);
</pre>

  <p>In your native RenderScript code, draw the built mesh to the screen:</p>
  <pre>
rs_mesh mesh;
...

int root(){
...
rsgDrawMesh(mesh);
...
return 0; //specify a non zero, positive integer to specify the frame refresh.
          //0 refreshes the frame only when the mesh changes.
}
</pre>

  <h2 id="shaders">Shaders</h2>

  <p>You can attach four program objects to the {@link android.renderscript.RenderScriptGL} context
  to customize the rendering pipeline. For example, you can create vertex and fragment shaders in
  GLSL or build a raster program object with provided methods without writing GLSL code. The four
  program objects mirror a traditional graphical rendering pipeline:</p>

  <table>
    <tr>
      <th>Android Object Type</th>

      <th>RenderScript Native Type</th>

      <th>Description</th>
    </tr>

    <tr>
      <td>{@link android.renderscript.ProgramVertex}</td>

      <td>rs_program_vertex</td>

      <td>
        <p>The RenderScript vertex program, also known as a vertex shader, describes the stage in
        the graphics pipeline responsible for manipulating geometric data in a user-defined way.
        The object is constructed by providing RenderScript with the following data:</p>

        <ul>
          <li>An Element describing its varying inputs or attributes</li>

          <li>GLSL shader string that defines the body of the program</li>

          <li>a Type that describes the layout of an Allocation containing constant or uniform
          inputs</li>
        </ul>

        <p>Once the program is created, bind it to the {@link android.renderscript.RenderScriptGL}
        graphics context by calling {@link android.renderscript.RenderScriptGL#bindProgramVertex
        bindProgramVertex()}. It is then used for all subsequent draw calls until you bind a new
        program. If the program has constant inputs, the user needs to bind an allocation
        containing those inputs. The allocation's type must match the one provided during creation.
        The RenderScript library then does all the necessary plumbing to send those constants to
        the graphics hardware. Varying inputs to the shader, such as position, normal, and texture
        coordinates are matched by name between the input Element and the Mesh object being drawn.
        The signatures don't have to be exact or in any strict order. As long as the input name in
        the shader matches a channel name and size available on the mesh, the run-time would take
        care of connecting the two. Unlike OpenGL, there is no need to link the vertex and fragment
        programs.</p>

        <p>To bind shader constructs to the Program, declare a struct containing the necessary
        shader constants in your native RenderScript code. This struct is generated into a
        reflected class that you can use as a constant input element during the Program's creation.
        It is an easy way to create an instance of this struct as an allocation. You would then
        bind this Allocation to the Program and the RenderScript system sends the data that is
        contained in the struct to the hardware when necessary. To update shader constants, you
        change the values in the Allocation and notify the native RenderScript code of the
        change.</p>
      </td>
    </tr>

    <tr>
      <td>{@link android.renderscript.ProgramFragment}</td>

      <td>rs_program_fragment</td>

      <td>
        <p>The RenderScript fragment program, also known as the fragment shader, is responsible for
        manipulating pixel data in a user-defined way. It's constructed from a GLSL shader string
        containing the program body, textures inputs, and a Type object describing the constants
        used by the program. Like the vertex programs, when an allocation with constant input
        values is bound to the shader, its values are sent to the graphics program automatically.
        Note that the values inside the allocation are not explicitly tracked. If they change
        between two draw calls using the same program object, notify the runtime of that change by
        calling rsgAllocationSyncAll so it could send the new values to hardware. Communication
        between the vertex and fragment programs is handled internally in the GLSL code. For
        example, if the fragment program is expecting a varying input called varTex0, the GLSL code
        inside the program vertex must provide it.</p>

        <p>To bind shader constants to this program, declare a struct containing the necessary
        shader constants in your native RenderScript code. This struct is generated into a
        reflected class that you can use as a constant input element during the Program's creation.
        It is an easy way to create an instance of this struct as an allocation. You would then
        bind this Allocation to the Program and the RenderScript system sends the data that is
        contained in the struct to the hardware when necessary. To update shader constants, you
        change the values in the Allocation and notify the native RenderScript code of the
        change.</p>
      </td>
    </tr>

    <tr>
      <td>{@link android.renderscript.ProgramStore}</td>

      <td>rs_program_store</td>

      <td>The RenderScript ProgramStore contains a set of parameters that control how the graphics
      hardware writes to the framebuffer. It could be used to enable and disable depth writes and
      testing, setup various blending modes for effects like transparency and define write masks
      for color components.</td>
    </tr>

    <tr>
      <td>{@link android.renderscript.ProgramRaster}</td>

      <td>rs_program_raster</td>

      <td>Program raster is primarily used to specify whether point sprites are enabled and to
      control the culling mode. By default back faces are culled.</td>
    </tr>
  </table>

  <p>The following example defines a vertex shader in GLSL and binds it to the RenderScript:</p>
  <pre>
    private RenderScriptGL glRenderer;      //rendering context
    private ScriptField_Point mPoints;      //vertices
    private ScriptField_VpConsts mVpConsts; //shader constants

    ...

     ProgramVertex.Builder sb = new ProgramVertex.Builder(glRenderer);
        String t =  "varying vec4 varColor;\n" +
                    "void main() {\n" +
                    "  vec4 pos = vec4(0.0, 0.0, 0.0, 1.0);\n" +
                    "  pos.xy = ATTRIB_position;\n" +
                    "  gl_Position = UNI_MVP * pos;\n" +
                    "  varColor = vec4(1.0, 1.0, 1.0, 1.0);\n" +
                    "  gl_PointSize = ATTRIB_size;\n" +
                    "}\n";
        sb.setShader(t);
        sb.addConstant(mVpConsts.getType());
        sb.addInput(mPoints.getElement());
        ProgramVertex pvs = sb.create();
        pvs.bindConstants(mVpConsts.getAllocation(), 0);
        glRenderer.bindProgramVertex(pvs);


</pre>

  <p>The <a href=
  "{@docRoot}resources/samples/RenderScript/MiscSamples/src/com/example/android/rs/miscsamples/RsRenderStatesRS.html">
  RsRenderStatesRS</a> sample has many examples on how to create a shader without writing GLSL.</p>

  <h3 id="shader-bindings">Shader bindings</h3>

  <p>You can also set four pragmas that control the shaders' default bindings to the {@link
  android.renderscript.RenderScriptGL} context when the script is executing:</p>

  <ul>
    <li>stateVertex</li>

    <li>stateFragment</li>

    <li>stateRaster</li>

    <li>stateStore</li>
  </ul>

  <p>The possible values for each pragma are <code>parent</code> or <code>default</code>. Using
  <code>default</code> binds the shaders to the graphical context with the system defaults. The
  default shader is defined below:</p>
  <pre>
("varying vec4 varColor;\n");
("varying vec2 varTex0;\n");
("void main() {\n");
(" gl_Position = UNI_MVP * ATTRIB_position;\n");
(" gl_PointSize = 1.0;\n");
(" varColor = ATTRIB_color;\n");
(" varTex0 = ATTRIB_texture0;\n");
("}\n");
</pre>

  <p>Using <code>parent</code> binds the shaders in the same manner as it is bound in the calling
  script. If this is the root script, the parent state is taken from the bind points that are set
  by the {@link android.renderscript.RenderScriptGL} bind methods.</p>

  <p>For example, you can define this at the top of your native graphics RenderScript code to have
  the Vertex and Store shaders inherent the bind properties from their parent scripts:</p>
  <pre>
#pragma stateVertex(parent)
#pragma stateStore(parent)
</pre>

  <h3 id="shader-sampler">Defining a sampler</h3>

  <p>A {@link android.renderscript.Sampler} object defines how data is extracted from textures.
  Samplers are bound to Program objects (currently only a Fragment Program) alongside the texture
  whose sampling they control. These objects are used to specify such things as edge clamping
  behavior, whether mip-maps are used, and the amount of anisotropy required. There might be
  situations where hardware does not support the desired behavior of the sampler. In these cases,
  the runtime attempts to provide the closest possible approximation. For example, the user
  requested 16x anisotropy, but only 8x was set because it's the best available on the
  hardware.</p>

  <p>The <a href=
  "{@docRoot}resources/samples/RenderScript/MiscSamples/src/com/example/android/rs/miscsamples/RsRenderStatesRS.html">
  RsRenderStatesRS</a> sample has many examples on how to create a sampler and bind it to a
  Fragment program.</p>

</body>
</html>