OpenGrok

Introduction:
-------------
The emugen tool is a tool to generate a wire protocol implementation
based on provided API.  The tool generates c++ encoder code that takes
API calls and encodes them into the wire and decoder code that decodes
the wire stream and calls server matching API.
The emugen tool includes additional functionality that enables to
generate an wrapper library. The wrapper library provides entry points
for the specified API, where each entry routes the call via a dispatch
table. The dispatch table may be initialized as required by a specific application.

The following paragraphs includes the following:
    * Wire Protocol Description
    * Input files description & format
    * Generated code description.


Note: In this document, the caller is referred to as Encoder or Client
and the callee is referred to as the Decoder or Server. These terms
are used interchangeably by the context.


Wire Protocol packet structure:
-------------------------------
A general Encoder->Decoder packet is structured as following:
struct Packet {
	unsigned int opcode;
	unsigned int packet_len;
	 parameter 1
	 parameter 2
};
A general Decoder->Encoder reply is expected to be received in the
context of the call that triggered it, thus it includes the reply
data only, with no context headers. In precise term terms, a reply
packet will look like:

struct {
	...// reply data
};

consider the following function call:
int foo(int p1, short s1)
will be encoded into :
{
	101, // foo opcode
	14, // sizeof(opcode) + sizeof(packet_len) + sizeof(int) + sizeof(short)
	p1, // 4 bytes
	s1 // 2 bytes
}

Since foo returns value, the caller is expected to read back the return packet from the server->client stream. The return value in this example is in thus the return packet is:
{
	int retval;
}


Pointer decoding:
----------------
The wire protocol also allows exchanging of pointer data
(arrays). Pointers are defined with directions:

	in : Data is sent from the caller to the calle
	out: Data is sent from the callee to the caller
	in_out: data is sent from the caller and return in place.

in and in_out encoded with their len:
{
	unsinged int pointer_data_len;
	unsigned char data[pointer_data_len]; // pointer data
}

out pointers are encoded by data length only:
{
unsigned int pointer_data_len;
}

out and in_out pointers data is returned in the return
packet. For example, consider the following call:

int foo(int n, int *ptr); // assume that data is in_out pointer which contains n ints

The caller packet will have the following form:
{
	101, // foo opcode
	xx, sizeof(opcode) + sizeof(datalen) + sizeof(int) + sizeof(unsigned int) + n * sizeof(int);
	n, // the n parameter
	n * sizeof(int), // size of the data in ptr
	 // n* sizeof(int) bytes
}

The return packet is;
{
	. // n * sizeof(int) bytes of data return in ptr
	retval // sizeof(int) - the return value of the function;
}

Endianess
---------
The Wire protocol is designed to impose minimum overhead on the client
side. Thus, the data endianness that is sent across the wire is
determined by the client side. It is up to the server side to
determine the client endianess and marshal the packets as required.


Emugen input files - protocol specification
-------------------------------------------
The protocol generated by emugen consists of two input files:

1. basename.in - A sepcification of the protocol RPC procedures. This
part of the specification is expected to be generated automatically
from c/c++ header files or similar.

basename is the basename for the protocol and will be used to prefix
the files that are generated for this protocol.  A line in the .in
file has the following format:

[prefix](retvalType, FuncName, <param type> [param name],...)
where
	retvalType - The function return value type
	FuncName - function name
	<param type> mandatory parameter type
	[param name] - optional parameter name
Examples:
GL_ENTRY(void, glVertex1f, float v)
XXX(int *, foo, int n, float, short)
XXX(void, glFlush, void)

Note: Empty lines in the file are ignored. A line starts with # is a comment

2. basename.attrib - Attributes information of the API.
This file includes additional flags, pointers datalen information and
global attributes of the protocol. For uptodate format of the file,
please refer to the specification file in the project source
tree. The format of the .attrib file is described below.

3. basename.types - Types information

This files describes the types that are described by the API. A type
is defined as follows:
<type name> <size in bits> <print format string> <is a pointer? true|false>
where:
<type name> is the name of the type as described in the API
<size in bits> 0, 8, 16, 32 sizes are accepted
<print format string> a string to format the value of the type, as
acceted by printf(3)
<is pointer?> true or false string species whether the type should be
treated as a pointer.

example:
GLint 32 %d false
GLint* 32 %p true
GLptr  32 %p true

Encoder generated code files
----------------------------
In order to generate the encoder files, one should run the emugen
tool as follows:

emugen -i <input directory> -E <encoder files output directory> <basename>
where:
	<input directory> containes the api specification files  (basename.in + basename.attrib)
	<encoder directory> - a directory name to generate the encoder output files
	basename - The basename for the api.

Assuming the basename is api, The following files are generated:

api_opcodes.h - defines the protocol opcodes. The first opcode value
is 0, unless defined otherwise in the .attrib file

api_entry.cpp - defines entry points for the functions that are
defined by the protocol. this File also includes a function call
setContextAccessor(void *(*f)()). This function should be used to
provide a callback function that is used by the functions to access
the encoder context. For example, such callback could fetch the
context from a Thread Local Storage (TLS) location.

api_client_proc.h - type defintions for the protocol procedures.

api_client_context.h - defines the client side dispatch table data
structure that stores the encoding functions. This data structure also
includes accessors methods such that library user can override
default entries for special case handling.

api_client_context.cpp - defines an initialization function for
dispatch table

api_enc.h - This header file defines the encoder data strcuture. The
encoder data structure inherits its functionality from the
client_context class above and adds encoding and streaming
functionality.

api_enc.cpp - Encoder implementation.

Decoder generated files
-----------------------
In order to generate the decoder files, one should run the emugen
tool as follows:
emugen -i <input directory> -D <decoder files output directory> basename
where:
	<input directory> containes the api specification files  (basename.in + basename.attrib)
	<decoder directory> - a directory name to generate the decoder output files
	basename - The basename for the api.

With resepct to the example above, Emugen will generate the following
files:

api_opcodes.h - Protocol opcodes

api_server_proc.h - type definitions for the server side procedures

api_server_context.h - dispatch table the decoder functions

api_server_context.cpp - dispatch table initialization function
api_dec.h - Decoder header file

api_dec.cpp - Decoder implementation. In addtion, this file includes
an intiailization function that uses a user provided callback to
initialize the API server implementation. An example for such
initialization is loading a set of functions from a shared library
module.

Wrapper generated files
-----------------------
In order to generate a wrapper library files, one should run the
'emugen' tool as follows:

emugen -i <input directory> -W <wrapper files output directory> basename
where:
	<input directory> containes the api specification files  (basename.in + basename.attrib)
	<wrapper directory> - a directory name to generate the wrapper output files
	basename - The basename for the api.

With resepct to the example above, Emugen will generate the following
files:

api_wrapper_proc.h - type definitions for the wrapper procedures

api_wrapper_context.h - dispatch table the wrapper functions

api_wrapper_context.cpp - dispatch table initialization function
api_wrapper_entry.cpp - entry points for the API


.attrib file format description:
-------------------------------
The .attrib file is an input file to emugen and is used to provide
 additional information that is required for the code generation.
The file format is as follows:

a line that starts with # is ignored (comment)
a empty line just whitespace of (" " "\t" "\n") is ignored.

The file is divided into 'sections', each describes a specific API
function call. A section starts with the name of the function in
column 0.

A section that starts with the reserved word 'GLOBAL' provides global
attributes.

below are few sections examples:

GLOBAL
  encoder_headers string.h kuku.h

glVertex3fv
	len data (size)
glTexImage2D
       len pixels (pixels == NULL? 0 : (format_pixel_size(internalformat) * width * height * type_size(type)))


Global section flags description:

base_opcode
    set the base opcode value for this api
    format: base_opcode 100

encoder_headers
    a list of headers that will be included in the encoder header file
    format: encoder_headers <stdio.h> "kuku.h"

client_context_headers
    a list of headers that will be included in the client context header file
    format: client_context_headers <stdio.h> "kuku.h"

decoder_headers
    a list of headers that will be included in the decoder header file
    format: decoder_headers <stdio.h> "kuku.h"

server_context_headers
    a list of headers that will be included in the server context header file
    format: server_context_headers <stdio.h> "kuku.h"


Entry point flags description:

 len
	desciption : provide an expression to calcualte an expression data len
	format: len <var name> <c expression that calcluates the data len>

custom_pack
	description: provide an expression to pack data into the stream.
	format: custom_pack <var name> <c++ expression that pack data from var into the stream>
	The stream is represented by a (unsigned char *)ptr. The expression may also refer
	to other function parameters. In addition, the expression may refer to 'void *self' which
	is the encoding context as provided by the caller.

 dir
	description : set a pointer direction (in - for data that goes
	to the codec, out from data that returns from the codec.
     	format: dir <varname> <[in | out | inout]>

 var_flag
 	 description : set variable flags
 	 format: var_flag <varname> < nullAllowed | isLarge | ... >

        nullAllowed -> for pointer variables, indicates that NULL is a valid value
        isLarge     -> for pointer variables, indicates that the data should be sent without an intermediate copy

 flag
	description: set entry point flag;
	format: flag < unsupported | ... >
	supported flags are:
	unsupported - The encoder side implementation is pointed to "unsuppored reporting function".
	custom_decoder - The decoder is expected to be provided with
		       	 custom implementation. The call to the
		       	 deocder function includes a pointer to the
		       	 context
    not_api - the function is not native gl api