Home | History | Annotate | Download | only in emugen
      1 Introduction:
      2 -------------
      3 The emugen tool is a tool to generate a wire protocol implementation
      4 based on provided API.  The tool generates c++ encoder code that takes
      5 API calls and encodes them into the wire and decoder code that decodes
      6 the wire stream and calls server matching API.
      7 The emugen tool includes additional functionality that enables to
      8 generate an wrapper library. The wrapper library provides entry points
      9 for the specified API, where each entry routes the call via a dispatch
     10 table. The dispatch table may be initialized as required by a specific application.
     11 
     12 The following paragraphs includes the following:
     13     * Wire Protocol Description
     14     * Input files description & format
     15     * Generated code description.
     16 
     17 
     18 Note: In this document, the caller is referred to as Encoder or Client
     19 and the callee is referred to as the Decoder or Server. These terms
     20 are used interchangeably by the context.
     21 
     22 
     23 
     24 Wire Protocol packet structure:
     25 -------------------------------
     26 A general Encoder->Decoder packet is structured as following:
     27 struct Packet {
     28 	unsigned int opcode;
     29 	unsigned int packet_len;
     30 	 parameter 1
     31 	 parameter 2
     32 };
     33 A general Decoder->Encoder reply is expected to be received in the
     34 context of the call that triggered it, thus it includes the reply
     35 data only, with no context headers. In precise term terms, a reply
     36 packet will look like:
     37 
     38 struct {
     39 	...// reply data
     40 };
     41 
     42 consider the following function call:
     43 int foo(int p1, short s1)
     44 will be encoded into :
     45 {
     46 	101, // foo opcode
     47 	14, // sizeof(opcode) + sizeof(packet_len) + sizeof(int) + sizeof(short)
     48 	p1, // 4 bytes
     49 	s1 // 2 bytes
     50 }
     51 
     52 Since foo returns value, the caller is expected to read back the return packet from the server->client stream. The return value in this example is in thus the return packet is:
     53 {
     54 	int retval;
     55 }
     56 
     57 
     58 Pointer decoding:
     59 ----------------
     60 The wire protocol also allows exchanging of pointer data
     61 (arrays). Pointers are defined with directions:
     62 
     63 	in : Data is sent from the caller to the calle
     64 	out: Data is sent from the callee to the caller
     65 	in_out: data is sent from the caller and return in place.
     66 
     67 in and in_out encoded with their len:
     68 {
     69 	unsinged int pointer_data_len;
     70 	unsigned char data[pointer_data_len]; // pointer data
     71 }
     72 
     73 out pointers are encoded by data length only:
     74 {
     75 unsigned int pointer_data_len;
     76 }
     77 
     78 out and in_out pointers data is returned in the return
     79 packet. For example, consider the following call:
     80 
     81 int foo(int n, int *ptr); // assume that data is in_out pointer which contains n ints
     82 
     83 The caller packet will have the following form:
     84 {
     85 	101, // foo opcode
     86 	xx, sizeof(opcode) + sizeof(datalen) + sizeof(int) + sizeof(unsigned int) + n * sizeof(int);
     87 	n, // the n parameter
     88 	n * sizeof(int), // size of the data in ptr
     89 	 // n* sizeof(int) bytes
     90 }
     91 
     92 The return packet is;
     93 {
     94 	. // n * sizeof(int) bytes of data return in ptr
     95 	retval // sizeof(int) - the return value of the function;
     96 }
     97 
     98 Endianess
     99 ---------
    100 The Wire protocol is designed to impose minimum overhead on the client
    101 side. Thus, the data endianness that is sent across the wire is
    102 determined by the client side. It is up to the server side to
    103 determine the client endianess and marshal the packets as required.
    104 
    105 
    106 
    107 Emugen input files - protocol specification
    108 -------------------------------------------
    109 The protocol generated by emugen consists of two input files:
    110 
    111 1. basename.in - A sepcification of the protocol RPC procedures. This
    112 part of the specification is expected to be generated automatically
    113 from c/c++ header files or similar.
    114 
    115 basename is the basename for the protocol and will be used to prefix
    116 the files that are generated for this protocol.  A line in the .in
    117 file has the following format:
    118 
    119 [prefix](retvalType, FuncName, <param type> [param name],...)
    120 where 
    121 	retvalType - The function return value type
    122 	FuncName - function name
    123 	<param type> mandatory parameter type
    124 	[param name] - optional parameter name
    125 Examples:
    126 GL_ENTRY(void, glVertex1f, float v) 
    127 XXX(int *, foo, int n, float, short)
    128 XXX(void, glFlush, void)
    129 
    130 Note: Empty lines in the file are ignored. A line starts with # is a comment
    131 
    132 2. basename.attrib - Attributes information of the API. 
    133 This file includes additional flags, pointers datalen information and
    134 global attributes of the protocol. For uptodate format of the file,
    135 please refer to the specification file in the project source
    136 tree. The format of the .attrib file is described below.
    137 
    138 3. basename.types - Types information
    139 
    140 This files describes the types that are described by the API. A type
    141 is defined as follows:
    142 <type name> <size in bits> <print format string> <is a pointer? true|false>
    143 where:
    144 <type name> is the name of the type as described in the API
    145 <size in bits> 0, 8, 16, 32 sizes are accepted
    146 <print format string> a string to format the value of the type, as
    147 acceted by printf(3)
    148 <is pointer?> true or false string species whether the type should be
    149 treated as a pointer.
    150 
    151 example:
    152 GLint 32 %d false
    153 GLint* 32 %p true
    154 GLptr  32 %p true
    155 
    156 Encoder generated code files 
    157 ----------------------------
    158 In order to generate the encoder files, one should run the emugen
    159 tool as follows:
    160 
    161 emugen -i <input directory> -E <encoder files output directory> <basename>
    162 where:
    163 	<input directory> containes the api specification files  (basename.in + basename.attrib)
    164 	<encoder directory> - a directory name to generate the encoder output files
    165 	basename - The basename for the api.
    166 
    167 Assuming the basename is api, The following files are generated:
    168 
    169 api_opcodes.h - defines the protocol opcodes. The first opcode value
    170 is 0, unless defined otherwise in the .attrib file
    171 
    172 api_entry.cpp - defines entry points for the functions that are
    173 defined by the protocol. this File also includes a function call
    174 setContextAccessor(void *(*f)()). This function should be used to
    175 provide a callback function that is used by the functions to access
    176 the encoder context. For example, such callback could fetch the
    177 context from a Thread Local Storage (TLS) location.
    178 
    179 api_client_proc.h - type defintions for the protocol procedures. 
    180 
    181 api_client_context.h - defines the client side dispatch table data
    182 structure that stores the encoding functions. This data structure also
    183 includes accessors methods such that library user can override
    184 default entries for special case handling.
    185 
    186 api_client_context.cpp - defines an initialization function for
    187 dispatch table
    188 
    189 api_enc.h - This header file defines the encoder data strcuture. The
    190 encoder data structure inherits its functionality from the
    191 client_context class above and adds encoding and streaming
    192 functionality.
    193 
    194 api_enc.cpp - Encoder implementation. 
    195 
    196 Decoder generated files
    197 -----------------------
    198 In order to generate the decoder files, one should run the emugen
    199 tool as follows:
    200 emugen -i <input directory> -D <decoder files output directory> basename
    201 where:
    202 	<input directory> containes the api specification files  (basename.in + basename.attrib)
    203 	<decoder directory> - a directory name to generate the decoder output files
    204 	basename - The basename for the api.
    205 
    206 With resepct to the example above, Emugen will generate the following
    207 files:
    208 
    209 api_opcodes.h - Protocol opcodes
    210 
    211 api_server_proc.h - type definitions for the server side procedures
    212 
    213 api_server_context.h - dispatch table the decoder functions
    214 
    215 api_server_context.cpp - dispatch table initialization function
    216 api_dec.h - Decoder header file
    217 
    218 api_dec.cpp - Decoder implementation. In addtion, this file includes
    219 an intiailization function that uses a user provided callback to
    220 initialize the API server implementation. An example for such
    221 initialization is loading a set of functions from a shared library
    222 module.
    223 
    224 Wrapper generated files
    225 -----------------------
    226 In order to generate a wrapper library files, one should run the
    227 'emugen' tool as follows:
    228 
    229 emugen -i <input directory> -W <wrapper files output directory> basename
    230 where:
    231 	<input directory> containes the api specification files  (basename.in + basename.attrib)
    232 	<wrapper directory> - a directory name to generate the wrapper output files
    233 	basename - The basename for the api.
    234 
    235 With resepct to the example above, Emugen will generate the following
    236 files:
    237 
    238 api_wrapper_proc.h - type definitions for the wrapper procedures
    239 
    240 api_wrapper_context.h - dispatch table the wrapper functions
    241 
    242 api_wrapper_context.cpp - dispatch table initialization function
    243 api_wrapper_entry.cpp - entry points for the API
    244 
    245 
    246 .attrib file format description:
    247 -------------------------------
    248 The .attrib file is an input file to emugen and is used to provide
    249  additional information that is required for the code generation.
    250 The file format is as follows:
    251 
    252 a line that starts with # is ignored (comment)
    253 a empty line just whitespace of (" " "\t" "\n") is ignored.
    254 
    255 The file is divided into 'sections', each describes a specific API
    256 function call. A section starts with the name of the function in
    257 column 0.
    258 
    259 A section that starts with the reserved word 'GLOBAL' provides global
    260 attributes.
    261 
    262 below are few sections examples:
    263 
    264 GLOBAL
    265   encoder_headers string.h kuku.h
    266 
    267 glVertex3fv
    268 	len data (size)
    269 glTexImage2D
    270        len pixels (pixels == NULL? 0 : (format_pixel_size(internalformat) * width * height * type_size(type)))
    271 
    272 
    273 Global section flags description:
    274 
    275 base_opcode
    276     set the base opcode value for this api
    277     format: base_opcode 100
    278 
    279 encoder_headers
    280     a list of headers that will be included in the encoder header file
    281     format: encoder_headers <stdio.h> "kuku.h"
    282 
    283 client_context_headers
    284     a list of headers that will be included in the client context header file
    285     format: client_context_headers <stdio.h> "kuku.h"
    286 
    287 decoder_headers
    288     a list of headers that will be included in the decoder header file
    289     format: decoder_headers <stdio.h> "kuku.h"
    290 
    291 server_context_headers
    292     a list of headers that will be included in the server context header file
    293     format: server_context_headers <stdio.h> "kuku.h"
    294 
    295 
    296 Entry point flags description:
    297 
    298  len
    299 	desciption : provide an expression to calcualte an expression data len
    300 	format: len <var name> <c expression that calcluates the data len>
    301 
    302 custom_pack
    303 	description: provide an expression to pack data into the stream.
    304 	format: custom_pack <var name> <c++ expression that pack data from var into the stream>
    305 	The stream is represented by a (unsigned char *)ptr. The expression may also refer
    306 	to other function parameters. In addition, the expression may refer to 'void *self' which
    307 	is the encoding context as provided by the caller.
    308 
    309  dir
    310 	description : set a pointer direction (in - for data that goes
    311 	to the codec, out from data that returns from the codec.
    312      	format: dir <varname> <[in | out | inout]>
    313 
    314  var_flag 
    315  	 description : set variable flags
    316  	 format: var_flag <varname> < nullAllowed | isLarge | ... >
    317 
    318         nullAllowed -> for pointer variables, indicates that NULL is a valid value
    319         isLarge     -> for pointer variables, indicates that the data should be sent without an intermediate copy
    320 
    321  flag
    322 	description: set entry point flag; 
    323 	format: flag < unsupported | ... >
    324 	supported flags are:
    325 	unsupported - The encoder side implementation is pointed to "unsuppored reporting function". 
    326 	custom_decoder - The decoder is expected to be provided with
    327 		       	 custom implementation. The call to the
    328 		       	 deocder function includes a pointer to the
    329 		       	 context
    330     not_api - the function is not native gl api
    331 
    332 
    333