1 # Regenerating project files 2 3 Prerequisites: `python`, `pip install mako`, `go` 4 5 ``` 6 # Regenerate the projects files using templates 7 tools/buildgen/generate_projects.sh 8 ``` 9 10 # Quick justification 11 12 We've approached the problem of the build system from a lot of different 13 angles. The main issue was that there isn't a single build system that 14 was going to single handedly cover all of our usage cases. 15 16 So instead we decided to work the following way: 17 18 * A `build.yaml` file at the root is the source of truth for listing all the 19 targets and files needed to build grpc and its tests, as well as a basic system 20 for dependency description. 21 22 * Each project file (Makefile, Visual Studio project files, Bazel's BUILD) is 23 a [YAML](http://yaml.org) file used by the `build.yaml` file to generate the 24 final output file. 25 26 This way we can maintain as many project system as we see fit, without having 27 to manually maintain them when we add or remove new code to the repository. 28 Only the structure of the project file is relevant to the template. The actual 29 list of source code and targets isn't. 30 31 We currently have template files for GNU Make, Visual Studio 2013, 32 [Bazel](http://bazel.io) and [gyp](https://gyp.gsrc.io/) (albeit only for 33 Node.js). In the future, we 34 would like to expand to also generate [cmake](https://cmake.org) 35 project files, XCode project files, and an Android.mk file allowing to compile 36 gRPC using Android's NDK. 37 38 We'll gladly accept contribution that'd create additional project files 39 using that system. 40 41 # Structure of `build.yaml` 42 43 The `build.yaml` file has the following structure: 44 45 ``` 46 settings: # global settings, such as version number 47 ... 48 filegroups: # groups of files that are automatically expanded 49 ... 50 libs: # list of libraries to build 51 ... 52 target: # list of targets to build 53 ... 54 ``` 55 56 The `filegroups` are helpful to re-use a subset of files in multiple targets. 57 One `filegroups` entry has the following structure: 58 59 ``` 60 - name: "arbitrary string", # the name of the filegroup 61 public_headers: # list of public headers defined in that filegroup 62 - ... 63 headers: # list of headers defined in that filegroup 64 - ... 65 src: # list of source files defined in that filegroup 66 - ... 67 ``` 68 69 The `libs` collection contains the list of all the libraries we describe. Some may be 70 helper libraries for the tests. Some may be installable libraries. Some may be 71 helper libraries for installable binaries. 72 73 The `targets` array contains the list of all the binary targets we describe. Some may 74 be installable binaries. 75 76 One `libs` or `targets` entry has the following structure (see below for 77 details): 78 79 ``` 80 name: "arbitrary string", # the name of the library 81 build: "build type", # in which situation we want that library to be 82 # built and potentially installed (see below). 83 language: "...", # the language tag; "c" or "c++" 84 public_headers: # list of public headers to install 85 headers: # list of headers used by that target 86 src: # list of files to compile 87 secure: boolean, # see below 88 baselib: boolean, # this is a low level library that has system 89 # dependencies 90 vs_project_guid: '{...}', # Visual Studio's unique guid for that project 91 filegroups: # list of filegroups to merge to that project 92 # note that this will be expanded automatically 93 deps: # list of libraries this target depends on 94 deps_linkage: "..." # "static" or "dynamic". Used by the Makefile only to 95 # determine the way dependencies are linkned. Defaults 96 # to "dynamic". 97 dll: "..." # see below. 98 dll_def: "..." # Visual Studio's dll definition file. 99 vs_props: # List of property sheets to attach to that project. 100 vs_config_type: "..." # DynamicLibrary/StaticLibrary. Used only when 101 # creating a library. Specifies if we're building a 102 # static library or a dll. Use in conjunction with `dll_def`. 103 vs_packages: # List of nuget packages this project depends on. 104 ``` 105 106 ## The `"build"` tag 107 108 Currently, the "`build`" tag have these meanings: 109 110 * `"all"`: library to build on `"make all"`, and install on the system. 111 * `"protoc"`: a protoc plugin to build on `"make all"` and install on the system. 112 * `"private"`: a library to only build for tests. 113 * `"test"`: a test binary to run on `"make test"`. 114 * `"tool"`: a binary to be built upon `"make tools"`. 115 116 All of the targets should always be present in the generated project file, if 117 possible and applicable. But the build tag is what should group the targets 118 together in a single build command. 119 120 121 ## The `"secure"` tag 122 123 This means this target requires OpenSSL one way or another. The values can be 124 `"yes"`, `"no"` and `"check"`. The default value is `"check"`. It means that 125 the target requires OpenSSL, but that since the target depends on another one 126 that is supposed to also import OpenSSL, the import should then be implicitely 127 transitive. `"check"` should then only disable that target if OpenSSL hasn't 128 been found or is unavailable. 129 130 ## The `"baselib"` boolean 131 132 This means this is a library that will provide most of the features for gRPC. 133 In particular, if we're locally building OpenSSL, protobuf or zlib, then we 134 should merge OpenSSL, protobuf or zlib inside that library. That effect depends 135 on the `"language"` tag. OpenSSL and zlib are for `"c"` libraries, while 136 protobuf is for `"c++"` ones. 137 138 ## The `"dll"` tag 139 140 Used only by Visual Studio's project files. "true" means the project will be 141 built with both static and dynamic runtimes. "false" means it'll only be built 142 with static runtime. "only" means it'll only be built with the dll runtime. 143 144 ## The `"dll_def"` tag 145 146 Specifies the visual studio's dll definition file. When creating a DLL, you 147 sometimes (not always) need a def file (see grpc.def). 148 149 150 # The template system 151 152 We're currently using the [mako templates](http://www.makotemplates.org/) 153 renderer. That choice enables us to simply render text files without dragging 154 with us a lot of other features. Feel free to explore the current templates 155 in that directory. The simplest one is probably [BUILD.template](BUILD.template) 156 which is used to create the [Bazel](http://bazel.io/) project file. 157 158 ## The renderer engine 159 160 As mentioned, the renderer is using [mako templates](http://www.makotemplates.org/), 161 but some glue is needed to process all of that. See the [buildgen folder](../tools/buildgen) 162 for more details. We're mainly loading the build.json file, and massaging it, 163 in order to get the list of properties we need, into a Python dictionary, that 164 is then passed to the template while rending it. 165 166 ## The plugins 167 168 The file build.json itself isn't passed straight to the template files. It is 169 first processed and modified by a few plugins. For example, the `filegroups` 170 expander is [a plugin](../tools/buildgen/plugins/expand_filegroups.py). 171 172 The structure of a plugin is simple. The plugin must defined the function 173 `mako_plugin` that takes a Python dictionary. That dictionary represents the 174 current state of the build.json contents. The plugin can alter it to whatever 175 feature it needs to add. 176
