1 // Copyright 2014 Google Inc. All rights reserved. 2 // 3 // Licensed under the Apache License, Version 2.0 (the "License"); 4 // you may not use this file except in compliance with the License. 5 // You may obtain a copy of the License at 6 // 7 // http://www.apache.org/licenses/LICENSE-2.0 8 // 9 // Unless required by applicable law or agreed to in writing, software 10 // distributed under the License is distributed on an "AS IS" BASIS, 11 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 // See the License for the specific language governing permissions and 13 // limitations under the License. 14 15 // The Blueprint bootstrapping mechanism is intended to enable building a source 16 // tree using a Blueprint-based build system that is embedded (as source) in 17 // that source tree. The only prerequisites for performing such a build are: 18 // 19 // 1. A Ninja binary 20 // 2. A script interpreter (e.g. Bash or Python) 21 // 3. A Go toolchain 22 // 23 // The Primary Builder 24 // 25 // As part of the bootstrapping process, a binary called the "primary builder" 26 // is created. This primary builder is the binary that includes both the core 27 // Blueprint library and the build logic specific to the source tree. It is 28 // used to generate the Ninja file that describes how to build the entire source 29 // tree. 30 // 31 // The primary builder must be a pure Go (i.e. no cgo) module built with the 32 // module type 'bootstrap_go_binary'. It should have the 'primaryBuilder' 33 // module property set to true in its Blueprints file. If more than one module 34 // sets primaryBuilder to true the build will fail. 35 // 36 // The primary builder main function should look something like: 37 // 38 // package main 39 // 40 // import ( 41 // "flag" 42 // "github.com/google/blueprint" 43 // "github.com/google/blueprint/bootstrap" 44 // "path/filepath" 45 // 46 // "my/custom/build/logic" 47 // ) 48 // 49 // func main() { 50 // // The primary builder should use the global flag set because the 51 // // bootstrap package registers its own flags there. 52 // flag.Parse() 53 // 54 // // The top-level Blueprints file is passed as the first argument. 55 // srcDir := filepath.Dir(flag.Arg(0)) 56 // 57 // // Create the build context. 58 // ctx := blueprint.NewContext() 59 // 60 // // Register custom module types 61 // ctx.RegisterModuleType("foo", logic.FooModule) 62 // ctx.RegisterModuleType("bar", logic.BarModule) 63 // 64 // // Register custom singletons 65 // ctx.RegisterSingleton("baz", logic.NewBazSingleton()) 66 // 67 // // Create and initialize the custom Config object. 68 // config := logic.NewConfig(srcDir) 69 // 70 // // This call never returns 71 // bootstrap.Main(ctx, config) 72 // } 73 // 74 // Required Source Files 75 // 76 // There are three files that must be included in the source tree to facilitate 77 // the build bootstrapping: 78 // 79 // 1. The top-level Blueprints file 80 // 2. The bootstrap Ninja file template 81 // 3. The bootstrap script 82 // 83 // The top-level Blueprints file describes how the entire source tree should be 84 // built. It must have a 'subdirs' assignment that includes both the core 85 // Blueprint library and the custom build logic for the source tree. It should 86 // also include (either directly or through a subdirs entry) describe all the 87 // modules to be built in the source tree. 88 // 89 // The bootstrap Ninja file template describes the build actions necessary to 90 // build the primary builder for the source tree. This template contains a set 91 // of placeholder Ninja variable values that get filled in by the bootstrap 92 // script to create a usable Ninja file. It can be created by running the 93 // minibp binary that gets created as part of the standalone Blueprint build. 94 // Passing minibp the path to the top-level Blueprints file will cause it to 95 // create a bootstrap Ninja file template named 'build.ninja.in'. 96 // 97 // The bootstrap script is a small script (or theoretically a compiled binary) 98 // that is included in the source tree to begin the bootstrapping process. It 99 // is responsible for filling in the bootstrap Ninja file template with some 100 // basic information about the Go build environemnt and the path to the root 101 // source directory. It does this by performing a simple string substitution on 102 // the template file to produce a usable build.ninja file. 103 // 104 // The Bootstrapping Process 105 // 106 // There are three stages to the bootstrapping process, each with a 107 // corresponding Ninja file. The stages are referred to as the "bootstrap", 108 // "primary", and "main" stages. Each stage builds the next stage's Ninja file. 109 // 110 // The bootstrapping process begins with the user running the bootstrap script 111 // to initialize a new build directory. The script is run from the build 112 // directory, and when run with no arguments it copies the source bootstrap 113 // Ninja file into the build directory as ".minibootstrap/build.ninja". It 114 // also performs a set of string substitutions on the file to configure it for 115 // the user's build environment. Specifically, the following strings are 116 // substituted in the file: 117 // 118 // @@SrcDir@@ - The path to the root source directory (either 119 // absolute or relative to the build dir) 120 // @@BuildDir@@ - The path to the build directory 121 // @@GoRoot@@ - The path to the root directory of the Go toolchain 122 // @@GoCompile@@ - The path to the Go compiler (6g or compile) 123 // @@GoLink@@ - The path to the Go linker (6l or link) 124 // @@Bootstrap@@ - The path to the bootstrap script 125 // @@BootstrapManifest@@ - The path to the source bootstrap Ninja file 126 // 127 // Once the script completes the build directory is initialized and ready to run 128 // a build. A wrapper script (blueprint.bash by default) has been installed in 129 // order to run a build. It iterates through the three stages of the build: 130 // 131 // - Checks to see if the source bootstrap Ninja file is newer than the 132 // one that is in the build directory, if so, update the build dir copy. 133 // - Run the Bootstrap stage 134 // - Run the Primary stage 135 // - Run the Main stage 136 // 137 // Previously, we were keeping track of the "state" of the build directory and 138 // only going back to previous stages when something had changed. But that 139 // added complexity, and failed when there was a build error in the Primary 140 // stage that would have been fixed if the Bootstrap stage was re-run (we 141 // would only evaluate which stages needed to be run at the end of the stage). 142 // So now we always run through each stage, and the first two stages will do 143 // nothing when nothing has changed. 144 // 145 // During the Bootstrap stage, <builddir>/.minibootstrap/build.ninja, the 146 // following actions are taken, if necessary: 147 // 148 // - Build all bootstrap_core_go_binary rules, and dependencies -- 149 // minibp and some test helpers. 150 // - Run minibp to generate .bootstrap/build.ninja (Primary stage) 151 // - Run minibp to generate .minibootstrap/build.ninja.in 152 // - Restart if .minibootstrap/build.ninja.in has changed 153 // 154 // During the Primary stage, <builddir>/.bootstrap/build.ninja, the following 155 // actions are taken, if necessary: 156 // 157 // - Build any bootstrap_go_binary rules and dependencies -- usually the 158 // primary builder and any build or runtime dependencies. 159 // - Run the primary builder to generate build.ninja 160 // - Run the primary builder to extract documentation 161 // 162 // Then the main stage is at <builddir>/build.ninja, and will contain all the 163 // rules generated by the primary builder. In addition, the bootstrap code 164 // adds a phony rule "blueprint_tools" that depends on all blueprint_go_binary 165 // rules (bpfmt, bpmodify, etc). 166 // 167 // Updating the Bootstrap Ninja File Template 168 // 169 // The main purpose of the bootstrap stage is to generate the Ninja file for the 170 // primary stage. The one additional thing it does is generate a new bootstrap 171 // Ninja file template at .minibootstrap/build.ninja.in. When generating this 172 // file, minibp will compare the new bootstrap Ninja file contents with the 173 // original (in the source tree). 174 // 175 // This scheme ensures that updates to the source tree are always incorporated 176 // into the build process. 177 // 178 package bootstrap 179
