1 Updating Platform Toolchains 2 ============================ 3 4 For the latest version of this doc, please make sure to visit: 5 [Android Clang/LLVM Toolchain Prebuilts Doc](https://android.googlesource.com/platform/external/clang/+/dev/ToolchainPrebuilts.md) 6 7 Picking a good upstream revision 8 -------------------------------- 9 10 We generally like to follow along with upstream LLVM's Google stable tag: 11 12 http://llvm.org/svn/llvm-project/llvm/tags/google/stable/ 13 14 You can use git/svn to query the latest tagged version to see what we will be 15 validating next. 16 17 18 Updating Toolchain Source 19 ------------------------- 20 21 The following process is done in the Android LLVM tree. To fetch the sources: 22 23 repo init -u https://android.googlesource.com/platform/manifest -b llvm 24 25 # Googlers, use 26 repo init -u \ 27 persistent-https://android.git.corp.google.com/platform/manifest -b llvm 28 29 Loop over llvm, clang, compiler-rt (in this order): 30 31 1. We are working from a separate untracked/merged branch called *aosp/dev*. 32 33 git branch -D working_dev 34 repo start working_dev . 35 36 2. **OPTIONAL FIXUPS**. 37 These aren't really necessary if you remember to always keep *aosp/dev* and 38 *aosp/master* synchronized otherwise, but very often someone will forget to 39 merge back a change. 40 41 1. Grab the squashed commit that went into *aosp/master* and mark it 42 committed to *aosp/dev* too. 43 44 **Note**: If there were changes to *aosp/master* before the squashed 45 commit, grab those changes (using step 2), before applying this step, 46 and finally repeat step 2 for changes after the squashed commit. 47 48 git branch -D clean_master 49 git checkout -b clean_master <SHA_FOR_SQUASH> 50 git checkout working_dev 51 git merge -s ours clean_master 52 git push aosp refs/heads/working_dev:refs/heads/dev 53 git branch -D clean_master 54 55 2. Grab all outstanding changes that went into *aosp/master* and put them 56 into *aosp/dev* too. 57 58 git branch -D clean_master 59 git checkout -b clean_master aosp/master 60 git checkout working_dev 61 git merge clean_master 62 git push aosp refs/heads/working_dev:refs/heads/dev 63 git branch -D clean_master 64 65 3. Merge the upstream branch. 66 Use `git log aosp/upsteam-master` to browse upstream commits and find a SHA. 67 68 git merge <upstream_sha> 69 70 4. Fix conflicts. 71 72 5. Update build rules and commit that patch on top. 73 74 6. Test everything before pushing. 75 76 7. Submit your work to *aosp/dev*. 77 78 git push aosp refs/heads/working_dev:refs/heads/dev 79 80 8. Squash your work for *aosp/master*. 81 82 repo start update_38 . 83 git merge --squash working_dev 84 git commit -a 85 repo upload . 86 87 9. Test everything before submitting the patch from the previous step. 88 89 10. Grab the squashed commit and replay it in *aosp/dev*. 90 91 repo sync . 92 git remote update 93 git branch -D clean_master 94 git checkout -b clean_master aosp/master 95 git checkout working_dev 96 97 Use `-s ours` to ensure that we skip the squashed set of changes. 98 If/when we forget this, we have to do it later. 99 100 git merge -s ours clean_master 101 git push aosp refs/heads/working_dev:refs/heads/dev 102 git branch -D clean_master 103 104 11. Clean up after our working branch. 105 106 git checkout --detach 107 git branch -D working_dev 108 109 This works better because we can keep full history in *aosp/dev*, while 110 maintaining easy reverts/commits through *aosp/master*. 111 112 113 Generating New Prebuilts 114 ------------------------ 115 116 1. Run the toolchain build script. This will perform a two stage build and 117 create a tarball of the final toolchain. 118 119 python external/clang/build.py 120 121 2. The just built toolchain can be tested in an existing AOSP tree by invoking 122 make with: 123 124 make \ 125 LLVM_PREBUILTS_VERSION=clang-dev \ 126 LLVM_PREBUILTS_BASE=/path/to/llvm/out/install 127 128 This will use the just built toolchain rather than the one in **prebuilts/**. 129 If you used something other than the default for `--build-name`, use 130 `clang-$BUILD_NAME` instead of `clang-dev`. 131 132 3. Once the updates have been verified, upload to gerrit, review, submit. The 133 build server will pick up the changes and build them. The LLVM build page is 134 http://go/clang-build. Sorry, Googlers only (for now) :( 135 136 4. To update the platform compiler, download the selected package from the build 137 server and extract them to the appropriate prebuilts directory. The new 138 directory will be named "clang-BUILD\_NUMBER". 139 140 5. Update `LLVM\_PREBUILTS\_VERSION` in `build/core/clang/config.mk` to match 141 the new prebuilt directory. We typically keep around two versions of the 142 toolchain in prebuilts so we can easily switch between them in the build 143 system rather than needing to revert prebuilts. This also allows developers 144 that need new toolchain features to take advantage of them locally while 145 validation for the new compiler is still in progress. 146 147 6. Rebuild/test everything one more time to ensure correctness. 148 149 Make sure you check *goog/master* as well as *aosp/master*. 150 151 There may be necessary fixups here, to handle .ll reading or other projects 152 where new warnings/errors are firing. 153 154 m -j48 checkbuild 155 156 6. Upload the changes produced in **prebuilts/clang/host**. 157 This may entail more than a simple `git commit -a`, so look at `git status` 158 before finally uploading/committing. 159 160 repo start updated_toolchain . 161 git add clang-BUILD_NUMBER 162 git commit 163 repo upload --cbr . 164 165 7. Submit CLs. 166 167 168 Testing Checklist 169 ----------------- 170 171 1. Do a checkbuild. 172 2. Go to **external/llvm** and run `./android_test.sh` (no known failures as of 173 2015-10-08). 174 3. Ensure successful build for all architectures: 32- and 64- bit ARM, x86 and 175 Mips. 176 4. Run ART host tests. 177 This was broken by a rebase once, and worth testing after every rebase. 178 179 croot && cd art && mma -j40 test-art-host 180 181 5. Run ART device tests. 182 183 croot && cd art && mma -j4 test-art-device 184 185 186 Checklist for CLs 187 ----------------- 188 189 The following projects will almost always have CLs as a part of the rebase. 190 Depending on the changes in LLVM, there might be updates to other projects as 191 well. 192 193 * External projects 194 195 * **external/clang** 196 * **external/compiler-rt** 197 * **external/llvm** 198 199 * Prebuilts 200 201 * **prebuilts/clang/host/darwin-x86/** 202 * **prebuilts/clang/host/linux-x86/** 203 * **prebuilts/clang/host/windows-x86/** 204
