1 <html devsite> 2 <head> 3 <title>Reducing OTA Size</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 <p>This page describes build changes added to AOSP to reduce unnecessary file changes between 26 builds. Device implementers who maintain their own build system can use this information as a 27 guide for reducing over-the-air (OTA) update size.</p> 28 29 <p>Android OTAs occasionally contain changed files that do not correspond to code changes but are 30 instead artifacts of the build system. This can occurs when the same code built at different 31 times, from different directories, or on different machines, produces a large number of changed 32 files. These excess files not only increase the size of an OTA, but make it difficult to 33 determine which code is changed in the OTA.</p> 34 35 <p>To make the contents of an OTA more transparent, AOSP includes build system changes designed to 36 reduce OTA size by eliminating unnecessary file changes between builds. The aim is to reduce 37 the size of OTAs to include only the files that relate to the patches contained in the OTA. AOSP 38 also includes a <a href="#the_build_diff_tool">build diff tool</a> that filters out common 39 build-related file changes and provides a cleaner build file diff.</p> 40 41 <p>The build system can create unnecessary file diffs in several ways. The following sections 42 discuss some of these issues and solutions, providing examples of fixes in AOSP when possible).</p> 43 44 <h2 id=file_order>File order</h2> 45 46 <p><strong>Problem</strong>: Filesystems dont guarantee a file order when asked for a list of files 47 in a directory, though its commonly the same for the same checkout. Tools such as <code>ls</code> 48 sort the results by default, but the wildcard function used by commands such as <code>find</code> 49 and <code>make</code> do not. Before using these tools, you must sort the outputs.</p> 50 51 <p><strong>Solution</strong>: Users of tools such as <code>find</code> and <code>make</code> with 52 wildcard must sort the output of these commands before using them. Use of 53 <code>$(wildcard )</code> or <code>$(shell find )</code> in Android.mk files should also be 54 sorted. Some tools, such as Java, do sort inputs so verify sorting is necessary.</p> 55 56 <p><strong>Examples:</strong> Many instances were fixed in the core build system using the builtin 57 <code>all-*-files-under</code> macro, which includes <code>all-cpp-files-under</code> (as several 58 definitions were spread out in other makefiles). For details, refer to the following CLs:</p> 59 60 <ul> 61 <li><a href="https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f">https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f</a> 62 <li><a href="https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410">https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410</a> 63 <li><a href="https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653">https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653</a> 64 <li><a href="https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c">https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c</a> 65 </ul> 66 67 <h2 id=build_directory>Build directory</h2> 68 69 <p><strong>Problem:</strong> Changing the directory in which things are built can cause the binaries 70 to be different. Most paths in the Android build are relative paths so <code>__FILE__</code> in 71 C/C++ isnt a problem. However, the debug symbols encode the full pathname by default, and the 72 <code>.note.gnu.build-id</code> is generated from hashing the pre-stripped binary, so it will 73 change if the debug symbols change.</p> 74 75 <p><strong>Solution:</strong> AOSP now makes debug paths relative. For details, refer to CL: 76 <a href="https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02">https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02</a>.</p> 77 78 <h2 id=timestamps>Timestamps</h2> 79 80 <p><strong>Problem:</strong> Timestamps in the build output result in unnecessary file changes. This 81 is likely to happen in the following locations:</p> 82 83 <ul> 84 <li><code> __DATE__/__TIME__/__TIMESTAMP__ </code> macros in C or C++ code.</li> 85 <li>Timestamps embedded in zip-based archives.</li> 86 </ul> 87 88 <p><strong>Solutions/Examples:</strong> To remove timestamps from the build output, use the 89 instructions in the sections below.</p> 90 91 <h3 id=date_time_timestamp_in_c_c>__DATE__/__TIME__/__TIMESTAMP__ in C/C++</h3> 92 93 <p>These macros always produce different outputs for different builds, so they shouldnt be used. 94 Here are a few options on how to eliminate these macros:</p> 95 96 <ul> 97 <li>Just remove them, they often arent necessary. For an example, refer to 98 <a href="https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f">https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f</a></li> 99 <li>To uniquely identify the running binary, read the build-id from the ELF header.</li> 100 <li>To know when the OS was built, read the <code>ro.build.date</code> (should work for everything 101 except incremental builds, which may not update this date). For an example, refer to 102 <a href="https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84">https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84</a></li> 103 </ul> 104 105 <p class="note"><strong>Note:</strong>We turned on <code>-Werror=date-time</code> so using 106 timestamps is a build error.</p> 107 108 <h3 id=embedded_timestamps_in_zip-based_archives_zip_jar>Embedded timestamps in archives (zip, jar)</h3> 109 110 <p>We fixed the problem of embedded timestamps in zip archives by adding <code>-X</code> to all uses 111 of the <code>zip</code> command, so the UID/GID of the builder and the extended Unix timestamp are 112 not embedded in the zip file.</p> 113 114 <p>A new tool, <code>ziptime</code> (located in <code> 115 <a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/">/platform/build/+/master/tools/ziptime/</a></code>) 116 resets the normal timestamps in the zip headers. For details, refer to the 117 <a href="https://android.googlesource.com/platform/build/+/master/tools/ziptime/README.txt">README 118 file</a>.</p> 119 120 <p>The <code>signapk</code> tool sets timestamps for the APK files that may vary depending on the 121 server timezone. For details, refer to the CL 122 <a href="https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028">https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028</a>.</p> 123 124 <h2 id=version_strings>Version strings</h2> 125 126 <p><strong>Problem:</strong> APK version strings often had the BUILD_NUMBER appended to the 127 hardcoded version. Even if nothing else changed in the APK, the APK would still be different.</p> 128 129 <p><strong>Solution:</strong> Remove the build number from the APK version string.</p> 130 131 <p><strong>Examples:</strong></p> 132 133 <ul> 134 <li><a href="https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27">https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27</a></li> 135 <li><a href="https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c">https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c</a></li> 136 </ul> 137 138 <h2 id=consistent_build_tools>Consistent build tools</h2> 139 140 <p><strong>Problem:</strong> Tools that generate installed files must be consistent (the same input 141 should always produce the same output).</p> 142 143 <p><strong>Solutions/Examples:</strong> Changes were required in the following build tools:</p> 144 145 <ul> 146 <li><strong>NOTICE file creator</strong>. The NOTICE file creator needed the changes. Refer to CL: 147 <a href="https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64">https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64</a></li> 148 <li><strong>Java Android Compiler Kit (Jack)</strong>. The Jack toolchain required an update to 149 handle an occasional change in generated constructor ordering. Refer to CL: 150 <a href="https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b">https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b</a></li> 151 <li><strong>ART AOT compiler (dex2oat)</strong>. The ART compiler binary required an update to 152 create a deterministic image. Refer to CL: 153 <a href="https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9">https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9</a></li> 154 <li><strong>The libpac.so file (V8)</strong>Every build creates a different 155 <code>/system/lib/libpac.so</code> file because the V8 snapshot changes for each build. The 156 solution is to remove the snapshot. Refer to CL: 157 <a href="https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29">https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29</a></li> 158 <li><strong>Application pre-dexoptd (.odex) files</strong>. The pre-dexoptd (.odex) files 159 contained uninitialized padding on 64-bit systems. Refer to CL: 160 <a href="https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029">https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029</a></li> 161 </ul> 162 163 <h2 id=the_build_diff_tool>Using the build diff tool</h2> 164 165 <p>For cases where it is not possible to eliminate build-related file changes, we supply a build 166 diff tool, 167 <code><a href="https://android.googlesource.com/platform/build/+/master/tools/releasetools/target_files_diff.py">target_files_diff.py</a></code> 168 for use in comparing two file packages. This tool performs a recursive diff between two builds, 169 excluding common build-related file changes, such as:</p> 170 171 <ul> 172 <li>Expected changes in the build output (for example, due to a build number change).</li> 173 <li>Changes due to known issues in the current build system.</li> 174 </ul> 175 176 <p>To use the build diff tool, run the following command:</p> 177 178 <pre class="devsite-terminal devsite-click-to-copy"> 179 target_files_diff.py dir1 dir2 180 </pre> 181 182 <p><code>dir1</code> and <code>dir2</code> are base directories that contain the extracted target 183 files for each build.</p> 184 185 </body> 186 </html> 187
