1 Name: Compact Language Detection 2 2 Short Name: cld_2 3 URL: https://code.google.com/p/cld2/ 4 Version: 0 5 License: Apache 2.0 6 Security Critical: yes 7 8 Description: 9 The CLD is used to determine the language of text. In Chromium, this is used 10 to determine if Chrome should offer Translate UX to the user. 11 12 13 Dynamic Mode 14 ============ 15 Prior to CLD2's trunk@155, Chromium has always built CLD2 statically. The data 16 needed for CLD2 to perform its language detection has been compiled straight 17 into the binary. This contributes around 1.5 megabytes to the size of Chrome 18 and embeds one or more large rodata sections to the executable. 19 20 Starting with CLD2's trunk@r155, there is a new option available: dynamic mode. 21 In dynamic mode, CLD2 is built without its data; only the code is compiled, and 22 the data must be supplied at runtime via a file or a pointer to a (presumably 23 mmap'ed) read-only region of memory. 24 25 Tradeoffs to consider before enabling dynamic mode: 26 27 Pros: 28 * Reduces the size of the Chromium binary by a bit over a megabyte. 29 * As the data file rarely changes, it can be updated independently. 30 * Depending upon the update process on your platform, this may also reduce 31 the size of Chromium updates. 32 * It is possible to run Chromium without CLD2 data at all (language 33 detection will always fail, but fails gracefully). 34 * Different types of CLD2 data files (larger and more accurate or smaller 35 and less accurate) can be dynamically downloaded or chosen depending 36 on runtime choices. 37 38 Cons: 39 * Data files must be generated and checked into source control by hand. 40 * At runtime a data file must be opened and its headers parsed before CLD2 41 can be used in any given process (this time should be negligible in most 42 circumstances). This will prevent language detection from working until 43 a data file has been loaded. 44 45 To enable dynamic mode in CLD2 itself, you must define "CLD2_DYNAMIC_MODE". 46 In Chromium, this is controlled by the 'cld2_data_source' variable in 47 ../../build/common.gypi. 48 49 50 Building a CLD2 Dynamic Mode Data File 51 ====================================== 52 Note: The cld_2_dynamic_data_tool target is not currently supported on Android. 53 The binaries that it generates are platform-independent, but to build 54 the target itself you'll need a desktop environment. 55 56 1. Configure your desired table size by setting the value of "cld2_table_size" 57 in ../../build/common.gypi. 58 2. Build the "cld_2_dynamic_data_tool" target. This will generate the tool: 59 ${BUILD_DIR}/cld_2_dynamic_data_tool 60 3. Run the tool with "--dump <file>" to generate a data file, e.g.: 61 ${BUILD_DIR}/cld_2_dynamic_data_tool --dump /tmp/cld2_data.bin 62 4. (Optional) Verify that the file was correctly written: 63 ${BUILD_DIR}/cld_2_dynamic_data_tool --verify /tmp/cld2_data.bin 64 65 The data file is suitable for use on all platforms.
