README.chromium
1 Name: Compact Language Detection 2
2 Short Name: cld_2
3 URL: https://code.google.com/p/cld2/
4 Version: 0
5 License: Apache 2.0
6 Security Critical: yes
7
8 Description:
9 The CLD is used to determine the language of text. In Chromium, this is used
10 to determine if Chrome should offer Translate UX to the user.
11
12
13 Dynamic Mode
14 ============
15 Prior to CLD2's trunk@155, Chromium has always built CLD2 statically. The data
16 needed for CLD2 to perform its language detection has been compiled straight
17 into the binary. This contributes around 1.5 megabytes to the size of Chrome
18 and embeds one or more large rodata sections to the executable.
19
20 Starting with CLD2's trunk@r155, there is a new option available: dynamic mode.
21 In dynamic mode, CLD2 is built without its data; only the code is compiled, and
22 the data must be supplied at runtime via a file or a pointer to a (presumably
23 mmap'ed) read-only region of memory.
24
25 Tradeoffs to consider before enabling dynamic mode:
26
27 Pros:
28 * Reduces the size of the Chromium binary by a bit over a megabyte.
29 * As the data file rarely changes, it can be updated independently.
30 * Depending upon the update process on your platform, this may also reduce
31 the size of Chromium updates.
32 * It is possible to run Chromium without CLD2 data at all (language
33 detection will always fail, but fails gracefully).
34 * Different types of CLD2 data files (larger and more accurate or smaller
35 and less accurate) can be dynamically downloaded or chosen depending
36 on runtime choices.
37
38 Cons:
39 * Data files must be generated and checked into source control by hand.
40 * At runtime a data file must be opened and its headers parsed before CLD2
41 can be used in any given process (this time should be negligible in most
42 circumstances). This will prevent language detection from working until
43 a data file has been loaded.
44
45 To enable dynamic mode in CLD2 itself, you must define "CLD2_DYNAMIC_MODE".
46 In Chromium, this is controlled by the 'cld2_data_source' variable in
47 ../../build/common.gypi.
48
49
50 Building a CLD2 Dynamic Mode Data File
51 ======================================
52 Note: The cld_2_dynamic_data_tool target is not currently supported on Android.
53 The binaries that it generates are platform-independent, but to build
54 the target itself you'll need a desktop environment.
55
56 1. Configure your desired table size by setting the value of "cld2_table_size"
57 in ../../build/common.gypi.
58 2. Build the "cld_2_dynamic_data_tool" target. This will generate the tool:
59 ${BUILD_DIR}/cld_2_dynamic_data_tool
60 3. Run the tool with "--dump <file>" to generate a data file, e.g.:
61 ${BUILD_DIR}/cld_2_dynamic_data_tool --dump /tmp/cld2_data.bin
62 4. (Optional) Verify that the file was correctly written:
63 ${BUILD_DIR}/cld_2_dynamic_data_tool --verify /tmp/cld2_data.bin
64
65 The data file is suitable for use on all platforms.