1 # gRPC (Core) Compression Cookbook 2 3 ## Introduction 4 5 This document describes compression as implemented by the gRPC C core. See [the 6 full compression specification](compression.md) for details. 7 8 ### Intended Audience 9 10 Wrapped languages developers, for the purposes of supporting compression by 11 interacting with the C core. 12 13 ## Criteria for GA readiness 14 15 1. Be able to set compression at [channel](#per-channel-settings), 16 [call](#per-call-settings) and [message](#per-message-settings) level. 17 In principle this API should be based on _compression levels_ as opposed to 18 algorithms. See the discussion [below](#level-vs-algorithms). 19 1. Have unit tests covering [the cases from the 20 spec](https://github.com/grpc/grpc/blob/master/doc/compression.md#test-cases). 21 1. Interop tests implemented and passing on Jenkins. The two relevant interop 22 test cases are 23 [large_compressed_unary](https://github.com/grpc/grpc/blob/master/doc/interop-test-descriptions.md#large_compressed_unary) 24 and 25 [server_compressed_streaming](https://github.com/grpc/grpc/blob/master/doc/interop-test-descriptions.md#server_compressed_streaming). 26 27 ## Summary Flowcharts 28 29 The following flowcharts depict the evolution of a message, both _incoming_ and 30 _outgoing_, irrespective of the client/server character of the call. Aspects 31 still not symmetric between clients and servers (e.g. the [use of compression 32 levels](https://github.com/grpc/grpc/blob/master/doc/compression.md#compression-levels-and-algorithms)) 33 are explicitly marked. The in-detail textual description for the different 34 scenarios is described in subsequent sections. 35 36 ## Incoming Messages 37 38  39 40 ## Outgoing Messages 41 42  43 44 ## Levels vs Algorithms 45 46 As mentioned in [the relevant discussion on the spec 47 document](https://github.com/grpc/grpc/blob/master/doc/compression.md#compression-levels-and-algorithms), 48 compression _levels_ are the primary mechanism for compression selection _at the 49 server side_. In the future, it'll also be at the client side. The use of levels 50 abstracts away the intricacies of selecting a concrete algorithm supported by a 51 peer, on top of removing the burden of choice from the developer. 52 As of this writing (Q2 2016), clients can only specify compression _algorithms_. 53 Clients will support levels as soon as an automatic retry/negotiation mechanism 54 is in place. 55 56 ## Per Channel Settings 57 58 Compression may be configured at channel creation. This is a convenience to 59 avoid having to repeatedly configure compression for every call. Note that any 60 compression setting on individual [calls](#per-call-settings) or 61 [messages](#per-message-settings) overrides channel settings. 62 63 The following aspects can be configured at channel-creation time via channel arguments: 64 65 #### Disable Compression _Algorithms_ 66 67 Use the channel argument key 68 `GRPC_COMPRESSION_CHANNEL_ENABLED_ALGORITHMS_BITSET` (from 69 [`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 70 takes a 32 bit bitset value. A set bit means the algorithm with that enum value 71 according to `grpc_compression_algorithm` is _enabled_. 72 For example, `GRPC_COMPRESS_GZIP` currently has a numeric value of 2. To 73 enable/disable GZIP for a channel, one would set/clear the 3rd LSB (eg, 0b100 = 74 0x4). Note that setting/clearing 0th position, that corresponding to 75 `GRPC_COMPRESS_NONE`, has no effect, as no-compression (a.k.a. _identity_) is 76 always supported. 77 Incoming messages compressed (ie, encoded) with a disabled algorithm will result 78 in the call being closed with `GRPC_STATUS_UNIMPLEMENTED`. 79 80 #### Default Compression _Level_ 81 82 **(currently, Q2 2016, only applicable for server side channels. It's ignored 83 for clients.)** 84 Use the channel argument key `GRPC_COMPRESSION_CHANNEL_DEFAULT_LEVEL` (from 85 [`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 86 valued by an integer corresponding to a value from the `grpc_compression_level` 87 enum. 88 89 #### Default Compression _Algorithm_ 90 91 Use the channel argument key `GRPC_COMPRESSION_CHANNEL_DEFAULT_ALGORITHM` (from 92 [`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 93 valued by an integer corresponding to a value from the `grpc_compression_level` 94 enum. 95 96 ## Per Call Settings 97 98 ### Compression **Level** in Call Responses 99 100 The server requests a compression level via initial metadata. The 101 `send_initial_metadata` `grpc_op` contains a `maybe_compression_level` field 102 with two fields, `is_set` and `compression_level`. The former must be set when 103 actively choosing a level to disambiguate the default value of zero (no 104 compression) from the proactive selection of no compression. 105 106 The core will receive the request for the compression level and automatically 107 choose a compression algorithm based on its knowledge about the peer 108 (communicated by the client via the `grpc-accept-encoding` header. Note that the 109 absence of this header means no compression is supported by the client/peer). 110 111 ### Compression **Algorithm** in Call Responses 112 113 **Server should avoid setting the compression algorithm directly**. Prefer 114 setting compression levels unless there's a _very_ compelling reason to choose 115 specific algorithms (benchmarking, testing). 116 117 Selection of concrete compression algorithms is performed by adding a 118 `(GRPC_COMPRESS_REQUEST_ALGORITHM_KEY, <algorithm-name>)` key-value pair to the 119 initial metadata, where `GRPC_COMPRESS_REQUEST_ALGORITHM_KEY` is defined in 120 [`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 121 and `<algorithm-name>` is the human readable name of the algorithm as given in 122 [the HTTP2 spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) 123 for `Message-Encoding` (e.g. gzip, identity, etc.). See 124 [`grpc_compression_algorithm_name`](https://github.com/grpc/grpc/blob/master/src/core/lib/compression/compression.c) 125 for the mapping between the `grpc_compression_algorithm` enum values and their 126 textual representation. 127 128 ## Per Message Settings 129 130 To disable compression for a specific message, the `flags` field of `grpc_op` 131 instances of type `GRPC_OP_SEND_MESSAGE` must have its `GRPC_WRITE_NO_COMPRESS` 132 bit set. Refer to 133 [`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 134
