Home | History | Annotate | Download | only in doc
      1 Interoperability Test Case Descriptions
      2 =======================================
      3 
      4 Client and server use
      5 [test.proto](../src/proto/grpc/testing/test.proto)
      6 and the [gRPC over HTTP/2 v2 protocol](./PROTOCOL-HTTP2.md).
      7 
      8 Client
      9 ------
     10 
     11 Clients implement test cases that test certain functionally. Each client is
     12 provided the test case it is expected to run as a command-line parameter. Names
     13 should be lowercase and without spaces.
     14 
     15 Clients should accept these arguments:
     16 * --server_host=HOSTNAME
     17     * The server host to connect to. For example, "localhost" or "127.0.0.1"
     18 * --server_host_override=HOSTNAME
     19     * The server host to claim to be connecting to, for use in TLS and HTTP/2
     20       :authority header. If unspecified, the value of --server_host will be
     21       used
     22 * --server_port=PORT
     23     * The server port to connect to. For example, "8080"
     24 * --test_case=TESTCASE
     25     * The name of the test case to execute. For example, "empty_unary"
     26 * --use_tls=BOOLEAN
     27     * Whether to use a plaintext or encrypted connection
     28 * --use_test_ca=BOOLEAN
     29     * Whether to replace platform root CAs with
     30       [ca.pem](https://github.com/grpc/grpc/blob/master/src/core/tsi/test_creds/ca.pem)
     31       as the CA root
     32 * --default_service_account=ACCOUNT_EMAIL
     33     * Email of the GCE default service account.
     34 * --oauth_scope=SCOPE
     35     * OAuth scope. For example, "https://www.googleapis.com/auth/xapi.zoo"
     36 * --service_account_key_file=PATH
     37     * The path to the service account JSON key file generated from GCE developer
     38     console.
     39 
     40 Clients must support TLS with ALPN. Clients must not disable certificate
     41 checking.
     42 
     43 ### empty_unary
     44 
     45 This test verifies that implementations support zero-size messages. Ideally,
     46 client implementations would verify that the request and response were zero
     47 bytes serialized, but this is generally prohibitive to perform, so is not
     48 required.
     49 
     50 Server features:
     51 * [EmptyCall][]
     52 
     53 Procedure:
     54  1. Client calls EmptyCall with the default Empty message
     55 
     56 Client asserts:
     57 * call was successful
     58 * response is non-null
     59 
     60 *It may be possible to use UnaryCall instead of EmptyCall, but it is harder to
     61 ensure that the proto serialized to zero bytes.*
     62 
     63 ### cacheable_unary
     64 
     65 This test verifies that gRPC requests marked as cacheable use GET verb instead
     66 of POST, and that server sets appropriate cache control headers for the response
     67 to be cached by a proxy. This test requires that the server is behind
     68 a caching proxy. Use of current timestamp in the request prevents accidental
     69 cache matches left over from previous tests.
     70 
     71 Server features:
     72 * [CacheableUnaryCall][]
     73 
     74 Procedure:
     75  1. Client calls CacheableUnaryCall with `SimpleRequest` request with payload
     76     set to current timestamp. Timestamp format is irrelevant, and resolution is
     77     in nanoseconds.
     78     Client adds a `x-user-ip` header with value `1.2.3.4` to the request.
     79     This is done since some proxys such as GFE will not cache requests from
     80     localhost.
     81     Client marks the request as cacheable by setting the cacheable flag in the
     82     request context. Longer term this should be driven by the method option
     83     specified in the proto file itself.
     84  2. Client calls CacheableUnaryCall again immediately with the same request and
     85     configuration as the previous call.
     86 
     87 Client asserts:
     88 * Both calls were successful
     89 * The payload body of both responses is the same.
     90 
     91 ### large_unary
     92 
     93 This test verifies unary calls succeed in sending messages, and touches on flow
     94 control (even if compression is enabled on the channel).
     95 
     96 Server features:
     97 * [UnaryCall][]
     98 
     99 Procedure:
    100  1. Client calls UnaryCall with:
    101 
    102     ```
    103     {
    104       response_size: 314159
    105       payload:{
    106         body: 271828 bytes of zeros
    107       }
    108     }
    109     ```
    110 
    111 Client asserts:
    112 * call was successful
    113 * response payload body is 314159 bytes in size
    114 * clients are free to assert that the response payload body contents are zero
    115   and comparing the entire response message against a golden response
    116 
    117 ### client_compressed_unary
    118 
    119 This test verifies the client can compress unary messages by sending two unary
    120 calls, for compressed and uncompressed payloads. It also sends an initial
    121 probing request to verify whether the server supports the [CompressedRequest][]
    122 feature by checking if the probing call fails with an `INVALID_ARGUMENT` status.
    123 
    124 Server features:
    125 * [UnaryCall][]
    126 * [CompressedRequest][]
    127 
    128 Procedure:
    129  1. Client calls UnaryCall with the feature probe, an *uncompressed* message:
    130     ```
    131     {
    132       expect_compressed:{
    133         value: true
    134       }
    135       response_size: 314159
    136       payload:{
    137         body: 271828 bytes of zeros
    138       }
    139     }
    140     ```
    141 
    142  1. Client calls UnaryCall with the *compressed* message:
    143 
    144     ```
    145     {
    146       expect_compressed:{
    147         value: true
    148       }
    149       response_size: 314159
    150       payload:{
    151         body: 271828 bytes of zeros
    152       }
    153     }
    154     ```
    155 
    156  1. Client calls UnaryCall with the *uncompressed* message:
    157 
    158     ```
    159     {
    160       expect_compressed:{
    161         value: false
    162       }
    163       response_size: 314159
    164       payload:{
    165         body: 271828 bytes of zeros
    166       }
    167     }
    168     ```
    169 
    170     Client asserts:
    171     * First call failed with `INVALID_ARGUMENT` status.
    172     * Subsequent calls were successful.
    173     * Response payload body is 314159 bytes in size.
    174     * Clients are free to assert that the response payload body contents are
    175       zeros and comparing the entire response message against a golden response.
    176 
    177 
    178 ### server_compressed_unary
    179 
    180 This test verifies the server can compress unary messages. It sends two unary
    181 requests, expecting the server's response to be compressed or not according to
    182 the `response_compressed` boolean.
    183 
    184 Whether compression was actually performed is determined by the compression bit
    185 in the response's message flags. *Note that some languages may not have access
    186 to the message flags, in which case the client will be unable to verify that
    187 the `response_compressed` boolean is obeyed by the server*.
    188 
    189 
    190 Server features:
    191 * [UnaryCall][]
    192 * [CompressedResponse][]
    193 
    194 Procedure:
    195  1. Client calls UnaryCall with `SimpleRequest`:
    196 
    197     ```
    198     {
    199       response_compressed:{
    200         value: true
    201       }
    202       response_size: 314159
    203       payload:{
    204         body: 271828 bytes of zeros
    205       }
    206     }
    207     ```
    208 
    209     ```
    210     {
    211       response_compressed:{
    212         value: false
    213       }
    214       response_size: 314159
    215       payload:{
    216         body: 271828 bytes of zeros
    217       }
    218     }
    219     ```
    220     Client asserts:
    221     * call was successful
    222     * if supported by the implementation, when `response_compressed` is true,
    223       the response MUST have the compressed message flag set.
    224     * if supported by the implementation, when `response_compressed` is false,
    225       the response MUST NOT have the compressed message flag set.
    226     * response payload body is 314159 bytes in size in both cases.
    227     * clients are free to assert that the response payload body contents are
    228       zero and comparing the entire response message against a golden response
    229 
    230 
    231 ### client_streaming
    232 
    233 This test verifies that client-only streaming succeeds.
    234 
    235 Server features:
    236 * [StreamingInputCall][]
    237 
    238 Procedure:
    239  1. Client calls StreamingInputCall
    240  2. Client sends:
    241 
    242     ```
    243     {
    244       payload:{
    245         body: 27182 bytes of zeros
    246       }
    247     }
    248     ```
    249 
    250  3. Client then sends:
    251 
    252     ```
    253     {
    254       payload:{
    255         body: 8 bytes of zeros
    256       }
    257     }
    258     ```
    259 
    260  4. Client then sends:
    261 
    262     ```
    263     {
    264       payload:{
    265         body: 1828 bytes of zeros
    266       }
    267     }
    268     ```
    269 
    270  5. Client then sends:
    271 
    272     ```
    273     {
    274       payload:{
    275         body: 45904 bytes of zeros
    276       }
    277     }
    278     ```
    279 
    280  6. Client half-closes
    281 
    282 Client asserts:
    283 * call was successful
    284 * response aggregated_payload_size is 74922
    285 
    286 
    287 ### client_compressed_streaming
    288 
    289 This test verifies the client can compress requests on per-message basis by
    290 performing a two-request streaming call. It also sends an initial probing
    291 request to verify whether the server supports the [CompressedRequest][] feature
    292 by checking if the probing call fails with an `INVALID_ARGUMENT` status.
    293 
    294 Procedure:
    295  1. Client calls `StreamingInputCall` and sends the following feature-probing
    296     *uncompressed* `StreamingInputCallRequest` message
    297 
    298     ```
    299     {
    300       expect_compressed:{
    301         value: true
    302       }
    303       payload:{
    304         body: 27182 bytes of zeros
    305       }
    306     }
    307     ```
    308     If the call does not fail with `INVALID_ARGUMENT`, the test fails.
    309     Otherwise, we continue.
    310 
    311  1. Client calls `StreamingInputCall` again, sending the *compressed* message
    312 
    313     ```
    314     {
    315       expect_compressed:{
    316         value: true
    317       }
    318       payload:{
    319         body: 27182 bytes of zeros
    320       }
    321     }
    322     ```
    323 
    324  1. And finally, the *uncompressed* message
    325     ```
    326     {
    327       expect_compressed:{
    328         value: false
    329       }
    330       payload:{
    331         body: 45904 bytes of zeros
    332       }
    333     }
    334     ```
    335 
    336  1. Client half-closes
    337 
    338 Client asserts:
    339 * First call fails with `INVALID_ARGUMENT`.
    340 * Next calls succeeds.
    341 * Response aggregated payload size is 73086.
    342 
    343 
    344 ### server_streaming
    345 
    346 This test verifies that server-only streaming succeeds.
    347 
    348 Server features:
    349 * [StreamingOutputCall][]
    350 
    351 Procedure:
    352  1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`:
    353 
    354     ```
    355     {
    356       response_parameters:{
    357         size: 31415
    358       }
    359       response_parameters:{
    360         size: 9
    361       }
    362       response_parameters:{
    363         size: 2653
    364       }
    365       response_parameters:{
    366         size: 58979
    367       }
    368     }
    369     ```
    370 
    371 Client asserts:
    372 * call was successful
    373 * exactly four responses
    374 * response payload bodies are sized (in order): 31415, 9, 2653, 58979
    375 * clients are free to assert that the response payload body contents are zero
    376   and comparing the entire response messages against golden responses
    377 
    378 ### server_compressed_streaming
    379 
    380 This test verifies that the server can compress streaming messages and disable
    381 compression on individual messages, expecting the server's response to be
    382 compressed or not according to the `response_compressed` boolean.
    383 
    384 Whether compression was actually performed is determined by the compression bit
    385 in the response's message flags. *Note that some languages may not have access
    386 to the message flags, in which case the client will be unable to verify that the
    387 `response_compressed` boolean is obeyed by the server*.
    388 
    389 Server features:
    390 * [StreamingOutputCall][]
    391 * [CompressedResponse][]
    392 
    393 
    394 Procedure:
    395  1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`:
    396 
    397     ```
    398     {
    399       response_parameters:{
    400         compressed: {
    401           value: true
    402         }
    403         size: 31415
    404       }
    405       response_parameters:{
    406         compressed: {
    407           value: false
    408         }
    409         size: 92653
    410       }
    411     }
    412     ```
    413 
    414     Client asserts:
    415     * call was successful
    416     * exactly two responses
    417     * if supported by the implementation, when `response_compressed` is false,
    418       the response's messages MUST NOT have the compressed message flag set.
    419     * if supported by the implementation, when `response_compressed` is true,
    420       the response's messages MUST have the compressed message flag set.
    421     * response payload bodies are sized (in order): 31415, 92653
    422     * clients are free to assert that the response payload body contents are
    423       zero and comparing the entire response messages against golden responses
    424 
    425 ### ping_pong
    426 
    427 This test verifies that full duplex bidi is supported.
    428 
    429 Server features:
    430 * [FullDuplexCall][]
    431 
    432 Procedure:
    433  1. Client calls FullDuplexCall with:
    434 
    435     ```
    436     {
    437       response_parameters:{
    438         size: 31415
    439       }
    440       payload:{
    441         body: 27182 bytes of zeros
    442       }
    443     }
    444     ```
    445 
    446  2. After getting a reply, it sends:
    447 
    448     ```
    449     {
    450       response_parameters:{
    451         size: 9
    452       }
    453       payload:{
    454         body: 8 bytes of zeros
    455       }
    456     }
    457     ```
    458 
    459  3. After getting a reply, it sends:
    460 
    461     ```
    462     {
    463       response_parameters:{
    464         size: 2653
    465       }
    466       payload:{
    467         body: 1828 bytes of zeros
    468       }
    469     }
    470     ```
    471 
    472  4. After getting a reply, it sends:
    473 
    474     ```
    475     {
    476       response_parameters:{
    477         size: 58979
    478       }
    479       payload:{
    480         body: 45904 bytes of zeros
    481       }
    482     }
    483     ```
    484 
    485  5. After getting a reply, client half-closes
    486 
    487 Client asserts:
    488 * call was successful
    489 * exactly four responses
    490 * response payload bodies are sized (in order): 31415, 9, 2653, 58979
    491 * clients are free to assert that the response payload body contents are zero
    492   and comparing the entire response messages against golden responses
    493 
    494 ### empty_stream
    495 
    496 This test verifies that streams support having zero-messages in both
    497 directions.
    498 
    499 Server features:
    500 * [FullDuplexCall][]
    501 
    502 Procedure:
    503  1. Client calls FullDuplexCall and then half-closes
    504 
    505 Client asserts:
    506 * call was successful
    507 * exactly zero responses
    508 
    509 ### compute_engine_creds
    510 
    511 This test is only for cloud-to-prod path.
    512 
    513 This test verifies unary calls succeed in sending messages while using Service
    514 Credentials from GCE metadata server. The client instance needs to be created
    515 with desired oauth scope.
    516 
    517 The test uses `--default_service_account` with GCE service account email and
    518 `--oauth_scope` with the OAuth scope to use. For testing against
    519 grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo"
    520 should
    521 be passed in as `--oauth_scope`.
    522 
    523 Server features:
    524 * [UnaryCall][]
    525 * [Echo Authenticated Username][]
    526 * [Echo OAuth Scope][]
    527 
    528 Procedure:
    529  1. Client configures channel to use GCECredentials
    530  2. Client calls UnaryCall on the channel with:
    531 
    532     ```
    533     {
    534       response_size: 314159
    535       payload:{
    536         body: 271828 bytes of zeros
    537       }
    538       fill_username: true
    539       fill_oauth_scope: true
    540     }
    541     ```
    542 
    543 Client asserts:
    544 * call was successful
    545 * received SimpleResponse.username equals the value of
    546   `--default_service_account` flag
    547 * received SimpleResponse.oauth_scope is in `--oauth_scope`
    548 * response payload body is 314159 bytes in size
    549 * clients are free to assert that the response payload body contents are zero
    550   and comparing the entire response message against a golden response
    551 
    552 ### jwt_token_creds
    553 
    554 This test is only for cloud-to-prod path.
    555 
    556 This test verifies unary calls succeed in sending messages while using JWT
    557 token (created by the project's key file)
    558 
    559 Test caller should set flag `--service_account_key_file` with the
    560 path to json key file downloaded from
    561 https://console.developers.google.com. Alternately, if using a
    562 usable auth implementation, she may specify the file location in the environment
    563 variable GOOGLE_APPLICATION_CREDENTIALS.
    564 
    565 Server features:
    566 * [UnaryCall][]
    567 * [Echo Authenticated Username][]
    568 * [Echo OAuth Scope][]
    569 
    570 Procedure:
    571  1. Client configures the channel to use JWTTokenCredentials
    572  2. Client calls UnaryCall with:
    573 
    574     ```
    575     {
    576       response_size: 314159
    577       payload:{
    578         body: 271828 bytes of zeros
    579       }
    580       fill_username: true
    581     }
    582     ```
    583 
    584 Client asserts:
    585 * call was successful
    586 * received SimpleResponse.username is not empty and is in the json key file used
    587 by the auth library. The client can optionally check the username matches the
    588 email address in the key file or equals the value of `--default_service_account`
    589 flag.
    590 * response payload body is 314159 bytes in size
    591 * clients are free to assert that the response payload body contents are zero
    592   and comparing the entire response message against a golden response
    593 
    594 ### oauth2_auth_token
    595 
    596 This test is only for cloud-to-prod path and some implementations may run
    597 in GCE only.
    598 
    599 This test verifies unary calls succeed in sending messages using an OAuth2 token
    600 that is obtained out of band. For the purpose of the test, the OAuth2 token is
    601 actually obtained from a service account credentials or GCE credentials via the
    602 language-specific authorization library.
    603 
    604 The difference between this test and the other auth tests is that it
    605 first uses the authorization library to obtain an authorization token.
    606 
    607 The test
    608 - uses the flag `--service_account_key_file` with the path to a json key file
    609 downloaded from https://console.developers.google.com. Alternately, if using a
    610 usable auth implementation, it may specify the file location in the environment
    611 variable GOOGLE_APPLICATION_CREDENTIALS, *OR* if GCE credentials is used to
    612 fetch the token, `--default_service_account` can be used to pass in GCE service
    613 account email.
    614 - uses the flag `--oauth_scope` for the oauth scope.  For testing against
    615 grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo"
    616 should be passed as the `--oauth_scope`.
    617 
    618 Server features:
    619 * [UnaryCall][]
    620 * [Echo Authenticated Username][]
    621 * [Echo OAuth Scope][]
    622 
    623 Procedure:
    624  1. Client uses the auth library to obtain an authorization token
    625  2. Client configures the channel to use AccessTokenCredentials with the access
    626     token obtained in step 1
    627  3. Client calls UnaryCall with the following message
    628 
    629     ```
    630     {
    631       fill_username: true
    632       fill_oauth_scope: true
    633     }
    634     ```
    635 
    636 Client asserts:
    637 * call was successful
    638 * received SimpleResponse.username is valid. Depending on whether a service
    639 account key file or GCE credentials was used, client should check against the
    640 json key file or GCE default service account email.
    641 * received SimpleResponse.oauth_scope is in `--oauth_scope`
    642 
    643 ### per_rpc_creds
    644 
    645 Similar to the other auth tests, this test is only for cloud-to-prod path.
    646 
    647 This test verifies unary calls succeed in sending messages using a JWT or a
    648 service account credentials set on the RPC.
    649 
    650 The test
    651 - uses the flag `--service_account_key_file` with the path to a json key file
    652 downloaded from https://console.developers.google.com. Alternately, if using a
    653 usable auth implementation, it may specify the file location in the environment
    654 variable GOOGLE_APPLICATION_CREDENTIALS
    655 - optionally uses the flag `--oauth_scope` for the oauth scope if implementator
    656 wishes to use service account credential instead of JWT credential. For testing
    657 against grpc-test.sandbox.googleapis.com, oauth scope
    658 "https://www.googleapis.com/auth/xapi.zoo" should be used.
    659 
    660 Server features:
    661 * [UnaryCall][]
    662 * [Echo Authenticated Username][]
    663 * [Echo OAuth Scope][]
    664 
    665 Procedure:
    666  1. Client configures the channel with just SSL credentials
    667  2. Client calls UnaryCall, setting per-call credentials to
    668     JWTTokenCredentials. The request is the following message
    669 
    670     ```
    671     {
    672       fill_username: true
    673     }
    674     ```
    675 
    676 Client asserts:
    677 * call was successful
    678 * received SimpleResponse.username is not empty and is in the json key file used
    679 by the auth library. The client can optionally check the username matches the
    680 email address in the key file.
    681 
    682 
    683 ### custom_metadata
    684 
    685 This test verifies that custom metadata in either binary or ascii format can be
    686 sent as initial-metadata by the client and as both initial- and trailing-metadata
    687 by the server.
    688 
    689 Server features:
    690 * [UnaryCall][]
    691 * [FullDuplexCall][]
    692 * [Echo Metadata][]
    693 
    694 Procedure:
    695  1. The client attaches custom metadata with the following keys and values:
    696 
    697     ```
    698     key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value"
    699     key: "x-grpc-test-echo-trailing-bin", value: 0xababab
    700     ```
    701 
    702     to a UnaryCall with request:
    703 
    704     ```
    705     {
    706       response_size: 314159
    707       payload:{
    708         body: 271828 bytes of zeros
    709       }
    710     }
    711     ```
    712 
    713  2. The client attaches custom metadata with the following keys and values:
    714 
    715     ```
    716     key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value"
    717     key: "x-grpc-test-echo-trailing-bin", value: 0xababab
    718     ```
    719 
    720     to a FullDuplexCall with request:
    721 
    722     ```
    723     {
    724       response_parameters:{
    725         size: 314159
    726       }
    727       payload:{
    728         body: 271828 bytes of zeros
    729       }
    730     }
    731     ```
    732 
    733     and then half-closes
    734 
    735 Client asserts:
    736 * call was successful
    737 * metadata with key `"x-grpc-test-echo-initial"` and value
    738   `"test_initial_metadata_value"`is received in the initial metadata for calls
    739   in Procedure steps 1 and 2.
    740 * metadata with key `"x-grpc-test-echo-trailing-bin"` and value `0xababab` is
    741   received in the trailing metadata for calls in Procedure steps 1 and 2.
    742 
    743 
    744 
    745 ### status_code_and_message
    746 
    747 This test verifies unary calls succeed in sending messages, and propagate back
    748 status code and message sent along with the messages.
    749 
    750 Server features:
    751 * [UnaryCall][]
    752 * [FullDuplexCall][]
    753 * [Echo Status][]
    754 
    755 Procedure:
    756  1. Client calls UnaryCall with:
    757 
    758     ```
    759     {
    760       response_status:{
    761         code: 2
    762         message: "test status message"
    763       }
    764     }
    765     ```
    766 
    767  2. Client calls FullDuplexCall with:
    768 
    769     ```
    770     {
    771       response_status:{
    772         code: 2
    773         message: "test status message"
    774       }
    775     }
    776     ```
    777 
    778     and then half-closes
    779 
    780 
    781 Client asserts:
    782 * received status code is the same as the sent code for both Procedure steps 1
    783   and 2
    784 * received status message is the same as the sent message for both Procedure
    785   steps 1 and 2
    786 
    787 ### special_status_message
    788 
    789 This test verifies Unicode and whitespace is correctly processed in status
    790 message. "\t" is horizontal tab. "\r" is carriage return.  "\n" is line feed.
    791 
    792 Server features:
    793 * [UnaryCall][]
    794 * [Echo Status][]
    795 
    796 Procedure:
    797  1. Client calls UnaryCall with:
    798 
    799     ```
    800     {
    801       response_status:{
    802         code: 2
    803         message: "\t\ntest with whitespace\r\nand Unicode BMP  and non-BMP \t\n"
    804       }
    805     }
    806     ```
    807 
    808 Client asserts:
    809 * received status code is the same as the sent code for Procedure step 1
    810 * received status message is the same as the sent message for Procedure step 1,
    811   including all whitespace characters
    812 
    813 ### unimplemented_method
    814 
    815 This test verifies that calling an unimplemented RPC method returns the
    816 UNIMPLEMENTED status code.
    817 
    818 Server features:
    819 N/A
    820 
    821 Procedure:
    822 * Client calls `grpc.testing.TestService/UnimplementedCall` with an empty
    823   request (defined as `grpc.testing.Empty`):
    824 
    825     ```
    826     {
    827     }
    828     ```
    829 
    830 Client asserts:
    831 * received status code is 12 (UNIMPLEMENTED)
    832 
    833 ### unimplemented_service
    834 
    835 This test verifies calling an unimplemented server returns the UNIMPLEMENTED
    836 status code.
    837 
    838 Server features:
    839 N/A
    840 
    841 Procedure:
    842 * Client calls `grpc.testing.UnimplementedService/UnimplementedCall` with an
    843   empty request (defined as `grpc.testing.Empty`)
    844 
    845 Client asserts:
    846 * received status code is 12 (UNIMPLEMENTED)
    847 
    848 ### cancel_after_begin
    849 
    850 This test verifies that a request can be cancelled after metadata has been sent
    851 but before payloads are sent.
    852 
    853 Server features:
    854 * [StreamingInputCall][]
    855 
    856 Procedure:
    857  1. Client starts StreamingInputCall
    858  2. Client immediately cancels request
    859 
    860 Client asserts:
    861 * Call completed with status CANCELLED
    862 
    863 ### cancel_after_first_response
    864 
    865 This test verifies that a request can be cancelled after receiving a message
    866 from the server.
    867 
    868 Server features:
    869 * [FullDuplexCall][]
    870 
    871 Procedure:
    872  1. Client starts FullDuplexCall with
    873 
    874     ```
    875     {
    876       response_parameters:{
    877         size: 31415
    878       }
    879       payload:{
    880         body: 27182 bytes of zeros
    881       }
    882     }
    883     ```
    884 
    885  2. After receiving a response, client cancels request
    886 
    887 Client asserts:
    888 * Call completed with status CANCELLED
    889 
    890 ### timeout_on_sleeping_server
    891 
    892 This test verifies that an RPC request whose lifetime exceeds its configured
    893 timeout value will end with the DeadlineExceeded status.
    894 
    895 Server features:
    896 * [FullDuplexCall][]
    897 
    898 Procedure:
    899  1. Client calls FullDuplexCall with the following request and sets its timeout
    900     to 1ms
    901 
    902     ```
    903     {
    904       payload:{
    905         body: 27182 bytes of zeros
    906       }
    907     }
    908     ```
    909 
    910  2. Client waits
    911 
    912 Client asserts:
    913 * Call completed with status DEADLINE_EXCEEDED.
    914 
    915 ### concurrent_large_unary
    916 
    917 Status: TODO
    918 
    919 Client performs 1000 large_unary tests in parallel on the same channel.
    920 
    921 ### Flow control. Pushback at client for large messages (TODO: fix name)
    922 
    923 Status: TODO
    924 
    925 This test verifies that a client sending faster than a server can drain sees
    926 pushback (i.e., attempts to send succeed only after appropriate delays).
    927 
    928 ### Experimental Tests
    929 
    930 These tests are not yet standardized, and are not yet implemented in all
    931 languages. Therefore they are not part of our interop matrix.
    932 
    933 #### rpc_soak
    934 
    935 The client performs many large_unary RPCs in sequence over the same channel. 
    936 The number of RPCs is configured by the experimental flag, `soak_iterations`.
    937 
    938 #### channel_soak
    939 
    940 The client performs many large_unary RPCs in sequence. Before each RPC, it 
    941 tears down and rebuilds the channel. The number of RPCs is configured by 
    942 the experimental flag, `soak_iterations`.
    943 
    944 This tests puts stress on several gRPC components; the resolver, the load 
    945 balancer, and the RPC hotpath.
    946 
    947 #### long_lived_channel
    948 
    949 The client performs a number of large_unary RPCs over a single long-lived 
    950 channel with a fixed but configurable interval between each RPC.
    951 
    952 ### TODO Tests
    953 
    954 #### High priority:
    955 
    956 Propagation of status code and message (yangg)
    957 
    958 Multiple thousand simultaneous calls on same Channel (ctiller)
    959 
    960 Metadata: client headers, server headers + trailers, binary+ascii
    961 
    962 #### Normal priority:
    963 
    964 Cancel before start (ctiller)
    965 
    966 Cancel after sent first message (ctiller)
    967 
    968 Cancel after received headers (ctiller)
    969 
    970 Timeout but completed before expire (zhaoq)
    971 
    972 Multiple thousand simultaneous calls timeout on same Channel (ctiller)
    973 
    974 #### Lower priority:
    975 
    976 Flow control. Pushback at client for large messages (abhishek)
    977 
    978 Flow control. Pushback at server for large messages (abhishek)
    979 
    980 Going over max concurrent streams doesn't fail (client controls itself)
    981 (abhishek)
    982 
    983 RPC method not implemented (yangg)
    984 
    985 Multiple thousand simultaneous calls on different Channels (ctiller)
    986 
    987 Failed TLS hostname verification (ejona?)
    988 
    989 Large amount of headers to cause CONTINUATIONs; 63K of 'X's, all in one header.
    990 
    991 #### To priorize:
    992 
    993 Start streaming RPC but don't send any requests, server responds
    994 
    995 ### Postponed Tests
    996 
    997 Resilience to buggy servers: These tests would verify that a client application
    998 isn't affected negatively by the responses put on the wire by a buggy server
    999 (e.g. the client library won't make the application crash).
   1000 
   1001 Reconnect after transport failure
   1002 
   1003 Reconnect backoff
   1004 
   1005 Fuzz testing
   1006 
   1007 
   1008 Server
   1009 ------
   1010 
   1011 Servers implement various named features for clients to test with. Server
   1012 features are orthogonal. If a server implements a feature, it is always
   1013 available for clients. Names are simple descriptions for developer
   1014 communication and tracking.
   1015 
   1016 Servers should accept these arguments:
   1017 
   1018 * --port=PORT
   1019 
   1020     * The port to listen on. For example, "8080"
   1021 
   1022 * --use_tls=BOOLEAN
   1023 
   1024     * Whether to use a plaintext or encrypted connection
   1025 
   1026 Servers must support TLS with ALPN. They should use
   1027 [server1.pem](https://github.com/grpc/grpc/blob/master/src/core/tsi/test_creds/server1.pem)
   1028 for their certificate.
   1029 
   1030 ### EmptyCall
   1031 [EmptyCall]: #emptycall
   1032 
   1033 Server implements EmptyCall which immediately returns the empty message.
   1034 
   1035 ### UnaryCall
   1036 [UnaryCall]: #unarycall
   1037 
   1038 Server implements UnaryCall which immediately returns a SimpleResponse with a
   1039 payload body of size `SimpleRequest.response_size` bytes and type as appropriate
   1040 for the `SimpleRequest.response_type`. If the server does not support the
   1041 `response_type`, then it should fail the RPC with `INVALID_ARGUMENT`.
   1042 
   1043 ### CacheableUnaryCall
   1044 [CacheableUnaryCall]: #cacheableunarycall
   1045 
   1046 Server gets the default SimpleRequest proto as the request. The content of the
   1047 request is ignored. It returns the SimpleResponse proto with the payload set
   1048 to current timestamp.  The timestamp is an integer representing current time
   1049 with nanosecond resolution. This integer is formated as ASCII decimal in the
   1050 response. The format is not really important as long as the response payload
   1051 is different for each request. In addition it adds
   1052   1. cache control headers such that the response can be cached by proxies in
   1053      the response path. Server should be behind a caching proxy for this test
   1054      to pass. Currently we set the max-age to 60 seconds.
   1055 
   1056 ### CompressedResponse
   1057 [CompressedResponse]: #compressedresponse
   1058 
   1059 When the client sets `response_compressed` to true, the server's response is
   1060 sent back compressed. Note that `response_compressed` is present on both
   1061 `SimpleRequest` (unary) and `StreamingOutputCallRequest` (streaming).
   1062 
   1063 ### CompressedRequest
   1064 [CompressedRequest]: #compressedrequest
   1065 
   1066 When the client sets `expect_compressed` to true, the server expects the client
   1067 request to be compressed. If it's not, it fails the RPC with `INVALID_ARGUMENT`.
   1068 Note that `response_compressed` is present on both `SimpleRequest` (unary) and
   1069 `StreamingOutputCallRequest` (streaming).
   1070 
   1071 ### StreamingInputCall
   1072 [StreamingInputCall]: #streaminginputcall
   1073 
   1074 Server implements StreamingInputCall which upon half close immediately returns
   1075 a StreamingInputCallResponse where aggregated_payload_size is the sum of all
   1076 request payload bodies received.
   1077 
   1078 ### StreamingOutputCall
   1079 [StreamingOutputCall]: #streamingoutputcall
   1080 
   1081 Server implements StreamingOutputCall by replying, in order, with one
   1082 StreamingOutputCallResponse for each ResponseParameters in
   1083 StreamingOutputCallRequest. Each StreamingOutputCallResponse should have a
   1084 payload body of size ResponseParameters.size bytes, as specified by its
   1085 respective ResponseParameters. After sending all responses, it closes with OK.
   1086 
   1087 ### FullDuplexCall
   1088 [FullDuplexCall]: #fullduplexcall
   1089 
   1090 Server implements FullDuplexCall by replying, in order, with one
   1091 StreamingOutputCallResponse for each ResponseParameters in each
   1092 StreamingOutputCallRequest. Each StreamingOutputCallResponse should have a
   1093 payload body of size ResponseParameters.size bytes, as specified by its
   1094 respective ResponseParameters. After receiving half close and sending all
   1095 responses, it closes with OK.
   1096 
   1097 ### Echo Status
   1098 [Echo Status]: #echo-status
   1099 When the client sends a response_status in the request payload, the server closes
   1100 the stream with the status code and messsage contained within said response_status.
   1101 The server will not process any further messages on the stream sent by the client.
   1102 This can be used by clients to verify correct handling of different status codes and
   1103 associated status messages end-to-end.
   1104 
   1105 ### Echo Metadata
   1106 [Echo Metadata]: #echo-metadata
   1107 When the client sends metadata with the key `"x-grpc-test-echo-initial"` with its
   1108 request, the server sends back exactly this key and the corresponding value back to
   1109 the client as part of initial metadata. When the client sends metadata with the key
   1110 `"x-grpc-test-echo-trailing-bin"` with its request, the server sends back exactly this
   1111 key and the corresponding value back to the client as trailing metadata.
   1112 
   1113 ### Observe ResponseParameters.interval_us
   1114 [Observe ResponseParameters.interval_us]: #observe-responseparametersinterval_us
   1115 
   1116 In StreamingOutputCall and FullDuplexCall, server delays sending a
   1117 StreamingOutputCallResponse by the ResponseParameters's `interval_us` for that
   1118 particular response, relative to the last response sent. That is, `interval_us`
   1119 acts like a sleep *before* sending the response and accumulates from one
   1120 response to the next.
   1121 
   1122 Interaction with flow control is unspecified.
   1123 
   1124 ### Echo Auth Information
   1125 
   1126 Status: Pending
   1127 
   1128 #### Echo Authenticated Username
   1129 [Echo Authenticated Username]: #echo-authenticated-username
   1130 
   1131 If a SimpleRequest has fill_username=true and that request was successfully
   1132 authenticated, then the SimpleResponse should have username filled with the
   1133 canonical form of the authenticated source. The canonical form is dependent on
   1134 the authentication method, but is likely to be a base 10 integer identifier or
   1135 an email address.
   1136 
   1137 #### Echo OAuth scope
   1138 [Echo OAuth Scope]: #echo-oauth-scope
   1139 
   1140 If a SimpleRequest has `fill_oauth_scope=true` and that request was successfully
   1141 authenticated via OAuth, then the SimpleResponse should have oauth_scope filled
   1142 with the scope of the method being invoked.
   1143 
   1144 Although a general server-side feature, most test servers won't implement this
   1145 feature. The TLS server `grpc-test.sandbox.googleapis.com:443` supports this
   1146 feature. It requires at least the OAuth scope
   1147 `https://www.googleapis.com/auth/xapi.zoo` for authentication to succeed.
   1148 
   1149 Discussion:
   1150 
   1151 Ideally, this would be communicated via metadata and not in the
   1152 request/response, but we want to use this test in code paths that don't yet
   1153 fully communicate metadata.
   1154