xref: /external/wycheproof

# Project Wycheproof
https://github.com/google/wycheproof

*Project Wycheproof is named after
[Mount Wycheproof](https://en.wikipedia.org/wiki/Mount_Wycheproof), the smallest
mountain in the world. The main motivation for the project is to have a goal
that is achievable. The smaller the mountain the more likely it is to be able to
climb it.*

[TOC]

## Introduction

Project Wycheproof tests crypto libraries against known attacks. It is developed
and maintained by members of Google Security Team, but it is not an official
Google product.

At Google, we rely on many third party cryptographic software libraries.
Unfortunately, in cryptography, subtle mistakes can have catastrophic
consequences, and we found that libraries fall into such implementation
pitfalls much too often and for much too long. Good implementation guidelines,
however, are hard to come by: understanding how to implement cryptography
securely requires digesting decades' worth of academic literature. We recognize
that software engineers fix and prevent bugs with unit testing, and we found
that cryptographic loopholes can be resolved by the same means.

These observations have prompted us to develop Project Wycheproof, a collection
of unit tests that detect known weaknesses or check for expected behaviors of
some cryptographic algorithm. Project Wycheproof provides tests for most
cryptographic algorithms, including RSA, elliptic curve crypto and
authenticated encryption. Our cryptographers have systematically surveyed the
literature and implemented most known attacks. We have over 80 test cases which
have uncovered more than [40 bugs](doc/bugs.md). For
example, we found that we could recover the private key of widely-used DSA and
ECDHC implementations.

While we are committed to develop as many attacks as possible, Project
Wycheproof is by no means complete. Passing the tests does not imply that the
library is secure, it just means that it is not vulnerable to the attacks that
Project Wycheproof tests for. Cryptographers are also constantly discovering
new attacks. Nevertheless, with Project Wycheproof developers and users now can
check their libraries against a large number of known attacks, without having
to spend years reading academic papers or become cryptographers themselves.

For more information on the goals and strategies of Project Wycheproof, please
check out our [doc](doc/).

### Coverage

Project Wycheproof has tests for the most popular crypto algorithms, including

- AES-EAX
- AES-GCM
- [DH](doc/dh.md)
- DHIES
- [DSA](doc/dsa.md)
- [ECDH](doc/ecdh.md)
- ECDSA
- ECIES
- [RSA](doc/rsa.md)

The tests detect whether a library is vulnerable to many attacks, including

- Invalid curve attacks
- Biased nonces in digital signature schemes
- Of course, all Bleichenbachers attacks
- And many more -- we have over 80 test cases

Our first set of tests are written in Java, because Java has a common
cryptographic interface. This allowed us to test multiple providers with a
single test suite. While this interface is somewhat low level, and should not
be used directly, we still apply a "defense in depth" argument and expect that
the implementations are as robust as possible. For example, we consider weak
default values to be a significant security flaw. We are converting as many
tests into sets of test vectors to simplify porting the tests to other
languages. We provide ready-to-use test runners for Java Cryptography
Architecture providers such as [Bouncy Castle](http://bouncycastle.org),
[Spongy Castle](https://rtyley.github.io/spongycastle/), and the default
providers in [OpenJDK](http://openjdk.java.net/).

### Usage

-   Install [Bazel](https://bazel.build/).

-   Install [Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
    Policy
    Files](http://stackoverflow.com/questions/6481627/java-security-illegal-key-size-or-default-parameters):
    this enables tests with large key sizes. Otherwise you'll see a lot of
    "illegal key size" exceptions.

-   Check out the tests

```
git clone https://github.com/google/wycheproof.git
```

- To test latest stable version of Bouncy Castle:

```
bazel test BouncyCastleAllTests
```

- To test other versions, e.g., v1.52:

```
bazel test BouncyCastleAllTests_1_52
```

- To test all known versions (warning, will take a long time):

```
bazel test BouncyCastleAllTests_*
```

-   To test a local jar, set the `WYCHEPROOF_BOUNCYCASTLE_JAR` environment
    variable:

```shell
$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/bouncycastle
$ bazel test BouncyCastleTestLocal
$ bazel test BouncyCastleAllTestsLocal
```

Note: bazel does not currently invalidate the build on environment changes. If
you change the `WYCHEPROOF_BOUNCYCASTLE_JAR` environment variable, run `bazel
clean` to force a rebuild:

```shell
$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/bouncycastle
$ bazel test BouncyCastleTestLocal
$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/other/jar
$ bazel clean
$ bazel test BouncyCastleTestLocal
```

- To test [Spongy Castle](https://rtyley.github.io/spongycastle/), replace
BouncyCastle with SpongyCastle in your commands, for example

```
bazel test SpongyCastleAllTests
```

- To test your current installation of
[OpenJDK](http://openjdk.java.net/):

```
bazel test OpenJDKAllTests
```

Note that OpenJDKAllTests expects that OpenJDK is your default JDK, so it might
refuse to run or its results might be incorrect if you are using some other JDK.
If you downloaded your JDK from Oracle or https://java.com, you're probably
using Oracle JDK, which should be compatible with OpenJDK, thus the tests should
run correctly.

Some tests take a very long time to finish. If you want to exclude them, use
BouncyCastleTest, SpongyCastleTest or OpenJDKTest -- these targets exclude all
slow tests (which are annotated with @SlowTest).

Most test targets are failing, and each failure might be a security issue. To
learn more about what a failed test means, you might want to check out [our
documentation](doc/bugs.md) or the comments on top of the corresponding test
function and test class.

### Hall of Bugs

Here are some of the notable vulnerabilities that are uncovered by
Project Wycheproof:

- OpenJDK's SHA1withDSA leaks private keys > 1024 bits
  - Test: testBiasSha1WithDSA in
[DsaTest](https://github.com/google/wycheproof/blob/master/java/com/google/security/wycheproof/testcases/DsaTest.java).
  - This bug is the same as
[CVE-2003-0971 - GnuPG generated ElGamal signatures that leaked the private key]
(https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2003-0971).

- Bouncy Castle's ECDHC leaks private keys
  - Test: testModifiedPublic and testWrongOrderEcdhc in
[EcdhTest](https://github.com/google/wycheproof/blob/master/java/com/google/security/wycheproof/testcases/EcdhTest.java).

### Maintainers

Project Wycheproof is maintained by:

- Daniel Bleichenbacher
- Thai Duong
- Emilia Kasper
- Quan Nguyen

### Contact and mailing list

If you want to contribute, please read [CONTRIBUTING](CONTRIBUTING.md) and send
us pull requests. You can also report bugs or request new tests.

If you'd like to talk to our developers or get notified about major new
tests, you may want to subscribe to our
[mailing list](https://groups.google.com/forum/#!forum/wycheproof-users). To
join, simply send an empty mail to wycheproof-users+subscribe (a] googlegroups.com.