Date: 2024-09-19

# Fun with bignum: how RSA encryption works

#### 2024-09-19

Primitive types such as `int` or `double` store numbers in exactly 4 or 8 bytes, with finite precision. This suffices for most applications, but cryptography requires arithmetic on very large numbers, without loss of precision. Therefore OpenSSL uses a bignum data type which holds arbitrary sized integers and implements all basic arithmetic and comparison operators such as `+`, `-`, `*`, `^`, `%%`, `%/%`, `==`, `!=`, `<`, `<=`, `>` and `>=`.

One special case, the modular exponent `a^b %% m` can be calculated using `bignum_mod_exp`, even when `b` is too large for calculating `a^b`.

    # create a bignum
    y <- bignum("123456789123456789")
    z <- bignum("D41D8CD98F00B204E9800998ECF8427E", hex = TRUE)

    # size grows
    print(y * z)

    ## [b] 34808613111804879213872650915812112647840354642904626774

    # Basic arithmetic
    div <- z %/% y
    mod <- z %% y
    z2 <- div * y + mod
    stopifnot(z2 == z)
    stopifnot(div < z)

RSA involves a public key and a private key. The public key should be known by everyone and is used for encrypting messages. Messages encrypted with the public key can only be decrypted in a reasonable amount of time using the private key. In RSA, this asymmetry is based on the practical difficulty of factoring the product of two large prime numbers.

### RSA key generation

An RSA key-pair is generated as follows (adapted from wikipedia):

- Choose two distinct prime numbers \(p\) and \(q\). Keep these secret.
- Compute the product \(n = p*q\). This \(n\) value is public and used as the modulus.
- Compute \(\phi(n) = (p − 1)(q − 1)\).
- Choose an integer \(e\) smaller than \(\phi(n)\) such that \(e\) and \(\phi(n)\) are coprime. OpenSSL always uses \(65537\).
- Compute a value for \(d\) such that \((d * e)\pmod{\phi(n)} = 1\).

OpenSSL has a key generator that does these things for us.

    (key <- rsa_keygen(512))

    ## [512-bit rsa private key]
    ## md5: 6a40c1ac4cb4fe2ebf0f16d6e9504a18
    ## sha256: 337c088b0e0d5e63bfafb34b36ca3e663ec29826fe2d30488cc91c35e4d2b9ef

    (pubkey <- key$pubkey)

    ## [512-bit rsa public key]
    ## md5: 6a40c1ac4cb4fe2ebf0f16d6e9504a18
    ## sha256: 337c088b0e0d5e63bfafb34b36ca3e663ec29826fe2d30488cc91c35e4d2b9ef

Usually we would use `rsa_encrypt` and `rsa_decrypt` to perform the encryption:

    msg <- charToRaw("hello world")
    ciphertext <- rsa_encrypt(msg, pubkey)
    rawToChar(rsa_decrypt(ciphertext, key))

    ## [1] "hello world"

Let’s look at how this works under the hood.

### How RSA encryption works

The `data` field of the private key extracts the underlying bignum integers:

    key$data

    ## $e
    ## [b] 65537
    ## $n
    ## [b] 10331723363597025163066924143546640053976139195464598876267296060902833058568180647720291320785951507251692022552397165484567261416234600082331382113017029
    ## $p
    ## [b] 108777857350402604418102580423565097487287524966475308213793297310590481504057
    ## $q
    ## [b] 94980022729403262936072006390031702892343539248801535767134392564509628335597
    ## $d
    ## [b] 5499521942400204812755995067635450787844387381689465977975565300282212051942320731832030436210617940247711733560570848349084928846347397064419143349265953
    ## $dp
    ## [b] 59584910829334621603147299000954627686591312674847751492545670081829923486185
    ## $dq
    ## [b] 89062702241676428894721593614187531769017194271874659795740985407622788649705
    ## $qi
    ## [b] 36396158442017237810656594886902483050771146314562802548257564763865806655675

You can verify that the equations above hold for this key. The public key is simply a subset of the key which only contains \(n\) and \(e\):

    pubkey$data

    ## $e
    ## [b] 65537
    ## $n
    ## [b] 10331723363597025163066924143546640053976139195464598876267296060902833058568180647720291320785951507251692022552397165484567261416234600082331382113017029

In order to encrypt a message into ciphertext we have to treat the message data as an integer. The message cannot be larger than the key size. For example convert the text `hello world` into an integer:

    m <- bignum(charToRaw("hello world"))
    print(m)

    ## [b] 126207244316550804821666916

To encrypt this message \(m\) into ciphertext \(c\) we calculate \(c = m^e\pmod n\). Using the public key from above:

    e <- pubkey$data$e
    n <- pubkey$data$n
    c <- (m ^ e) %% n
    print(c)

    ## [b] 4246445085192329498141187521401900754227848296790587799639941636284486581787845331496358029289794124759605053848489542374218463487427900589642951424807646

This number represents our encrypted message! It is usually exchanged using base64 notation for human readability:

    base64_encode(c)

    ## [1] "URQw8yZgYNzN06RET5waOu8UbwCiIwKKYPXV5N+NKopkDn4plDAaARYQa6yzOhr7PF7FeE89zGPhM7xyF20S3g=="

The ciphertext can be decrypted using \(d\) from the corresponding private key via \(m = c^d \pmod{n}\). Note that `c^d` is too large to calculate directly so we need to use `bignum_mod_exp` instead.

    d <- key$data$d
    out <- bignum_mod_exp(c, d, n)
    rawToChar(out)

    ## [1] "hello world"

The only difference with the actual `rsa_encrypt` and `rsa_decrypt` functions is that these add some additional padding to the data.