Heartbleed, for anyone who doesn’t read the papers, is a serious bug in the popular OpenSSL security library. Its effects are particularly bad, because OpenSSL is so popular, used to implement the secure bit of https: secure web sites on many of the most popular web servers such as apache, nginx, and lighttpd.

A few people have suggested that the problem is that OpenSSL is open source, and code this important should be left to trained professionals. They’re wrong. The problem is that writing and testing cryptographic software is really, really hard.

Writing and testing any sort of security software is hard, because the goals are more or less the opposite of normal software. For normal software, the main goal is to do the right thing with correct input. If it’s a word processor or spreadsheet, you want it to compute and display the right results with reasonable input, but you don’t much care what happens with unreasonable input. If you tell a spreadsheet to open a file full of garbage, and you get a strange screen display or the program crashes, that is at worst mildly annoying.

With security software, though, the entire value is to make sure that it rejects every incorrect input, and doesn’t erroneously reveal secure material. This distinction is not one that is well understood in the computer industry. I can recall far too many reviews of desktop file encryption programs where the reviewer went on at great length about the speed of encryption and decryption, the ease of use of the various screen displays, but never bothered to check that it rejected attempts to decode data with the wrong password. Since the number of possible invalid inputs is stupendously greater than the number of valid inputs, ensuring that security software does what it is supposed to do presents a severe debugging and testing problem.

Public key (PK) cryptography, the core functions for which everyone uses OpenSSL, is doubly difficult because the programming is really tricky. All PK algorithms depend on mathematical operations which are relatively easy to do, but very difficult to reverse. A well-known example is multiplying two prime numbers to get their product vs. finding the two primes if you only know the product. All current algorithms involve arithmetic on very large numbers, much larger than any computer can handling without using special arithmetic libraries.

Even the relatively easy PK operations are still pretty slow, so the practical way to use PK is to use conventional shared key cryptography to protect the web page or mail message or whatever, and only use the PK for one end to tell the other the shared key to use. Cryptography has advanced a lot in the decade that OpenSSL has been in use, with both new PK cryptographic algorithms and shared key algorithms, so there are now about two dozen combinations of initial PK and session shared key schemes that OpenSSL has to support.

People have added extra complication to try to make things faster; one trick is to note that if you fetch a secure web page from a server, you’ll probably fetch other stuff from the same server, so with the agreement of both ends they can leave the session open after the page is complete, and reuse the same session and same shared key on subsequent requests. The heartbleed bug is in “heartbeat” code that periodically checks to see if the other end of an open session is still there. (One way to fix heartbleed is just to turn off the session saving feature, in which case everything will still work, just slower since there will be more sessions to to create.)

As if all this weren’t complex enough, public keys by themselves are just large pseudo-random numbers, which have to be securely associated with domain names for web servers, or e-mail addresses for S/MIME mail, and there’s a whole Public Key Infrastructure (PKI) in which well known entities can assert that a particular key belongs to a particular name using digital signatures, essentially PK encryption run backwards. The package of key, name, and a bunch of other stuff is known as a certificate, which is encoded using a system called ASN.1. The encodings and options in ASN.1 are so complicated that it is notoriously difficult to implement correctly, and has led to multiple security issues just from encoding errors.

Any software package that does what OpenSSL does has to do all the stuff I described above. OpenSSL has a few issues of its own. One is its history; it evolved from an earlier package in the 1990s called SSLeay which was apparently originally an experimental implementation of the large number arithmetic needed for PK cryptography. SSLeay turned into OpenSSL in 1998, so there is now close to 20 years of evolutionary cruft in the half million lines of OpenSSL code. It is written in the C programming language, which remains the lingua franca of the software community, in that no matter what language your application is written in, there’s always a way for that language to connect to and use C libraries.

C grew up in the 1970s on small computers, notably 16-bit PDP-11s, where every bit of code and data space was precious, so it doesn’t have much in the way of defensive programming features to detect and prevent buffer overruns and other bugs. Modern C applications can use libraries that provide much of this defensive programming, but rewriting old C code to be defensive is tedious, and doing so without introducing new bugs is hard, so people rarely do.

So all in all, it’s a miracle that OpenSSL works at all, and a tribute to the skill and diligence of the handful of people who’ve been working on it all these years. Experience with heartbleed shows that while open source doesn’t prevent bugs, it makes it very fast to fix them. It’s been little more than a week since we learned about heartbleed, and surveys show that the majority of vulnerable systems are already fixed. (I fixed mine last Tuesday.)

With all these problems, why do people use OpenSSL?

• It’s free.
• It’s written in C and is fairly fast, so you can use it in all sorts of projects.
• Its license allows commercial binary-only applications (as opposed to the GPL.)
• It’s free.

The key to the solution to OpenSSL’s fragility is easy: it’s MONEY, which will allow them to pay more people to work on it, and do testing and certification and improve the code and everything else that would make it more reliable. None of this is incompatible with open source, except for the fact that nearly all open source projects are starved for resources, because people’s willingness to use free software vastly exceeds their willingness to send back even a little money to ensure that it remains available. (The exceptions are special cases, like the Mozilla Foundation, which gets a large revenue stream from commissions on ads shown when users do Google searches from the default home page, and the Apache Software Foundation which has a high enough profile to attract multiple sponsors at $40K and $100K/year.)

If we agree that OpenSSL is important, which it surely is if the panic about one bug in it is any indication, the question is how to persuade the people who depend on it to voluntarily pay for it. Any suggestions?