Root Causes 312: Why You Shouldn't Roll Your Own Crypto

Episode Transcript

Lightly edited for flow and brevity.

• 

  Tim Callan

  So, Jason, I’m gonna start with a question and based on many things you’ve said on this podcast in the past, I think I know the answer to that question, but I’m just gonna ask you. Should one roll one’s own crypto?

• 

  Jason Soroko

  Short answer – no. In fact, avoid it at all costs. I even know people in the industry, you do too, who really know this stuff and they don’t either. So, don’t do it.

• 

  Tim Callan

  So, this has now moved out of the realm of opinion and into the realm of academically established, peer reviewed fact. I am referencing a paper that you called my attention to originally, Jay, from July 2021 – so it took me a little while to get to it – and the title is You Really Shouldn’t Roll Your Own Crypto. An Empirical Study of Vulnerabilities and Cryptographic Libraries. It’s from three researchers from MIT – Jenny Blessing, Michael Specter and Daniel Weitzner. So, congrats Jenny, Michael and Daniel.

  And, basically, what these researchers did is research the nature of vulnerabilities in cryptographic systems. They went to a set of commonly available, commonly used crypto systems. There’s a list here. Cryptographic libraries - OpenSSL, GnuTLS, Mozilla NSS, WolfSSL, Botan, Libgcrypt, LibreSSL and BoringSSL. They took this set of cryptographic libraries, and they needed a common database of vulnerabilities, and so what they wound up with was the common vulnerabilities and exposure CVE list that goes through lists for libraries and what they wanted to do. That’s from the National Vulnerability Database in particular. They took the CVE list from the National Vulnerability Database and analyzed these vulnerabilities by their nature. What is the nature of these vulnerabilities inside these cryptographic libraries?

  I think you know what this paper says, Jay, because you sent it my way originally but try to channel the common person. What do you think the average informed IT person would imagine would be the main source of vulnerabilities and problems with cryptographic libraries?

• 

  Jason Soroko

  Geez, Tim. If I try to put myself in the head of just about anybody who is looking at this, anybody who is informed, I’m going back to experiences of technical journalism where cryptography kind of was a problem. We’ve had things to deprecate due to insufficient key lengths. We know that there are some algorithms completely that have been deprecated and so there are standard problems that are just caused by good ideas that aged out, but I think in the case of cryptographic libraries, you know, the real source of issues probably comes down to – and this is me, you know, playing the roll of the guesser. It really comes down to there probably are some sort of implementation or procedural issues that seem to be the most common. I do happen to know though, Tim, there has been technical journalism with respect to some people have chosen, people who are even professionals building cryptographic algorithms sometimes will make errors in assumption about how factorization works and things like this and those can lead to problems, but the most common problems are typically related to the implementation.

• 

  Tim Callan

  So what I’ll do if it’s fair is roll all these up and call them cryptographic implementation problems. It might have to do with keys or algorithms or deprecated processes or things along those lines.

• 

  Jason Soroko

  Yes.

• 

  Tim Callan

  Ok. So the most common problem according to this research in cryptographic systems was memory management problems. The most common vulnerability was memory related vulnerabilities. Had nothing to do with cryptography. It was software coding error.

• 

  Jason Soroko

  Yes.

• 

  Tim Callan

  And we know that from Heartbleed.

• 

  Jason Soroko

  We know that from Heartbleed.

• 

  Tim Callan

  The most devastating crypto-based attack that ever occurred wasn’t a problem with the cryptography.

• 

  Jason Soroko

  Correct.

• 

  Tim Callan

  It was a buffer override attack.

• 

  Jason Soroko

  We have had podcasts specifically about the usage of the RUST language, which purports to be memory safe compared to say coding directly in C or C++ with pointer problems. It’s very, very, easy to hang yourself with memory issues when you are dealing with languages that are not naturally memory safe. And people might wonder, well, why in the world would a language not be memory safe? Well, because flexibility was a lot more important at the time C first came out. Flexibility of the language was a lot more important than being memory safe and a lot of our cryptographic algorithm implementations are written in those, dare I say it, legacy languages that are not memory safe and therefore, you’ve had problems like Heartbleed.

• 

  Tim Callan

  Exactly. And Heartbleed is just such as a case study of this. Blessing, Specter, Weitzner, here’s a quote from their paper. “Systems level bugs are a greater security concern than the actual cryptographic procedures.” That’s it in black and white. To put some numbers on it, 27.2% of the analyzed vulnerabilities were actually what they call cryptographic issues (and they have a methodology for classifying them), while 37.2% were memory safety issues. They go on to analyze the level of complexity of a cryptographic system and the number of CVEs and the conclusion they are basically coming to is the thing about cryptographic systems is they are more complex than your average bear. They are more complex than a lot of what we are running just by the nature of what they are doing and as a consequence of that, there’s just more opportunity for errors. And because there’s more opportunity for errors, there’s more opportunity for errors that are vulnerabilities and there’s more opportunities for those vulnerabilities to be discovered and exploited. And they see it as being really that simple.

• 

  Jason Soroko

  Yes. It is. It’s amazing. When we talk about implementation problems, you could say that the actual coding of a cryptographic algorithm is part of an implementation. It certainly, absolutely is. It doesn’t have anything to do with usage. In other words, a lot of times implementation people will consider it to be the way I installed something, where I installed something, the procedures I followed to install something, when in reality, you could have installed something perfectly and yet it was really the underlying code, the framework in which the cryptographic algorithm was framed, which was leaking information. Enough information so that the bad guy could get in there and do funny things. Heartbleed is an example, but there are a lot of other examples where being memory safe is important. We’ve podcasted about this, Tim, and I swear, one of these days – it doesn’t have to be RUST, it could be other languages that’ll come up in the future or other techniques – but being memory safe is absolutely critical.

• 

  Tim Callan

  Code is complex. We know this. Software errors happen. They are part of life and that means they also happen in things that are really essential and really important, and in just a few recent episodes we’ve talked about direct attacks and vulnerabilities that stem from that kind of thing. And here in this case, we are looking at I think a pretty compelling case that in cryptographic systems the problems aren’t fundamentally all that different.

  I have a breakdown here of the different types: Cryptographic issues are 25.8%, memory buffer issues 19.4%, resource management errors 17.7%, exposure of sensitive information 11.3%, improper input validation (of all things) 11.3%, numeric errors (hmm, I wonder what that means) 5.3%, and then other is a 9.2%. So, you know, a small number of things dominate but the memory error issues are actually worse on the whole than the actual cryptographic issues even in these complicated, very esoteric, cryptographic systems.

• 

  Jason Soroko

  Right on. That numeric one, Tim, let’s just point that out for a second and it could be something as simple as, you know the most heinous version of that would be you intend to calculate something and round it to many decimal places and therefore you need a floating point variable and - -

• 

  Tim Callan

  Oh, it’s been a long time since we talked about floating points.

• 

  Jason Soroko

  And if your variables happen to be integer, well, you are gonna be rounding off a lot of information. So, there could be - - an example of a numeric issue.

• 

  Tim Callan

  Thank you. That helps.

  Now, I’m gonna show you one other thing which is – and I just found this interesting – it’s the exploitable lifetime. This is Table 3 of this paper. Exploitable lifetimes in years of vulnerabilities in cryptographic libraries. So, they include three. They use OpenSSL, GnuTLS and Mozilla NCSS, and that was all they could get data for. They’ve got a bunch of columns. They got two columns I’m gonna focus on – median lifetime and average lifetime, because we know average and median of course are not the same. OpenSSL – this is median lifetime of an exploitable vulnerability in years. Should I make you guess or should I just tell you?

• 

  Jason Soroko

  Please, tell me.

• 

  Tim Callan

  4.14 years median lifetime. Average lifetime, 6.16 years. GnuTLS is doing much better. It’s got a median lifetime of a mere 1.65 years and an average lifetime of a mere 2.35 years and Mozilla NSS has a median lifetime of 12.43 years and an average lifetime of 9.43 years. So, WOW!

• 

  Jason Soroko

  That’s cool.

• 

  Tim Callan

  These vulnerabilities are just sitting out there. Then, like I said, they have a whole analysis where they basically compare a number of bugs to code complexity and they have a method for that and the conclusion they come to is that it’s pretty correlative that the more complex the code, the more the bugs you get and that really code complexity is the ultimate source of this problem. The reason that cryptographic systems seem to have more bugs that other systems is they just also have more code and more complex code.

• 

  Jason Soroko

  It makes sense.

• 

  Tim Callan

  Yes. It makes sense. I’m not sure how directly actionable it is, but things like this are always interesting and I love to look at them and think about them and think about what they imply.

• 

  Jason Soroko

  You know, Tim, just to back up to the very high level thinking about don’t roll your own crypto, a lot of software systems are built that, yes, you want them to be secure but there is a certain amount of risk that is accepted because it is performing some kind of a function that requires interoperability and usability, and the problem with cryptographic systems, the problem with implementing a cryptographic algorithm specifically, is it really has no other purpose than to be secure. Therefore, the risk level that you can take, the type of developer that you need, the type of experience that you require in order to build these kinds of systems is so much more important. This is why we have 12-year-old crypto systems that are sitting out there and it’s because very few people are writing their own, and they depend on the experts, they depend on a ton of peer review going on in terms of the open-source community and even they have a hard time getting it right all the time. In fact, we have a long history of these kinds of problem, but it really shows, I don’t think anybody really is in a position to write core cryptographic systems correctly or well. Just stay away from that completely.

• 

  Tim Callan

  Yeah. It’s hard. It’s even hard for the people who do this professionally.

• 

  Jason Soroko

  That’s right.

• 

  Tim Callan

  So, anyway, thank you to the three MIT researchers for this. It certainly was an interesting read and wanted to make sure we share it. I don’t have any data behind this but just having been around the world, I think you correctly, in my opinion, channeled what the average person would have said which is, oh, cryptography is, you know, there’s all this esoteric stuff and people are gonna get the esoteric stuff wrong. And we see that. You and I see examples for that on a routine basis. But what’s interesting here is just to say, hey, code is hard and complicated code is extra hard and complicated code that is fundamentally meant to be secure is a place to be ultra careful. It’s interesting and enlightening to see that backed up by the evidence.

• 

  Jason Soroko

  Don’t roll your own crypto, Tim. I know you were tempted but don’t do it.

• 

  Tim Callan

  I was going to. I was just gonna sit down and roll my own crypto, but instead in guess I won’t.

• 

  Jason Soroko

  Alright.

• 

  Tim Callan

  Alright.