Root Causes 113: What Is Certificate Pinning?

Episode Transcript

Lightly edited for flow and brevity.

• 

  Tim Callan

  Today we are going to talk about certificate pinning.

• 

  Jason Soroko

  Geez, that old chestnut. It’s been around almost 10 years as a concept.

• 

  Tim Callan

  Yes, maybe a little more than 10.

• 

  Jason Soroko

  Tim, as you always ask me, maybe we could get into ‘what’s the pithy definition?’. Then I’ll get into the even more pithy: Here’s the reason why it was considered a good idea at one time. To define this, let’s get into the real world, because even if you’re not a technician you’ll understand this analogy.

  You are a company that builds mobile applications, native applications. That native application is talking to some sort of API backend. As you and I always say Tim, it’s best to have some sort of TLS-based authentication to that backend; that’s a certificate-based authentication. What happens if a certificate that is not fully trusted, is the one that’s being used to do that authentication? In other words, how could that happen? That could happen if somehow you were socially engineered to install a self-signed certificate that was in possession of the bad guy. Then, all the sudden your communication to that backend can essentially be man-in-the-middled. In other words, even though that traffic is encrypted back and forth, it’s being decrypted and read by somebody in the middle, because of the fact that there is a self-signed certificate that is being trusted at the client side.
  

• 

  Tim Callan

  Therefore, they could be doing a lot of things. They could be pulling out information, they could be changing commands that are being given to the backend. Let’s say it’s your mobile banking app, they can use it to steal your log-in credentials, or they could just use it directly to tell the banking app to wire money to their fraudulent account.

• 

  Jason Soroko

  There’s a ton of bad things that can be done, and in the past it has happened; It’s not a theoretical attack. This whole idea of being socially engineered to install a self-signed certificate; I remember studying in a lot of depth a decade ago. It’s not difficult at all to install on iOS a profile which includes a certificate, or to install a certificate on an Android device. What was interesting in the past was the user interface. When you were prompted to do such an act, as a non-technical layperson you’re being asked, “Hey, do you want to do your banking or do you want to go to this social media site?” That’s fantastic, install this certificate first.

• 

  Tim Callan

  And these screens are getting in my way and I’m just hitting, Ok, Ok, Ok, so I can make the screens go away and do what I want to do, which is look at cat videos.

• 

  Jason Soroko

  Back in the day, Apple were the geniuses of making such an easy-to-use user interface. Android made it very easy as well. The takeaway here is the "go ahead and install it” button was huge, and the "no, don’t do this, please cancel" button was almost imperceptibly tiny.

• 

  Tim Callan

  It was hidden in the corner.

• 

  Jason Soroko

  We now know a lot more about these risks, and one of the big things that has changed was the user interface around installing these certificates and profiles, which would include a certificate. Therefore, I wouldn't say it's impossible to be socially engineered. But certainly, the types of warnings you now get; especially on a desktop browser, but also on a mobile device; would be one of the biggest things that did change over time. Which is potentially why you don't hear about this problem as often anymore.

• 

  Tim Callan

  To summarize your point, you are saying once upon a time, when this social engineering risk was quite real, you might address that problem by pinning to a specific certificate?

• 

  Jason Soroko

  It was the immediate response to a state of these incidents of people being socially engineered, absolutely. So, certificate pinning became quite popular for a period of time and in fact, still is used to solve this exact problem. Because the beauty of it is that if you are trying to connect with this illegitimate certificate, it simply won't happen and therefore you are protected.

• 

  Tim Callan

  At that level it's easy to say ah-ha, I can see a bunch of benefits. There's a security benefit to walk through the scenario just described. There is also a governance benefit: I can control which certificates are connecting to my application, and make sure they are compliant with my own policies and other compliance requirements that I have. In general I feel like the whole world is more controlled and more control is better. At that level one might say, “hey, great.” So, Jason, how come we are not still doing this?

• 

  Jason Soroko

  Well, towards the end of when certificate pinning was really popular, one of the things that was popular wasn't to just pin the whole certificate, but to pin the public key. That gave you a bit more flexibility because, any time the underlying certificate changed you didn't have to redistribute the app, because the public key wasn't changing as often.

• 

  Tim Callan

  And certificates expire. So, you don't want to make everybody download a new app once a year. Absolutely.

• 

  Jason Soroko

  Yes, that solved that problem. Then another problem was solved in that people realized, ‘hey, I can put multiples.’ I can put a backup public key. I can, in fact, put a few.

• 

  Tim Callan

  As long as it's one of these 100 and I control all 100, it's just fine.

• 

  Jason Soroko

  Right, a couple things happened. We've reported on this podcast several times, about native applications that went entirely dark, because of expired certs. Therefore, the certificate pinning was hardcoded and there was just no simple fix; that's just a simple case of human error. This isn't a pedantic computer science problem; this is just somebody didn't follow the policies properly and realize the implications of not updating these applications at a given critical point in time, which is the expiry of these certificates. Additionally, we've seen through the cab forum, through the Mozilla board, sometimes mistakes are made in the issuance of certificates, so a large number of certificates have to be revoked. Some of those certificates may cover even the intermediates, or whatever it is that root up from that certificate that's being used in the pinning. That's when really bad things happen, if that's revoked, you go dark. You are not able to connect with that application.

• 

  Tim Callan

  Now in principle I could mitigate this problem vastly, right? If I’m really buttoned up and I really had my act together, I could do what we talked about. I give it a set of alternatives to pin to and I make sure that they don't co-expire, and I put various other measures in place. I bring them in from different CA’s so even in a worst-case scenario, like a DigiNotar, CA-goes-out-of-business kind of scenario, I'm still ok.

• 

  Jason Soroko

  We've seen mistakes made over and over and over again. I'm certain that there are many organizations that have done this very well; even though we always highlight the bad ones.

  The conclusion, Tim, for certificate pinning is that the costs to agility are just too high now.

• 

  Tim Callan

  Yes, the agility price tag is very high. The point I'm trying to make is if you had best-of-breed, World-Class certificate and PKI management chops in your organization, and you could be 100% confident in your ability to execute on a complex certificate pinning scenario; then you could probably feel secure. But anybody who isn't at the Olympic level is putting themselves at unacceptable risk.

• 

  Jason Soroko

  Yes, that's the extreme. But to temper that a little bit, I would say so many mobile applications are built by contractors, third-parties, and sometimes just 20-year-old people who are on a gig job. Their first and foremost job is to get the app built and make it work. Their thinking is not the security aspect of it, and that's where the problem lies.

• 

  Tim Callan

  When it passes QA, they get their check, and what happens a year from today ain't their problem.

• 

  Jason Soroko

  If you're an organization that has a Risk Officer or a dedicated person or people who obsess over this, then it could be done. You don't necessarily have to be an Olympic level governance organization that has 15 auditors looking in every orifice of your organization. You simply need to have at least some mechanism in the company that's obsessed with getting it right. It doesn't take a massive amount of time, I don't even think that's a full-time job. It could be depending on how many apps you have, but for a singular important application you just need to have somebody who is doing the checks and balances.

• 

  Tim Callan

  I know I'm playing both sides of the fence on this. But now, I'm going to come around and I'm gonna flip on what I said earlier. I’ll say ‘yeah, but we see people can't even renew their certificates.’ So, if that's the bar and we can't even get over that bar, then why on earth would you set yourself a considerably higher bar?

• 

  Jason Soroko

  Maybe that's the reason why certificate pinning became a big topic a decade ago. It's a solution to a problem that we saw a while back; it still exists but it used to be a bigger problem. The gut feel that I have about it is ‘if you just do your Ps and Qs right, it can be done.’ But time and time and time again we've seen that it isn't. Therefore, I have to admit maybe my gut feel is incorrect and I'm willing to concede that.

• 

  Tim Callan

  The boiled down viewpoint would be, if you're going to do certificate pinning, you are setting yourself a task. You're giving yourself a much higher standard, which you have to be fully confident that you're gonna be able to meet. It better be worth it and almost all the time it probably isn't. Is that a good assessment?

• 

  Jason Soroko

  The boiled down viewpoint would be, if you're going to do certificate pinning, you are setting yourself a task. You're giving yourself a much higher standard, which you have to be fully confident that you're gonna be able to meet. It better be worth it and almost all the time it probably isn't. Is that a good assessment?

• 

  Tim Callan

  This also comes back to risk profiles. You start talking IoT and IoT means so many things. Sometimes, IoT is a refrigerator and in the worst-case scenario my refrigerator is used for a botnet, or maybe someone turns my freezer off and my meat spoils. But on the other end of the spectrum, what if it's a nuclear submarine. Maybe in that scenario we are really going the extra mile and there it's warranted. But on your refrigerator, you’re probably just asking for trouble.

• 

  Jason Soroko

  Yes, I would love to see a refrigerator right now that's doing full blown X.509 certificate management anyway.

• 

  Tim Callan

  Fair enough. Bad example, but yes.

• 

  Jason Soroko

  On the other hand, you are right. Any kind of critical infrastructure, your risk for downtime is unacceptable. Certificate pinning in that scenario has its own dangers. In other words, is it worth it to restrict which certificate is being used? That agility cost, Tim, is something that just keeps rearing its head regardless of the use case.

• 

  Tim Callan

  Yes so we’d have to know the specific circumstances, but highly unlikely that we would recommend certificate pinning for almost all use cases.

• 

  Jason Soroko

  You better really know why you are using it and have a hopefully a limited or highly controllable user base. That I think is the key. In other words, if it's a user base that may or may not update their app, it's probably not a good idea.

• 

  Tim Callan

  Either way you better have confidence, not only that you have good control over your PKI, but that you're going to continue to have that for the foreseeable future. It's not just what you did today; it's what does it look like in 18 months, and what does it look like in five years?

• 

  Jason Soroko

  I would say if you are the type of shop that outsources your native app development to a group of people that you don't completely control or know or trust. Other than the fact that they tick off the QA at the end and the app just works. Then absolutely avoid certificate pinning, it will probably bite you in the butt.

• 

  Tim Callan

  Somewhere along the line, when it happens you won't know why, and it will take a while to figure it out. In the meantime, you are just gonna be down.

• 

  Jason Soroko

  No Tim. It's just amazing how long this topic has been around. The reason why we revisited it is to remind people that it's a really good idea for a narrower use case than we originally envisioned it for.