Root Causes 284: 90-day SSL Certificates Are on the Way

Episode Transcript

Lightly edited for flow and brevity.

• 

  Tim Callan

  What we want to talk about today is a very significant communication from the Google Chrome program and where you can find this is on the Moving Forward, Together page of the Chromium project. If you search on “Chromium Moving Forward, Together” it will be the first thing you’ll find. Ok. I won’t bother reading you this URL because it’s a little long. But this is a long document and in this document, the Chromium Project, Google Chrome, tries to telegraph their intentions for what they want with their browser and they put this out there where people can look forward and understand what they are doing and in particular, this is the the Root Store aspect of it where they want to say this is what’s coming, this is what we are working on so that the community has an opportunity to think about it, react to it, plan for it, start doing the work they are gonna need to do to support these new vision that Google has. They try to give us as much notice as they can so people have a chance to adjust.

  Recently, we saw an update – and I’m gonna focus on a little bit of this page with significantly more material than just what we will focus on – but there’s a section here called, Encouraging Modern Infrastructures and Agility. I’ll sample from that and read just a little of it, ok? There’s more to it than this, but here’s the passage:

  “In a future policy update or CA/Browser Forum Ballot Proposal, we intend to introduce:

  A reduction of TLS server authentication subscriber certificate maximum validity from 398 days to 90 days.”

  In other words, what Google has just said is they intend to reduce the maximum validity of an allowable public SSL certificate to 90 days.

• 

  Jason Soroko

  That’s a big deal, Tim.

• 

  Tim Callan

  That’s a big deal. Just to refresh everybody, right now we are at 398 days, which is 13 months and this is going down to 90 days which is not quite 3 months. So, in that sense, yes, that is a big deal.

• 

  Jason Soroko

  Folks, you know, I’m gonna put it into oversimplified laymen’s terms which is something I attempt to do on some of these podcasts and what you just heard Tim say was if you are managing your SSL certs off a spreadsheet, you know, which is kind of a set-it-and-forget-it thing back in the days of the 5-year certs and the 3-year certs and even 1-year certs. You say well that’s a year from now and whatever, I’ll just put it on maybe a little calendar, a little bell and it’ll go off and I’ll go back to my magical spreadsheet that looks at all my certs and then we are good to go. Well, now it’s gonna be 90 days, guys. 90 days.

• 

  Tim Callan

  Yep.

• 

  Jason Soroko

  I can’t imagine. Tim, you and I have had so many podcasts about outages because of expired certificates and it happens to the big guys and it happens to everybody and those outages are horrendously expensive and it’s because things are not tracked properly and those of you who are doing things in spreadsheets, well, God love ya, but you know you’ve made mistakes and you’ve had close calls. And that’s with 1-year, 2-year, 3-year certs and more. What’s gonna happen when it goes to 90 days because it’s coming. Tim just said to you, 90 day certs are coming.

• 

  Tim Callan

  Yeah. So let’s hit a little bit of the nuances of the language here and then I’d like to talk about the implications which you brought up, Jay, which I think are extremely important.

  So, the first thing, I’m gonna go back to the beginning where it says, “In a future policy update or CA/Browser Forum Ballot Proposal, we intend to introduce:” Now let me break that down because that actually is meaningful. A CA/Browser Forum Ballot Proposal. In other words, we will introduce a ballot that will say the maximum term will be 90 days. We will take that to the CA/Browser Forum. As a little reminder for those of you who haven’t been following the subtleties of all of these things or listening to every single episode of this podcast, there was a process whereby on two occasions ballots were introduced to the CA/Browser Forum to reduce the term of a certificate to one year of an SSL Certificate and both of those ballots were voted down by the majority of the CAs in the CA/Browser Forum. And even in the second ballot, if my memory serves, every single browser consumer voted for the ballot. The ballot still passed because the CAs declined to put it through. After which, Apple declared this was going into their root policy and it was a done deal. It was a fait de accompli, after which the CA/Browser Forum passed a ballot.

  So, you know, I think what Google is saying here if you read this wording is we are gonna put out a ballot, we are gonna give you guys a chance to vote for it. In the event you don’t vote for it, we are gonna make it into a root program requirement anyway and it won’t really matter if you voted for it. Right? That is reading between the lines. So this signals a couple things.

  One, it signals Google’s seriousness that they’ve made up their mind and this isn’t something they are toying with, it’s something that they’ve decided.

  The second thing that it signals is that they’d like to do it through a ballot. They’d like to follow the CA/Browser Forum and do it through a ballot and have a good community and have the community agree, but in the event that it doesn't they are willing to flex their muscles.

  The last thing that signals to me is that they don’t want this lingering for years. That is something that says we will move deliberately and reasonably rapidly to make this happen. I see all of three things built into that language there, and I think those are important takeaways for all of us to bear in mind.

• 

  Jason Soroko

  It’s very important. We’ve seen this before. You know, Tim, I hear you and I know where that’s coming from. We’ve seen this before. We’ve seen even, you know, Apple and some of the other big players from the browser vendor community who have not waited for CA/Browser Forum ballots to pass. Sometimes things just get forced through because the browsers want it.

  And as we said on previous podcasts, Tim, any one of the major browsers, if they make a rule change, typically a lot of them will follow suit and certainly who has to follow suit is the CAs because of the fact that no CA can survive not being completely compliant with every single browser. Every single percentile is just so important.

• 

  Tim Callan

  100% correct, Jay. On all of the above.

  Now the second subtlety that I want to point out is that – and I’m gonna go down to that latter passage that I read: It’s “a reduction of TLS server authentication subscriber certificate maximum validity from 398 days to 90 days.” And as I said – well, think about it this way. Hey, Jay? How many days in a year?

• 

  Jason Soroko

  365 most years.

• 

  Tim Callan

  And what’s 90 X 4?

• 

  Jason Soroko

  90 X 4 is about that.

• 

  Tim Callan

  It’s 360. So, this actually four of these is less than a year.

• 

  Jason Soroko

  Right.

• 

  Tim Callan

  So, one of the other interesting implications about this is if you use the full 90 days on your certificates, you are going to get date creep. Right? You can’t say I always do this on April 1 because you are gonna have outages. You are actually gonna have a date that moves over time. Which was interesting. I see why they picked 90 days. It’s a nice round number but it does have the consequence that you can’t say I’m gonna do this once a quarter or I’m gonna do this every three months because you actually need more than 90 days each quarter to get it done.

• 

  Jason Soroko

  It’s almost like lunar cycles versus calendar cycles.

• 

  Tim Callan

  Yes. Exactly. So, in addition to the fact that I have to do this four times a year, that also means I can’t say I’m doing it on the first of these four months. It’s even worse than that

• 

  Jason Soroko

  I’m just thinking about myself. I’m thinking about being the Linux administrator and just setting that little bell like I said earlier and setting the bell for the first of a month or the first of a third month. It’s not gonna work. It’s not gonna work in this case.

• 

  Tim Callan

  It’s not gonna work. Exactly. You can’t just do a simple alert here – unless you are doing it every 60 days. Then you can. Sure.

  The last thing I’m gonna point out about that is there’s no cushion. So, we went to 398 days was the old “one-year” cert. Why 398 days? Well, it’s actually 13 months if you picked the very longest month, a 31-day month and you add a Leap Year day. Right? That’s how you get to the 398 days. So it’s the very most days that a 13-month timespan can ever be under any circumstances. That’s why Apple picked that, right? We don’t see any cushion time in here at all. It’s not, eh, we want you to do it every 90 days so we are gonna give you 100 days. It’s 90 days. Which means that in reality, if you don’t want to take things down to the last day – which you don’t – you are probably replacing your cert every 80 days, every 75 days, you know, something materially less than that. And so, in reality, even this thing that I’ve been hearing people say: oh well, you only have to do it four times a year. No. You are really probably doing it five times a year

• 

  Jason Soroko

  This screams the need for machine automation, Tim. Every form of human remembering is out the door. This really does call for machine automation.

• 

  Tim Callan

  Absolutely. This is making the case for automation just so much stronger than it was. In fact, that’s even built into some of Google’s rationale. Let me read, again, from the same document a little bit of their rationale for this. “Reducing certificate lifetime encourages automation and the adoption of practices that will drive the ecosystem away from baroque, time-consuming, and error-prone issuance processes.” So that’s the point you made right there. They want to drive people toward automation. Now why do they want to drive people toward automation? Let’s keep going with this same passage. “These changes will allow for faster adoption of emerging security capabilities and best practices and promote the agility required to transition the ecosystem to quantum-resistant algorithms quickly.” It goes on but that’s a second passage that’s worth looking at.

  So, what are we looking to get out of this? We are trying to create crypto agility. Or what I like to call certificate agility is a form, a subset of crypto agility. And they are trying to do that. They are trying to say, we’ll be able to move the new cryptography in really quickly. A good example being quantum-resistant certs. Once it’s possible for me to issue a quantum-resistant cert, if there are certs out there that are lasting for a year then it’s gonna take an entire year to get those cycled out. But on the other hand, if all the certs are only 90 days in length, then, you know, not more than 90 days from today 100% of the certs are gonna be supportive or able to support the new algorithms. So, that’s the kind of thing that they are looking for and this isn’t new. This has been in the public discussion from multiple major browsers for years and years. The need for crypto agility, the need to remove error-prone and difficult manual processes, these are deemed to be essential to the ongoing health and security of the web PKI by multiple thinkers including, based on what I see here, the people who run the Google Root Store Program.

• 

  Jason Soroko

  Bravo. Look, you know, I could speak for different entities here but I’ll speak for myself mostly here. I think this is a good idea, Tim. I will tell you. I don’t know if any of you are old enough to remember - this is gonna show just how unbelievably old I am – but, Tim, you might recall an old British comedy, Yes Minister & Yes Prime Minister. That show. Nigel Hawthorne and all those great actors and there was a famous line that was used, a famous tagline in that show that basically Nigel Hawthorne’s character said, “well, that’s courageous.” And that always woke up the Prime Minister to be like, ok, that’s something you should be worried about. Whenever Nigel Hawthorne said, “that’s courageous.” And you might argue, right?

  Well, what Google is doing here – that’s courageous because there’s risk here and the risk is - - I mean I can see the pushback. Oh my God, we can’t go to 90-day certs because I don’t have a CLM.

• 

  Tim Callan

  Yeah. And I mean, yeah, I’m gonna make even a bolder statement. I think it is a 100% assured outcome that there will be an increase in certificate-related outages as soon as this rule rolls into effect.

• 

  Jason Soroko

  I agree.

• 

  Tim Callan

  90 days after this rule goes into effect. At that point, we will see a measurable uptick in these outages because there will be people who did not get on the automation train who tried to do it manually and some of them are gonna get it right but some of them are gonna get it wrong and that will occur and that will be measurable. I think that is guaranteed and I’m sure the folks over there put a lot of brain cycles into this, right? They are not doing these things casually and they are not doing these things without thinking them through. I think they know that, and I think they probably would say that that’s regrettable but it’s unavoidable because there’s no other way to bring about a world of crypto agility and automated certificate processes and that’s the world we just need to get to.

• 

  Jason Soroko

  Shorter certificate lifespans just make so much sense, Tim. We’ve covered that a number of times. Maybe we can even cover it again because there’s so much good that’ll come out of this wholesale change but I completely agree with you. There’s gonna be some pain in the meantime for those of you who don’t have a CLM in place or automation in general.

• 

  Tim Callan

  Let me point out that this also is related to another initiative from Chrome which is alternately to make OCSP not be a requirement anymore and really to encourage the use of very short-lived certs, 10 days or less, by removing the need for any of the revocation mechanisms under those circumstances. Once you imagine that vast numbers of certificates become automated and if they are automated anyway, then the difference between 90-day certs and 10-day certs may not be that important and at that point, why don’t we go down to 10-day certs or 5-day certs or 2-day certs. That’s the kind of direction that you see them trying to drive the world to get large numbers of certificates rolling on shorter and shorter lifespans over time because clearly, the folks at the Chrome program and a lot of other places believe that shorter-lived are just safer and better and more agile. You see both of those two initiatives kind of flying together working in tandem to drive that trend toward getting automated and getting shorter-lived certs.

• 

  Jason Soroko

  Tim, thanks so much for reading out what Google has said to the world. Everybody, you gotta listen up. I think that’s probably the clearest explanation anybody has ever given.

• 

  Tim Callan

  The last thing is when. I think the one other question that you’ll see here is when and I will note that they don’t say when but if you can indulge me, Jason, I want to walk through a little bit of math.

• 

  Jason Soroko

  Let’s hear it.

• 

  Tim Callan

  This is just me speculating. I don’t have any inside information on this. Maybe someone will tell me, well, actually, we want to do it this date and if so, I’ll report it. But, for now, this is what I’ve got.

  This has been announced. This is being telegraphed by Google early in the year of 2023. They’re not doing that because they don’t intend to take action in 2023. We already know because Google has told us that they are going to move forward with this OCSP ballot. This OCSP ballot is gonna be very important and one of the things it’s gonna do is put another motivator in place for automation and shorter-lived certificates. That ballot I predict will pass. I think that ballot will be very popular and after that passes, that will clear the slate for Google to move on to their next initiative. So, I am predicting that sometime this year – I’m gonna say summertime – we see a ballot that proposes 90-day certificates. Now that may not pass. However, we’ve also seen rhetoric from Google that suggests that in the event that that doesn’t pass that they will move forward with a pronouncement anyway.

  Let’s look at those two scenarios. Either it passes or it doesn’t. At the end of the day, the effective date probably doesn’t change in a meaningful way. Let’s just pretend it doesn’t pass. Summer of 2023, it fails. Google probably gives it a little bit of respectful time and then they probably come out and say, well, here’s a pronouncement and here is what we are gonna do. We are gonna make this a root store requirement and make it on this date.

  Now, they tend to make these pronouncements at the CA/Browser Forum face-to-faces. So, it will probably happen at the fall face-to-face. Usually, these guys give us about a year. Ok, you got a year. Get your systems worked out CAs because in a year this is gonna happen. So, if you assume that happens now we are looking at essentially Quarter 4 of 2024.

  So I’m gonna throw out a date of October 15. For me, the over/under on this is that October 15, 2024 there will be a mandatory requirement to move to 90 day certificates. And, I may be wrong but I just walked through the rationale of how I got there and if someone who is on the inside and really knows what the decision is gonna be and wants to state otherwise, obviously, we will trust them but until that occurs, that is my prediction for when this happens. I’m predicting that it’s very likely, damn near a fait de accompli that in that timeframe we will have that requirement.

• 

  Jason Soroko

  Bold predictions but that’s coming from the voice of experience and I think those are well thought out estimates.

  I think what the listener of this podcast needs to - - your takeaway is, guys, you are not far away from absolutely needing to automate your certificate lifecycle management. Period.

• 

  Tim Callan

  Right. So, let’s say it’s a year-long project to get yourself automated on all your SSL certificates. Ok. You can maybe drag your feet a little but you better get going on that project this year and you are even betting on my math. What if they are more aggressive than I said?

  And so, this really is the time to try to understand what is my profile on automation, what do I have to do to get there and what is preventing me from getting there? If you need to secure some budget, this is the time to be looking to secure that budget. If you need to reallocate some roadmap and some resources, this is the time to be looking at reallocating that roadmap and resources. Get going on it, guys. Google deliberately told us this ahead of time so we all had a chance to prepare. If we don’t prepare, that’s not on them. It’s on us.

• 

  Jason Soroko

  Tim, I think this is important enough that I thank you so much for bringing it up on this podcast, but I think we are gonna have to repeat this a few more podcasts coming up just so you guys get the message. This is super important. You know - - this is a huge change and Google is ripping the Band-Aid and therefore, we all have to jump.

• 

  Tim Callan

  100% and, you know, I do think that probably we will continue to unpack this as implications of it become clear and, obviously, we will also report on any news updates. So, for instance, sometime in the future if there is a ballot or there is a pronouncement or if there is a hard date, obviously, we will come back and tell you guys about it right away.

• 

  Jason Soroko

  Don’t be the company 90 days after you get your first 90-day cert are the people that Tim and I have to report on having outages. Please don’t be that - - if you are listening to this podcast, don’t be in the club of people who have an outage.

• 

  Tim Callan

  Big news and certainly something that I think everybody who uses public SSL certificates - and I mean everybody – should attend to.