Security Now 982 transcript

Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show

0:00:00 - Leo Laporte
It's time for security now. Steve Gibson is here coming up, a response from Entrust to getting yanked from Google and other Chrome and other browsers. We'll also talk about how you can protect yourself using port knocking from the OpenSSH flaw we talked about last week. The biggest story, though, is the big bullet the Internet dodged. It could have been a bad one. The story of Polyfillio. All that and more coming up next with Steve on Security. Now Podcasts you love.

0:00:39 - Steve Gibson
From people you trust.

0:00:41 - Leo Laporte
This is Twit. This is Twit. This is Security. Now with Steve Gibson, episode 982, recorded Tuesday, july 9th 2024. The Polyfillio attack. It's time for Time to record a show. Oh yes, the big red record button, please. We now have copies of the show on every streaming service, so the chance of losing a show is very low. It's time for Security Now, the show where we cover the latest security privacy news. Tell you a little bit about computers' work. Sometimes talk about sci-fi with this guy right here, mr Steve Gibson. What do you see? What do you know? It's time for the steve gibson show. How do you like it?

0:01:30 - Steve Gibson
we're thinking that no, yeah, maybe we should wait till year 20 before we deploy the corny jingle we'll never get to 999 if you keep that up later, okay, well, uh, this is the week I've been waiting for, because I have a photo that just had to be a picture of the week. It had to be given the label Windows Patch Tuesday. And so here we are second Tuesday of July, patch Tuesday. Don't know what the patches are going to be, but doesn't really matter, because this photo says it all.

Today's podcast is titled the Polyfillio attack and, oh baby, we, the internet, we the internet dodged another potentially devastating bullet. A really interesting story, though, about the backstory and history of this. But first we're going to look at, not surprisingly, entrust responded to Google's decision which was our topic last week, to refuse the trust of any of their newer TLS certificates, refuse the trust of any of their newer TLS certificates. We're going to look at their response and also at how the other CAs have responded to what they perceive as a new opportunity. And I was thinking, leo, about how you mentioned last week that somebody in the forum I guess in Discord or wherever you have the chat dialogue commented that Entrust was not a major, you know, not one of the top tier certificate providers.

0:03:11 - Leo Laporte
I don't remember if it was 1% or 0.1%, it was pretty low, but it was ninth on the list, so it wasn't.

0:03:17 - Steve Gibson
Well, yes, and I was going to say that, given the total number of certificates, even 1% is a lot of certificates, that of certificates that you'd like to suck up?

0:03:28 - Leo Laporte
Yeah?

0:03:28 - Steve Gibson
well, or that other CAs would love to take off of Entrust's hands. You don't need all those certificates. Let us handle those for you, because you seem unable to carry them all. Also, we're going to answer the question what's a passkey redaction attack and how worried should we be? And, speaking of passkeys, why not just have a website? One of our listeners asks hold as many of them as we need.

And also, regarding the OpenSSH attack, wouldn't adding port knocking in front of that serious OpenSSH flaw which we discussed, prevent this problem? And if so, what's the larger lesson there to be learned? And what about blocking NIP after some number of failed attempts or nailed I guess Number of failed, that would be nailed attempts. What about that? And, as I said, we're going to then wrap up with the Internet once again, dodging a potentially devastating bullet. But what's really interesting is how we just walked right into this thing. I mean, it's just like so obvious. Other people saw it coming, but nah, anyway, the good news is lessons to be learned, uh, and I think some great takeaways from this week's uh podcast, the polyfillio attack.

0:04:59 - Leo Laporte
I like, I like, whatever that is polyfill, just sounds, good sounds like something to hold your dentures in place. No, that's another. Actually, I think there is a product called polydent no, that's polydent, yes.

And I have written in the past. I have coded flood fills. That's kind of a fun little coding routine, but a polyfill I don't know what that is. We'll find out. Coding routine, but a polyfill I don't know what that is, we'll find out. First, though, let's take a little time out to take a look at one of our fine sponsors, in this case, a company you've heard us talk about many times Lookout.

Today, every company is a data company and today always right. Data is everything as soon as we went digital. That was it. Right. Data is everything as soon as we went digital. That was it.

But all your data means your company's at risk. Right data can be exfiltrated, breached, encrypted. There's cyber threats. There's. These are the new norm, and cyber criminals are getting better at it all the time.

At a time when no more boundaries to where you work and where your data lives exist, what it means for your data to be secure has really changed. But that's why you need Lookout. From the first phishing text to the final data grab, lookout stops modern breaches as swiftly as they unfold, whether on a device in the cloud, across networks, even working remotely, or your local coffee shop. Lookout gives you clear visibility into all your data, whether it's at rest or in motion. You can monitor, assess and protect without giving up in this important productivity for security With a single unified cloud platform. Lookout simplifies and strengthens reimagined security for the world that we'll be today. Visit Lookoutcom today to learn how to safeguard data, secure hybrid work and reduce IT complexity. It's easy to find Just go to Lookoutcom. We thank them so much for supporting security now and the fine work done by this fellow right here.

0:07:10 - Steve Gibson
I am prepared to show you the consequences of many patch tuesdays. Oh, one could argue, too many patch tuesdays oh this is just so good do you want to describe it?

0:07:21 - Leo Laporte
do you need to?

0:07:22 - Steve Gibson
describe it. What we appear to have, given the somewhat blurry background, is like an oil storage tank of some sort. It sort of looks like that a big barrel painted red got a big valve on it, but then, coming from it in the background, right up to crystal clarity in the foreground, is a pipe coming from the bottom of the. You know the output of this barrel, which then makes a right-hand turn and a heads off the screen, but not before we are treated to what you can only describe as something must have sprung a leak and the people were desperate to not lose any of the precious fluid that was contained in this tank. This is just. This is a, a patched pipe from hell. I mean it's got like shims and cardboard and black tape wrapped and then metal screw tightenable strangulators. I mean it's just wonderful.

0:08:31 - Leo Laporte
It ain't going anywhere, that's for sure.

0:08:35 - Steve Gibson
The moment I saw it I thought, oh, this is our Patch Tuesday picture of the week week. So, yeah, uh. So, for those who are listening, you'll uh if you have signed up to grc's obviously free, uh email system. Well, not, obviously.

It's good that you do make that for free is free, of course 5782 of our listeners, so we're approaching 6782, received the email from me a couple hours ago containing a little thumbnail size of this wonderful picture which, if you click on it, will reveal it in all its full-size glory. So wow, this is a great Patch Tuesday picture. So, wow, this is a great Patch Tuesday picture. Okay, and again a reminder that if you would like to get the summary of the show, a link to the show notes and also the picture of the week in your inbox every Tuesday morning as soon as I have them, the system is working well.

0:09:43 - Leo Laporte
You know, that number is actually pretty good. That's almost 10% of your audience is subscribed, so that's pretty good, good job.

0:09:57 - Steve Gibson
But that means there's still 9 out of 10 can subscribe.

0:09:59 - Leo Laporte
Well, and you know, realistically, that's what I'd expect.

0:10:01 - Steve Gibson
Yes, I'm sure that a lot of people are like, hey, I listened to it, why do I need text? You know I've already that a lot of people are like, hey, I listened to it, why do I need text? I've already got it buzzing into my ears, so I understand. Okay, so Entrust responds. We get a letter from El Presidente, followed by an interesting FAQ.

So Todd Wilkinson, the president and CEO of Entrust you can imagine this all got his attention the day after Google's announcement. Of course, we were right on top of it. The day of he wrote an open letter to the industry essentially titled Thoughts on the Google Chrome Announcement and Our Commitment to the Public TLS Certificate Business, and again by Todd Wilkinson would no longer include Entrust Root CA certificates in the Chrome Root program. Not technically accurate, but then you know, todd is the president, so we're not expecting complete technical accuracy from a president and CEO. But he's, you know, he's got the gist. As we know, the Ro roots are still staying in chrome. It's just that the signature date of anything signed by those roots is now being examined and if that date falls after this coming Halloween, which is to say the end of October, then they will not be trusted, but anyway Todd's got the idea. Then they will not be trusted, but anyway Todd's got the idea.

He says we are disappointed by this decision. One can imagine and want to share how we intend to move forward. He says we understand what led us here. We are committed to improvement and Entrust continues to have operational capabilities to serve our customers' public and private digital certificate needs. These capabilities extend beyond the issuing routes in question. Our recent mis-issuance incidents arose out of a misinterpretation we made of CA browser form compliance requirements. And let me just stop right there to note that. And when they found out that they had been misissuing certificates, they did not stop doing so. They instead started arguing with the CA browser forum, while continuing to issue misissued certificates and refusing to revoke any because they didn't want to upset their customers. Ok, anyway, he says, in our attempt to resolve this issue, our changes created additional non-security related misissuances. Boy wow.

0:13:01 - Leo Laporte
We started digging a hole and then we got tangled up there, todd, wow, we started digging a hole and then we dug it, get a little tangled up there, todd, even a little deeper.

0:13:08 - Steve Gibson
Yeah, throw that shovel away. In our attempt to provide additional flexibility to our customers, we provided extensions and delays in revocations that were not supported by the CA browser form requirements.

In other words, we weren't supposed to do that because you know there's no latitude here. Right, you've got to follow the rules which you voted on. Anyway. He says which mandate five-day revocation for all certificate misissuances. So you know he's covering part of this, but he's conveniently failing to mention that we didn't stop misissuing certificates, and that's also bad dog. So he says this create an environment in which the community scrutinized past and trust incidents. Whoops, we looked further back. How'd that go? He says this identified past and trust commitments which, if fully implemented, could have helped to prevent these incidents.

But of course we didn't fix those either back then. So whoops our bad. He says we agree that there are opportunities for us to improve. In other words, they agree with everybody else and we've completed a thorough assessment of our CA operation in the past few months. As a result of this assessment, we made changes in our organization Attempted to insert here which no one has seen yet, no, In our organization, processes and policies. For example, we've moved the CA product compliance team into our global compliance and operations teams to fully leverage the more robust capabilities of this larger organization. Well, thank goodness, Apparently, the people that were in the sidecar, Jeez.

0:15:10 - Leo Laporte
Louise, wow, we moved them. We moved them over, yeah.

0:15:15 - Steve Gibson
They weren't doing it right. So now we've moved the people that weren't doing it right in with the people who hopefully are Right and maybe they won't contaminate the rest of the group. He said we've instituted a cross-functional change control board, Because that's what you want in your change control boards, Leo, is some cross-functionality.

0:15:34 - Leo Laporte
Hereinafter, deck chairs will be rearranged by a committee of people in charge.

0:15:40 - Steve Gibson
And bolted down with a seatbelt and bolted down with a seatbelt. So they've instituted a cross-functional change control board whatever that is and a technical change review board to catch similar issues in the future. Boy, this all sounds great. We're accelerating R&D for TLS, certificate compliance and automation-related work, while also improving the tracking of our public commitments oh, where they're going to be tracking their public commitments? That's good, so they don't wander off course and revising our public incident response practices to ensure such issues do not occur again.

I'll talk about showing up after the party's over. He said I want to assure you that we're committed to continuing to serve as a public CA, even though we'll have no customers, and that we will complete open issues and promised improvements in a timely manner. We're working with Chrome and the other browser root programs to address the raised concerns, while also providing continuity for customers. Well, continuity, they had that because they were never getting any certificates revoked While we execute these changes. We have the expertise to do this, as demonstrated by our ability to deliver our many products and solutions designed to meet demanding global compliance requirements.

Wait, it was the demanding global compliance requirements that they were not meeting, which is what they've demonstrated over the years. But OK, finally, he says Entrust has been a public and this is true a publicly trusted CA for over two decades and has contributed to stronger web PKI capabilities globally. Again, that's true. We continue to have operational capabilities to serve customers' certificate needs today, and while he says we'll do so in the future, that remains to be seen because they'll have no customers. But he said, we respectfully ask for your patience as we work to ensure that you have no disruptions to the service you have come to expect from Entrust.

Okay well, so he said what he could, right. I mean, do you need it at this point? Is this where you need the president and CEO to say something? So he did. Of course, he failed to explain that the real concern was their history of like exactly this right Now. This is not the first time. This is like the 12th time the industry has heard this a history of repeatedly broken promises to repair what's broken and then excuses after the fact that you know, oh like why it was that they weren't these changes weren't made and attempt to justify their actions. So okay, and of course, they keep putting their customers' needs in front of their duties as a CA, which was the beautiful conclusion that one of the techies came to last week. So, anyway, I found also their four question FAQ regarding this incident to be interesting, especially number three, but here's all four. So this is what they asked themselves and then answered Question what does it mean when Google says you've been distrusted?

Ouch, they say. This means that public TLS certificates issued from nine Entrust routes after October 31st 2024, will no longer be trusted by Google Chrome by default. Tls certificates issued by those Entrust routes through October 31, 2024, will still be trusted by default through their expiration date. Got that one, nailed it, okay. Question number two Other CAs have said that after November 1, your TLS certificates won't be valid in the Google Chrome browser. Is that true? No, that's not true. All Entrust TLS certificates issued through October 31st 2024, like we just told you, will be trusted by default by Google through their expiration date. Be trusted by default by Google through their expiration date. After October 31st, we will have the operational capabilities to serve customers' certificate needs. But I don't think they're saying but not issue any new ones? Oh no, with alternative or even partner routes if necessary, or even partner routes if necessary. Our goal is to ensure that customers have no disruptions and continue to receive global support. Certificate lifestyle management oh, sorry, life cycle management. Being nice to have some lifestyle management and expert services to meet their digital certificate needs.

Question three does Google's decision mean Entrust is out of the digital certificate business? Oh, by no means. No, no, it means the public TLS certificates issued from nine Entrust routes after October 31st 2024, will no longer be trusted in Chrome's browsers by default 31st 2024, will no longer be trusted in Chrome's browsers by default. Google's decision has no impact on our private certificate offerings. Well, private certificate, not public Right, including our PKI, pki as a service and managed PKI, nor our code signing, digital signing and email SMIME and VMC offerings.

Now, these are the things that I was referring to last week as the second order casualties of being forced away from the public certificate business. It's just like well, you know you don't seem able to make simple web certificates. Should we trust you to do all this other stuff that's you know, in question? They finished answer three saying we have the operational capabilities to serve customer certificate needs now and in the future. And here they're repeating from question two, with alternative or partner routes if necessary. Our goal is to ensure that customers have no disruptions and continue to receive global support, certificate lifestyle life cycle management and expert services to meet their digital certificate needs. And finally, fourth question does this impact other Entrust solutions? Answer this announcement has no impact on the broad range of Entrust identity-centric solutions, of course, other than whether anyone is going to want them.

Okay, so, as I said, I thought their most interesting response was three, noting that even after Google begins to conditionally mistrust certificates signed by any of their root certs, nothing prevents them and this is true from riding on someone else's coattails. You know nothing that is other than Google releasing another update to Chrome which then also blocks that Right. Remember that Entrust originally bootstrapped themselves into the commercial SSL TLS browser certificate business by having Thought sign their intermediate certificate, which is how they were then able to sell end certificates and that allowed them to ride the coattails of the trust that thought had earned. So this leaves two questions Would another CA think it was worth their while to sign an intermediate certificate for Entrust, knowing that their own reputation would then become entangled with Entrust's?

And if this were done, what would Google do? Would they move to also block Entrust's intermediate certificates? And actually they could probably do that preemptively in Chrome. That is, they could look's intermediate certificates. And actually they could probably do that preemptively in Chrome. That is, they could look at intermediate certificates, see if the signer is Entrust and essentially make the block a little stronger than just blocking certificates signed by the routes directly. You know you could do a second order also. So we're going to see what Halloween brings this year and I imagine behind the scenes Entrust is probably begging to be allowed to use these intervening four months They've got July, august, september, october to fix and demonstrate fixes and not have that update to Chrome on November 1st make this happen.

0:25:12 - Leo Laporte
I wonder how much they care. I mean, they really are in a lot of different. They have fingers in many different pies. I don't know how big a part of their business this is. Todd came to the company from DataCard which acquired Entrust, and DataCard's business was printing credit cards and the mag stripes on the back first credit card imprinting like the raised lettering on it and then the mag strips on the back. So I don't think he has deep roots in the CA community. Clearly you know it's just a management issue at this point and maybe they don't even care about that business that much it's kind of amazing.

0:25:51 - Steve Gibson
You know, somebody wrote a nice letter for him. Yeah, so they put his name on.

0:25:58 - Leo Laporte
There's a big company. They have 2,500 employees. They're a giant company, yeah.

0:26:03 - Steve Gibson
They've been around since 50 years. Yeah, company, yeah, they've been around since 50 years. Yeah, yes, yeah, and, as we noted, they were acquired by private equity Right, so don't know that that had any effect, but it happened. Repeating certificate authorities were not slow to report themselves on the news of Entrust's fall from grace and to offer to provide their services in replacement. Sadly, not all CAs did this in a conscientious fashion. For example, the CA Sectigo misstated the future. For example, the CA Sectigo misstated the future. Now, remember that Sectigo is not one of this podcast's favorite CAs. In fact, they would likely be the last CA I would ever choose.

Long-time listeners may recall from whose loins Sectigo sprung. That would be none other than Komodo. Essentially, komodo so badly damaged their own reputation due to a series of famous major certificate authority screw-ups that they decided to rebrand themselves in the apparent hope that no one would notice. In the apparent hope that no one would notice. Remember that first spyware that I found. That was the. It was. I'm trying to remember whether it was. I know that they were Oriate, but they renamed themselves Radiate or something you know. They again got so they were. Their reputation was so hit by the discovery that they'd snuck into everyone's pcs that they said, oh, I think maybe this time to change our name, yeah, that's right.

So okay, in any event, komodo slash sectigo uh, gave their rapacious posting the title quote Google to distrust and trust SSL TLS certificates what this means for the industry. And then they tried to tell us they said they wrote in a significant move to enhance digital certificate security, google has announced its decision to distrust all public SSL certificates issued by Entrust, effective after October 31st 2024. Okay, they got that part right. This announcement has sent not just ripples but waves through the industry, particularly among Entrust customers, who now face the urgent task of transitioning to new certificate authorities. Okay, except there's nothing whatsoever urgent about anything here. You know, as we are all very well aware, all currently issued certificates remain valid through their current lifetime and everyone has July, august, september and October to choose another CA when it comes time. I mean it could be a year from this coming October when it comes time to replace any Entrust SSL TLS browser certificate. So not so urgent there, sectigo. Then they said under the heading the catalyst for distrust, google's decision is rooted in a series of compliance failures by Entrust. Uh-huh, look who's talking.

Over the past several months, entrust has experienced significant issues, including extremely delayed revocations and multiple lapses in meeting established security standards. Google's security blog noted quote over the past six years we've observed a pattern of compliance failures, unmet improvement commitments and the absence of tangible, measurable progress in response to publicly disclosed incident reports. Yeah, rub their face in it. This lack of progress and ongoing issues justified the revocation of trust in Entrust's public roots, the revocation of trust in Entrust's public routes. To be a trusted browser, a CA must comply with specific requirements defined by the CA Browser Forum, and I guess they would be an authority on that. Transparency is crucial, as CAs are expected to work in good faith with browsers to fix and prevent issues. Recent root program audits indicated a lack of confidence in Entrust's TLS certificate issuance practices, so this news wasn't completely unexpected to the industry and prompted Google's decision to distrust Entrust certificates in the Chrome browser.

Implications for business For businesses using Entrust certificates this development necessitates immediate action. Okay, no, it deliberately doesn't. But okay, any website using an Entrust certificate issued starting November 1st will be treated as an unsecured site on Google Chrome, and likely other major browsers will follow suit. Companies must source a new certificate authority before the deadline Again, no, to avoid their websites being flagged as untrusted Again. Just you know, no companies must source a new certificate authority before their present Entra certificate dies a natural death by having reached its expiration date.

Then they finally they finish with choosing a reputable certificate authority. They said, considering Entrust's failings and, of course, our renaming, businesses must reassess their relationships with CAs. A reputable CA should demonstrate robust compliance with industry standards, transparent operations and a proven track record of security and reliability. Companies like Sectigo, which offers comprehensive certificate life cycle management solutions, present viable alternatives. Now, okay, as I said, these guys would not be my first choice, but we know who my first choice would be. That would be none other than DigiCert. So I went over to see what DigiCert had to say about this. Again, this end of end trust. Trust does represent a significant marketing opportunity for every other CA in good standing, opportunity for every other CA in good standing. Now, rather than going point by point through DigiCert's similar marketing piece, I'll just say that they did the honorable thing, as I was hoping they would.

Under their topic what does that mean for Entrust customers? They had three bullet points. First, public TLS certificates issued off of Entrust roots, whose earliest SCT that's, the signed certificate timestamp is after October 31st 2024, will no longer be valid on Google Chrome 100% correct. Second, those Entrust certificates will be treated as an unsecured site, right again. And third, any TLS certificate with a signed certificate timestamp dated before October 31, 2024 will be valid for its term. Okay now, with a slight exception that they meant before November 1st rather than before October 31st. Digicert got it exactly right Valid for its term. And then they finished with.

Of course, we're here to help. We understand this incident poses significant risk of business disruption to a large number of organizations. As the world leader in globally trusted PKI and TLS SSL solutions, we're committed to making our services and solutions available to help you maintain critical operations and ensure uninterrupted business continuity during the transition from Entrust and beyond. As it happens, grc's DigiCert certificate expires at the end of this month and, thanks to their excellent service, I already have its replacement standing by, which I'll be switching all of GRC's servers over to shortly. So anyway, I guess we will see in the fullness of time what transpires.

Time what transpires, and I guess I'm most curious of all out of all of this, to see whether the decision is reversible, whether I mean and I don't know how it would be really, because all they can do is make more promises, but their promises are only good when measured against practice, and that takes time, and they've been given time, time and time again and have not come into compliance. So, in order to have no disruption, google would have to become convinced that this time they really mean it, when they sounded like they really meant it all the other times too, and nothing changed. So, you know, are these one-way decisions that never change, decisions that never change, and I guess this is the point Once they're out of the business, they don't have the capability of proving that they should be in the business because they're out of the business. So interesting, leo, we're 30 minutes in. Let's take a break. Let's do it, and then I'm going to talk about something new Pass key redaction attacks. You don't want your pass key to be redacted.

0:36:22 - Leo Laporte
I don't even know what that means. It means covering it up with a black magic marker.

0:36:27 - Steve Gibson
An interesting guy coined the term Okay and he likes to refer to himself in the third person, sort of like Trump, so yeah, Well, this Leo Laporte is going to do an ad right now, and we'll find out what that. Steve gibson and steve will continue with this podcast after this brief commercial announcement from vanta.

0:36:51 - Leo Laporte
Whether you're starting or scaling your company's security program, demonstrating top-notch security practices and establishing trust is more important than ever. With you, know regulatory authorities, with your customers, with your partners, vanta makes it easy. They automate compliance for SOC 2, iso 27001, and more, saving you time and money and helping you build customer trust. Plus, you can streamline security reviews by automating questionnaires and demonstrating your security posture with a customer-facing trust center. All of this powered by Vanta AI. You know, it's not a surprise. 7,000 global companies, big ones you've heard of Atlassian, quora, flow Health use Vanta to manage risk and prove security in real time. Get $1,000 off Vanta right now. Wow. When you go to vantacom slash security now V-A-N-T-A. Vantacom slash security nowanta. Vanta dot com slash security now one thousand dollars off. I love their slogan compliance. That doesn't suck too much. Vanta, vanta dot com, slash security now. All right, steve, let's hear about this okay action?

0:38:12 - Steve Gibson
thing? Yeah, a number of our listeners.

So, yeah, a number of our listeners picked up on a weird story that was featured on the Dark Reading site. Dark Reading is a good site, but maybe they were starved for news last week. The story's headline was Passkey Redaction Attacks Subvert GitHub and Microsoft's authentication. Okay now, naturally that raised my eyebrows too. The tagline beneath the headline was adversary. In the middle, attacks can strip out the passkey option from login pages that users see, leaving targets with only authentication choices that force them to give up their credentials. Okay now. So first I just need to pause here for an old fart. Get off my LAN. Get off my LAN. Comment.

0:39:10 - Leo Laporte
LAN is better Get off my LAN.

0:39:12 - Steve Gibson
I like it. Get off my LAN. I got my shotgun ready. It's what they're calling an adversary in the middle attack. Okay, so it appears that the politically correct PC police have decided that the use of the traditional and time-honored term man in the middle is no longer proper, as if man in the middle was actually a reference to some man somewhere, and that, as a consequence, all men should now be taking offense at this slur to our gender. Well, I am a man and I'm not the least bit offended, because I, and I'm sure everyone else, have always understood that the use of the term man in man in the middle was always meant as an abstraction. So if any of our listeners were also rolling their eyes at the use of this new replacement term adversary in the middle. Anyway, at least know that you are not alone.

0:40:11 - Leo Laporte
Well, you're not going to like this, but I understand. The evil maid attack has been rebranded the evil sanitation worker attack. So I think that really there's just. The world has changed. The world has changed.

0:40:25 - Steve Gibson
There's just no hope. I don't know, leo, maybe 999 should be it, because I'm just kidding everybody, no 999 will not be it.

0:40:33 - Leo Laporte
Don't old man, don't give up. Give up, yet it's not too late, I can adjust.

0:40:37 - Steve Gibson
Okay, I can adjust, although I'm sticking with man in the middle. Okay, you know Okay.

0:40:46 - Leo Laporte
Okay, so that's out of my system.

0:40:47 - Steve Gibson
Yes, and I'm completely distracted from the article M-I-T-M. That's what you call it, m-i-t-m yes, dark reading refers to a piece from a site I've never encountered before called E-Sentire, e-s-e-n-t-i-r-e. I don't know why it's called that. I think all the good names were taken. So everything about this is a bit odd, since the article was written by a guy named Joe Stewart and in the introduction which he wrote he refers to himself, as I mentioned before, in the third person. Okay, I'm only going to share the first bit in his introduction because I'll take it from there, but Joe writes of himself. He said in the past year, the uptake of passkeys has surged, with industry giants such as Apple, microsoft and Google championing their adoption.

Unit TRU has been reviewing many of the leading software providers' implementations of PassKey technology and their current authentication process.

Regrettably, stewart found that cybercriminals can still break into accounts protected by PassKey technology on many online platforms, such as banking, e-commerce, social media, website domain name administration, software development platforms and cloud accounts, by using adversary-in-the-middle AITM phishing attacks.

Stewart explains in this and I guess he should know, since he wrote it blog. I guess he should know, since he wrote it blog how pass keys are designed to work, how threat actors could currently circumvent pass key security and the steps that online software providers and websites that use or intend to use pass key technology must take to ensure that their pass key implementation is secure from threat actors. Okay now, as I said, I read through Joe's article looking for this new problem that he had discovered, and there isn't one that I could find. All of this boils down to if someone goes to a phishing site that's pretending to be the authentic site, the miracle that is passkeys won't protect you because the fake site won't offer passkey authentication authentication. Instead, it will offer any of the traditional authentication fallbacks that are still completely prone to interception by phishing see, this shows you how good pass keys are right.

0:43:47 - Leo Laporte
In a way, this is proof. I mean yeah right, pass keys work.

0:43:52 - Steve Gibson
Yes, otherwise they wouldn't turn them off and it's unclear who would have ever believed otherwise. But Joe wants to be sure that everyone is on the same page with this. So you know for what it's worth. We have a new term, the passkey redaction attack, which, when you think about it, that's what you'd call this. You know it's a subset of phishing, since the FIDO2 WebAuthn PassKeys technology is immune from phishing. The phishing site simply removes the PassKeys login option and gets you the old-fashioned way.

0:44:34 - Leo Laporte
They probably don't want to use a YubiKey to log in either.

0:44:38 - Steve Gibson
Oh no, I can't do any of that. You just got to say give me your password, buddy, what's my pet's first name? Or what's my porn? Forget the password.

0:44:47 - Leo Laporte
Whatever it is. What are your secret questions? That's what you need. That's right.

0:44:54 - Steve Gibson
Wow Okay.

So those those, wow Okay. So those of our listeners who were worried. You know, basically, if you go to a fishing site, folks, you're in deep doo-doo, right. I mean, you clicked on a link in email, you're at a bogus site and nothing is going to help you. So don't do that and nothing is going to help you, so don't do that.

Ahmad Kayat from Riyadh, saudi Arabia, said Hello Steve, instead of synchronizing passkeys, isn't it more secure to have a passkey per device locked into that device's TPM or equivalent facility? Instead of backing up passkeys, have backup passkeys on additional devices. Moreover, it's provably more feasible to convince I'm sorry, probably more feasible to convince sites to support multiple passkeys per user than to convince Google, apple and Microsoft to support passkey portability. I completely agree with that latter. The big problem here is that there's no way to know which sites support multiple passkeys and which don't. You know. You're just going to try to associate another passkey with a site from a different device and the device says sorry, you already got one, use that one, but you can't because it's on a device you're not using right now.

The passkeys spec states that passkey supporting sites should provide many to one PassKeys to account mappings but as we know today, not all do, and it only takes one to ruin your lunch and running into a site. That doesn't means that the user cannot add another device to that site, which is a breakdown of the PassKeys promise. Hopefully this will eventually change. But it's also true that having Apple, google and Microsoft performing their own cross-device synchronization of passkeys, just as they've always done for passwords, takes the pressure off of sites to improve their passkeys implementations, because right works for everybody using apple, google and microsoft. What's wrong with you?

So for the time being, the only practical solution is to either have that be your, your complete and total solution, remain within one of the closed ecosystems which is provided by the big three, or use a third-party password manager, such as 1Password or Bitwarden, both sponsors of the Twit network, which will provide the kind of cross-platform compatibility that PassKeys was intended to provide, but doesn't yet universally.

And PassKeys was intended to provide it the right way, by having sites provide a many-to-one PassKeys-to-account mapping, then giving the user a user interface where they could see all the pass keys which are currently registered on their account and administrate them. You know, say yes and no to various pass keys. You know like, remove pass keys for devices they're no longer using or don't want, or have given a device to a family member but they shouldn't have access to the family's banking site because you know they're not old enough yet and so forth. Anyway, we don't have that, unfortunately. We can hope that we get that moving forward. Max finally said I'm guessing oh, I love this that because I was pronouncing it nouveau mailer.

0:49:00 - Leo Laporte
He says I'm guessing that n-u-e-v-o m-a-i-l-e-r is well, he was saying nuevo, right um as in spanish, for new, for new, yes, nuevo mailers, yeah.

0:49:17 - Steve Gibson
so now that he says it and now that you've said it, leo, and now that I've heard it, I'm sure that's the case Nuevo Mailer Nuevo. So, thank you, max. And I did get a nice email back from Panos, nuevo Mailer's author, saying that he'd heard from some of our listeners. So, believe me, as I said, I gave it an absolutely zero restraint endorsement, because this thing I'm just. You know, when you send out email to thousands of people in 30 minutes, it's a puckering sort of experience and I keep expecting something to go wrong, but it just works. It's great. So, nuevo mailer, and now I know how to even pronounce it, so okay. So, uh, john, who's? Uh? His twitter handle is psy, and I did get actually this from Twitter this week. He said hey, steve, listening to you two discussing the OpenSSH bug in SecurityNow 981. Last week he said I note that a port knocking setup would completely mitigate this front door camping. Love listening to you guys and excited for 999 and beyond. On a side note, while I am all signed up for email, I cannot see what address to send SN feedback on. Oh, so he fell back to Twitter because he didn't know how to email me. I'll get to that in a second.

So John's correct that port knocking would prevent exploitation of the OpenSSH flaw and I suppose if one were really a belt and suspenders type, then adding that additional layer of pre-authentication authentication would make sense. But the problem is open SSH is supposed to be super secure and the only protection anyone needs all by itself. Leo, you and I both authenticate our open SSH instances with passwords and certificates. A certificate is like a 2048 bit secret key which cannot possibly be brute-forced. Then a password is added to that just to keep anyone who might obtain local access to the certificate from using it. So my point is open ssh already provides so much security that anyone could be forgiven for not deploying an additional outer layer of protection.

But having said that, there is a larger, more general lesson to be learned here and I'm glad John asked. And that's the inherent problem with the security of any monoculture. Relying entirely upon security from any single source is never going to be as safe as using multiple different sources of security. Or, stated another way, using a layered security solution is able to provide the strongest security by far. And if you're going to go to the trouble of layering, it's even better when the various layers have nothing in common with one another. So John's observation that adding a layer of port knocking security is exactly that, and the advantage of port knocking is that it does not require a fixed IP for the knocker. The whole point of knocking is that the secret knock identifies the IP of the knocker dynamically, whose traffic is then allowed to progress into the next layer of security.

I've often noted that I have all of the links between my various sites protected not only with their own native security, whatever that may be, and about which each of their vendors will boast at great length, but also with point-to-point IP address filtering, so that none of those links are even visible to anyone else out on the Internet. Their own protection may be strong, but why take an unnecessary risk? Of course, ip filtering is only feasible when the IP addresses of the endpoints are relatively static, but there really is no stronger security than adding a layer of simple IP address filtering to internet traffic whenever and wherever possible. If anyone is listening to this who has publicly accessible endpoints which have fixed IPs, where IP-based filtering is not currently in place but could be, allow me to urge you to consider preventing anyone else from even being able to see those ports. I mean it's almost magic. It's really worth it, the best protection there is Now.

John also mentioned that he signed up for our email system but didn't see his inbound email, our inbound email address, listed anywhere. That's mostly by design, to tell the truth, because I would prefer to only give SecurityNow listeners direct access to this mailbox. The address is shown on GRC's old feedbackhtm page and since many others have mentioned this, I've just changed the reply to address for the mailings to the Security Now list to that correct address, which is securitynowatgrccom. So everyone who has already received today's podcast email all 5,782 of you containing this show's summary, the picture of the week, the full show notes link, will see that it was sent from securitynowatgrccom. So simply hitting reply to any of that Security Now email will properly address a reply directly to me.

And speaking of heterogeneous layers of security, Dr Brian of London wrote fail to ban F-A-I-L numeral 2 B-A-N. He said I can't believe any serious sysadmin is not running this and he provides a link to its GitHub page githubcom slash. Fail to ban slash. Fail to ban F-A-I-L numeral 2 B-A-N. Twice. He said that completely negates this SSH vulnerability, as far as I can tell. And as far as he can tell, is exactly correct. After being installed into any Linux system, failed to ban creates a daemon running in the background.

The project describes itself by writing. The project describes itself by writing. It said Failed to ban. Scans log files like var log, authlog, and bans IP addresses conducting too many failed login attempts. It does this by updating system firewall rules to reject new connections from those IP addresses for a configurable amount of time. Fail to ban comes out of the box ready to read many standard log files, such as those for SSHD and Apache, and is easily configured to read any log file of your choosing for any error you wish.

So cool solution. It provides another layer of security based upon dynamic IP address filtering, and it's another layer that any mature security solution ought to have. Why would any service allow any single remote IP address to be pounding away with repeated authentication failures? It's just nuts, but sadly that's still not built in. So this sort of protection you know it wouldn't block a widely distributed attack where attempts are coming from a huge botnet, for example, with each bot having a different IP. But in this instance, the exact timing required to exploit this recent SSH flaw means that it would be very difficult to launch such an attack through a widely distributed network of attackers. So again, failed to ban. If you've got publicly accessible important services, that can be intolerant and should be utterly intolerant of many failed login attempts and you're running Linux or a compatible OS for fail to ban. It's free, it's open source. I can't imagine what the downside would be.

0:59:07 - Leo Laporte
What a good idea. I mean, I've a lot of services have that turned on. My Synology, I have that turned on. So after 10 failed attempts, it's like yeah, like yeah, no, you're not. You're not getting in here ever again. Uh, I also block china and russia, which is probably a good idea, since I don't think I'll be logging in from there anytime soon no, and in fact I have the same thing on my email server, right?

0:59:30 - Steve Gibson
if because they're just, you know the, they're not spamming valid addresses, they're just going, they're doing a dictionary attack.

0:59:39 - Leo Laporte
First names.

0:59:40 - Steve Gibson
A through Z, yeah, yes, and so after a server tries I think it's five times with bogus accounts, I just put them on a blacklist for I don't know some length of time and just say look, I'm just not even going to entertain your guesses anymore, because they could guess right at some point and I'd rather not have that crap for them.

1:00:03 - Leo Laporte
It's a simple and effective defense.

1:00:07 - Steve Gibson
Let's take another break, and then we're going to talk about the polyfillio attack.

1:00:13 - Leo Laporte
And oh, baby, it sounds. Now that I think about it, I think it's a pokemon. Is what is what it is? You don't know what pokemon are probably of course I know pokemon.

1:00:26 - Steve Gibson
I have several in the refrigerator, oh uh are they tasty?

1:00:33 - Leo Laporte
there is a. There is a pokemon with that name actually.

1:00:36 - Steve Gibson
I was always the uncle who was able to get for his nieces and nephews the unavailable Christmas present.

1:00:48 - Leo Laporte
So you do know a little bit about all this. Oh yeah, that's a good uncle. This portion of the show brought to you by Bitwarden. You know, and we've talked about this. You know, I'm of the opinion. If you're Bitwarden We've talked about this I'm of the opinion.

If you're going to use encryption, it should be open source. It just seems obvious to me, because you don't want a backdoor in there. You want to make sure that other eyes maybe not yours, but eyes that know what they're looking at are looking at it. I think the same holds for password manager, because what is the most fundamental bit of the password manager? It's the thing that encrypts your vault. It also helps to have a good random number generator. I would recommend that as well. As we learned last week, bitwarden does it all. They are the number one open source password manager that offers services for individuals, businesses, teams, families. It's a cost-effective solution that I think you know, as a listener to this show, can dramatically improve your chances of staying safe online. I bet it would be your secret shame, but I bet there are a few listeners to this show who do not use a password manager, who are making up passwords as they go or, worse, reusing them. You got to stop. You got to stop. And maybe you're saying, oh well, as soon as passkeys becomes common, then I'll be doing it. Well, good news Bitwarden fully supports passkeys. In fact, they just added full support for passIs on browser extensions and mobile devices, so they've had it for a long time in the desktop app.

I love logging in with PASCIs in Bitwarden. When I go to githubcom, I click the button that says yeah, I'll use my PASCI. Press a button, that's it. No, the simplest login ever. But, as we know, it is secure. Pascis available on Bitwarden's mobile app on iOS. It's an open beta on Android. Stay tuned for more.

We did mention that a couple of months ago was World Password Day and Bitwarden did a survey of users from the US, uk, australia, france, germany and Japan. Actually, I don't know if it was users, because they were asking about their current password practices, and I know it wasn't users because 31% of the US respondents almost a third said they reuse passwords across sites. We should do this survey. It'd be very interesting to hear how Security Now listeners do it. But in this survey, 42% of the US respondents incorporate personal information in their passwords your birth date, mother's maiden name.

That's not good, not good, not good at all. 58% of respondents rely on their memory. No, 34% use pen and paper for password management. Well, that's fine at home, but this is 34% of do it at work, which means if you open the drawer, the desk, right there there's the pad, or, worse, maybe it's as it was in war games, underneath the blotter. Oh, look, here's the passwords. The password is 1138. Look at that. Or, worst case of all, it's a post-it note on the monitor.

No, a quarter of the people Bitwarden talked to admitted their workplace security habits are risky 45% no, they store passwords in security. 44% no, know, they use weak credentials. Is this your business? There is some room for improvement.

As they say, bitwarden is the way to go. It empowers enterprises, developers and yes, individuals to store and share sensitive data safely. In fact, if you've got a family member that's not using a password manager, tell them about Bitwarden, because it's free for individuals. Free forever, unlimited passwords, un devices, passkey support too. So there's no excuse when they say, well, I don't want to spend the money, no, it's free, you don't have to spend any money. It's also a really good, affordable choice for businesses and very, very secure with their transparent, open source approach to password management, bitwarden makes it easy for users to extend robust security practices to all their online experiences. If you're a developer, you will love the Bitwarden feature that allows you to store secrets you know API keys and like your Amazon secrets right in the password manager and protect them so that they're not accidentally uploaded to GitHub.

Has that ever happened to you? It's happened to me, it's happened to everybody and it's one of the big sources of breaches Bad guys know. I just scan through the repositories and GitHub. Let's see what we can find. Get Bitwarden. Stops that cold. Get started with a free trial of Bitwarden's teams or enterprise plans or, as I said, free across all devices and as an individual user. This is the password manager for you. I love it. Bitwardencom slash twit B-I-T-W-A-R-D-E-N. Bitwardencom slash twit. We thank them so much for supporting security. Now they believe in security, that's for sure. All right, steve. Continue on, soldier on my friend.

1:06:06 - Steve Gibson
This is big Uh-oh. We recently examined the consequences of the removal of a cloud-hosted service whose DNS CNAME record continued to point to the missing service. This then allowed Nerda Wells to establish their own replacement cloud service under the abandoned domain name as a means of hosting their malicious services within someone else's parent domain. That's not good, but bad as that is, it's not the worst thing that can happen. Arguably the worst thing that could happen would be for a domain which is hosting a highly popular, often downloaded code library to fall into nefarious hands. That would be awful. It would be awful and I would not be talking about this this week if it had not just happened, oh boy. So first I'll back up a bit and explain what's going on here.

An unhealthy design pattern that's developed is websites hosting pages which dynamically pull significant chunks of code from third-party sites. Back in the good old days, a website provided all of the pieces of the web page that it would present to its visitors. Naturally, advertising ended that with ads being filled in by third-party advertising servers on the fly, and though we've never had the occasion to talk about it directly, the same thing has become commonplace with code libraries. A perfect example came to mind, with GRC's own Zenforo web forums. With GRC's own Zenforo web forums. Zenforo mixes a bunch of different technologies, with PHP being its primary implementation language on the server side, but the forum's browser side polish things like fading button highlights. Some fancy animations, pop-up, floating dialogues and so forth are provided courtesy of JavaScript running in the user's browser. And, rather than reinventing the wheel, zenforo, like most websites, takes advantage of the wildly popular jQuery library.

Wikipedia explains. They said jQuery is a JavaScript library designed to simplify HTML DOM. You know document object model, tree traversal and manipulation, as well as event handling, css animations and AJAX. You know AJAX being the browser's web page code itself making queries to other resources, typically back to the server, as the user does things on the site. Wikipedia said it is of jQuery. It is free, open source software using the permissive MIT license. Open source software using the permissive MIT license as of August 2022, get this.

Jquery is used by 77% of the 10 million most popular websites, so just shy of four out of every five of the top 10 million websites are using jQuery. Web analysis indicates that it is the most widely deployed JavaScript library by a large margin, having at least three to four times more usage than any other JavaScript library. Okay, so jQuery has become so widely and universally, the benefits of any bugs that are fixed, speed improvements or other optimizations as soon as they become available. So rather than hosting the jQuery library yourself, you instead ask the browser to download the latest version of jQuery from the Internet. The question is where should it be obtained? I've included a screenshot of one of the configuration dialogues for the Zenforo forum system. The option is labeled jQuery source and it has four radio buttons, so we get to pick one from among the four. The choices are locally hosted, meaning that the server you're visiting provides the jQuery library itself Microsoft CDN at ASPNETCDNcom, jquery CDN at jQuerycom or GoogleAjaxAPICDN at GoogleAPIscom, and the default from the Zenforo folks was the final choice to obtain the library on the fly every time it's needed, which is all the time, like every page, directly from Google.

Now we might wonder why Microsoft and Google are interested in being benefactors by providing access to code libraries on their content delivery networks. But remember 77% of the top 10 million websites are all using jQuery. In a world where third-party cookies are able to track their users every time, any web browser anywhere fetches a copy of the jQuery code library from Microsoft or Google, that browser's cookie for that CDN is sent along with the site the user's browser is visiting and the user's current IP address, because that's where it's connecting from. So Microsoft or Google will obtain some information to add to the pile they already have about every users whereabouts and when abouts. In return, our browsers probably don't need to wait very long to receive that library, since both of those CDNs are extremely high performance. Okay, so the main point here is that a model has gradually evolved over time where websites we visit are no longer providing all of the content that runs in their pages and that, in addition to ads and Gravitars and web fonts and a bunch of other stuff and Gravitars and web fonts and a bunch of other stuff today's web pages also contain significant code libraries from third-party servers.

At the beginning of our discussion of this, I noted that arguably the worst thing that could happen would be for a domain which is hosting a highly popular and often downloaded code library to fall into nevarious hands, and I would not be talking about this if it hadn't happened. Armed now with this bit of background, we can easily understand the consequences of this and of how serious it could be. So what happened? First, we need to answer the question what's a polyfill?

There are times when a device or a library may not be the latest and greatest, like when someone is still using an older version of Internet Explorer which may not support the latest HTML standards.

What could be done in such cases is that a so-called polyfill library can be used.

The fill is it's filling in for missing APIs. So a website that wishes to be able to run on the widest range of browsers while still being able to take advantage of the latest features of the most recent browsers can choose to have its web pages download and invoke a Polyfill library. Then, when the page's code runs, the Polyfill library will check to see whether the underlying web browser supports the various features that the site wishes to use and if the answer is no, the polyfill library itself will make up the difference. It'll fill in the gap. It'll fill in for the down version browser by emulating the functionality that's natively missing from that browser. So, in short, polyfilling is another popular practice and library. It's mostly used to fill in for missing JavaScript features, but both PHP and Python have available polyfills as well. So, just as with the wildly popular jQuery library. Polyfill libraries, while not as wildly popular as jQuery, are still in wide use. That makes it, let's just say, very bad when a Chinese company purchases the polyfillio domain and its associated GitHub account.

1:16:05 - Leo Laporte
Uh-huh.

1:16:07 - Steve Gibson
And then proceeds to do bad things with them. Essentially, this company purchased a position of extreme power and responsibility and then chose to misbehave. Exactly two weeks ago, on June 25th, the forensics team at the security site Sansec broke the news with the headline polyfill supply chain attack hits more than 100,000 sites. And before I go on, I'll just note that, as we'll see, their initial estimation of more than 100,000 sites fell short by around 280,000. The real number has turned out to be more than 380,000 websites. They wrote polyfilljs is a popular open source library to support older browsers, and they said 100,000 plus. We now know it's 380,000 plus. Sites, they said, embed it using the cdnpolyfillio domain. Notable users are JSTOR, intuit and World Economic Forum. However, in February this year, a Chinese company bought the domain and the GitHub account. Since their purchase, this domain was caught injecting malware into mobile devices via any site that embeds cdnpolyfillio. Any complaints were quickly removed from the GitHub repository.

The polyfill code is dynamically provided based on the HTTP query headers. So multiple attack vectors are likely. So multiple attack vectors are likely. Sansec decoded one particular malware which redirects mobile users to a sports betting site using a fake Google Analytics domain, and I love this. Get this Leo G-I-E GoogieAniytics A-N-A-I-Y-T-I-C-Scom. So get your GoogieAniytics here. The code has specific protection against reverse engineering and only activates on specific mobile devices at specific hours. It also does not activate when it detects an admin user. It also delays execution when a web analytics service is found presumably to not end up in the stats.

The original Polyfill author recommends that Polyfill should no longer be used at all, as it is no longer needed by modern browsers. Meanwhile, boast fastly and we'll see where fastly figures in here, because it's significant, and cloudflare have put up trustworthy alternatives if you should still need it. This incident is a typical example of a supply chain attack. Now they followed up this report with a series of four updates, one per day, starting the same day, so on the 25th of June, 26th, 27th and 28th. First, they updated Google has already started blocking Google ads for e-commerce sites that use polyfillio. The next day, someone launched DDoS attacks against our infrastructure and bleeping computer. Who was the first to cover our research? The day after that, cloudflare has implemented real-time rewrites of cdnpolyfillio to their own version. A little later, namecheap has put the domain on hold altogether, which eliminates the risk. For now, however, you're still recommended to remove any polyfillio references from your code. And finally, on June 28th, we are flagging more domains that have been used by the same actor to spread malware since at least June so a year ago of 2023. 23, bootcdnnet, bootcsscom, staticfilenet, staticfileorg, unionadjscom, xhsbpzacom, unionmacomsla and newcrbcpccom. So a big problem with this code.

And then Census weighed in with some terrific reporting based upon their extensive visibility into the Internet. I should note that my server logs are full of port probes from Census. They want to know what's going on and who's running what, where they're another site like Shodan. But the nice thing, for me at least, is that their reverse DNS resolves to their domain name. So at least if I'm curious, I can see who's knocking at the door. I still don't let them in, but it's marginally less creepy to know that it's not an attacker, you know, trying to actually get in. It's just somebody, as their name suggests, doing a census of the Internet. Anyway, last Friday, under the headline Polyfillio Supply Chain Attack Digging into the web of compromised domains, census wrote On June 25th 2024, the Sansec forensics team published a report revealing a supply chain attack targeting the widely used polyfillio JavaScript library.

The attack originated in February 2024 when FunNull, a Chinese company, acquired the previously legitimate Polyfillio domain and GitHub account. Shortly thereafter, the service began redirecting users to malicious sites and deploying sophisticated malware with advanced evasion techniques. On the 27th of June, namecheap suspended the malicious polyfillio domain, mitigating the immediate threat. For now, however, census still detects 384,773 hostsding a polyfilled JS script linking to the malicious domain as of July 2, 2024. These hosts include websites associated with major platforms like Hulu, mercedes-benz and Warner Brothers Hulu, mercedes-benz and Warner Brothers. Security experts strongly advise website owners to remove any references to polyfillio and its associated domains from their code base as a precautionary measure. Cloudflare and Fastly have offered alternative secure endpoints for polyfill services as a workaround.

Further investigation has uncovered a more extensive network of potentially compromised domains. Researchers identified four additional active domains linked to the same account that owns the polyfillio domain. Census detected 1.637 million hosts referencing one or more of these endpoints, at least one referencing these domains. I mean, they've got to be thinking that they're pulling something from them. It's just mind-boggling. This incidence, they write, highlights the growing threat of supply chain attacks on open source projects. So, just to be clear, 1,637,160 website hosting servers are currently emitting web pages that are causing their visitors' web browsers to download resources from these malicious domains. Speaking of the hosts that they have found attempting to pull from polyfillio, census added the presence of domains like wwwfeedthefuturegov in these top results also highlights the use of Polyfieldio across various sectors, including government websites. In total, census observed 182 affected hosts displaying a gov domain. Displaying a gov domain, in other words, 182 gov websites were pulling code from polyfillio, owned by a malicious Chinese company, which is where we interject what could possibly go wrong. While estimates of the scale of affected websites vary widely between sources they said SanSec reported 100,000, while CloudFlare suggested tens of millions it's clear that this supply chain attack has had a widespread impact. Further investigation has uncovered an extensive network of potentially related domains. A Twitter user discovered that the maintainers of the Polyfill GitHub repo had leaked their Cloudflare API secrets within the repo Whoops. It was thereafter discovered that the leaked Cloudflare API key is still active and shows four additional active domains linked to the same account Bootcdnnet, bootcsscom, staticfilenet and staticfileorg. And I should just mention that we're all able to put polyfillio and those other four domains into our hosts file or into whatever local resolver we have and black hole them so that for no reason would anyone in our own networks or our browsers successfully download code from those sites where they never would be wanting to. They wrote. Bootcsscom has been observed engaging in malicious activities that are very similar to the polyfillio attack, with evidence dating back to June of 2023. Mollyfieldio attack, with evidence dating back to June of 2023. The two main malicious domain names involved were both registered at the end of May.

The evil jQuery was introduced by highlightjs hosted on cdnbootcsscom. When the request for highlightjs has a specific referrer and mobile user agent, the server will return highlightjs with malicious code. Otherwise, it returns normal code, which is highly disguised. Moreover, there are specific conditions for whether malicious jQuery is introduced when this code is executed. Okay, in other words, matching on a specific referrer enables the code to only target users visiting specific websites, and matching on the mobile device's user agent causes it to only target users using specific device types. So a highly targeted attack which, because it's delivering benign code most of the time, would easily go unseen. So this is all really quite bad.

The one person whose reactions we've not yet heard from is Andrew Betts, the originator of Polyfill Not Polyfillio, just Polyfill. February 25th, after the change of ownership of polyfillio had occurred and before all this came to a head, andrew tweeted. He said this is the originator of the polyfill process, the polyfill library for web browsers. He tweeted if your website uses http//polyfillio, remove it immediately, all caps. He says I created the Polyfill service project but I have never owned the domain name and I have had no influence over its sale, so he wants to fully disavow himself of any association or responsibility for what might happen. He said no website today requires any of the polyfills in the polyfillio library.

Most features added to the web platform, meaning the polyfill web platform, are quickly adopted by all major browsers, with some exceptions that generally cannot be polyfilled anyway, like WebSerial and WebBluetooth Domains that serve popular third-party scripts. So listen to this, this is Andrew. Domains that serve popular third-party scripts are a huge security concern. The team at Google Analytics dot com, for example, could read or modify almost any website in the world, like govuk or wellsfargocom. If you own a website, loading a script implies an incredible relationship of trust with that third party. Do you actually trust them? And I should explain that, as I said, andrew never endorsed the distribution of his polyfill library via a third party. He intended to have the hosting site that needed to use the library host it themselves, just like in the good old days. He's clearly aware of the havoc that would ensue if any major third party like Google, with their ubiquitous analytics code being injected into every website there is, were to go rogue or be compromised by a sophisticated attack. But somewhere along the way, someone not Andrew had the bright idea of hosting Polyfill at polyfillio, and over time, everyone started to use it rather than host it themselves, and this was the crucial mistake that so many in the industry made.

Think about it. You've purchased TLS certificates to protect your site, right? You have electronic and physical security, maybe even hardware security modules to protect your secrets. You've implemented pass keys to offer the latest state-of-the-art public key login authentication. You've gone through the popular checklists of all the things you need to do to be secure, and everything is checked off on them. You've done everything you can think of Downloading and running active JavaScript code that runs with full permissions in a first party context from an unknown entity in China that turns out to be malicious. What's wrong with this picture? What's right with this picture? It's nuts.

1:33:59 - Leo Laporte
So what happened in this case? And Leo, after sharing our final sponsor, we day from panoptica. Panoptica is cisco cloud's application security solution and you need to know about it because it provides end-to-end life cycle protection for cloud native applicationative application environments. Panoptica empowers organizations to safeguard their APIs, serverless functions, containers and Kubernetes environments. Panoptica ensures comprehensive cloud security compliance and monitoring at scale, offering deep visibility, contextual risk assessments and actionable remediation insights for all your cloud assets.

Powered by graph-based technology, panoptica's Attack Path Engine prioritizes and offers dynamic remediation for vulnerable attack vectors, helping security teams quickly identify and remediate potential risks across cloud infrastructures. A unified cloud-native security platform minimizes gaps from multiple solutions, providing centralized management and reducing non-critical vulnerabilities from fragmented systems. Panoptica utilizes advanced attack path analysis well, there you go root cause analysis and dynamic remediation techniques to reveal potential risks from an attacker's viewpoint. This approach identifies new and known risks and emphasizes critical attack paths and their potential impact. You need this info. Panoptica provides several key benefits for businesses at any stage of cloud maturity, including advanced CNAP, multi-cloud compliance, end-to-end visualization, the ability to prioritize with precision and context, dynamic remediation and increased efficiency with reduced overhead. I like it. Visit panopticaapp to learn more P-A-N-O-P-T-I-C-A panopticaapp. Thank you, panoptica, for supporting the work we're doing at Security Now. Now Steve's going to explain what the Financial Times has to do with this mess.

1:36:12 - Steve Gibson
So what happened? Without some deeper investigation, it's impossible to say exactly. I took a look back in time thanks to the Internet Archive's Wayback Machine. The Wayback Machine began taking snapshots of the polyfillio site back in 2013, so 11 years ago, and it certainly all looked above board and solid. For years it was a service being brought to the world by the Financial Times, with bandwidth and CDN services provided by fastly and the public domain.

Polyfill code present on github. An early version of the site's homepage said just the polyfills you need for your site, tailored to each browser. Copy the code to unleash the magic. And then there's a simple line. It's got an open angle bracket, bracket that says script space source equals open quote HTTPS colon slash slash CDN dot polyfill dot. Io. Slash V2 slash polyfill dot min dot JS. And then all that's closed. And then you close the script that will cause the browser that reads that line to go to that domain through the CDN, pull polyfillminjs, de-minify it and execute it, and what it then chooses to do is entirely up to it. Entirely upped it. You've loaded it because of the privileges it needs as a first party in your own domain's code. It can do anything to your page it wants. It can take the S's off of the HTTPs. It can log the username and password that are entered into your page's forms. It can do whatever it wants. They said. Polyfillio reads the user agent header of each request and returns polyfills that are suitable for the requesting browser. Tailor the response based on the features you're using in your app and see our live examples to get started quickly.

The Polyfill service is developed and maintained by a community of contributors led by a team at the Financial Times. It evolved from a previous service developed by Jonathan Neal, and our cdnpolyfillio domain routes traffic through fastly, which makes it available with global, high availability and superb performance no matter where your users are. So at this point, the Financial Times is referring to CDN dot polyfieldio as their domain, but things changed and it's interesting to trace the site's evolution through the years. The Wayback Machine makes that easy. What I discovered was that something happened toward the end of the year last year, on November 1st of 2023. Last year, on November 1st of 2023. The day before on October 31st, Halloween of last.98 trillion bytes worth of polyfills. And that was just in the past 30 days 61.2 billion requests offering nearly 300, just shy of 300 trillion bytes in polyfills, and Fastly's name was still proudly highlighted in red against a black background, but then the next day, on November 1st, the site's pages changed. Gone was the activity tracking, the reference to GitHub or any mention of Fastly.

So, as I said, it would take interviewing the parties involved to learn more. One guy who would likely know the whole story is Jake Champion, whose Twitter handle is at Jake Champion. His personal bio, at jakechampionname, mentions his involvement with the Polyfill project. He's at Financial Times and his copyright can be found at the bottom of the earlier pages, but he has his Twitter account locked for viewing only by approved followers, so I wasn't able to say whether he may have been tweeting something about what had been going on.

What's clear is that maintaining the 100% free polyfill service was doubtless becoming expensive, and it was the definition of thankless bankless 11 years earlier. It would have been much easier than it was today, and with nearly 300 terabytes of polyfills being served every 30 days, that's 10 terabytes per day, day in and day out, in return for nothing. So it's not difficult to imagine someone coming along and offering to purchase that massive burden, or maybe just take it off of the Financial Times hands. We don't know. I've not found anything about that yet. Maybe some of our listeners will know or will find out. I'd love to know. But however, it happened in the blink of an night suddenly, 384,773 websites or tens of millions if you take Cloudflare's number were suddenly inviting an unknown, stranger's JavaScript code into and onto their previously super secure and secured pages, thus bypassing and making a joke of everything else they had done to create secure websites.

While digging around for all of this interesting information and backstory, I ran across one other chilling bit. This was posted over on Medium by Amy Blankenship, who describes herself as a full-stack developer at a financial technology company. She primarily writes about React, JavaScript, TypeScript and testing and under the headline, those scary stories about polyfillio. They're just the beginning. She posted the following to Medium. They're just the beginning. She posted the following to Medium. She said the Internet exploded this week about how Polyfillio was injecting malicious code into websites that were linking to it from the CDN at the URL where it has been served for years. The thing is, ownership of the GitHub repo and the download domain were transferred in February. They didn't wait until this week to start making changes. Here's how I know.

In February we started to get mysterious errors. When some of our users tried to log in the stack trace, the Sentry error boundary was giving us didn't make any sense. It was deep within the Okta React library code we were using, but we'd not recently upgraded our Okta React, Okta JS or Sentry Code or the code that called it. Long story short, in the course of stepping through the code in the debugger, I discovered that some code in Okta was expecting to receive an object that had iterable properties. Sometimes, when it received the object, those properties could not be iterated. Sometimes, when I received the object, those properties could not be iterated. Digging further, I found that the object was some deeply embedded dependency upon the polyfillio library which it was pulling from polyfillio, thus from china.

1:45:12 - Leo Laporte
Oh oh good on amy blankenship for finding that.

1:45:18 - Steve Gibson
That's amazing, yep wikipedia reminds us, okta inc is an american identity and access management company based in san francisco. It provides cloud software that helps companies manage and secure user authentication into applications and for developers to build identity controls into applications, website web services and devices. It was founded in 2009 and had its IPO offering in 2017, reaching a valuation of over $6 billion. That's right. At its core, okta React and Okta JavaScript library code was pulling from a library that is now being supplied by a clearly hostile and malicious company named FunNull, acquired the polyfillio domain and GitHub account.

The Okta library itself began mysteriously acting up, and this, of course, brings us to, at the top of page 17 of the show notes, randall Munroe's wonderful XKCD cartoon showing a large and ungainly collection of stacked blocks reaching up to the heavens. We would call it a house of cards if it were composed from playing cards rather than blocks. That detailed and stacked assortment is then labeled all modern digital infrastructure, and then, down near the very bottom off to one side, is a crucial twig whose presence is holding up the entire edifice, the whole assembly, such that without it, everything would come tumbling down. And that little twig is labeled a project some random person in Nebraska has been thanklessly maintaining since 2003. Unfortunately, in this case, as the result of a number of unfortunate events, it would be labeled control acquired by a hostile and malicious Chinese company.

Now Leo and I live in California. This state has a major seismically active geological fault running through it, known as the San Andreas Fault fault running through it, known as the San Andreas fault. The San Andreas fault runs north-south about 750 miles through California, forming part of the tectonic boundary between the Pacific plate and the North American plate.

This is the same fault that Lex Luthor was determined to trigger and use to turn California's eastern neighboring state of Nevada into beachfront real estate by sinking all of California into its Pacific Ocean. Fortunately, well, we have Superman, so that hasn't happened yet. The relevance of this is that the somewhat precarious state of our Internet security infrastructure puts me in mind of the nature of seismic faults and the earthquakes that accompany them. As tectonic plates slowly shift, pressure builds up, and that's not a good thing. So here in California, what we're looking for and we're hoping for is a more or less continuous series of small earthquake tremors where no one is hurt and the china plates remain on their shelves.

People text each other asking hey, did you just feel that one? That's much better than a long period of quiet which allows massive pressures to build up, only to be released in a single massive event. So what we want here in California is lots of small, barely noticeable events. How many times on this podcast, especially over the last five years, where it seems to be happening more frequently have I noted that we all just dodged a bullet, that the Internet just experienced another tremor, we discovered a serious problem that did not take everything down with it. So here again, the industry just received another wake-up call, with no known actual, let alone horrific, damage being done, and an important lesson was hopefully learned that was not expensive. Let's all hope that things remain like this, with a continuing series of small and harmless little earthquakes rather than, as it's referred to in California, the big one hitting this could have been the big one, big time.

1:50:42 - Leo Laporte
This could have been really bad. We don't know how many sites chose polyfillio to load that library, though.

1:50:53 - Steve Gibson
Yes, yes, it could have been 1.67 billion. We do 1.67 million sites. It could have been 1.67 billion. We do 1.67 million sites. We have a count.

1:51:02 - Leo Laporte
And got it from polyfillio.

1:51:05 - Steve Gibson
They got it from the CDNs. Looking at the pages and seeing how many Right and also right the various sites that are pulling from polyfillio right now after taking control of that domain, Wow, Millions, millions of different websites.

1:51:25 - Leo Laporte
So somebody benign now controls it, so we're okay.

1:51:29 - Steve Gibson
Yes, namecheap took the site down. They took the domain name off the air?

1:51:33 - Leo Laporte
Do they offer the library or no? They?

1:51:37 - Steve Gibson
took the domain name. Do they offer the library or no? Both Cloudflare and Fastly are hosting the final benign version for anybody who needs it, but, as its author said, nobody does anymore. So a lot of this again. This is also stupid inertia Running on automatic. All that crap was left in the pages because no one's sure if we need it or not, so we better leave it there. Meanwhile, china owns the domain and it's pulling JavaScript as a first party into every one of those pages. Code that could do anything.

1:52:15 - Leo Laporte
It's the big one. It's the big one, but it isn one. But it isn't. We dodged a bullet. Amazing story, what a great story. And a cautionary tale, as you point out. That's why you listen to this show, so you know, you're prepared and you're keeping an eye out. And again, credit to amy blankenship, who noticed a weird anomaly with her octalog ins and said I, you know this, something's right. Not right here. And that's a note to everybody when, when, little weird anomalies happen, they're not necessarily the big ones, just the little ones, that's. That's. That's how Clifford Stoll noticed the bad guy in his network, cause there were one or two cent discrepancies in the balance sheet and actually I bet our listeners are as twitchy as I am.

1:52:58 - Steve Gibson
Now I see like some spike in bandwidth.

1:53:00 - Leo Laporte
I go whoa, what's going on? What's going on? It's out of the ordinary. It needs to be investigated. Wow, what a story. You'll hear more stories like that every week.

On Security Now Steve Gibson is the man in charge. You can get on his mailing list or send him email by going to GRCcom slash email. Then you can verify, validate your address, sign up if you wish for the free newsletters, including, by the way, the show notes, so you get a peek at the picture of the week ahead of time. Grccom slash email. Now, while you're there, you can also buy a copy of Steve's bread and butter. Didn't mention it this week, but Spinrite pays the bills for Steve and it is a very useful must-have, frankly, tool for anybody with mass storage, for both performance, reliability and maintenance and recovery if necessary. Grccom, look for Spinrite. Lots of free stuff there, including shields, up uh validator, valid drive rather, which is really a great drive, yeah, uh. And of course, this show.

Steve has the usual 64 kilobit audio. You know the kind of traditional version. He also has a non-traditional 16 kilobit version for people with very little bandwidth and no really hearing. Uh, if you could put up with it, you know you'll save a lot of bits. Many bits will not have to die for your, your listening. Uh, we, he made it for elaine ferris, who does those wonderful transcripts of every episode. All that's at grccom, along with the show notes. We have the 64 kilobit audio at our site, but we also have the video that's at twittv slash SN if you want to see Steve's smiling face.

We put also a copy of the show on YouTube. There's a dedicated channel for that and you can subscribe in your favorite podcast player and get it automatically If you'd like to do it live. There's a small number, but devoted, who like to watch a show the minute you know, like as we do it. That's probably so they can be in our Discord and chat with us in the club. And, by the way, what we've done is completely expanded the number of places you can watch live. So we do the show right after Mac break, weekly, about one 30 Pacific, four, 30 Eastern on Tuesday, that's 2030 UTC. Uh, if you uh are watching live, uh on YouTube, there's a chat room there on discord, of course, in the club there's a chat room there, twitchtv, the chat rooms open there. So lots of places you can chat. We see the chat. I see the chat Steve does. I see all the chats, can't respond to them all. We're also on LinkedIn now and Facebook and X Everywhere, everywhere, and I think that's a great thing. So you can see more of the show live pretty much anywhere and chat at the same time with other people watching. Let's see what else. Oh, club Twit, don't forget.

Your support makes this show possible. Yes, we have advertisers, but it is more. It costs more to do the show than we make and you make up the difference and we really want to keep doing this show and all the shows we do. So we need your participation. If you listen to this show or maybe another show on the network and you want to support it, it's seven bucks a month. You can pay more if you want. You don't have to. That's that's kind of the standard. Seven bucks a month.

You get ad-free versions of all the shows. You get video on shows that we don't put out in video in public, like hands-on Macintosh, hands-on Windows, home theater geeks, the Untitled Linux show, all that. What else do you get? Participation in the club-only activities Stacy's Book Club, for instance, micah's Crafting Corner all of that Seven bucks a month. That's nothing. Go to twittv, slash clubtwit to learn more. And a very special thanks to all our club twit members who make this show possible. We really, really appreciate it. Steve, I appreciate you. Thank you for doing this show every week. Please come back next week and deliver us from evil once again. Oh, will do See you then, my friend See you.

1:57:14 - Steve Gibson
Bye Security Now.