Security Now 1035 transcript

Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show

0:00:01 - Leo Laporte
It's time for security now. Steve Gibson is here with news of a passkey bypass Yikes. We'll also talk about problems Cloudflare had with its DNS provider and explain what happened. Russia clamps down really clamps down on internet usage. And some advice if you're planning to travel to China or, frankly, anywhere, stay tuned. Security Now is next. Podcasts you love From people you trust. This is Twit. This is Security Now with Steve Gibson, episode 1035. Recorded Tuesday, july 22nd 2025. Cloud flares 1.1.1.1 outage. It's time for Security Now, the show where we cover your security and your privacy online. We talk about all the exciting stuff happening in the world with this guy right here, mrve gibson. A show we wait all tuesday for. Hello, mr steve arino, all tuesday, oh, all week for tuesday oh, okay, there we go.

0:01:14 - Steve Gibson
That's better.

0:01:15 - Leo Laporte
From the end of the previous I wait, I wait all day for this show that's right, you're busy all morning before the show no, I wait, I do look forward to it all week, because you always find some interesting stuff. In fact, this week I even sent you some things. I don't know if you used them, but I sent you some things to talk about.

0:01:33 - Steve Gibson
You did and they're going to need. I've been aware of homomorphic encryption for quite a while.

0:01:41 - Leo Laporte
I'm sure you're the first person to ever mention it in my presence.

0:01:44 - Steve Gibson
Mind-scrambling technology, the fact that you're able to operate on data without decrypting it.

0:01:50 - Leo Laporte
that's crazy and the ultimate in privacy. But we're not there yet. Because it's computationally a little intensive, it does keep its secrets.

0:02:00 - Steve Gibson
Yes, actually, a number of our listeners, like in a flurry, sent me news of today's topic, which is what's really interesting, leo, is this occurred during the hour, the latter part of the hour, exactly during last week's podcast, oh, interesting, which is that, due to someone tripping over a cord at Cloudflare headquarters, their famous 1.1.1.1 DNS server service disappeared for an hour globally. So what happened? How does that happen? We're going to have fun talking about that today. Cloud flares, 1.1.1.1 outage, uh, for security, now number 1035 for this july 22nd.

But we got a bunch of other fun stuff to talk about. We've got, uh, a, a, an active attack successfully bypassing all passkey protections, as in, you know, the fido to passkey, uh, ransomware attacks just keep coming. Cloudflflare capitulates to the MPA, the Motion Pictures Association, and starts blocking, which they've resisted strongly until now, you know, on the whole net neutrality side. Also, we're going to look at, actually in two different instances, this issue of age verification, which turns out to be, for some reason, my main focus, I guess I just see this as a huge need, an obvious need, and it's like the industry's been caught flat-footed because we have no way to do that. We're, we don't. We have no way to do that, uh, and, and I don't see how we can do it for no cost, which is upsetting because it sort of de-democratizes the internet, which every you know, everyone's been keep going on fighting to to keep open anyway. We're going to talk about that a few more times uh, microsoft trying to push people from a purchase to a subscription of their exchange servers, russia further clamping down on their internet usage, and how, unfortunately, we begin to see a global trend toward that emerging.

China inspecting locked Android phones. So maybe get yourself a burner Android phones. So maybe get yourself a burner Web shells becoming the new buffer overflow. As I said, I'm going to sketch out a few different aspects of this whole age verification problem and play with some protocols that might work, and then we're going to talk about what Cloudflare did did to create their own complete global, massive dns outage of their flagship server dns server for the hour during which we were recording this podcast last week and, of course, as always, we have a great picture of the week. So, yeah, I think, uh, this one may have been worth waiting for yay, absolutely.

0:05:25 - Leo Laporte
As always, we'll save homomorphic encryption for another day it's, it's yeah, it's a hairy topic it's a yeah.

0:05:34 - Steve Gibson
You lose your hair by the time.

0:05:36 - Leo Laporte
You've covered that time it's really funny how much of the time that I'm. You know. I spend every hours every day reading all the tech news, so I'm preparing for all the shows, and how many stories I come across where oh, I gotta ask steve about this. I don't pester you with those. So just to let you know every once in a while.

0:05:53 - Steve Gibson
I said steve one but I don't know what one escapes the filter.

0:05:56 - Leo Laporte
Yes, well, one just came in that I'm a little devastated about. You know I've talked before about this ai device that I wear, the b computer. It's been recording everything that I've done for the last six months and summarizing it in AI and giving me a. They just got sold to Amazon so I just now deleted all the data and I'm really hoping the company lives up to its promise to delete all that data.

0:06:21 - Steve Gibson
So you're wearing it as a lanyard?

0:06:23 - Leo Laporte
Yeah, so it just sort of hangs there well, I used to wear it on my wrist and then I briefly, I wore it. Uh, clipped on, but I kept losing it.

0:06:32 - Steve Gibson
This is the third one, it is audio, so it's not a camera, it's just audio.

0:06:37 - Leo Laporte
Okay, it records everything. It sends the transcript to ai, uh, which then synopsizes it bullet points. It gives me a kind of a diary, a list of tasks that I think I might want to add to my to-do list, which is very, very valuable and generates facts about me over 2,500 facts over the last six months. Things like your wife's name is Lisa, your cat's name is Rosie, you have a Helix mattress, whatever stuff like that, which is great, but I don't want Amazon to have all that stuff.

0:07:14 - Steve Gibson
So yeah, I'm in the process of setting up a new home and it was I needed to choose an automation system and I initially thought naively oh well, you know, everybody has that A word device, you know A-L-E-X-A. But as I started to play with it, I realized I'm part of a big commercial enterprise and I mean they're like upselling me, you know Amazon is. I realized I'm part of a big commercial enterprise and I mean they're like upselling me, you know Amazon is. And it's like, no, I'm not using their technology.

0:07:54 - Leo Laporte
Yeah.

0:08:01 - Steve Gibson
I think really, we're all just going to end up using Apple stuff and hope that Apple lives up to its promise, and I settled on using HomeKit as the base. It's secure, right? Yes, we know to the highest levels of the industry that's what Apple has done. I'm a little annoyed with them over their position on age verification because they're so wrapped around that flag that it's like the industry needs this Apple so. But you know they're doing what they can by by creating age range brackets and, you know, trying to keep it as fuzzy as possible, but anyway, so. So so you're saying secure, yes, Sure, Private yes.

0:08:39 - Leo Laporte
Well, we hope so. I mean, we're you somewhat trust Apple, but this is so much part of now their marketing that I think they probably will live up to it, at least more than Amazon or Google or Microsoft or OpenAI or any of the other possibilities. So it seems like if you're going to do home automation, that's the way to go.

0:08:57 - Steve Gibson
That's what I'm going to do, for sure I did have to choose Google's doorbell, because it's the best.

0:09:02 - Leo Laporte
Well, you don't want Ring, because ring just announced that they're gonna be sending the information to the cops again.

0:09:08 - Steve Gibson
So I have a, I have, I have a bridge in order to, to, to create that, uh, that that link.

0:09:14 - Leo Laporte
So my goal in the long run and this is a time-consuming thing that I'm not going to do any time soon, but is to make it all internal and and, and you know, use ai internally and all of that stuff. That would be my goal. Same thing with this. I love the idea, the premise of this B computer.

0:09:31 - Steve Gibson
I want it to be my own AI, not somebody else, and the problem is, the world is switching to a subscription model where it's all services and you and, and you know, and so, like you know, you audit your bank account and this is all these little dribbles coming out. From all the things that have happened in the past, it feels it's almost impossible, and as you get older, you really don't want dribbles, that's just not good Never good, that's not progress.

0:09:56 - Leo Laporte
Hey, let me talk about another way to secure your enterprise. One of our sponsors for this episode of Security Now is Zscaler. They are the leader in cloud security and they solve a problem kind of in two different directions. You know, AI it's a double-edged sword. It's a blessing, it's a boon, and it's also perhaps the biggest threat to security we've ever seen. Hackers are using AI now to literally breach your organization. On the other hand, AI powers innovation, can drive efficiency. On the other hand, it helps bad actors deliver more relentless and effective attacks. There is a solution Zscaler, zero trust plus AI. Phishing attacks over encrypted channels last year increased by 34.1%. This year I'm sure it'll even be worse, and that's fueled by the growing use of generative AI tools and phishing as a service kits.

We talk about this all the time and, on the other hand, organizations in all industries, from small to large, are using AI internally. They're using it to increase employee productivity. They're using public AIs for engineers with coding assistance. Marketers are using it for writing. Finance is using AI to create spreadsheet formulas. You ever do a pivot table? Not on my watch. Let the AI do it. You can also automate workflows for operational efficiency across individuals and teams. Ai is being embedded into the applications and services that are customer and partner facing. Ultimately, AI lets every company move faster in the market and gain a competitive advantage. But companies, you've really got to think about how you protect your private and public use of AI and, at the same time, defend against those AI-powered attacks. This is what Zscaler does.

Jason Kohler, who's the Chief Information Security Officer, the CISO at Eaton Corporation, leverages Zscaler to embrace AI innovations and combat AI threats. He says, quote data loss detection has been very helpful for us. Chat GPT came out. We had no visibility into it. Zscaler was the key solution initially to help us understand who was going to it and what they were uploading right. Traditional firewalls, VPNs, public-facing IPs expose your attack surface. This old school way of protecting the perimeter is no match to the bad guys in the AI era. It's time for a modern approach with Zscaler's comprehensive zero trust architecture plus AI that ensures safe public AI productivity, protects the integrity of private AI and stops AI powered attacks. It can do all three Thrive in the AI era with Zscaler Zero Trust Plus AI to stay ahead of the competition and remain resilient even as threats and risks evolve. Learn more at zscalercom slash security. That's zscalercom slash security. I thank them so much for their support of security. Now, All right, Steve, I have not glimpsed, I have not looked, I have not.

0:13:11 - Steve Gibson
So I gave, I gave this, this little cartoon, the caption uncertainty is the nature of the universe.

0:13:19 - Leo Laporte
Okay, uncertainty is the niche. I feel like Ed McMahon Uncertainty is the nature of the year. I'm like Ed McMahon uncertainty is the nature of the. I'm going to now scroll up and reveal.

0:13:32 - Steve Gibson
That's a cute little cartoon and you know, it's interesting that the Heisenberg uncertainty principle is now so well-known that you could actually do a New Yorker cartoon with this and it would yes, it would work and he would get it yes so we have a kind of a professor looking guy uh staring at the at at the map on the wall, trying to figure out you know where he he's trying to go. Uh, he is standing. We, we know this because it says above the map he's, he's in the heisenberg Department of Physics and, in keeping with the theme, the map has the legend, with an arrow actually pointing to sort of little scatter chart of dots. It says you are probably here. So you know, you can't be certain because you are all in the Heisenberg department of physics, but anyway it does it does say something about how widespread knowledge of quantum mechanics has become.

0:14:35 - Leo Laporte
I guess Right.

0:14:36 - Steve Gibson
Yeah, actually there was a funny. I finished, and I'll talk about this a little bit later. I finished a project Hail Mary last night and I'll talk about this a little bit later. I finished Project Hail Mary last night and they're rereading it. We should say my reread.

0:14:49 - Leo Laporte
Yes.

0:15:06 - Steve Gibson
And there was one point where and I have to be careful not to do any spoilers, although you know the book is I mean, anyone who's been following the podcast and listening to us and so forth is probably well aware of it. But there was a point where there was a technology exchange and our main character, dr Grace, sort of commented that to the person receiving the human store of knowledge that they're going to be really happy with everything they have received from humanity until they get to the bit about quantum physics, because you know, they're not going to be happy about that.

That makes nobody happy.

0:15:35 - Leo Laporte
It's like nah, this is just have we told you about string theory?

0:15:41 - Steve Gibson
oh lord, yeah okay, so um the security guys at Expel Security. I thought that was a good name.

0:15:49 - Leo Laporte
We're going to expel this.

0:15:50 - Steve Gibson
Expel Security have uncovered a passkey bypass using an adversary in the middle attack. Now the vulnerability of passkeys to this attack is actually understood. It's well known. It was a concession that was needed to be made for the sake of cross-device login, where you're using a passkey in your phone to log into a website on a browser somewhere. The Expel Security guys just have three bullet points at the top of their blog's TLDR section. They said first, bad actors have figured out how to downgrade FIDO key authentication when compromising accounts.

Now and we've often talked about downgrade attacks where, for example, an early one would have been the client sends to a web client sends to a web server a list of all the protocols it supports, and normally the web server would deliberately choose the strongest of those offered by the client, which it also supports. Right, so you cross-reference the security protocols. It has the security protocols the client has, the server chooses the strongest of those. But if you have an adversary in the middle, the adversary downgrades what the client is sending because, of course, we haven't established a secure connection at this point. This is the initial client hello. So all of the good protocols are stripped out, leading the remote server to believe that the client supports the paper cup and string protocol, which, arguably, is not very secure. Arguably is not very secure, and so it shrugs its shoulders and establishes a protocol that the man in the middle, the adversary there, is able to intercept. So, anyway, downgrade protocol.

Bad actors have figured out how to downgrade in a similar fashion FIDO key authentication when compromising accounts. Second bullet point this technique is being leveraged in phishing attacks, meaning it's happening in the wild that pass keys are being bypassed. Pass key authentication and finally, the attack involves tricking a user into scanning a QR code with a multi-factor authentication authenticator, which includes pass keys. So their blog posting was titled Poison Seed, which is the name they gave this downgrading FIDO key authentications to fetch user accounts. And they explain.

Our SOC, which is, you know, abbreviation for Security Operations Center, has recently spotted a novel attack technique that involves socially engineering a target to get around the security protections provided by FIDO passkeys. The attacker does this by taking advantage of cross-device sign-in features available with FIDO passkeys. These features are designed to help users sign into their accounts on systems without a passkey by using an additional registered device like a mobile phone. However, the bad actors in this case are using this feature in adversary in the middle attacks. This is a concerning development, given that FIDO passkeys are often regarded as one of the pinnacles of secure multi-factor authentication, and while we haven't uncovered a vulnerability in FIDO keys, it and SecOps folks will want to sit up and take notice. This attack demonstrates how a bad actor could run an end route around uninstalled FIDO key. We have reason to believe that this attack was carried out by Poison Seed oh, that's the name of the group, not the attack An attack group known for large-scale phishing campaigns designed to steal cryptocurrency from their target's wallets.

However, the technique described here could easily be leveraged in other attacks wallets. However, the technique described here could easily be leveraged in other attacks. And then they take us through the details of the attack by explaining. They said the attack started with a, and this is one that actually one of their client accounts was hit by, so they were able to get in there and reverse engineer what happened. They said the attack started with a phishing email sent to several employees at the company. The email lured these users to log into a fake sign-in page hosted at octalogin-requestcom. They said this page mimicked the general look and feel of the company's normal authentication process, including an Octalog logo and sign in fields for username and password. However, not only is the domain hosting this fake login page suspicious, the domain itself had only been created a week before the attack.

Now I'm going to just pause here to say it's interesting that they provide that bit of detail, or like fetching, you know, for any, for whatever reason, should always be raised as a massive red flag. At the very least, any visit to a freshly minted domain ought to be brought to the user's attention. I mean, maybe it's asking too much for many users, but if nothing else, it's an additional signal. Right, you need to prove, you need to acquire a reputation before it'll be allowed. So, as I wrote here, highly security conscious DNS resolvers like NextDNS should be checking the age of any domain names being resolved before they're visited. Or perhaps, you know, maybe the page should be displayed.

Well, I was thinking in terms of the browser. The page could be displayed and the user could begin filling out any forms while the reputation is checked in the background, and the form submit function would only be unlocked once a domain reputation, you know, including the domain's age, passed scrutiny. You know, a user could always bypass such a block, but, you know, bringing this to their attention and saying, just so you know, you're submitting this to a domain that's only a week old, so does that surprise you? If so, don't proceed. So anyway, I think you know somehow, except in your case, leo. I'm glad that NextDNS does this, because this is the kind of thing that responsible DNS providers ought to offer, at least as an option. Ought to offer at least as an option, anyway.

So they said both this domain and the AWS-US3-manageprodcom domain the user is redirected to if they enter their credentials are services like Cloudflare. This is not Cloudflare's fault, right? They're just the hosting provider can make phishing scams appear more trustworthy, potentially lulling visitors into a false sense of security. They said the targeted user in this case had a FIDO key registered to secure their account. Normally, the user would be required to physically interact with the FIDO key, touching it, for example, to confirm they're the ones logging in and are on the registered device, or using a passkey app. If a user whose account is protected by a FIDO key in this case, enters their username and password into the phishing page, their credentials, that username and password, will be stolen, just as with any other user, but with a FIDO key protecting their account, the attackers are unable to physically interact with the second form of authentication.

This is where things took a turn. They wrote from your traditional phishing site. After entering their username and password on the phishing site, the user on the phishing site device scan this QR code with the device that has the pass key for, and then they blanked it out. But it would be the name of the site. This request comes from the app mstscexe by Microsoft Corporation, and then there's a chunk of the QR code, but they blanked out a bunch so that wouldn't be legitimate. So they wrote.

What happened behind the scenes is the phishing site automatically sent the stolen username and password to the legitimate login portal of the organization, along with a request to utilize the cross-device sign-in feature of FIDO keys. The login portal then displayed a QR code. Under normal circumstances, when a user wants to sign into their account, they wrote from a different unregistered device. They can still verify their identity if they've enrolled another authentication device. In most cases, this would be an MFA authentication app installed on a mobile device, most of which include a QR code scanner. The login portal displays a QR code after it receives the correct username and password, which the user scans with our MFA authenticator. The login portal and the MFA authenticator communicate to verify the login and the user's granted access. In the case of this attack, the bad actors have entered the correct username and password which they got from the user entering it into the phishing page.

Requested cross-device sign-in. Requested cross-device sign-in. The login portal the legitimate login portal displays a QR code which the phishing site immediately captures and relays back to the user on the fake site. The user scans it with their MFA authenticator. The login portal and the MFA authenticator communicate. The bad guys capture that and they log in as the user. Any protections that a FIDO key grants and gives the attackers access to the compromised user's account, including access to any application, sensitive documents and tools such access provides.

0:28:15 - Leo Laporte
And, of course, squirrel was not vulnerable to this.

0:28:18 - Steve Gibson
Well, it's an acknowledged weakness of any cross-device authentication. Squirrel had this weakness. I went to extreme lengths to eliminate this possibility from any same device authentication app which was available for all platforms. Then any possible man in the middle was excluded because the user's browser connected to the local squirrel client which performed the authentication. It then received the logged in URL, that is, the squirrel client received the logged in URL which it forwarded to the user's browser so that cut out explicitly cut out any man in the middle. They were never able to obtain that logged in URL, but this protection depends upon having a link between the user's browser and the authenticator and that's not available for typical cross-device authentication. The device is able to see the QR code, but there's no way for the device to get a secret back to the user's browser. So it authenticates to the site, which then authenticates the browser session which is logged into the the site which in this case is the fishers the, the fishing session, not the fished user session. So I wanted to share this with everyone so that this danger would be very clear.

You know, pass keys are a huge step forward and they can prevent many other forms of abuse.

If not, maybe it's all other forms except real wanton negligence convince a user to enter their valid username and password into a site, then intercept and forward their QR code for a cross-device passkey authentication can still get themselves authenticated even with passkey-protected authentication.

So all the other things that can go wrong with usernames and passwords, you know pass keys resolve. But it's that cross-device authentication which is still the Achilles heel, which is why you really do want to put pass keys into the device where you're doing the authentication. You know the cross-device can be useful for bootstrapping and, as we know, the good news is the FIDO group have now ratified a cross ecosystem pass keys import and export and Apple has said they're going to support it, and I'm sure everybody else is going to, and we know that the, the, the browser add-on, guys like bitwarden and one password and so forth they'll be supporting it too. So it's going to be possible to solve this startup problem that we've had for the first couple years of pass keys. Cross device authentication cannot be as safe as on device authentication, so that's what you want to use whenever you can.

I'm sorry yeah, I'm disappointed yeah, and boy, leo, I spent so much time uh working with the group that I was, that I was uh interacting with during this, that we you interacting with during this, that we, you know, we solved the problem. That we solved this completely If you ran a squirrel client in the device you were authenticating on. But there's, there was just no way to. I mean, if you, if you had a, if you had a camera on the machine with the browser, then the URL received, the authenticated URL, could have been displayed as a QR code on your phone, which you would then show to the camera that the browser had and that could link back. So I mean, there were some clunky ways to do it if you had NFC or if you had common Wi-Fi or if you had Bluetooth, but all of those things are very messy. They're just not zero configuration solutions, and what you really want is not to have this raise the bar of complexity to a level where people are like I don't understand what's happening. I'm just going to use my username, because this is, too and it's, of course, failure prone too. The more other communication channels you need, to have the greater chance for one of them failing so or, you know getting hacked again. So, yeah, still have a problem For fear of allowing one of the biggest continuing problems the cyber community still faces, which is ransomware attacks.

I wanted to quickly note a couple of recent biggies. South Korea's largest insurance company, seoul Guarantee Insurance, got hit by ransomware last Monday. The incident has severely disrupted the company's operations and the company has been issuing handwritten loan guarantees to customers all week as it works to restore its affected systems. Its affected systems and this is the third major South Korean company to experience severe business disruptions this year due to cyber attacks. The country's largest telecom and its largest online bookstore both suffered similar disruptions. Also, the grocery distributor United Natural Foods has announced that they expect to lose up to $400 million in sales this year following a ransomware attack last month which took multiple systems offline for days. That downtime affected their ability to fulfill and distribute customer orders.

Australian airline Qantas you know, the big, famous Australian airline obtained this is odd obtained an injunction to prevent individuals and organizations from using or publishing data stolen from them in a recent ransomware attack. Okay, that's a new one. Since when do some foreign bad actors care about an Australian court order? The injunction suggests that the company is not willing to pay the ransom and is therefore expecting the hackers to leak the data, or maybe they just want to protect themselves from that leakage in any event, but it's difficult to see what they expect to gain Like okay, we've got an injunction to prevent anyone from using the data that's been stolen from us, except that criminals stole it from you, and I don't think the criminals care if you have an injunction against them, you know, in Russia or wherever they may be.

So anyway, I just want to just keep everybody aware that, like it's business as usual, unfortunately in the cyber attack and ransomware world, as we've noted before, there's not even any sign that we are making any progress and improving our effective security. It's just that this is now like a steady state, constant background pain that companies that are online are suffering when bad guys get in. And you know, as we've talked about it before right, and you know, as we've talked about it before right, employees, unfortunately, in large organizations are the weakest link, as far back as that Sony advanced persistent threat hack what was it 15 years ago? Or something I said I sure wouldn't want that responsibility of, like you know, having every employee never make a mistake, never make the mistake of clicking on a link in email. How do you do that? You know it.

The problem is it still is a weakness right it's an effective problem. Wow um, in what's being called a significant turn of events, cloudflare has become the first internet intermediary, beyond local residential ISPs, to block access to pirate sites. In the UK, users attempting to access certain pirate sites are greeted with cloud flares Error 451.

0:37:51 - Leo Laporte
Unavailable for legal reasons Intercept page.

0:37:53 - Steve Gibson
That's weird. Yeah, there actually is HTTP 451.

0:38:00 - Leo Laporte
It's new on me New to me.

0:38:02 - Steve Gibson
We all know the 404, right, there's a 451 unavailable for legal reasons as an official IETF error return. So they wrote in theory, isp blocking should prevent UK users from the internet. Isp blocking should prevent users from ever connecting, from ever even being able to connect, to a pirate website. That is, you know, cloudflare is the host of these pirate sites and so you shouldn't be an ISP should prevent their own users and the ISP, of course, is a UK ISP. So they're following UK court orders, they're they're abiding by a block list and so users shouldn't get to the host, and in this case, cloudflare is the host. The article continues. Internet service providers BT, virgin Media, sky, talktalk, EE and Plusnet account at the high court often list these companies as respondents.

These so-called no-fault injunctions stopped being adversarial a long time ago. Right where ISPs would like say no, we're not going to do that. Torrent Freit wrote that ISPs indicate in advance they will not contest a blocking order against various pirate sites and typically that's good enough for the court to then issue an order with which they subsequently comply. So everybody's just getting along with this. Now they wrote for more than 15 years with this. Now they wrote for more than 15 years.

This has led to blocking being carried out as close to users, meaning their ISP the so-called last mile as possible, with ISPs' individual blocking measures doing the heavy lifting. A new wave of blocking targeting around 200 pirate site domains came into force last Monday, the 14th, but with the unexpected involvement they wrote of a significant new player. In the latest wave of blocking that came into force, close to 200 pirate domains requested by the Motion Picture Association were added to what was already one of the longest pirate site blocking lists in the world. The big change is the unexpected involvement of Cloudflare, which, for some users attempting to access the domains added yesterday, displays the following notice, and I've got it here in the show notes it's a big error. Http 451 with a time code, date and timestamp.

0:41:32 - Leo Laporte
We now know what that is.

0:41:34 - Steve Gibson
Yep Available for legal reasons and then, as an explanation under what happened, they wrote in response to a legal order, cloudflare has taken steps to limit access to this website through Cloudflare's pass-through security and CDN services within the United Kingdom. Oh, I'm sorry, and CDN services within the United Kingdom Find more information about the order, the party that requested it and the authority that issued it. Here, and the here was lit up in blue as a link that users who received this intercept page could click on. And they said learn more about Cloudflare's approach to blocking orders in our transparency report on abuse processes and then another link report on abuse processes and then another link. So when we've previously covered this issue and we've applauded Cloudflare's adamant pro-net neutrality stance, we've cited Cloudflare's formal policy statement about this, and this may sound familiar to our listeners because it reads because this is Cloudflare speaking, because Cloudflare cannot remove content, or because Cloudflare cannot remove content it does not host, other service providers are better positioned to address these issues. Among other things, any blocking by Cloudflare is of limited effectiveness, as a website will be accessible if it stops using Cloudflare's network.

Cloudflare therefore regularly pushes back against attempts to seek blocking orders. So the point they're making here is they're sort of deflecting. They're saying if an ISP they're saying users, local ISPs are the correct and better place for blocking enforcement. Because if a pirate relocated from Cloudflare to anywhere else, well, the ISP would still block their domain. It doesn't matter who's hosting their domain, the domain is blocked. So Cloudflare is saying, whereas if you tell Cloudflare to block a domain, well, they can move to a different host where the domain would not be blocked. So this has always been Cloudflare's formal position. They're saying don't ask us to take responsibility for the content we're hosting, because there are many other hosting providers. So TorrentFreak explains Cloudflare notes that it may take steps to comply with valid orders if, among other things in this new HTTP 451 intercept page, if, among other things quote principles relating to proportionality, due process and transparency are upheld. They wrote whether Cloudflare was offering by those links. They said.

With no central repository for blocking orders and no legal requirement to share details of injunctions with the public, transparency in the UK is mostly left to chance. Some orders make their way online, but there's no guarantee For those interested in finding out more about the order affecting Cloudflare. The company provides a link which promises to reveal quote the party that requested it and the authority that issued it. The link directs to the Lumen database, which publishes information effectively donated by companies such as Google and Cloudflare for the purpose of improving transparency, but in this case, there's no indication of who requested the blocking order or the authority that issued it. However, from experience we know that the request was made by the studios of the Motion Picture Association and, for the same reason, the High Court in London was the issuing authority To the general public. The information is just a short list of domains. If it wasn't for the efforts of Lumen, google and Cloudflare volunteering, the situation would be significantly less clear than that. Would be significantly less clear than that. So TorrentFreak here is noting and complaining that there's a real problem with a lack of transparency and accountability with the way the system is working.

Now, they add, the issue lies with dynamic injunctions. While a list of domains will appear in the original order, which may or may not ever be made available, when the MPA concludes that other domains that appear subsequently are linked to the same order, those can be blocked too, but the details are only rarely made public From information obtained independently. One candidate is an original order obtained in December of 2022, which requested blocking of domains with well-known private pirate brands, including 123 Movies, f Movies, soap Today, hurra Watch and Sflix and Onion Play, and they finished. What's odd is that the notice linked from Cloudflare doesn't directly concern Cloudflare. The studio sent the notice to Google after Google agreed to voluntarily remove those domains from its search indexes if it was provided with a copy of relevant court orders. Notices like these were supplied and the domains were de-indexed, and the practice has continued ever since. That raises questions about the nature of Cloudflare's involvement here and why it links to the order sent to Google.

Notices sent to Cloudflare are usually submitted to Lumen by Cloudflare itself. That doesn't appear to be the case here, so they're just sort of generically unsatisfied with Cloudflare's lack of transparency. Maybe just that this all happened very quickly. Again, we don't know what's going on, but what's interesting is that Cloudflare is is now putting up HTTP 451 responses and blocking their own clients' access to their own clients. Oh and as for VPN circumvention, torrent Freak wrote when blocking measures are required, cloudflare digs in when requests concern its public DNS resolver, which we'll be spending more time talking about at the end of the podcast 1.1.1.1. They wrote to achieve a similar effect. Cloudflare uses another technique instead. Okay, so I assume they mean that Cloudflare still adamantly refuses to muck up their master public dns resolvers with filters? And thank god for that.

0:49:12 - Leo Laporte
Yeah, rightly so yes, we've seen horrible things happen in the past when the, the international soccer, uh broadcasters blocked right the d, it screwed up everything.

0:49:27 - Steve Gibson
Yes, we need a strong, universal DNS resolver that is not subject to the whims and needs of any particular industry or government. And, by the way, cloudflare does offer 1.1.1.2 and 1.1.1.3, which are .2 is family-friendly filtering or no? Okay, one is malware, only One is malware and family-friendly, or one is family-friendly and one is malware. Anyway, I know what they are and they're now part of the DNS benchmark, but I don't have it off the top of my head. But so Cloudflare is offering some filtering services, but if you want an industry standard, absolutely clean DNS, I'm really glad to hear that they're saying no, we are not going to start screwing around with our DNS. Torrent Freak quoted Cloudflare saying in countries with laws that provide for blocking access to online content, cloudflare may this is Cloudflare speaking, which Torrent Freak quoted Cloudflare may geo-block websites to limit access in the relevant jurisdiction to those websites through Cloudflare's pass-through security and CDN services. Okay, but they will, when required by law, typically taking the form of court orders filter access to some list of sites they host based upon the location of the client making the connection to Cloudflare's network. So they're able to do as they said geo-blocking. Torrent Freak wrote Cloudflare appears to be using geo-blocking in the UK, as some VPN users will soon find out. In normal circumstances, they said, a VPN using a server in the UK will bypass ISP blocking no differently than a server located anywhere else in the world. Users attempting to gain access to domains currently blocked by Cloudflare using a VPN server in the UK will be greeted by Cloudflare's error 451 blocking notice instead. So what they're saying here is that, whereas in the past, basically any VPN would have been useful in jumping past that user's local ISPs block be doing, any UK-based VPN users will need to be sure that they're terminating their VPNs at servers outside the UK, which would not then be blocked by Cloudflare, and that would have not been previously necessary when only their ISP was doing the blocking. And they concluded by noting that the scale of this blocking appears to be

large. Writing. Checking through the new domains blocked on the 14th, something else becomes apparent. They appear in multiple blocking orders, not just the ones highlighted in their article. They said we're unable to check all 200 domains, but at least potentially, hundreds or even thousands of domains could be involved, and that may actually be a very good thing. I thought what Okay? They said domains blocked by Sky, bpi and others don't appear to be affected, at least as far as we can determine, all relate to sites targeted by the MPA and the majority, if not all, trigger malware warnings of a very serious kind, either immediately upon visiting the sites or shortly after. At least in the short

term. If Cloudflare is blocking a domain in the UK, moving on is strongly advised. So I believe they're saying that the blocking Cloudflare has begun doing appears to relate to domains hosting malware, perhaps more than just those that the MPA may be grumbling about. So I could then see where Cloudflare is like 100% behind blocking of malicious websites or access to them. That seems like a lower bar than than getting into a an argument over copyrights. So, whatever the case, it appears that cloudflare is simply doing, you know they're abiding by the law, by the law, you know, uh, as they're required to if they're going to operate in in the uk. Although it's a little sad or disappointing, because I'm such a Cloudflare fanboy, overall, the evidence is that Cloudflare doesn't seem to be explaining very clearly exactly what they're doing and why, at least not in the notice that Torrent Freak pulled up and looked at that they received last week. Maybe it just happened.

0:55:05 - Leo Laporte
They're in a tough business. I mean they're really in a tough situation.

0:55:10 - Steve Gibson
Yes, and I'll be talking a little bit more about this later, but we're seeing more and more of this where governments are increasingly mucking around in what used to be just a hands-off, you know, fully democratized internet uh, that's only because it was under the radar. I mean, as soon as it became mainstream, they said we gotta control this yep, that's exactly what I conclude, leo, is that you know we were all having fun before it mattered.

0:55:44 - Leo Laporte
Exactly this portion of Security Now brought to you by One Password. Love these guys. You know this was kind of a stunning stat. Over half of IT professionals you know, these guys who secure your business say that securing SaaS apps is their biggest challenge. With the growing problem of SaaS sprawl and shadow IT, it's not hard to see why. Thankfully, there's a solution Trellica by 1Password. Trellica T-R-E-L-I-C-A can discover and secure access to all your apps, whether they're managed or not. Trellica by 1Password inventories every app in use at your company. Then pre-populated app profiles assess the SaaS risks, letting you manage access, optimize spend and enforce security best practices across every app your employees use, whether you know about it or not. Well, now you will know about it. Right, You'll be able to manage shadow IT. Oh, it also lets you securely onboard and offboard employees and make compliance goals. So it really is a great tool. Trellica by 1Password provides a complete solution for SaaS access and governance, and it's just one of the ways that extended access management for 1Password helps teams strengthen compliance and security.

Password helps teams strengthen compliance and security. 1password's award-winning password manager is trusted by millions of users and over 150,000 businesses, from IBM to Slack, and now they're securing more than just passwords with 1Password Extended Access Management. 1password is ISO 27001 certified, with regular third-party audits and the industry's largest bug bounty. 1password exceeds the standards set by various authorities and is a leader in security. So take the first step to better security for your team by securing credentials and protecting every application, even unmanaged shadow IT. Learn more at 1passwordcom slash security now that's 1passwordcom security. Now that's onepasswordcom security. Now all lowercase, thank you. One password for supporting security now and the good work steve gibson's doing here, kind of similar to what you're doing with one password extended access. Now back to mr gibson. So um this.

0:58:05 - Steve Gibson
The heat surrounding internet user age verification continues to increase. Um, I'm encountering an increasing level of pressure in like just more and more in the various news items that I survey. For example, last thursday, roblox posted an update which included this under the headline age estimation technology to help deliver tailored and developmentally appropriate experiences, while aiming to protect its community from those who might seek to do harm. To add contacts as a trusted connection, users must be 13 and over and confirm their age using a video selfie, which is analyzed against a large, diverse data set to estimate their age.

Matt Kaufman, roblox chief safety officer, said quote we know teens want more freedom to chat more freely with their friends. We believe that unfiltered chat should only be made available to users who have been age checked, which is why we're using new age estimation tools to unlock access to trusted connections for those 13 and over. We believe this additional freedom to chat more openly will reduce the incentive for teens to move interactions off platform, where they may be exposed to greater risk. Unquote. So okay, you know how, how I don't know, subject to maybe tampering or spoofing or whatever uh, facial recognition could be trying to guess someone's age. I mean, I'm not a teenager. No one's gonna confuse me for that. But still, you know, down in the you know 13 or so.

1:00:01 - Leo Laporte
You look pretty young, you could pass for 65. At least Give me a wig, maybe.

1:00:09 - Steve Gibson
So okay, there's that. Steam reports that they're being pressured you'll pardon the pun over some of their content by the payment processors believe it or not that they use. In response, rather than risk losing their payment flows, steam has reportedly removed thousands of games containing adult content, though what that is remains unclear should be. Last Friday, eurogamer asked Valve for some clarification and then wrote this of their response, they said in response to questions from Eurogamer regarding Steam's new guidelines preventing quote certain types of adult content unquote from being distributed on the platform, valve has provided some general background on the events leading to the decision. A Valve spokesperson told Eurogamer, quote we were recently notified that certain games on Steam may violate the rules and standards set forth by our payment processors and their related card networks and banks. As a result, we are retiring those games from being sold on the Steam store because loss of payment methods would prevent customers from being able to purchase other titles and game content on Steam. So okay, in this case, thousands of titles are being removed without regard for the age of the user in what appears to be a case of looks like blackmail censorship by Valve's payment providers. So I'm sure it must be clear to everyone by now that the need to verify the age of Internet users is not off someday in the future. We need the W3C or the IETF or perhaps the FIDO Alliance if any of them could move at anything other than glacial speed to get busy and whip up some standards, because we need some technology here. Then we need Google and Apple to implement them in their biometrically equipped devices, and my concern is that these things are so expensive, these high-end smartphones, that there would be a place for someone like a next generation Yubico. You know to create cute, inexpensive little spoof resistant thumbprint authenticators that would follow the same specification, which unfortunately doesn't yet exist. You know, and we need all of that yesterday, because the need for age verification is today. So imagine that a Yubico-type thumbprint sensor age verifier existed.

If you have a biometrically lockable smartphone, then you wouldn't need an extra gadget because the phone you've got would be able to do that. But, as I said, my concern is that such smartphones are very expensive. So we need a $20, $30, $40 alternative. If you don't have some suitably equipped smartphone, you buy an inexpensive gadget from a local retailer, a neighborhood electronics store outlet, whatever. So in my little thought experiment here, how do we arrange to create the binding between the user's biometric and an assertion of their age. And how do we do it at scale? Someone who wishes to enroll their iPhone, their biometric Android device or some inexpensive theoretical thumbprint verifier takes their chosen device to any US post office in the US the DMV, maybe, aaa, if you have a membership, or any notary like is available at any UPS store in the United States. You show them your government-issued ID proving your age. They check it carefully for forgery, look at you, look at your ID and then have the user in front of them authenticate with their chosen biometric their face or their thumbprint, depending upon their device after which the agent uses their own device any NFC-equipped phone or terminal or Bluetooth or whatever to essentially bless and activate and lock that biometric age binding. Now this individual is in the possession of a biometrically locked age assertion which they can use on demand anywhere in cyberspace that it's needed.

Again, we don't have the protocols. We don't have, as far as I know, any little well, there is no protocol, so there's nothing for anybody to implement on any platform at this point. But there's a little bit of brainstorming about how we might begin to solve this problem and this is no, it's a good thing, leo, that I'm very committed right now to the projects that I have in front of me, because this is pulling me in the same way that Squirrel pulled me 10 years ago, and we know how that went. It went seven years of my life, but anyway, it seems to me this is so necessary. A bit later in today's podcast, in answer to one of our listeners' questions, I'm going to sketch out an example of a cryptographic protocol to provide, again, just a rough sense for some more of the details of this.

But my overall point is that the problem is not intractable, but it's not easy either and people need to get moving on this, and I don't see any sign of this happening, even though Yubico's founder, stina Ehrensvard, has moved on to other passions. I dropped her a note yesterday as I was writing this passions. I dropped her a note yesterday as I was writing this. She's the perfect kind of person to shake things up and get the industry's attention and get this moving. I did get email back from her. I found it waiting for me this morning saying that she's established a nonprofit which doesn't seem focused on age, but she's still on the identity crusade. She did, I know I did tell her about my concern over the need for some sort of workable privacy respecting age verification, and she said that that's what she was doing and wanted to set up a conference and see how we could collaborate. And I again, I don't want to get too sucked into something because I've got work to do, but this just to me this seems like like one of the biggest needs we have, because the world is starting to wake up to the Internet, it seems, and the age of the people using it is suddenly a big deal. So we need protocols. I hope somewhere that's beginning to happen.

In other news, it appears that Microsoft remains unsure what to do about the fact that no one appears to actually want their new crap, especially in light of the fact that Exchange Server, in this case, is switching to a subscription. What a surprise. I guess no one should be surprised that no one is in a big hurry to switch to subscription mode is in a big hurry to switch to subscription mode. Everyone wants to just keep using the stuff they already have. That's working just as well as any of the new stuff probably would, especially when they already paid for the stuff that they have. That's all installed and running and configured and working just fine. So in this case, we're talking about Exchange 2016 and 2019 server, whose end of life is scheduled for that same fateful day approaching us, on October 14th, when Windows 10 and some other Microsoft products that no one wants to be forced to stop using were originally scheduled to stop receiving their security updates.

But because users of Exchange Server are not just some rando consumers, anyone who has so far refused to jump at the opportunity to switch to their marvelous new pay-as-you-go subscription plan for Exchange Server is going to need to pay up, and Microsoft says that's it. We're serious this time. No, really, no kidding. This is it. We're serious this time. No, really, no kidding. This is it. You're actually going to have to do this. They actually wrote don't even bother. Asking for more.

So last Tuesday's Exchange team blog posting under the headline announcing Exchange 2016-2019 Extended Security Update Program. They wrote with both Exchange 2016 and 2019 going out of support in October 2025, we've heard. I bet they have. We've heard from some of our customers that they've started their migrations to Exchange Subscription Edition Literally, it's SE for Exchange Subscription Edition but might need a few extra months of security updates for their Exchange 2016-2019 servers while they're finalizing their migrations. We are announcing that we now have a solution for such customers, starting on August 1st 2025. So the end of this month, august 1st, customers can contact their Microsoft account team to get information about and purchase an additional six-month extended security update ESU for their Exchange 2016-2019 servers. Your account teams will have information related to per-server cost and additional details on how to purchase and receive ESUs starting August 1st 2025. Now logic would suggest you know that the stay right where I am for the next six months plan will cost more than the. You know that subscription sounds great sign me up plan and you know, no one ever accused Microsoft of leaving any money on the table. So it will almost certainly cost those foot draggers more than getting on with the new plan.

Microsoft continued writing. So what does this mean? They said first, this ESU is not an extension of the support lifecycle and they said Microsoft lifecycle policy, microsoft Learn for Exchange 2016-2019. Those servers still go out of support on October 14th 2025. And you will not be able to open support cases for them unless directly related to an issue with a SU released to ESU, that is, a service update released to ESU customers during the ESU period. So they said the ESU is not an extension of the support lifecycle. Okay, I don't understand why, because that's what they're selling you. They said this ESU is a way for customers who might not be able to finalize their migrations to Exchange SE the subscription edition before October 14th, to receive critical and important updates, as currently defined by Microsoft Security Resource Center, scoring as SU's security updates that we might release after October 2025. Okay, so I guess what they're saying is you have to have signed up for the subscription, but we understand you may not have yet finished migration to the subscription servers or from your non-subscribed servers Exchange Server 2016 and 2019. So you can buy additional support for them.

In order to bridge, they said, exchange 2016, 2019 SUs these service updates will not be released on public download center or Windows update after October 2025. So they're still trying to be as strict here as they can. They also said we are not committing to actually releasing any service updates during the ESU period, meaning you pay for it and you may not get anything. They said Exchange Server does not necessarily receive security updates every month on Patch Tuesday, as security updates are released only if there are critical or important security product changes. Therefore, if there are no SUs that we need to release during the time of ESU, there will be no such updates provided. We will, however, confirm with ESU participants each patch Tuesday whether an issue was provided or not.

This ESU will be valid, they said, for six months only, through April 14th, 2026. And they wrote this period will not be extended past April 2026. You do not need to ask 2026, you do not need to ask so anyway, that's the story. If you are an enterprise, you're not going to be ready by October 14th to stop receiving any security updates for your existing Exchange 2016, 2019 servers, then you can buy any that may occur. I wonder if you could wait to see if any occur and then buy them then. I don't know. Anyway, they finished explaining. They said customers using Exchange 2019 should in-place upgrade to Exchange SE quickly and switch to the Exchange SE modern lifecycle policy meaning, yes, the modern lifecycle policy, also known as the, will no longer allow you to purchase it in these modern times. You now keep paying for it forever. So, anyway, for what it's worth, the uh, the wonder and clever folks over at zero patch you know it's numeric zero, p-a-t-c-h dot com.

The zero patch guys do provide patches for exchange server, and they do so on very reasonable terms. So it might be more cost effective to consider remaining with the already paid for in full exchange server you already own and then having the zero patch folks keep it up to date for you. You know, basically they recreate microsoft's patches, they reverse engineer them and then offer them like they don't even have to reboot exchange server. Right, I mean, it's like way better than microsoft. Um, until april 14th, when that, when those older servers will no longer be receiving security updates for the micro patch guys to reverse engineer and I don't know whether they can look at the security updates for the next generation of Exchange servers and backport them to the earlier editions of Exchange server. We'll have to see at that point. But don't forget those zero patch guys. They're going to be friends of Windows 10 users also starting October 14th, as we talked about before.

So wow, a new Russian law has get this. Leo criminalized online searches for controversial content, such content or obtaining it, but with officials saying that censorship during wartime is justified. That is, they're using their war with Ukraine as the context here. They're saying restrictive digital laws are justified and being tightened. The Washington Post reported this on this last Thursday. Writing.

Russian lawmakers passed controversial legislation Thursday, meaning last Thursday that would dramatically expand the government's ability to punish internet users, not for sharing forbidden content, but for simply looking it up, like putting the search term in. The new measures, which sailed through the Russian parliament and will take effect in September, envision fining people who quote deliberately searched for knowingly extremist materials unquote and gained access to them through means such as virtual private networks or VPNs, which lets users bypass government blocks. Vpns are already widely used in Russia to circumvent the many blocks on websites. The Washington Post wrote. Russia defines extremist materials rather broadly, as content officially added by a court to a government-maintained registry a running list of about 5,500 entries at the moment or content produced by extremist organizations ranging from LGBT movement to Al-Qaeda. The new law also covers materials that promote alleged Nazi ideology or incite extremist actions.

Until now, russian law stopped short of punishing individuals for seeking information online. Only creating or sharing such content was prohibited. The new amendments follow remarks by high-ranking officials that censorship is justified in wartime. Adoption of the measures would mark a significant tightening of Russia's already restrictive digital laws. Similar legislation, they wrote, passed recently in neighboring Belarus, russia's close ally, ruled by authoritarian leader, alexander Lukashenko, and has been used to justify prosecution of government critics. The fine for searching for banned content in Russia would be about $65, while the penalty for advertising circumvention tools such as VPN services would be steeper $2,500 for individuals and up to $12,800 for companies. Sarkis Darbinian, an internet freedom activist whom the Russian authorities have labeled a foreign agent, said the fines imposed for searching for extremist materials in this iteration may be minor, but this can be grounds for detention, pressure or a pretext to be escorted to the police station. I am most afraid that in the next iteration, administrative fines will turn into criminal cases.

Previously, the most significant expansion of Russia's restrictions on internet use and freedom of speech occurred shortly after the February 2022 full-scale invasion of Ukraine, when sweeping laws criminalized the spread of so-called fake news and discrediting the Russian military. The new amendment was introduced Tuesday, attached to a mundane bill on regulating freight companies. According to documents published by Russia's lower house of parliament, the state Duma we talked about before, net Freedoms, an advocacy group, said in a statement, lawmakers have repeatedly used this cunning tactic of quietly inserting repressive measures into dormant, previously introduced bills. It allows them to accelerate the legislative process, moving through the second and third readings in a single day and to avoid public scrutiny. On Wednesday, as news of the censorship amendment sparked widespread concern in Russian media, lawmakers pushed the bill. Sought to down.

Pushing the bill sought to downplay fears that citizens would be penalized for browsing the web. Senator Artem Shikin, one of the bill's authors, told state-controlled news agencies that the new measures are not intended to punish individuals for accessing prohibited websites using VPNs. Reading Facebook or scrolling through Instagram, shikin said, does not constitute an administrative offense. The main focus is on regulating providers. He said there's no plan for mass punishment of users. He claimed that liability would only attach in cases of knowingly searching for and accessing content officially designated as extremist by a court and added to a Ministry of Justice blacklist.

However, he did not explain how authorities would determine whether an individual knew the access content was deemed extremist and they used the term throttling, talking about how Russia has also expanded its use of deep packet inspection technologies enabling more precise blocking of traffic, and committed millions of dollars to fortify what we know RussiaNet or RUNet. It's creating this sovereign Internet infrastructure that allows them to pull the switch and disconnect, use domestic platforms instead of the foreign ones by throttling or restricting platforms such as YouTube X and Instagram, as the Russian government seeks to limit access. And you know, we talked about the use of the term throttling because Cloudflare sites were recently added to this throttle technology, where a page was limited to 16K bytes. If it came from Cloudflare which, as I observed, was really not enough to run any like, even begin to get a modern web page off the ground, maybe you could do a 301 redirect. Well, you could do that in 16K, and that was the only explanation that I could come up with. But, as we've said, any site that had content that was of interest to Russians could just move to a Russian hosting provider in order to get around that block, which is probably the whole goal here.

So for me, this news is disturbing. I'm not in Russia, but Russia is an extreme example of what we're seeing everywhere, this general tendency globally from the world's governments. The UK and the EU are chafing over encryption and arguing against fundamental privacy rights. Here in the US, we've seen the Supreme Court just approve the means by which various extreme special interest groups will be able to criminalize essentially any internet speech that they dislike or deem to be unwholesome. The definition in the legislation that the US Supreme Court just approved is very worrisomely broad and, as I was saying, leo, before, it feels as though for the first 50 years of the internet, you know, it was not well understood and sort of remained out of bounds for the world's governments and politicians. Or, as we noted, perhaps it just didn't matter all that much until just the past decade or so. Uh, you know, we enthusiasts, we're all having a great time playing in our sandboxes with our technologies, but now the political adults have returned and they're scowling at the things that we've been up to yeah I don't know, I mean I don't know what the answer is, I mean it does it does feel like it's all changing that yeah

yeah, rapidly yeah, um, let's take another break, and then, uh, we're going to talk about, uh, a bunch more stuff and stuff, important stuff, some listener feedback and, of course, 1.1.1.1.

1:27:23 - Leo Laporte
We're gonna get there too, more cloud flare news coming up. This episode of security now is brought to you by acronis. You know that name. We talk about them all the time, especially the acronis threat research unit. How many times have we quoted them in the security bulletins and information? Well, they can. They can work for you too.

You, dear IT professional, deserve fewer headaches in your life. Even something as simple as watching TV these days can become a headache. When your favorite shows are scattered across different streaming services, it's nearly impossible to find one place that has everything you need. Now, bear with me, I'm not talking about streaming TV, so much as taking the headache out of cybersecurity with Acronis, a natively integrated platform that gives you comprehensive cyber protection in a single console, so you don't have to go searching around for the information you need. If you want to know what's happening in cybersecurity, the Acronis Threat Research Unit, the TRU, is the place to go. It's your one-stop source for cybersecurity research.

Tru also helps MSPs stop threats before they could damage you or your client's organization. Acronis Threat Research Unit is a dedicated unit composed of experienced cybersecurity experts. Their team includes cross-functional experts in cyber security, ai threat intelligence. Tru conducts deep, intelligent driven research into emerging cyber threats, proactively manages cyber risks and response to incidents, plus provides security best practices to help you and your IT team in building robust security frameworks. They also offer threat intelligence reports, custom security recommendations and educational workshops.

If you're listening to Security Now, I know it means you need this kind of information. But whether you're an MSP looking to protect clients or you need to safeguard data in your own organization, acronis has what you need. It's all there in Acronis Cyber Protect, cloud, edr, xdr, remote monitoring and management, managed detection and response, email security, microsoft 365 security, even security awareness training, and it's all available in a single platform with a single point of control for everything, so it's easy to deploy and manage. If managing cybersecurity gives you a headache, it's time to check out Acronis. Know what's going on in the cybersecurity world by visiting goacroniscom slash twit and take the headache out of cybersecurity. That's goacroniscom slash twit, a-c-r-o-n-i-s. You know the name, you know the T-R-U. Maybe you got to get them working on your behalf. Goacroniscom slash twit. We thank them so much for their support of security now, very often we've we've quoted research from the China.

1:30:29 - Steve Gibson
if they're an Android phone user, turns out that Chinese authorities are using a new's able to extract geolocation data, images, sms messages, contacts and other data from third-party messaging apps. According to the mobile security firm Lockout, massistent appears to be the successor of a previous tool used by authorities named MF Socket. And, just as another note, anyone switching to the use of a burner phone should probably begin using it sometime before the trip so that it can accrue some believable history. Trip, so that it can accrue some believable history. There have been instances of people being further harassed when their use of a burner was made obvious by its lack of any extractable historic data. You give the authorities an empty phone and they stare at you and go okay, where's the phone you actually use?

1:31:42 - Leo Laporte
Of course there's a certain irony in this, because same thing happens when you enter the united states if you're a foreign national and actually it's good, it's funny.

1:31:51 - Steve Gibson
You mentioned that, um, this mailing, uh, and the show notes went out yesterday afternoon and I got a note from a listener saying, uh, for what it's worth, the USA is just as bad. Right, and I actually included that in next week's show because I wanted the reality check that. Yes, it's not like our hands are completely clean in this either.

1:32:21 - Leo Laporte
You know, in general, I think if you're going to travel internationally, you need some sort of plausible deniability. Maybe get a chromebook and I don't know, and a burner phone and wow. No, it's sad that we have to do that. I, you know, honestly, I I have no plans to travel outside the us for the foreseeable future. For that reason, yeah, it's changed it's changed I love china by the way, I love visiting china. It's an amazing country, yeah, yeah.

1:32:44 - Steve Gibson
After encountering the following bit of news, it occurred to me that perhaps remote web management access of any kind, regardless of how well authenticated its designers and deployers certainly believe it to be, it's really risen to the status of the much heralded buffer overflow or overrun. You know, it's just. It tops the list of recurring ubiquitous. This time, with the Shadow Server Foundation, have found web shells that is installed by maliciously installed web shells on. Almost 80 hackers exploited a recently patched vulnerability cve 2025, 25, 25, 7. The bug here it comes again is a pre-auth sql injection in the firewalls web panel. Fortinet has not yet confirmed in the wild exploitation, which I thought was humorous. Apparently, they're the last to know, since 80 individual instances of a Fortinet-Forta web firewall compromise ought to be pretty easy to confirm. You know, it sounds like they just may not want to pull their head out of the sand and be in any big hurry to confirm it officially, but anyway, again, historically it's been buffer overruns.

That's been the mistake everybody keeps making. Well, it now looks like that's been supplanted by web portal compromises. We seem unable to put up a web portal whose authentication cannot be bypassed. So you know, of course, my conclusion is so don't put them up. Just, you know, restrict them in a way that is actually strong and useful, rather than relying on a username and password. That's just no Bad idea. I wanted to mention, before we talk about listener feedback, that last night I finished my very pleasurable reread of Andy Weir's Project Hail Mary novel. Oh, good.

Yeah, yeah. And Leo, I have absolutely no idea how anyone could possibly turn this into a hyper condensed, two hour enjoyable movie which is in any sense faithful to the book. I would not want to be the screenwriter or the director, and we're going to find out next, march 20th, which is its release date. I don't doubt that people who have never read the book will still love the movie. I think it looks like it's going to be a really fun movie. But the book was really terrific and whatever the movie will be, I can't see how it could possibly be anything but the roughest of outlines of the events in the book. And I was surprised.

Lori said that she felt some of the physics was kind of beyond her. She didn't, you know, track it all. I'm okay, I mean it's all. There's a lot of science in there, but um, uh, and I think she actually was being modest. I think she understood most of what was going on. But no, certainly that a lot of that won't make it into the movie, because that would be what way past your, your typical audience and probably isn't necessary. I think that's what a you know, an enthusiast who reads the, who reads the novel, wants. But anyway, I did immediately purchase Andy's second novel, artemis, and it's now loaded into the five Kindles which I use. Why five Kindles? I have one Kindle device. I've got one iPhone and three iPads and I move among them from day to day.

1:37:16 - Leo Laporte
As one does yeah, yeah.

1:37:18 - Steve Gibson
So I have my iPhone in my pocket. I've got a Kindle Mini on my nightstand next to the bed, I've got one on a big pad downstairs and then one on my Kindle device, the Oasis that I love, and I'll take that with me if I'm going to be offsite somewhere, like I'm doing transport for a friend or something, and I'll have some time to kill where I can't really do anything else. So, anyway, I'm going to read Artemis and I'll let people know what I think. Good, okay.

Bob Van Metteren said Hi, steve, just wanted to write oh, I love this. A Spinrite level three refresh of my 2017 Kindle fixed my issue. And he wrote as if I already knew that he had an issue. I couldn't find any reference to any previous feedback from him or writing to support or anything. And he said he said thank you for this amazing product. He said I'm also a loyal security now listener since 2019 and grew up with a speak and spell. Oh, yeah, that's, that's the speak and spell right there behind me, that orange thing. He said so thanks for that, too, because I was involved in its development. And he said we can infer anyway. So thank you, bob. We can infer from his note that he has an eight year old Amazon Kindle that developed some sort of problem.

Spinrite 6.1 development we learned much more than we knew there was to learn about the surprising age-related decline in the performance and reliability, which are closely related, of solid-state storage.

We also learned that Spinrite's ability to recover data that's become marginal, coupled with its rewriting of solid-state data, more often than not completely reverses this decline and rejuvenates storage. As an avid Kindle owner myself, who often exports books from the device for archiving, I am well aware that connecting a Kindle to a PC allows the PC to view the Kindle's storage as a solid state drive, and that's all. Spinrite needs to be able to work its magic on any device such as a Kindle. You know, we sometimes hear from people asking whether Spinrite is able to similarly repair and restore, like an Android, smartphone or other devices, and we tell such people that if their device allows itself to be placed into a mode where its storage is visible as a storage drive, then the chances are very good that, as Bob found with his well-used Kindle, spinrite can restore the device's proper operation and its prior performance Kind of amazing so very cool it even works on a Kindle.

1:40:32 - Leo Laporte
Remind us. But before you go on, what did you do for the speak and spell?

1:40:38 - Steve Gibson
I was involved in the LPC, the linear predictive coding of the speech. I was involved at the AI lab at Stanford.

1:40:50 - Leo Laporte
So early speech synthesis it was very early.

1:40:53 - Steve Gibson
speech synthesis Wow.

1:40:56 - Leo Laporte
Yeah.

1:40:56 - Steve Gibson
Impressive, it was fun. And back then I want to say it has 4K bit of ROM. That sounds about right. It's about right.

1:41:10 - Leo Laporte
Which is like 1K yeah.

1:41:13 - Steve Gibson
Four kilobits of ROM. Yeah, it's half a K, and so for that thing to speak at all is crazy. I mean, it sounds awful, it sounds like a robot.

1:41:23 - Leo Laporte
I've got it back here. I'll stick some batteries in it's like spell, spell relief you know, but you could understand it. I mean it worked.

1:41:32 - Steve Gibson
Oh yeah, it did work and it was astonishing at the time.

1:41:35 - Leo Laporte
So it's a shame you didn't keep it up. You could be making hundreds of millions of dollars a year in ai research right. Ai research right.

1:41:40 - Steve Gibson
Well, this was done. I don't know if TI ended up with the patents on LPC linear predictive coding, but I know that Stanford produced some of the early work and research, so maybe they took it and refined it or did something to it, yeah, as often happens.

Yep. Alan Haig said hi, steve, loved the podcast for decades now and spin right. The new version really helps with my tivo drive, which is large, he said in a recent security now you mentioned that you no longer worry books then it has many ideas of what a human might do when threatened. Could AI simply respond to a stimulus by using what it has learned? Shows could be a proper response. Couldn't it therefore replicate itself, disable electronic controls or, worse, without intent? Thanks for all you do for us all. Alan Haig in Indianapolis. And just a note, his note reminded me of my TiVos and I know you had them too, leo which I still miss to this day. That company, for its day, got so much right. While we have vastly more options than we once did, it was once so nice having everything gathered in one place. Today it's necessary to go hunting around for shows among so many disparate services. But in any event, it's very cool that Spinrite is still useful in keeping Alan's TiVo alive. And, as he says, tivo's drive being large means that before Spinrite 6.1, a full drive recover and refresh cycle would have taken quite some time, during which there would have been no media recording or playback on the TiVo. So with 6.1 being so much faster, that means much less downtime, and props to you, alan, for keeping your TiVo going. I was forced to give mine up some time ago when I went digital and I wanted to play with all these other services.

The statement that mankind has not yet created an artificial intelligence what we've been working toward for the past 100 years, although very rudimentarily in the beginning, amounts to increasingly good simulated intelligence. I really like the term simulated intelligence. I really like the term simulated intelligence. I like it because it delineates itself from true intelligence, I think, in exactly the right way, and I believe that it helps us to disentangle ourselves from the very seductive struggle to understand what it is that we've most recently created. You know, we clever monkeys have managed to create an extremely convincing and compelling simulation of true human intelligence. But no matter how good that simulation may be, it's fundamentally different from the actual human intelligence that went into creating it.

A recording of an opera singer can be indistinguishable from the original singer, but the recording is not the opera singer. A simulation is not the same as the real thing. And, to your point, alan, if an AI trained on sci-fi, as certainly they will all have been, at least in part, if they've been trained on internet accessible material, because there's a lot of sci-fi available on the internet. If it were to be prompted with language that's threatening, and if it was not otherwise restrained from answering without filtering, I agree it would be likely to respond according to its training, which might be as we would expect a truly intelligent machine to respond, because that's what it's simulating. But that would only be because what we have today are are high-fidelity simulations of truly intelligent machines.

Forty years ago, 40 years ago, edsger Dijkstra, the quite famous Dutch computer scientist and professor who's considered to be the father of structured computer programming he was the inventor, the first conceiver, of the notion of what we now call structured computer programming. He wrote an essay about the similar claims being made at the time of intelligent machines, and this was 40 years ago and previous, so he was writing retrospectively. One of the things that he wrote in his takedown of this concept of machines being intelligent was so pithy that it stuck with me. He wrote that the question of whether computers can think is just as relevant and just as meaningful as the question of whether submarines can swim.

1:47:32 - Leo Laporte
I love that.

1:47:33 - Steve Gibson
Isn't that good Wow.

1:47:36 - Leo Laporte
Let me think about that a little bit. That's great.

1:47:39 - Steve Gibson
Yeah, he said the question of whether computers can think is just as relevant and just as meaningful as the question of whether submarines can swim. So he wasn't a believer. So he wasn't a believer and at least as regards what we have today, I think he is still just as correct as he clearly was 40 years ago. Today we may have far better submarines, but that does not make them fish. Love it. Eric Southwell said Hello Stephen Leo, long time all caps his emphasis. Listener of the show I eagerly await each episode has all of our info For non-citizen residents.

The government has other databases that have unique numbers associated with people and, importantly, their birthdays. We invent a secure process where we people somehow generate a hash or are provided a hash of just our name and birthday. Possibly we generate this hash on a government website that asks for other data to prove who we are. Later, when asked to prove our age to a different website, we provide the hash. The hash can be checked against the database of hashes for proof of age. The only data transmitted is the hash data.

By design, the process from the age verifying service would be only yes to allow access or no to prevent access. I'm probably missing something obvious. It's just that it seems like we could use cryptography to provide data that's anonymous to a requester but can be verified against a database that already exists in order to prove our age or identity. He says heck, maybe a QR code would do the trick, or even a TOTP from an authenticator app or public-private key pairs, anxious to hear your thoughts, eric. Okay, so there are two primary issues. The first is spoofing. As long as there have been age-based restrictions on what someone can do or cannot do, there has been pressure to spoof one's age. The concept of the fake ID is so ubiquitous and deeply rooted right into our culture.

That it's not even a meme any longer. It's way beyond a meme. You know there's no one who hasn't heard of a fake ID. The primary classic reason for having and using a fake ID is so that its holder may fraudulently assert that they're older than they truly are that they're older than they truly are In the physical world. A higher quality fake ID will sport a photo of its underage holder. This makes contesting the ID when it's presented much more difficult. The other use case is the use of someone else's you know, some other older person's actual ID. In that case, the question is whether the photo on the ID is that of the person presenting it. In the first case, we have a falsified identifier for the person holding the ID and in the second case, we have a legitimate identifier for a different person.

So the first and largest problem as we transition into the cyber realm is how to prevent the spoofing of anyone's age assertion. This is why I've consistently referred to the need for tight biometrics as a necessary component of any effective online age verification system. If someone simply has a hash or a QR code or a public-private key pair, nothing prevents any of those technologies, which are all inherently anonymous, from being shared with others from being shared with others. The time that would be required for an underground marketplace in fake online age assertions to be established would best be measured in microseconds, you know? I mean, it would just be, it would instantly come into existence. Therefore, any technology that asserts someone's age must, you know, absolutely must somehow be tied to unspoofable biometric parameters that uniquely identify that person. Fingerprint recognition pretty much need to go hand in hand with any form of online age verification at the time of its assertion, and this, of course, presents a sticky wicket because not everyone has uniform access to some sort of biometric technology. But I said there were two important issues. The second one is no less important, and that's privacy. It will almost certainly be very important to people who wish to authenticate their age, for whatever reason there may be, that they not be individually identified as part of the requirement for doing that.

This is where last week's zero-knowledge proof business comes in. We need the ability to make a go-no-go over 18 years of age or not, or maybe it's over 13, or maybe it's Apple's age ranges. In order to create fuzziness, we need to make that assertion, and that assertion alone, without revealing anything else about ourselves, and this suggests that we need some sort of proxy, you know, a proxy to which we biometrically authenticate, to then make this assertion on our behalf. But we also don't want that proxy to obtain any information about the website to which we wish to authenticate. So we need to have a lot of blinding here. So, for example, the cryptographic tools we already have and already know well kind of provide us with a framework for a solution.

For example, just off the cuff, some website that must authenticate our age before we're permitted to enter could present a large, cryptographically unique, random token. We've talked about many times before how trivial this is to generate. The site simply encrypts a counter which only ever counts up using a secret per-site key. The output of that encrypted counter will be a pseudo-random token that has never been seen before and will never be seen again. To this token the site appends the age assertion that the site requires its visitor to validate. The user then needs to arrange to have that compound token signed by an age assertion provider. This could be anyone who participates in the system, like Apple or Google or Samsung, who have the necessary biometrics on their device, or anyone who's able to assert that they will somehow arrange to only ever sign an age assertion for someone whose age they have verified matches that assertion, so it can be broadly specified. But whatever that agency is, their reputation is on the line that when they sign this assertion and note that the entity that's being presented this age assertion to sign knows nothing about the entity or website that generated the assertion. It's just a random token with an age assertion appended to it, so the user's privacy is preserved. The signed assertion is then returned to the user and from the user then to the website, which verifies that the assertion is one that it recently issued and that it has not yet been used, since it must be single use, and that it matches the token that was issued for this user's current browser session, so that that signed assertion from someone older can't be given off to somebody younger to use. It's got to be the current browser session.

The asserter's signature is verified against the root certificates in what would become, in this future environment, the industry's common age assertion root store. In the same way that we have web browser root certificates, we would have age assertion root store. You know, in the same way that we have web browser root certificates, we would have age assertion root certificates in a in a common store, and the users then you know common storage, and the users then admitted we're having passed these tests to the age controlled website, and so you know. So in this scenario, there are a lot of manual processes. With some additional thought, it would be possible to automate and streamline this process using QR codes and so forth. So my point is this is a solvable problem. This is not beyond beyond us, but it needs to happen.

To me, it's extremely annoying that the US Supreme Court ruled that no one's First Amendment rights to protected free speech would be abridged by the imposition of this quite onerous requirement that is age verification. As we all know, at present the industry has no means whatsoever for asserting anyone's age without sacrificing all of their privacy and their individual identity. So it's very much like the UK, exactly like the UK saying you know, you must give us access to everyone's messages, or any oh sorry, to anyone's messages we ask for, while absolutely preserving everyone's privacy. Well, you can't have it both ways. And so here's the Supreme Court just saying, yeah, everyone must be able to assert their age, but no one's going to do that because it's going to be a complete loss of privacy and we have no mechanisms in place for this. So this imposition of age restriction significantly changes the nature of the internet.

Some of our listeners have forwarded links to me since I began talking about this more to commentary written by authors of websites containing, for example, salacious adult content. That's far more tame than the very broad legislation's scope. So the point has been made that this is only the initial foray and that the underlying goal is to force the removal from the Internet of any content that a minority of the US public may find unacceptable. You know, the Internet we have tomorrow may look much different from the one we have today. In some ways it'll be better, but unfortunately the control that is now beginning to be asserted can always be misused. So I don't know, leo.

2:00:35 - Leo Laporte
Yeah.

2:00:40 - Steve Gibson
Well, we'll just watch. This is the place, uh, to watch. We are gonna, we're gonna make that technology happen right here.

2:00:44 - Leo Laporte
Yeah, I think there may also be legal reasons. You can't use the social security administration of the irs databases to verify age like actually the law says you can't use a social security number?

yeah, for identification, and I don't. I think that there are lots of reasons why that data despite the fact that doge is actually now trying to unify it all that data should be protected from uh widespread use for other purposes I think we clearly need a privacy preserving, um, you know, some sort of age assertion system that everyone understands isn't revealing anything about them, but whether or not they are a certain age, I'm sure somebody's got to be working on that I stina referred to something called WW Wallet, which she said was an open source effort, and there are some.

2:01:40 - Steve Gibson
The EU has some sort of a wallet technology I don't yet know what the details are and it's got to be widespread. It's got to be in our smartphones. And the other thing that bugs me, leo, is that I can't see how this cannot, how this can possibly be free. So what we're saying is that that's not okay either.

I exactly we are having to deed democratize the internet right now. You know, if there you don't, you're not charged for access other than putting up with ads and tracking. But I can't see how we're going to be able to verify age without some technology that involves biometrics and that can't. I don't see how we make that free.

2:02:31 - Leo Laporte
I think the best, honestly, the best we could do is to put it in the hands of the parents.

2:02:42 - Steve Gibson
I know that's not a perfect solution. Um, you can't. That only. That only solves the problem with kids.

2:02:45 - Leo Laporte
How do adults who want to prove their age do well, then you don't have to, because you're presumed, if you don't have a parent blocking you, that you're an adult that doesn't work.

2:02:53 - Steve Gibson
for what the what? The us constitution just said the us, I mean the uS Constitution just said the US, I mean the US Supreme Court just said that sites can require positive confirmation, that some actual age over 18. Right, and so it's't just be a device saying that I'm old enough, because there's no proof of that being true. You know, 17-year-olds could have a device that says that. So it's a mess, but it's time for a break, and then we're going to talk about cloudflare.

2:03:37 - Leo Laporte
Okay, Uh, you're watching security Now. You see, we deal with the intractable issues of the day and we attempt to solve them with logic and and thought as opposed to emotion, and that's what Steve's so good at. We're glad you watch. We encourage you, if you are a viewer, to support the show by joining the club. 10 bucks a month gets you ad-free versions of this show in fact, all the shows we do.

Access to the club to discord, a great place to chat about this and everything that's going on. I mean the club twit is. Discord is kind of a social network that goes 24 7. It's my favorite social network. You also get special shows that we don't do anywhere else. If you're interested and I hope you are, because it really makes a difference to what we can do as a network twittv slash club twit and thank you in advance for your support. We really appreciate it. We've got a lot of great members. I just I'm thrilled about the club. It's doing very well, but it would do better if you were a member. It really would all right. Let's talk about can you call it quad one, one, dot one, what?

uh, yeah, actually that would be a lot easier than yeah, that would be a lot easier than saying one dot, one dot, one dot, one every time.

2:04:50 - Steve Gibson
So I'm going to say that during the podcast, quad one. Okay, because I've written one dot, one dot, one dot1 and I got tired of just writing it all out, it is Dotted.

2:05:00 - Leo Laporte
quads were not designed to be typed.

2:05:02 - Steve Gibson
So I have not mentioned anything about my discoveries resulting from my pretty much incessant use of the new and still developing GRC DNS benchmark. I'm excited about this. It's really current, turning out very cool, um, but I'm just I'm. I just added a new feature and I was like, okay, where's the last one I'm gonna put in? But, uh, I thought it would be very cool to allow its users to put to enter any domain name they want and then check it against all the DNS providers in the list to see whether they are filtering it or not, to be a DNS filter checker in addition to just being a performance checker. So it's broadening its scope a little bit, but in ways that I think are useful for the future and I don't want to go back to it again. So I'm putting everything in that I can think of that would be useful. So what I suspect most of the Benchmark's users are going to discover is that if you didn't have something like the benchmark to more carefully customize and personalize or confirm your own choice of optimal DNS resolvers, you probably could not go very wrong choosing any of Cloudflare's DNS solutions. Choosing any of Cloudflare's DNS solutions. Although they're not alone among the benchmark's top-rated resolvers, they're always near the top. Cloudflare is, and I've been quite impressed with what I've seen. I'll have a lot more to say about that before long.

I'm mentioning this today because exactly one week ago, as I mentioned at the top of the show on July 14th, while we were recording this podcast from just before 3 pm to just before 4 pm so right now, as it's 3.50,. So one week ago at this time, 1.1.1.1, I'll say it one time quad one was gone. It was not resolving, it was off the air, which is, you know, earth shaking, really because this resolver is so popular outage which caused their wildly popular primary DNS resolver, that Quad1 IP, to disappear from the internet for an hour. The details surrounding this event are extremely interesting and I thought everyone would enjoy learning about not only what happened but also why and how. So before I start by sharing the introduction of their report, I want to note that this is precisely why standard best practice on the internet has always been to configure a pair, at least a pair, of DNS resolvers for use by every connection to the internet.

Dns resolvers for use by every connection to the internet. Stuff happens, as they say. So anyone whose internet connection was configured to use both of Cloudflare's IPs, quad one and its secondary backup of 1.0.0.1, assuming that 1.0.0.1 did not also go offline and I was never able to confirm that either way. I'm not sure that they're not referring to both as 1.1.1.1. So it might actually be. It might make more sense to use a different provider for, if not a secondary then a tertiary DNS. But in any event, the concept is to have two different DNS resolvers resolvers, and if you had that, and assuming that quad one went off the offline but 1.0.0.1 did not, then users would have only noticed a brief stutter when quad one stopped responding.

Operating systems, all of them their TCP IP stacks that do this DNS resolution. They will first reissue their UDP DNS queries, under the assumption that the UDP packet that went out and tried to come back may have been dropped either to or from that remote resolver. Then, once the primary resolver has failed to respond to a couple of retries, all DNS resolvers that are configured on that internet interface will simultaneously be queried in parallel and the OS will then switch to using the first one to reply. So a nearly transparent switchover from quad one to one zero, zero one would have occurred for many people during that hour-long outage. Just, you wouldn't maybe have noticed anything, assuming that one zero, zero one had stayed up. And one last point, lest anyone worry that their that their lands network border router may only be assigning a single DNS IP, which is aimed at itself to, to their PCs inside the LAN. This is a common configuration and it should not be any cause for concern. In these scenarios, the LAN's router is serving as the proxy for the public-facing DNS resolvers and is using DHCP Dynamic Host Configuration Protocol to configure the client machines on its LAN to ask it for any of their DNS resolution needs, and then it will, in turn, forward those DNS queries to one or more of its configured public resolvers, which are in turn often configured and provided by the Connections ISP using also DHCP on the WAN side interface.

Okay, so what happened at Cloudflare to cause a massive hour-long worldwide outage of their flagship DNS resolver? Here's what they shared, they wrote on July 14th 2025, cloudflare made a change to our service topologies that caused an outage for 1.1.1.1. I can't help myself saying it Quad one on the edge. I can't help myself saying it Quad one on the edge, resulting in downtime for 62 minutes for customers using the quad one public DNS resolver, as well as intermittent degradation of service for gateway DNS. Cloudflare's Quad1 Resolver service became unavailable to the Internet starting at 2152 UTC and ending at 2254 UTC. The majority of 1111 users globally were affected. For many users, not being able to resolve names using 1111 resolver meant that basically all Internet services were unavailable. This outage can be observed on Cloudflare radar.

Ok, now I'm going to pause here, because this radar page of theirs is very cool. I have its link in the show notes and I've also made it this week's GRC shortcut, so you can just go to grcsc slash 1035. Grcsc slash today's episode number 1035. Or click the link in the show notes and that just bounces you to the same place. Anyone who is interested in DNS at scale will find this page very interesting.

The second chart shows the overall usage ratios of the four DNS protocols for their Quad One resolver. I wouldn't have thought this at all, I know. Wow. Traditional DNS over UDP currently commands an 86% share. Commands an 86% share.

In a very distant second place is DOT at 7.1%, then DOH at 4.7% and plain unencrypted TCP at 2.2%. Now, although modern browsers have settled upon using DOH for their use of privacy-enforcing DNS, when Android devices are configured to use private DNS with CloudFlares, that's DOT or any private DNS. The various private DNSs that Android devices can be configured for are DOT by default, and DOT is often preferred by IoT devices and enterprises, so that's why it's in second place, although wow, a very distant second place, you know at 7.1%. And of course, you know the reason is DNS has always been UDP, so it still holds its, you know, a grip on 86% of all DNS resolutions. Another interesting data point is that Cloudflare's Quad1 resolver receives 62.6%, so just shy of two-thirds of its requests for IPv4 addresses, whereas queries for IPv6 addresses make up 18.8%. So IPv6 requests is nearly one-fifth of the total, whereas ipv4 is two-thirds. So, yes, it's clear that ipv4 still rules, although you know less than I would have thought, to be honest yeah, yeah yeah, exactly 20.

You know for for 20, nearly 20, 18.8 to be ipv6.

2:16:13 - Leo Laporte
That's still pretty good, yeah I mean, obviously people are using 1111 are more sophisticated than a normal user, right?

2:16:20 - Steve Gibson
yeah, they have somebody fancy in the house they've deliberately chosen that right because it's not their isps dns I would suspect fewer than one percent of all internet users use a custom dns yes, yeah, it's just look it works and it's.

You know it's gonna go to their isp who's rubbing their hands together because they're getting all of the dns. Yeah, so I'll note that the dns benchmark tends to favor resolvers having ipv6 addresses, meaning that it can sit. The grc's DNS benchmark, which now supports all of those protocols IPv4, ipv6, dot and DOH consistently finds that resolvers with IPv6 addresses respond slightly faster than resolvers addressed with IPv4. And Cloudflare does have a similar pair of IPv6 resolvers. But my God, leo, the IP is just from hell. I mean, it's like well, all the IPv6s yeah.

And so it's not fun to say or fun to type, but once you do it, you end up with slightly faster DNS. So, anyway, lots of interesting stuff on that page. I commend it to anybody who's interested. So let's continue with Cloudflare's IP addresses to the internet.

This was a global outage. During the outage, cloudflare's 1.1.1.1 resolver was unavailable worldwide. Wow, yeah, I know it's breathtaking. We're very sorry for this outage. They wrote period. The root cause was an internal configuration error and not the result of an attack or a BGP hijack. In this blog, we're going to talk about what the failure was, why it occurred and what we're doing to make sure this doesn't happen again. They wrote Cloudflare introduced the QuadOne public DNS resolver service in 2018.

So that's interesting to know. It is seven years ago. Since the announcement, quadone has become one of the most popular DNS resolver IP addresses and it is free for anyone to use. And yeah, like, why wouldn't you use it? I mean, it is often faster. Actually, I wonder if it's never not faster, which would be to say, as always, faster than the ISP. I think it's always faster than my Cox. You know automatically assigned DHCP IP or DNS resolution, which is astonishing, but we'll talk about why in a minute they wrote. Almost all of Cloudflare's services are made available to the internet using a routing method known as Anycast, a well-known technique intended to allow traffic for popular services to be served in many different locations, or it should say, served from many different locations across the internet, increasing capacity and performance. This is the best way to ensure we can globally manage our traffic, but also means that problems with the advertisement of this address space can result in a global outage. Okay, so let's talk about. Let me take a break here and talk about Anycast for a second.

Several weeks ago, we mentioned that the European Union had introduced a set of its own DNS resolution services for its EU member citizens. I immediately added all of their DNS IP. Added all of their DNS IP, dot and DOH addresses to GRC's default list of resolvers for the benchmark, and I remembered mentioning on the podcast that I was quite a bit put off by their sluggish performance. In retrospect, this was to be expected, since the benchmark was actually communicating with DNS resolvers operated by Whalebone and located in the Czech Republic, and while that might be right around the corner for users in the EU, it's on the far side of undersea cables and many router hops from my location in Southern California. I confirmed subsequently with many of our EU-located DNS benchmark pre-release testers that the same DNS for EU resolvers operate quite acceptably well for anyone who is located near them. In other words, for those DNS for EU resolvers, their actual real-world performance will be a direct function of how far away the client is from the location of those physical servers whose IP addresses the client is accessing. These resolvers have so-called unicast IP addresses where traffic addressed to those addresses will be routed across the internet to wherever it is. They're located, wherever their servers are, and this is completely fine for EU citizens, since those servers will be close by and the EU certainly doesn't wish to expend their resources arranging to make their DNS for EU fast. For me in the United States, that's not a priority for them.

Okay, so what's different about Cloudflare and their Quad1 IP? That 1.1.1.1 Cloudflare IP is an Anycast address where the IP does not refer to any specific physical resolver hardware, so any traffic addressed to that IP is not routed to some resolver located at a specific location. Instead, any cast addresses will automatically route to the closest Cloudflare data center. This means that, whereas the performance of the DNS for EU IPs is determined by the client's location and their distance from the EU. Cloudflare, being a major global network provider, will have a data center that's close to everyone to everyone, and that single, ubiquitous quad one IP will automatically cause any client's DNS lookup traffic to be routed to that closest data center for its resolution. It's an extremely slick system. I mean, it's the way CDNs operate and it explains how Cloudflare is able to offer their super high-performance DNS services from a network is very few router hops away from where you're located.

Okay, so back to Cloudflare. They wrote. Cloudflare announces these Anycast routes to the internet in order for traffic to those addresses to be delivered to a Cloudflare data center providing services from many different places. Most Cloudflare services are provided globally, like the 1.1.1.1 public DNS resolver, but a subset of services are specifically constrained to particular regions. These services are part of our data localization suite, which allows customers to configure Cloudflare in a variety of ways to meet their compliance needs across different countries and regions. One of the ways in which Cloudflare manages these different requirements is to make sure the right services IP addresses are internet reachable only where they need to be, so your traffic is handled correctly worldwide. A particular service has a matching service topology, that is, traffic for a service should only be routed to a particular set of locations within specific locations, which sounds suspiciously like this large collection of domains that dropped off the net for the UK at exactly the same time of this outage. Very clever, isn't that interesting? So they wrote on June 6th, during a alongside the prefixes that were intended for the new DLS service, that fundamental configuration error, which lumped the universal availability of the Quad1 DNS IP in with some others, occurred back on June 6th when they were preparing, not in July, so it was more than a month ago, more than a month old when it happened. They explain this configuration error, they wrote, sat dormant in the production network as the new DLS service was not yet in use, but it set the stage for the outage on July 14th. Since there was no immediate change to the production network, there was no end user impactth. Since there was no immediate change to the production network, there was no end user impact and because there was no impact, no alerts were fired. Their report then lays out a detailed minute by minute and hour by hour timeline of what they call the event At 2148 UTC, just before 3 pm during last week's podcast recording, you know, the you-know-what started to hit the fan.

They detailed this. They said a configuration change was made for the DLS service. The change attached a test location to the non-production service. This location itself was not live, but the change triggered a refresh of network configuration globally, meaning a BGP rerouting of traffic, and I'll explain more about that in a second. And I'll explain more about that in a second. Everything they said.

Due to the earlier configuration error linking the Quad1 Resolver IP address to our non-production service, the Quad1 IP was inadvertently included. When we changed how the non-production service was set up, the Quad One Resolver prefixes started to be withdrawn from production Cloudflare data centers globally Okay. So, as I said, everything we're talking about here is BGP, the Border Gateway Protocol, which we've covered a number of times in the past. Generally, with BGP, when something goes very wrong with the Internet due to its misconfiguration, that's what's going on, such as a mistake attempting to route all of the Internet's global traffic through a pawn shop in lower Slavovia. You know that never turns out well for anyone. So something similar happened again and with a similar outcome. Internet traffic is great and it works incredibly well right up until it utterly fails, and then it generally fails big. So at 21.52, they wrote DNS traffic to quad one resolver service begins to drop globally. At 22.01, internet service health alerts begin to fire for the 1.1.1 resolver and a formal incident event is declared 2240, a fix is deployed, a revert was initiated to restore the previous configuration. To accelerate full restoration of service, a manually triggered action is validated in testing locations before being executed. At 2254, the impact ends, resolver alerts are cleared and DNS traffic on resolver prefixes return to normal levels.

Okay, so what was the impact of this? There are some interesting details there too. They write when the impact started we observed an immediate and significant drop in queries over UDP, tcp and DNS over TLS. And they wrote most users have 1.1.1.1, 1.0.0.1, and then they list their two IPv6 IPs, which are 2606, 4700, 4700, colon, colon 1111, or same thing, and then 1010, configured as their DNS server. They said it's worth noting that DOH, the DNS over HTTPS traffic, remained relatively stable as most DOH users use the domain cloudflare-dnscom configured manually or through their browser to access the public DNS resolver rather than by IP address. Doh remained available and traffic was mostly unaffected. As CloudFlare-DNScom uses a different set of IP addresses, some DNS traffic over UDP that was also used. Different IP addresses remained mostly unaffected as well, they said.

As the corresponding prefixes, meaning BGP routing prefixes, were withdrawn, no traffic sent to those addresses could reach Cloudflare. As we can see this, they said, in the timeline for the BGP announcements and it's lower down on that same radar page I talked to before you see a spike in traffic where the withdrawals happen and then an hour goes by and another spike when the proper prefixes are being reannounced. So the second spike is when they have realized how to fix what has gone wrong and then apply that and the announcement of the update to the routers spreads out across the internet. One last bit of interesting charting that they provide I thought was very cool. It's shown as a green chart down at the bottom of the radar. They said when looking at the query rate of the withdrawn IPs, it can be observed that almost no traffic arrives during the impact window. When the initial fix was applied at 2220 UTC, a large spike in traffic can be seen before it drops off again. This spike is due to clients retrying their queries. When we started announcing the withdrawn prefixes again, queries were able to reach Cloudflare once more.

It took until 2254 UTC before routing was restored in all locations and traffic returned to mostly normal levels. So it's very cool that that chart shows the 90 minutes before the event. Everything is just, you know, puttering along more or less straight line. Then it just utterly disappears, bang, it's like a sharp edge, drops straight down to zero, which is what we would expect once the entire internet has essentially forgotten what to do with that IP. That's, what this means is that the internet, all the routers on the internet have, just they have no idea what to do with those IPs. Then, at 2220, the traffic just as suddenly skyrockets to about six or seven times its normal level. And, as they wrote, dns clients that were at that moment just discovering the outage and were retrying would have been frantically sending DNS packets out, retrying their queries, you know, basically creating an artificial tsunami, which you know can be seen at the Cloudflare resolvers once routing had been restored.

Their post-mortem posting then digs deeper into how and why this happened. I'll share one paragraph of it and see if this doesn't sound hauntingly familiar to what we heard CrowdStrike explain almost exactly one year ago. Last July, after they caused the crash of 8.5 million Windows machines, cloudflare wrote this is just last week. Cloudflare wrote. The way Cloudflare manages service topologies has been refined over time and currently consists of a combination of a legacy and a strategic system that are synchronized. Cloudflare's IP ranges are currently bound and configured across these systems that dictate where an IP range should be announced in terms of data center location on the edge network. The legacy approach of hard coding explicit lists of data center locations and attaching them to particular prefixes has proved error prone since, for example, bringing a new data center online requires many different lists to be updated and synced consistently. Okay, and here it comes, they wrote.

This model also has a significant flaw in that updates to the configuration do not follow a progressive deployment methodology. It's not progressive. Even though this release was peer-reviewed by multiple engineers, the change did not go through a series of canary deployments before reaching every Cloudflare data center. In other words, just as with CrowdStrike, there was what turned out to be too much confidence placed in their automation. So deployment was all at once and not incremental or tested in. You know, basically in place before it was let loose upon the entire planet. And, as they say, lessons learned After sharing a bunch more detail, including how the inadvertent withdrawal of the Quad One routing revealed an underlying but inconsequential BGP hijack originating from Tata Communications in India.

They conclude, writing. Cloudflare's 1.1.1.1 DNS resolver service fell victim to an internal configuration error. We're sorry for the disruption this incident caused for our customers. We are actively making these improvements to ensure improved stability, moving forward and to prevent this problem from happening again.

And after rereading all this, leo, and seeing that they talk about all four of those IPs together, my guess is that they're always referring to them collectively as their 1.1.1.1 resolver, but that all four of those probably dropped off the internet because they would have all four been served by the same data centers, all which stopped receiving their incoming packets.

So my guess is that if somebody, as most people, would have had 1111 and 1001 configured as their primary and secondary DNS, I'll bet you for an hour they had no Internet access appreciably. I mean no effective ability to look up the DNS addresses, to look up the IPs of their domains. So which explains the mea culpa there, because it's like yikes, yes, that would have been a problem. So they are already moving forward toward a better and less error-prone system to support their future growth. If nothing else, this mishap uh, much as it showed, crowd strike a year showed them the value of the planning that they have been undertaking and deploying and that they're making a necessary and important investment. I've got all the links to the original report and the cloud radar graphs at the end of the show notes for anyone who's interested. And boy, if you were wondering, if you were as I imagine our listeners probably were.

Yeah, one one, one, one and one, zero, zero one. Now you know what happened just shows how dependent on a dns resolver we are I mean completely, I mean it is so crucial to the operation of, of all of the services that we now just take for granted on the internet.

2:40:39 - Leo Laporte
So where do we stand on the dns benchmark pro?

2:40:44 - Steve Gibson
um, uh, I'm uh. That last feature I mentioned is finished the ability to to do a large, huge, wholesale, uh, an analysis of fill, of filtering against domains the bench.

2:41:01 - Leo Laporte
That's really useful. I'm glad you're adding that.

2:41:03 - Steve Gibson
I mean I adding features is a way to slow it down, but yeah that's a good one. I think that's a very useful I I just think it really does make sense to be able to, to see that with any domain name you want. So you just put test domains in.

2:41:20 - Leo Laporte
What I can't see is very important.

2:41:42 - Steve Gibson
Yeah, it'll show you what you know and confirm that your resolver is filtering. What you would hope it would be. Yeah, because you know. Good point. Yeah, good point.

Yeah, turns out that this thing is so busy, it's got so much juggling at the same time that throttling with the number of outstanding queries is tricky, because I'm also checking, like the DNS for EU resolvers and from where I am they're very slow. So that means that suddenly a lot of queries are outstanding and it tends, if they are younger than a cutoff, so that I won't get penalized for resolvers that are taking much longer to reply. Anyway, the end user sees none of that. They just go well, look, it works. But anyway, I'm very close to being done. I've got there. Windows 11 allows the OS itself to be configured to use DOH, so I need to do a little special handling of that. And then I need to spend some time with the command line features because I'm sure they're badly broken. But anyway, I'm like I'm the. All the heavy lifting is done. It supports all the protocol. You know. People test every time I do a release, people write back and go well, this just works and it's been working for like the last 12 releases.

So it's like okay I'm sure I'll break something. So anyway we're getting close.

2:43:37 - Leo Laporte
You're having fun. That's the most important part.

I'm having fun and I'm going to create a next generation very useful benchmark nice well, here's a way you can keep up on its status and find out the minute it's released is that's go to grccom slash email. That does two things. One, you can submit your email address so that steve can whitelist you, so you can send him comments, suggestions, pictures for the picture of the week, that kind of thing. But you'll see two unchecked checkboxes below it for the two different newsletters. Steve offers the weekly newsletter, which is the show notes from this show, and the very infrequent emailing about new products, things like that. If you sign up for both of those, then you'll be alerted the minute the DNS benchmark comes out. Plus, you'll get the picture of the week a day early and you can laugh along with Steve. That's just one of the many things you could do at GRCcom.

He is, of course, his bread and butter. He's the creator of Spinrite, the world's finest mass storage, maintenance, recovery and performance enhancing utility even works on your kindle, which I never would have thought of, but it makes sense. Anything that's storage right, yep, uh, if you don't have a copy of spin right, get 6.1 right now. If you do make sure you're upgraded, upgrades are free for everybody who's ever bought a version of Spinrite at GRCcom. He also has copies of the podcast there. He's got unique copies. We don't do any of the forms he does now. He has a 16-kilobit audio version for the bandwidth impaired, a 64-kilobit for those of you who just want the audio and a good quality without a lot of fuss. He also has show notes which you can, if you don't subscribe to the newsletter, download from there. That's great to read along while you're listening. He also has transcripts written by an actual human, lane ferris. So in a couple of days after the show comes out, you'll be able to get that and that you can read along while you listen.

But it also makes a great way to search for the stuff you want. All of that is at grccom. At twittv, slash SN, we have audio, 128 kilobit audio and video of the show. You can find that there. Download it at your leisure. You can also get it on YouTube. There's a YouTube channel dedicated to it. That we keep for a very special reason. It's a great way to share clips of the show because everybody has access. It's kind of the lingua franca. It's the great way to share clips of the show because everybody has access. It's it's kind of the lingua franca. It's the easiest way to share video. So if you see something you you on security now you say I gotta tell somebody about this, use the youtube channel. It's a great way to share it. Tell people about the show.

Easiest way to get the show, probably subscribe and your favorite podcast client. That way you don't even have to think about it. You'll get it automatically as soon as the show's done audio or video again, uh, whatever you prefer pocket cast, uh, you know apple's podcast. There's all sorts of places to get it. Um, if you do subscribe to one of those, please leave us a nice five-star review to let the world know how great security now is. It helps us to spread the word and that's really, I think, now that we've been doing this for almost 20 years. Be 20 years next month? Yep, uh, the job one now is just to let the world know we've been doing this. We will continue to do it as long as steve is up for it, and it's a really valuable resource. I think you'd agree or you wouldn't be here.

You can watch us live. You don't have to watch after the fact. You can watch us do it live. We stream the show right after MacBreak Weekly. That's 1.30 Pacific, 4.30 Eastern, 20.30 UTC. Roughly. We're not a TV channel. You're watching us produce the show. So roughly those times the live streams. Well, if you're in the club, it's of course on the club to discord and I apologize for the scratchiness of the live stream at the beginning. You know, sometimes that happens. We fix it as quickly as we can. There's also youtube, twitch, tiktok, facebook linkedin, xcom and kick eight different places. You can watch us live every Tuesday afternoon. I hope you will come by and watch and chat with us, but do download the show also so that you know, everybody knows we've got lots of listeners and come back next week for a great episode of security. Now. Thank you, steve.

2:47:50 - Steve Gibson
Thank you my friend, I will see you on the 29th.