Security Now 944, Transcript

Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show.

Leo Laporte (00:00:00):
It's time for Security Now, Steve Gibson is here and we have a very big show for you. Lots of information, but most importantly we're going to talk about the largest DDoS attack of all time, how it happened, why it happened, and what companies and more importantly server makers can do to stop it. That's next on Security Now,

Steve Gibson (00:00:26):
Podcasts you love from people you trust. [00:00:30] This is twit.

Leo Laporte (00:00:37):
This is security now with Steve Gibson. Episode 944 recorded October 17th, 2023: Abusing HTTP 2 Rapid Reset. This episode of Security Now is brought to you by Melissa. More than 10,000 clients worldwide rely on Melissa for full spectrum data quality and ID verification [00:01:00] software. Make sure your customer contact data is up to date this holiday season. Get started today with 1000 records cleaned for free at melissa.com/twit and by Duo. protect against breaches with a leading access management suite, providing strong multilayered defenses to only allow legitimate users in. For any organization concerned about being breached and in need of a solution, fast Duo quickly enables strong security and [00:01:30] improves user productivity. Visit cs.co/twit today for a free trial and by bitwarden, get the open source password manager that can help you stay safe online, get started with a free teams or enterprise plan trial or get started for free across all devices as an individual user at bitwarden.com/twit. It's time for security Now, the show we cover your [00:02:00] security and safety and privacy and all that good stuff online, sometimes offline as well with this man. Now I usually point to over my left shoulder, but he's over to my right now. No, my left. Here he is. Steven Gibson grc.com. For those that don't watch the video, I am at my mom's house doing the show from there instead of the usual studio. So we're a double box, you and I.

Steve Gibson (00:02:25):
Yes we are. And for anyone who's wondering, wait, wait, wait, wait, wait, [00:02:30] wait, wait, wait. What happened to part two of the top 10 most significant cybersecurity misconfigurations was promised for this week? Well, I did say that unless something even bigger and more important and timely came along, that's what we would be doing and it, that's what happened. There is a serious new way of [00:03:00] abusing the most, I wasn't going to say the most recent HTTP protocol, but that's actually http slash three. This is, which hasn't, has not yet been widely deployed. HTTP slash two is, and it's been given the big cloud providers a run for their money and for a while now. So it is a serious new [00:03:30] form of DDoS attack against http slash two capable servers.

Leo Laporte (00:03:37):
We had that story on Sunday on TWIT that the Google had suffered the largest DDoS attack of all time

Steve Gibson (00:03:45):
By hugeness. I mean, and I

Leo Laporte (00:03:47):
Think that was HTTP two, right?

Steve Gibson (00:03:50):
That was that story. So we're going to cover a bunch of our listeners feedback and questions, which brings us some interesting [00:04:00] topics such as how have valid drives first 10 days of life been going and what more have we learned about the world of fraudulently fake U SS B thumb drives? Should pakis be readily exportable or are they better off being kept hidden and inaccessible? Why can't a web browser be written from scratch? Can security now listeners have spin rights 6.1 early like now? [00:04:30] What was that app for filling a drive with crypto noise and what's my favorite iOS one-time password app and couldn't Google Docs H T M L exported links being redirecting be of user privacy feature. After we address those terrific questions and a few more posed by our listeners, we're going to take a look, as I said, at the surprise emergence [00:05:00] of a potent new HTTP two specific DDoS attack. Is it exploiting a zero day vulnerability as CloudFlare claims or is that just deflection?

Leo Laporte (00:05:18):
Oh, that's interesting. CloudFlare one of the biggest DDoS mitigation companies in the business, so they would have some inside information

Steve Gibson (00:05:26):
With the, they caught with their DDoS down baby.

Leo Laporte (00:05:29):
They might have some [00:05:30] incentive for dissembling

(00:05:32):
Interesting stuff. And we also have one of the funniest pictures of the week ever. I snuck, I snuck ahead, I confess I did. We'll get to all that in just a second. But first a word from our sponsor and we thank him so much, Melissa, you've heard me talk about Melissa, the contact data quality expert this holiday season. It's even more important than ever to let Melissa help your business meet online shopping expectations. [00:06:00] Increase R o I reduce waste and costs associated with lost and undeliverable packages and improve overall customer satisfaction. We probably do more mailings over the holidays than any other time of the year, right? And if you're a retailer, early preparedness is extremely important. It's not too early to start thinking about this. Here we are in October. Here are some of Melissa's tips to help you get started preparing for the busy season.

(00:06:27):
Start by cleaning out [00:06:30] your contact data with Melissa's data cleansing solutions. You want to get rid of all the old stale, outdated data and you don't want to get rid of those customers, but you want to replace the outdated data with verified accurate information, replace old addresses for people who've moved, add new emails, people change emails all the time or phone numbers. Make sure that you're getting a hold of the customer. By the way, I always say customers, but it's also suppliers, it's vendors, [00:07:00] it's the people you deal with, right? All those databases wear out, they go bad, they go stale over time. Customers of course want to seamless experience with delivery. I mean, this is the one time you don't want a punt of delivery, right? Make sure your business is meeting their expectations next day and two day delivery implementation in high demand.

(00:07:21):
As always this year, militia insurers addresses are verified and standardized at checkout using their auto complete tool. By the way, this is [00:07:30] one of the main places that stuff gets messed up. You get a transposed digit in a zip code or a phone number or worsen address, and sometimes those packages do just go into the wrong place. Not only does having a verified address to checkout ensure that that address is correct, you also are verifying that it's deliverable. You don't want to take an order for a address you can't deliver to. It also cuts down errors in keystrokes, 75% [00:08:00] so your customer, you've seen it. You probably used vendors that use Melissa. It's very popular and you've seen it when you start entering the address and Melissa already knows and we'll even correct it and get the zip code plus four and all of that.

(00:08:14):
That's Melissa, makes your experience as a customer faster and easier and it makes the supplier's experience much better because they're going to send it to the right address, right? One of the things Melissa can help with offering bundles and cross-selling [00:08:30] to existing customers, right? It's hard to get new customers, but you've already got a huge customer database. People who love your service, who love your product, who are going to maybe even order again, being able to reach them effectively as we approach the holiday season is very important, especially since many of those people are shopping for others. Hey, I love this lawnmower. I got to get it from my buddy's. Not a good example. Profiling the data in your customer database can give you a better understanding of your customer base [00:09:00] and bestselling products that can help you create more effective marketing strategies. Melissa can also enrich your customer data so you know more about their socials, their marital status, their demographics.

(00:09:11):
That's very valuable when it comes to marketing to them. Matching and de-duping data cleans the database as well, gives you a complete 360 degree view of each customer with more personalized marketing, a better customer experience and de-duping is really important. It eliminates multiple mailings to the same person that annoys [00:09:30] them and it wastes your time and money. Since 1985, Melissa has specialized in global intelligence solutions and contacts data quality. Melissa takes care of your data, of course, like it's their own. They undergo continuous independent security audits. They publish those audits on their webpage and they are SOC two. HIPAA and G D P are compliant. Make sure your customer contact data is up to date. Get started today with 1000 records cleaned for free at melissa.com/twi [00:10:00] melissa.com/twit and now our guy on the job for security, and I guess you're probably the number one place people send silly pictures these days.

Steve Gibson (00:10:16):
I think we've pretty much cornered the market on that. This one.

Leo Laporte (00:10:21):
So

Steve Gibson (00:10:22):
Good. So what we have is at least a four story building because we can see [00:10:30] four stories worth of it, although the picture's cropped a little bit and you know how code requires that You have often two exits from a domicile in case a fire broke out near the front door. You need to have a back way out like the traditional fire escape. Well, that appears to be what we have here because we see a line of back doors. [00:11:00] There's no numbers on the doors. They don't look like front doors. They look like

Leo Laporte (00:11:04):
Back doors. They're escape doors. There are escape doors. Yeah.

Steve Gibson (00:11:05):
Yes, unfortunately if you were to open the door and run out, that would be bad.

Leo Laporte (00:11:14):
Wiley coyote off the cliff

Steve Gibson (00:11:18):
Because for a reason that's difficult to explain. The stairs are shifted off to the side. So [00:11:30] there's like a spiral staircase where each of the landings of the spiral would correspond with the height at the bottom of the door like you'd expect, except that they're about what, six feet off to the side. And Leo, what occurred to me is that they're lined up with the windows because there's also windows off to the side. And anyway, I gave this picture the title slavish [00:12:00] devotion to the specifications as if maybe somebody was looking at the plans and got the windows and the doors exchanged. I could see the memo. They didn't stop to ask why are we putting the stairs over here where there's no doors?

Leo Laporte (00:12:22):
You can see the memo to tenant saying, now just a reminder, when you wrote the fire escape, take a sharp. Right? Okay. Very, very important. [00:12:30] Somebody in the chat room said, and this might actually be right, that maybe they're prefabbed and they're moving those into place and some smart photographer got the shot before they actually

Steve Gibson (00:12:40):
Got, oh, that's good. That

Leo Laporte (00:12:43):
Would make sense. They do prefab those things. I know

Steve Gibson (00:12:46):
That would explain it. So they tilted it up and it's against the wall.

Leo Laporte (00:12:52):
Now they're it over. Now they're going to move it

Steve Gibson (00:12:53):
Over and then anchor

Leo Laporte (00:12:55):
It

Steve Gibson (00:12:55):
To something. At

Leo Laporte (00:12:56):
Least I hope that's the case. If they built it like that,

Steve Gibson (00:12:59):
Wow,

Leo Laporte (00:13:00):
[00:13:00] Somebody ought to get fired.

Steve Gibson (00:13:03):
Yeah, I don't think that would pass the code inspection. Let's see how how's your exit? Well, don't

Leo Laporte (00:13:10):
Walk through and that's easy. When the inspector wants to see the exit, you just show them the door.

Steve Gibson (00:13:14):
That's right. Straight

Leo Laporte (00:13:16):
Through there.

Steve Gibson (00:13:17):
See you later.

Leo Laporte (00:13:18):
That's right.

Steve Gibson (00:13:19):
Okay, so I wanted to follow up a bit on valid drive. We are 10 days out from its release and we're currently seeing a seven day moving average, [00:13:30] which is what my server produces of twenty four hundred and forty three downloads per day.

Leo Laporte (00:13:36):
Wow.

Steve Gibson (00:13:37):
With a total of 27,603 total when I looked this morning.

Leo Laporte (00:13:43):
That's amazing.

Steve Gibson (00:13:44):
That download rate currently places it at the top among GRCs freeware goodies. It has managed to push GRCs always. Number one, which is the d n s benchmark [00:14:00] that's currently being downloaded 1,867 times per day. It's pushed it to the number two spot, although, yeah,

Leo Laporte (00:14:08):
But by the way, that's been out for years. That's pretty impressive

Steve Gibson (00:14:11):
Still. Yes, I was going to. That is the d n s benchmark has demonstrated somewhat astonishing endurance.

Leo Laporte (00:14:18):
It's

Steve Gibson (00:14:18):
Got more than eight and a half million downloads

Leo Laporte (00:14:22):
Since

Steve Gibson (00:14:22):
Its release. So Well, I still

Leo Laporte (00:14:24):
Recommend it all the time. I mean

Steve Gibson (00:14:26):
It's really, well there isn't really an alternative. It is the right one and it does run under [00:14:30] wine. So Linux people are able to run it through.

Leo Laporte (00:14:33):
Well, that's one problem I have with valid drive. You really have to do it on Windows, right?

Steve Gibson (00:14:36):
Yes, you do. And that's one of the questions that we'll be addressing here in just in a second. But anyway, so I think it's fair to say that the D n s benchmark has become a staple on the internet, and I expect that valid may have similar long-term legs, although I would expect at probably a lesser average rate once the news of its existence [00:15:00] has settled down a bit. Anyway, I just

Leo Laporte (00:15:01):
Hope that the people selling crappy hard drives on Amazon, don't figure out where you live because you're kind of cutting into their business there, Steve.

Steve Gibson (00:15:10):
Well, so for example, I got some feedback from someone whose handle is QQ Q one. I don't know why it's easy to type over there on the left-hand side of the keyboard.

Leo Laporte (00:15:21):
It is actually. That's funny. Anyway,

Steve Gibson (00:15:23):
He said, I knew these would be trash. He said, free gifts as in gifts [00:15:30] with stuff we order. We have a large pile of them. And then he sent me in his tweet, a screenshot of the valor drive results. So the drives declared size is 32 gig, the valid size is 512 meg. And so you can see in the graphic there's a little strip of green where there's actual storage [00:16:00] and then just the sea of red where there's nothing. And

Leo Laporte (00:16:04):
The point here that people who didn't hear your original discussion of this and why you did it is it's really easy to have the firmware report a larger size than is actually available.

Steve Gibson (00:16:16):
Exactly. Exactly. So

Leo Laporte (00:16:19):
You can make a cheap 500 meg drive and have the firmware say, oh no, this is 32 gigs. And the sad thing is a lot of software won't even question it and will just write to these nonexisting sectors.

Steve Gibson (00:16:29):
No [00:16:30] software will, I mean it will accept a format because formats no longer actually go out and read the data. A long format does, but even a long format won't catch it because the drive reports no error when you read from this non-existent memory. So okay, it pretends to be a 32 gig drive, but it only contains [00:17:00] half a gig of non-volatile storage. So what I was wondering was, okay, it's a free gift drive, why lie? Why lie about his size? It's free and it's

Leo Laporte (00:17:18):
A gift. It's really a gift horse and you should look it in the mouth.

Steve Gibson (00:17:22):
Yes.

Leo Laporte (00:17:23):
So

Steve Gibson (00:17:24):
As we know the drives electronics proactively lie as you said about the amount [00:17:30] of storage that it contains. So it cannot be a mistake. This is deliberate. It has to be deliberate fraud. Then I had the question, but a fraud upon whom, and in thinking about this further, in the case of such freebie giveaway drives,

Leo Laporte (00:17:58):
It

Steve Gibson (00:17:58):
Occurred to me that the first [00:18:00] target of this fraud is not those of us who received these freebies. We are the victims, but we're the secondary victims. The primary victim is the entity, whomever it was who purchased these as gifts in this instance above, they believed they were purchasing probably a huge quantity of 32 gigabyte drives, doubtless at a bargain quantity discount [00:18:30] to be used as appealing 32 gig giveaway drives. So the fraud was actually perpetrated upon them by the people from whom they purchased these drives. No one wants a free 512 megabyte drive anymore. It's like, what am I going to do with this? Can I Thanks a lot. Store one photo on it these days. So anyway, our takeaway is [00:19:00] to be very cautious about the actual use of any drives that you buy for a bargain at the checkout stand as you're moving your groceries along and there's a little like a fish bowl of drives and you can have one for a buck or a drive that you may have ever received as a thoughtful thank you gift thrown in with something else

Leo Laporte (00:19:27):
For a long time. I said, drives are getting so cheap you're going to get 'em in boxes [00:19:30] of cap and crunch. That

Steve Gibson (00:19:31):
Actually

Leo Laporte (00:19:32):
Probably is true, but it won't be the drive you think you're getting.

Steve Gibson (00:19:35):
Exactly.

Leo Laporte (00:19:36):
Wow.

Steve Gibson (00:19:36):
It'll be marked 32 gig, it'll format as 32 gig and it'll look like it's storing your 32 gigs of files, but they're not there when

Leo Laporte (00:19:46):
It comes to. So it rides without error and then it's only when you try to read it and there's nothing there that you go, oh,

Steve Gibson (00:19:53):
And even

Leo Laporte (00:19:53):
Then you might not suspect. You might just think, oh I'm, it

Steve Gibson (00:19:55):
Will be a even. So in order for Valla Drive to show, because [00:20:00] I have read error and write error colors in the map, in order for it to show red, it had to valla drive wrote and red with no error. So no error was returned. The drive accepted the data from the operating system said, thanks, I got it. And then when the operating system said, give it to me back, it said, here you go. Unfortunately all Well, how

Leo Laporte (00:20:26):
Does vdrive know that it didn't work?

Steve Gibson (00:20:28):
Because it uses [00:20:30] a crypto random pattern in order and puts that out and then verifies it when it asks for it back.

Leo Laporte (00:20:39):
So the drive will say here, but what you get is nothing or some random bits that don't match the bits you wrote

Steve Gibson (00:20:46):
It. It is typically all zeroes or all ones.

Leo Laporte (00:20:49):
Okay. Okay. Interesting.

Steve Gibson (00:20:51):
And the problem is, so the drive will give you the file back.

Leo Laporte (00:20:54):
You won't get an error. You won't get an error.

Steve Gibson (00:20:57):
Try opening that in your image viewer and it's [00:21:00] not going to be happy. This is not a valid file. So anyway,

Leo Laporte (00:21:03):
It's amazing how widespread this is. Last week you talked about how many drives on Amazon you bought that. Almost

Steve Gibson (00:21:09):
All of them. I bought a dozen. They were all bad,

Leo Laporte (00:21:11):
Every

Steve Gibson (00:21:12):
Single one of them.

Leo Laporte (00:21:13):
This is terrifying. Now, you said you're going to talk about other operating systems, so I'll let you go on. I would like, I don't have a PC running windows.

Steve Gibson (00:21:23):
Well, so Elaine said, hi Steve. Thank you for your work on vdrive. I'd really like to test a few U S B sticks [00:21:30] with it, but I only have access to Linux systems. Is it possible to get this running on Ubuntu or some other distro? Okay, so Elaine and our listeners, I understand the D n s benchmark that I wrote was deliberately wine compatible. I went out of my way to make sure that I wasn't doing anything there. That wine did not support on its windows compatibility layer, and I [00:22:00] spent some time on that. But valid drives ui, which uses dynamic U SS B insert and removal events to determine which drive the user wishes to test uses features that are outside of wine's windows compatibility set. So if it was crucial for vdrive to run under wine, I could probably arrange to somehow make that happen, but I've already spent more time on it than I planned [00:22:30] to and I'm chomping at the bit to get back to finishing spin, right? If it turns out to be seeing strong long-term use, I would be inclined at once spin, right? Six one is finished to return to Vdrive and give it a significant version 2.0 update. There are some other things I could do that for the sake of expediency and not spending another few months on it. I said, no, no, no, [00:23:00] no, no, let's not go there. There are many people in the news group who are saying, Hey, how about this and how about this and how about that?

Leo Laporte (00:23:07):
I'm sure somebody's asked this, but the one thing I would say would be useful if you just wrote a paragraph on the methodology. You don't have to say how to do it, but just what works to verify. We're going to write this, whatever it is, and then maybe somebody can duplicate it on another platform. You wouldn't mind that, right? You're not making money on

Steve Gibson (00:23:28):
It. No, I wouldn't mind that.

Leo Laporte (00:23:29):
I think [00:23:30] that what we need to do is understand there are ways you could do this that would not be a valid test. Like for instance saying, can I read it? Oh, I can read it. It's good.

Steve Gibson (00:23:39):
Oh yeah, so

Leo Laporte (00:23:41):
You went the extra mile. So what is it that we need to do to really have a valid test? You've got all this data now as well and

Steve Gibson (00:23:47):
What works and what doesn't work. One thing I should mention is that it does run, for example, under Parallels vm, even on an arm Mac.

Leo Laporte (00:23:57):
Oh, that's good to know. Oh, well I would do that,

Steve Gibson (00:24:00):
[00:24:00] Yes.

Leo Laporte (00:24:01):
Oh, okay.

Steve Gibson (00:24:01):
It absolutely will. Well

Leo Laporte (00:24:03):
Then I'm set,

Steve Gibson (00:24:05):
Yeah, there is something known as safe emulation mode, which at least on an arm Mac, which is

Leo Laporte (00:24:13):
What I have,

Steve Gibson (00:24:14):
We have to turn on in order to get it to run.

Leo Laporte (00:24:18):
So you turn on safe emulation mode. I thought it was really masking the reads and writes, but of course it can't be because you have to read and write data to a drive. You have to get out of the way and let that happen. So [00:24:30] that makes sense. So if I run Windows on arm in emulation mode on my Mac using parallels, I will still be able to run it. Oh, yep.

Steve Gibson (00:24:41):
We've

Leo Laporte (00:24:42):
Got

Steve Gibson (00:24:42):
People who are running it all the time.

Leo Laporte (00:24:44):
That's good to know. That's great.

Steve Gibson (00:24:46):
Yep. Okay, so some feedback from our listeners. Mark Smo said just visiting X to contact you and he's Mark [00:25:00] CMO slash at Marcos at twit social, by the way.

Leo Laporte (00:25:04):
Yeah, a good Mastodon user. Yeah,

Steve Gibson (00:25:08):
So he said regarding Pasky and their lack of easy exportability QR codes, et cetera, have you ever thought that those exact features would make passkey vulnerable for phishing and other social engineering attacks? They're hidden from normal users for a very good reason. Someone should now figure [00:25:30] out what could be a phishing resistant way to move them across ecosystems or we could all start using squirrel

Leo Laporte (00:25:39):
Because you solved this already.

Steve Gibson (00:25:41):
I did. So I completely agree with Marco's observation. It is definitely a double-edged sword to make the pass keys less mysterious. And I have to say, Leo, your comment about the [00:26:00] lackluster success, to put it to phrase it oddly of pasties heartened me a little bit, not because I don't want the world to have a public key based network authentication system, but because the reason my squirrel system has gone nowhere wasn't just me. Of course that is the reason it's never going to go anywhere, [00:26:30] but at least passkey hasn't yet taken the world by storm. Or if it had, I would've at least expected a little bit more from squirrel, and I use the word yet, hasn't taken the world by storm since I really do fully expect that this Fido two based pass keys public key system will someday supplant the use of secret passwords, but probably not in my lifetime. [00:27:00] Seriously? Not in my lifetime. All of the evidence suggests, and I'm feeling great right now, so I'm planning to be around for a while.

Leo Laporte (00:27:09):
All the evidence it doesn't take off in the next 10 years. It's going to be something else. You think it's going to be around?

Steve Gibson (00:27:15):
No, I think they did it right. I think, and my God, if it takes 10 years to get this thing going, we don't have another 10 years to go for plan B. So seriously, all the evidence suggests that implementing [00:27:30] this sort of sweeping change really will be that slow. But the reason I took, I put all that time in a squirrel was I said, well, no one had done it, and if no one does, then it's never going to happen. Well, my system's never going to happen anyway, but at least a public key system option now exists, and the problem is it just from the user standpoint, [00:28:00] there's nothing that's obviously better about it. As you said, Leo, I mean we already have password managers. This problem has been solved. It's not solved well, and we're kind of limping along with a few arrows in our back, some of us, but it's at least it's obvious how it works.

Leo Laporte (00:28:20):
Yeah, everybody's already gone the extra mile to get that working and if they have it, that's even going to be harder to get them to switch over. They're just entering their pet's name and their birthday [00:28:30] on every site and they say, well, what's wrong? No, they don't see a problem, so what are you fixing?

Steve Gibson (00:28:37):
Yeah, so Michael Mullany, he tweeted Art PAs keys meant to be a device verification like SS s h Keys, while you could copy them between devices, best security is to generate them on the device you need to authorize. So if that device gets stolen [00:29:00] or hacked, you can revoke the key for that device. Also, if the device is hacked, logs are way more useful if it shows Paul's iPhone rather than Paul's key, and then he has to redo every passkey device he set aside from backups, I hope passkey stay non-transferable. When you put your pin or fingerprint in your phone, you're basically unlocking all the sites. This pass key has access [00:29:30] to same as when you sign into your PC with your password that has all your SS s H keys. Okay, so that's true and I think it's a good point. This would be roughly equivalent to having multiple separate username password pairs for logging into one's bank account with a different username password pair stored in each device. Then if a device was compromised, the usernames and passwords [00:30:00] that were being used by the compromised device could be independently disavowed and disabled. Now, while that's good in theory, that really raises the level of management complexity

Leo Laporte (00:30:13):
To

Steve Gibson (00:30:13):
A whole new level. Yes, there's a lot of power in that approach, but also a huge amount of responsibility for keeping track of who's on first. And okay, yes, a crazy power user could do it, [00:30:30] but we can't even get anybody to use a pass key, let alone manage them all separately. Okay. Brian M. Franklin said at SS G G R C, maybe it is still possible to write a new web browser with even better features. And then he quoted a thread in X, which I'm now calling it where from [00:31:00] someone named A K H I L who tweeted, here's the multiplayer browser engine I've been working on in action. And I have to say I love the name because it's multiplayer. He named it the braid browser as in B R A I D, like something braided, which I think is a very cool name for it. If [00:31:30] anyone who's curious could go to braid browser.com, B R A I D B R O W S E r.com. I

Leo Laporte (00:31:37):
Wish it didn't sound so much like brave.

Steve Gibson (00:31:40):
Yes,

Leo Laporte (00:31:41):
Brave. Like in hair, there is

Steve Gibson (00:31:42):
A collision there.

Leo Laporte (00:31:43):
Yeah.

Steve Gibson (00:31:44):
Okay, so okay, here we've got a guy who wrote a browser. Well, there's browsers and then there's browsers. I love the idea of the creation of hobby browsers. It's a modern take on the idea [00:32:00] of a hobby operating system and a bunch of hobby operating systems have been created. In fact, I carefully considered basing spin right's future upon a few of them, but lack of support for booting on U E F I only systems ultimately led me to the embedded R toss 32 oss that I've talked about before. Similarly, while I think it's possible to create a functional outline of a [00:32:30] modern browser, the task, as I've said, has grown in size such that it's similar to that of creating a fully functional operating system. Yeah, there's hobby browsers and then there's production scale, chromium, Firefox and Safari, and that's it. So anyway, the guy, I don't know what this thing actually does. I played the video that was there in X [00:33:00] and there was two cursors moving around separately.

Leo Laporte (00:33:05):
What a cool feature.

Steve Gibson (00:33:07):
I don't what if you're ambidextrous, maybe you don't

Leo Laporte (00:33:12):
Keep

Steve Gibson (00:33:12):
One cursor over on the left to one on the right that you don't have to go so far. I don't know. Anyway, whatever it's,

Leo Laporte (00:33:19):
That's the kind of thing a hobbyist browser would do.

Steve Gibson (00:33:21):
That's right, because that's what we really need. We need more cursors on our browser.

Leo Laporte (00:33:28):
I'd be very careful because [00:33:30] really the browser is the primary vector for malware into your machine.

Steve Gibson (00:33:33):
I know,

Leo Laporte (00:33:33):
And it's really easy to make a mistake in a browser. Maybe people are using chromium as their base for all the rem.

Steve Gibson (00:33:42):
This guy claims it's a million lines of c plus plus and Leo, every one of them is perfect. Every one of them million lines,

Leo Laporte (00:33:51):
That scares me.

Steve Gibson (00:33:52):
That

Leo Laporte (00:33:53):
Scares me.

Steve Gibson (00:33:54):
Yeah.

Leo Laporte (00:33:54):
There's a new browser called arc A R C from arc, the browser company, but it's [00:34:00] chromium and then they're doing the gooey on top of it, the chrome as it were, and that's a little more doable. I don't know if I'd trust an individual to write a browser. That's a Arc.

Steve Gibson (00:34:10):
Zi. We got chromium, we got Safari.

Leo Laporte (00:34:13):
I

Steve Gibson (00:34:13):
Think that's it. Tech Gift Horse tweeted. Hi Steve. Longtime listener, first time DMR regarding episode 4 93 the last week where you introduced that Google was embedding tracking links in Google Docs [00:34:30] exports. I recently noticed that Google is also doing this in calendar invites no privacy. I'm not sure if they were trialing it in Google Docs, H G M L exports and is now it is expanding, but I thought it was noteworthy. I also wanted to say that you and Leo have inspired me to start my master's in cybersecurity.

Leo Laporte (00:34:55):
Oh, that's great. Oh, that's great. So

Steve Gibson (00:34:57):
Please keep the pods coming and keep up [00:35:00] the great work. Can't wait for your forthcoming email option so I can delete this bogus Twitter account. Sincerely,

Leo Laporte (00:35:07):
Sean, people are going over to Twitter just to talk to you.

Steve Gibson (00:35:12):
Yeah, they are. I mean, I'd seen enough of that and that was why when I said I'm leaving Twitter, what I meant really was I'm not going to make everyone use Twitter as a means of contacting me. That's the problem. So just as soon as I have spin, right, six one ready for rollout, [00:35:30] I'll let everyone here know of course to get the official final. That's the super easy communication path, Leo, that you and I have built with all of our listeners every week. Then I'll begin work on bringing up a new email facility so that I'm able to get the word out to all of six point OH'S current owners, and part of that work will be to create a new means for me to send announcements to my Twitter followers and [00:36:00] also create a means for them to send me the equivalent of private dms. Brent Russell said, hi, Steven, being formal an calls me Mr. Gibson. So yeah, this is different. This is, hi, Steven

Leo Laporte (00:36:15):
Doesn't call me Mr. LePort. He calls me Leo, but everybody else is Mr. Which is

Steve Gibson (00:36:19):
Mr. Gibson.

Leo Laporte (00:36:20):
Well brought up southern boy

Steve Gibson (00:36:21):
Here we have Hi Steven, and it actually is my given name, S E E V E N. So very formal.

Leo Laporte (00:36:27):
We didn't know that.

Steve Gibson (00:36:28):
I realize this is a request out [00:36:30] of the blue, but my hard disc is failing on my main machine. I own a copy of spin, right? And then he provided in the tweet his code, which is the spin right serial number, but the disc is set up as G P T, which six oh does not support any chance. You can give me a pre-release of 6.1 to run, please. There's nothing critical on the drive, but if it dies, it will take some time to bring it all back. [00:37:00] Of course, Brett, and this goes for everyone. If you go to grc.com/prerelease.htm, just P R E R E L E A S E dot htm, what you'll get is a little form, enter your current spin, right 6.0 serial number, which is presented whenever Spin right is run. And I just verified that Brett's code works perfectly for him.

(00:37:27):
So when you do that, you'll receive a link [00:37:30] to download your own personalized and licensed copy of the latest pre-release of version 6.1. What you'll get at this point is the Doss executable, since I haven't yet built it all into the Windows boot prep thing that's coming next. So just place that Doss executable, I think it's 95 K onto your spin right boot drive and let her rip. The current pre-release expires at the end of November, [00:38:00] so that's six weeks from now. And I'm sure we'll have updated releases before then, so you can just come back and get an update at or just retard the machine's date if you want to buy yourself some more time. At this point, I'm 100% certain that spin right is safe to use. The reason is I just checked and we have exactly 800 registered spin, right pre-release testers who have been using [00:38:30] and pounding on all of the spin right pre-releasees.

(00:38:33):
I've taken great pains to be very careful along the way and no one has ever experienced any data loss at any point. So this thing's ready for the world. I just need to tie up a bunch of loose ends, which any three year project of this complexity is going to have. So that's where we are. But absolutely all of our listeners, anybody who has spin, right, you're welcome to go grab six one. It's all but done. [00:39:00] Just a few last things we'll need to get changed. Ray Franklin tweeted, Steve, I've looked through the current show notes for the reference wipe drive with encryption. What's the program to effectively wipe a drive with random data? And that's Vera Crypt, and I tweeted him the link. It's at vera crypt fr because it's French and you could [00:39:30] use it as we've talked about, basically to fill your drive with cryptographically secured noise.

(00:39:36):
It's actually encrypting what you have already there, but if you throw away the key, no one's ever going to get it. Marcus, who's handle is badass bg, he said, hi, Steve. I just had a quick question. I heard you say that you do not use auie for two-factor authentication and [00:40:00] I was wanting to know what you do use. Thanks. So being an Apple ecosystem person for mobile, I use an app called O T P Auth and I love it. It's clean, simple and is very widely praised and for anyone who can't find it easily, I got a link in the show notes.

Leo Laporte (00:40:21):
I also just, I'll plug for something that's both iOS and Android, which I've been using also free, also open source two Fast or [00:40:30] maybe, I don't know, T number two, that's S tufa two f a s. And what's nice about it is it will back up an encrypted blob with your secrets to either Google Drive or iCloud. So it makes, this is an issue for me and one of the reasons we used authe is so that we can move. I get a new phone, many new phones all the time, and so the ability to move is very, very important to me

Steve Gibson (00:40:57):
And especially be able to move cross platform.

Leo Laporte (00:41:00):
[00:41:00] Yes, exactly.

Steve Gibson (00:41:01):
Nice.

Leo Laporte (00:41:01):
Yeah, I'll just install both a new Pixel eight Pro and a new iPhone 14 pro and two F A s worked on both and I was able to import those secrets and it was really nice and it looks good here. I'll show you. It's pretty, it's a pretty, I don't know about yours, but this one's pretty, and it uses face ID to, there's all my, oh, very nice. By the way. There's all my two factors.

Steve Gibson (00:41:21):
Yeah, we're not going to be, no,

Leo Laporte (00:41:23):
They're useless in 30 seconds. Use 'em quick kids.

Steve Gibson (00:41:26):
That is very pretty.

Leo Laporte (00:41:28):
It's very pretty. You click it and it will copy [00:41:30] it to the clipboard. It goes red when it's about to expire, and then it

Steve Gibson (00:41:33):
Generates a new one. Oh, I like that. OAuth does not do that,

Leo Laporte (00:41:37):
And it can sort alphabetically or in other ways, which is also nice. Yeah, and it's just the one I use, but two f a s to each is own. The nice thing is you don't have to use a paid or a solution or a solution from Google or Microsoft. This is a standard, T O T P is a standard and there are lots of people that make software.

Steve Gibson (00:42:00):
[00:42:00] And finally, Henning said, dear Steve, when listening to SED 9 43 again last week, I have a different explanation for why Google is rewriting links in Google Drive documents publishes H T M L pages prevent the leakage of the documents U R L through referrer information. Though the drive documents are public, they're hidden by their long obscure U R L by the referrer info can reveal the U R [00:42:30] L in the server logs of the embedded links. The additional step through a second Google Service hides this information. So first of all, Henning, that's a neat thought that hadn't occurred to me, but these links were present in offline H T M L exports of the dock. So wherever the H T M L was being served from would be appearing in a referrer header. Also, if this was a favor [00:43:00] that Google was doing for us to protect our privacy, then their referring link URLs would not have been loaded down with unique tracking parameters in order to see exactly what link it is that someone has clicked on in the future. So much as I love the idea that they might be protecting us, that doesn't appear to be their intent here.

Leo Laporte (00:43:25):
There's been a lot of conversation about Google rewriting search queries [00:43:30] as well.

Steve Gibson (00:43:30):
Drives me nuts. And they even show you the true link, but then that's not what you get when you click on it.

Leo Laporte (00:43:37):
Yeah, yeah. I think that, let's face it, they're an advertising based business and the need to monetize through advertising kind of permeates everything, all these free services. And as a result, I think if you want privacy, maybe, yeah, I pay 25 bucks a month for a advertising free search

Steve Gibson (00:44:00):
[00:44:00] A month.

Leo Laporte (00:44:01):
Well, it doesn't have to be a month.

Steve Gibson (00:44:03):
Okay,

Leo Laporte (00:44:04):
Yeah, no, it is. I think you can get it for five bucks. I pay a little more. They have a browser and stuff, but CGI is very good results. Google quality results, but no ads, no targeting, none of that. But they don't have docs, they don't have Google Drive, they don't have all these other great features and that's the problem. But all that stuff's free and you got to monetize it. I understand.

Steve Gibson (00:44:26):
So Leo, let's take our second break and then we're going to

Leo Laporte (00:44:29):
Get into [00:44:30] ads.

Steve Gibson (00:44:32):
Speaking of monetizing,

Leo Laporte (00:44:35):
I am not against ads. We wouldn't exist if it weren't for advertising, and we love our, we're very careful. But the other thing that, and I think we've talked before about the fact that podcast advertising is getting tough, and I think a lot of that is because advertisers and especially ad agencies are expecting the kind of tracking Google, Facebook, and Apple do, and we won't do that. We can't do it really with [00:45:00] a podcast our SSS feeds. The only thing we know about you is that your IP address, when you download the podcast, we can't get anything else. We do put redirects in, but either

Steve Gibson (00:45:12):
In order to count,

Leo Laporte (00:45:13):
Yeah, in order to count. And I should explain because people might wonder that we use also a service called pod sites that we think is the best compromise between getting advertisers. Look, they won't buy ads at all if we don't do this, [00:45:30] but what happens is pod sites gets, we redirect through them so they get all our IP addresses and then the advertiser sends them, does not get those IP addresses that's kept secret by pod sites, but sends to pod sites, their logs from their landing pages, and then pod sites goes, well, 43% of the people who downloaded that podcast went to your website, but neither side gets the others information. We don't get the advertisers IP addresses and [00:46:00] they don't get our IP addresses pod sites. We have to trust them that keeps that, but that's their business keeps that secret. That is tracking and we have to do it. That's how advertisers, if they don't get some sort of feedback, it used to be for a long time, just using that special U R L was sufficient. It's been a long time since that was sufficient. And frankly, their podcast network's going out of business right now, W N Y C is just basically shutting down its entire podcast operation because advertisers are, [00:46:30] they say, well, we can get so much more from Google,

Steve Gibson (00:46:32):
We get so

Leo Laporte (00:46:33):
Much more from Facebook, why can't you give us this? Well, we can't. It's a podcast.

Steve Gibson (00:46:38):
I guess we really are in a world where metrics is what it's about is like measure, measure, measure.

Leo Laporte (00:46:43):
Absolutely. And I think to some degree it's mistaken. I understand that advertisers want to know, well, how effective is my ad? But tracking, I'm not convinced that ad tracking gives you better ad results to be honest.

Steve Gibson (00:46:58):
So

Leo Laporte (00:46:58):
I think, but the problem [00:47:00] is it's what they believe. Good luck convincing 'em otherwise. So we have to go along to get along. We give them as little information as possible. We do everything we can to protect your privacy as that becomes less and less valuable to them. We're losing advertisers constantly.

Steve Gibson (00:47:16):
We subscribe in our household to YouTube tv and sometimes there will be a commercial break that'll instead be just some relaxing,

Leo Laporte (00:47:25):
Whatever.

Steve Gibson (00:47:26):
It's

Leo Laporte (00:47:26):
Enjoy this moment of Zen.

Steve Gibson (00:47:27):
Yes, exactly. The moment of Zen and [00:47:30] Lori first asked me, she said, what is that about? I said, well, Google knows who we are and they've decided they don't want to pay to show us that ad because we're not buying diapers. So

Leo Laporte (00:47:42):
Yeah, that's part of it. Advertisers don't want to, Wannamaker famously said, I know 50% of my ads work, but I just don't know which 50%.

Steve Gibson (00:47:52):
Well, now Lori's all creeped out. Whenever one of those comes up. She says they know who we are going. Well, I'll

Leo Laporte (00:47:57):
Tell you, you could tell her this is what's going on. [00:48:00] You're watching YouTube TV probably. And YouTube TV is in effect a cable company. Cable gets ad avails, local ad avails that they'll put in your local plumber. So when you watch your cable tv, they'll be miked the plumber in there after the ad for Toyota. That's a local ad sell made by the local cable company. Well, YouTube TV's not selling that. So it's a hole in the programming where somebody's ad's supposed to go. You're right. It's probably, [00:48:30] I don't know how hard Google is working to sell it, but they haven't sold it for one reason or another. Maybe they don't want to reach you, but it's not about you. It's about that's a local ad that they just don't have sold.

Steve Gibson (00:48:42):
Well, and the other thing I've noticed is if you back up over an ad you've seen, then you'll get the moment of Zen. They won't show you the at

Leo Laporte (00:48:50):
Again. Oh, that's clever. Oh, maybe they're smarter than I realized. I

Steve Gibson (00:48:53):
Think they're pretty smart.

Leo Laporte (00:48:54):
They don't want to show it twice then the They have to pay

Steve Gibson (00:48:57):
Exactly.

Leo Laporte (00:48:58):
Twice.

Steve Gibson (00:48:58):
Exactly.

Leo Laporte (00:49:00):
[00:49:00] It's a mess out there. It's one of the reasons we, it's one we push Club twit so hard. I think you remember when we started, I really wanted this to be listener supported, but we never were able to get more than eight or $9,000 a month, which wasn't enough to pay me and you so have a studio and all that stuff. So eventually we did have to, and this was the first show that had advertising on it. Thank you, Steve. We did have to turn to that, but increasingly I think we're going to [00:49:30] have to turn back to the audience and say, look, if you like these shows and you want to keep these shows going and you want to keep twit going, join Club Twit seven bucks a month. That's all it takes. That's all we make. That's all an advertiser would pay us for you.

(00:49:44):
In fact, it's a lot, frankly, probably a lot. I don't know. I have to think about it. I don't know if it's less or more. We didn't want to charge you more. Maybe we should. Facebook says they want 14 bucks a month, not to show you ads in the eu. They're going to do that. So maybe we're not charging [00:50:00] enough, but it's most, we feel fair charging. You get ad free versions of this show and all the shows. You get the Discord channel and you get all the other stuff we do in the club, including the book club and all of that.

Steve Gibson (00:50:13):
And then you don't get the following commercial break.

Leo Laporte (00:50:17):
You don't get this. You don't even get an ad for Club Trick because you're

Steve Gibson (00:50:20):
Already a member. I don't want to advertise. You enjoy this

Leo Laporte (00:50:22):
Moment of zen. If I had a bear catching salmon in his stream, I would show you that right now, but I don't [00:50:30] actually, somebody the other day wrote to us, said, how can I get rid of the ads in the live show? And I said, I can't. You're

Steve Gibson (00:50:43):
Without

Leo Laporte (00:50:44):
A time

Steve Gibson (00:50:44):
Machine.

Leo Laporte (00:50:45):
You want a moment of Zen maybe without a time machine. We don't have a way of doing that. Sorry. Just listen to the podcast and you'll get that free twit tv slash club twit. If you're not a member, it would really help us out. I appreciate it. Thank [00:51:00] you for letting me take some time out of this show. Now let me talk about Cisco and Duo. Oh, duo's great for security. It protects against breaches with the leading access management suite. Look, strong, multilayered defenses and innovative capabilities only allow legitimate users in and they keep bad actors out. For any organization concerned about being breached, that needs protection fast duo quickly enables strong security. But [00:51:30] here's the thing that's cool, while also improving user productivity. Now, you might scratch your head when you hear that, but let me explain. DUO prevents unauthorized access with multilayered defenses and modern capabilities that thwart sophisticated malicious access attempts.

(00:51:47):
And here's the key. Duo enables high productivity by only requiring on authentication when needed. If it knows you're you or for the risks levels low, it will [00:52:00] let you in a lot faster. It gives you swift, easy, secure access, increases authentication requirements in real time. When the risk increases and uses all the technologies you probably could convince 'em to use squirrel. It's using strong M F a passwordless single sign-on, trusted endpoint verification, all of this so that they can give you high security without getting in the way, without reducing productivity. Duo helps you [00:52:30] implement zero trust principles by verifying users and their devices in the least intrusive but the most secure way possible. Start your free trial, sign up today at css.co/twit css.co/twit

Steve Gibson (00:52:48):
Back to you. Okay, so abusing HTTP two, rapid reset. As I said last week, we would be continuing with a second part of our look into the top 10 cybersecurity [00:53:00] misconfigurations. If something bigger and more important didn't bump it to the following week. Well, something did indeed. So we'll do the second part of the top 10 cybersecurity misconfigurations. If something does it again, bump it into the following week. But today we're going to talk about an interesting surprise. The headline about this, which first caught my attention was CloudFlare, [00:53:30] was on Cloudflare's blog. That headline read http slash two zero day vulnerability results in record breaking DDoS attacks.

(00:53:43):
So yeah, what the idea of there being a newly discovered zero day vulnerability in a core internet protocol says HTTP two would be huge news if it were true [00:54:00] for the rest of the podcast, we're going to explore what has happened and decide to what degree this is actually true. And as always, the devil is in the details and these details are both interesting and important. So a few lines into their coverage, CloudFlare wrote, CloudFlare has mitigated a barrage of these attacks in recent months. So my first reaction was to question their use of the term zero day [00:54:30] if they've been doing something for several months. But after wondering whether it might be in an abuse or an overuse of the term, I suppose it's correct. If they first learned of this unsuspected problem through an attack, we've settled upon the definition of zero day as being the exploitation of any previously unknown vulnerability or stated another way.

(00:54:54):
The first indication we have of there being some specific vulnerability is not when [00:55:00] we find it through examination or when it's reported by a responsible researcher, but when we're surprised by something happening that we didn't believe to be possible. So yeah, it would probably be correct for this to be considered a zero day vulnerability in the version two H T T P protocol if indeed it was a vulnerability in the HTTP two protocol, which surprisingly is not [00:55:30] the case. So it is to cloud flare's benefit as LEO exactly as you said at the top of the show since they are in the DDoS protection business. To characterize this as a surprising zero day vulnerability in HTTP two that deflects the blame to the protocol when it appears that the true culprit is implementation, that [00:56:00] the HTTP two protocol is not to blame because there's a difference between a vulnerability and the abuse of a feature that's exacerbated by cloud flares internally, decoupled serial processing, MultiPro architecture.

Leo Laporte (00:56:20):
This is interesting.

Steve Gibson (00:56:21):
That's what made it such problem and it applies to the cloud providers because the way they've distributed [00:56:30] the workflow interacted with this feature. So as I think the evidence will show, and Google was much more forthcoming as we'll see, this is more about something that cloud flares behind the scenes request processing architecture was ill suited to handle than the exploitation of any previously unknown vulnerability in HTTP two. It's [00:57:00] a clever and in retrospect, probably foreseeable and maybe even inevitable leveraging that is the attack is of a feature in HTTP two. So I don't mean to jump on cloud FLA too much about this, but characterizing this as a zero day vulnerability in HTTP two is really not accurate or fair. Okay, so let's back up a bit. [00:57:30] Let's start with a very quick background review of layered network design and then a quick history of H T T P.

(00:57:41):
We refer to H T T P, the hypertext transfer protocol, thus H T T P as an application level protocol, also known as a layer seven protocol as high as the layers go because after we've finally finished building [00:58:00] up layer upon layer of the various underlying enabling protocols, we finally get to H T T P, which was the whole point. And it's there where something finally happens and gets done. So what do we mean by layer upon layer at the bottom most level or layer, we have an agreement about wires and voltages and clock rates and signal [00:58:30] transition times, and that's those are used to signal the transmission of individual discrete zeros and ones that's typically referred to as the physical layer. That's as physical as it can get, like what's the voltage on the wire and what does it mean. Then with that agreed upon, we describe the groupings of those ones and zeros to form packets of data [00:59:00] which are addressed between physical hardware endpoints on a single local network.

(00:59:09):
This is then the ethernet protocol. And those ethernet packets in turn contain IP internet protocol packets, which carry the addresses of endpoints on the global internet. And inside those IP packets are T C P packets where the T C [00:59:30] P protocol is used to manage the numbered sequential bite oriented, reliable flow of data between those specific virtual ports on the devices specified by their IP addresses on the global network. And finally, the data that flows back and forth under the watchful eye of the T C P protocol is formatted as specific queries and responses in accordance with a top level application protocol H T T P. So [01:00:00] that's the layering all the way up from the physical layer to layer seven, the application layer. And over the decades since their initial definitions, all of these protocols have seen some tuning, some tweaking and evolution in their definition as they interacted with the real world.

(01:00:23):
And that's certainly been true for http with a bit of editing by me for clarification, here's [01:00:30] how Wikipedia briefly sums up HTTPS evolution. They wrote development of H T T P was initiated by Tim Berners-Lee at CERN in 1989 and summarized in a simple document describing the behavior of a client and a server using the first H T T P version, which was 0.9. That version was subsequently developed eventually becoming the public. [01:01:00] 1.0 development of early http RFCs began a few years later in a coordinated effort by the I E T F, that's our engineering task force and the W three C W three meaning worldwide web and C for consortium with work later moving exclusively back to and now at the I E T F HTTP slash one. And it's shown as a slash one [01:01:30] rather than a V one because http slash one is actually what's sent over the wire with each query to specify the format of everything that will follow.

(01:01:42):
So you need to specify the version right up front so that the recipient is able to know how to interpret everything that's going to come. So version 1.0 was finalized and fully documented in 96. It quickly evolved into version 1.1 [01:02:00] a year later in 1997, after which its specifications were updated in 99 20 14, and even just last year in 2022, its secure variant. H G T P S is now used. As we know by more than 80% of websites writes Wikipedia. Now, HTTP two, which is the topic for today, published first in 2015, [01:02:30] so it's already eight years old. It provides a more efficient expression and we'll be talking about the sufficiency in a minute of http semantics on the wire. As of April, 2023, HTTP two is used by 39% of websites. So as of April this year, just shy of 40% of all websites offer HTTP two, and it's supported by almost all web browsers representing [01:03:00] over 97% of web users.

(01:03:02):
So all the popular browsers do it now. It's also supported by major web servers over T L s using an application layer protocol negotiation, known as an A L P N extension where t L s 1.2 or newer is required. And just to round this out for the sake of completion, we actually [01:03:30] drop T C P as the transport protocol for H T T P in favor of the faster to set up yet less feature rich U D P when we're using HTTP three, which is technically out there now though not yet widely adopted. HTTP three of course is the successor to HTTP two [01:04:00] that was published just last year. It's now available from one quarter of all websites, says Wikipedia, and it's at least partially supported by most web browsers. As I mentioned, unlike all of its TCP based predecessors, HTTP three does not run on top of T ccp. Instead it uses Q U I C, which runs on top of U D P support for HTTP three was added to CloudFlare [01:04:30] and Google Chrome first and is also enabled in Firefox. H TDP three has lower latency for today's webpages, which are pulling information together through many separate widespread sources. If H TDP three is enabled on the server, such pages such Wikipedia will load faster than with HTTP two, which is in turn faster than http 1.1 in some cases as much as [01:05:00] three times faster than http 1.1.

(01:05:06):
So now that we have the lay the land, let's look at what took CloudFlare and others by surprise a few months back. There's a bit of, I guess it's marketing bragging going on here from CloudFlare, but mostly a lot of good information and I'll insert a little bit of editorializing as we go. So CloudFlare wrote earlier today, [01:05:30] and this was posted exactly one week ago on the 10th of October. They said CloudFlare, along with Google and Amazon a w s, disclosed the existence of a novel zero day vulnerability, dubbed the http slash two rapid reset attack. And they wrote, CloudFlare wrote this attack exploits a weakness in the HTTP [01:06:00] two protocol to generate enormous hyper volumetric distributed denial of service DDoS attacks. CloudFlare, they said, has mitigated a barrage of these attacks in recent months, including an attack three times larger than any previous attack we've ever observed, which exceeded 201 million requests per second since [01:06:30] the end of August, 2023.

(01:06:32):
So that's just a couple months ago. CloudFlare has mitigated more than 1100 other attacks with over 10 million requests per second and 184 attacks that were greater than our previous DDoS record of 71 million RRP SS requests per second. Again, now these new attacks 201, so nearly [01:07:00] three times those 184 attacks that were greater than their previous DDoS record of 71 million. And I should note that a similar stat from Google about this. Put it at just shy of 400 million requests per second. I think it's three 90 something as I recall. So I'll pause here. Just a note that there are several different [01:07:30] ways to measure or characterize DDoS attacks. Traditionally attacks leveraged bandwidth, flooding and saturation, just pure bandwidth. The incoming network links to a server would be so saturated with packet traffic that a site's upstream routers would be overwhelmed as they were aggregating the traffic being aimed at a destination [01:08:00] server so overwhelmed with bogus packets that legitimate packet traffic stood little chance to get through to the server.

(01:08:07):
It was just a matter of numbers. If you've got a hundred bogus packets for every good one, there's a 1% chance that the good one's going to get through. So essentially the server's services are being denied and thus the term denial of service of course. But as we know, some time ago webpages shifted [01:08:30] from being simple static dumps of a previously written textual page. For example, my own G R C site remains that way. To this day it's just simple static H T M L, but most web services switched over to being the public facing aspect of a content management system, A C M S of some sort. Under this new website paradigm, any incoming H [01:09:00] T T P query is fielded by some sort of code running on the server. That code examines the query details and assembles the page, which will be returned to the client on the fly.

(01:09:14):
And this in turn typically involves multiple queries to a backend relational database of some kind, typically a SQL server. The point of this is that under this new content management paradigm replying to a query [01:09:30] for a simple page what looks like a standard H T M L page consumes server side compute resources in order to pull the various components of the page together for its return to the client. The other thing is that in the earliest days of this new approach, these on the fly page assembly systems were not often very efficient. So each query, especially deliberately complex queries, which [01:10:00] might be requiring a lot of backend database work to pull a complex page together could be quite costly to answer or to reply to naturally. It didn't take the bad guys long to figure this out, after which they switched their attack tactics from flooding a server's raw bandwidth pipeline with what could well just be noise.

(01:10:21):
It didn't really matter what the packets contained to loading the server down with actual valid queries, [01:10:30] each of which could be quite expensive for it to answer. Thus, we went from thinking of DDoS attacks in terms of bits per second to requests per second because it was the request that was expensive to handle, not just a bandwidth flood. Now I take issue with cloud flare's characterization of this as an attack, which exploits a weakness in the HTTP two protocol. [01:11:00] That's not what's been happening. As we'll see these new attacks exploit some new features of HTTP two, which were added to make the protocol much faster overall. But it turns out that these new features are subject to abuse and if a server's query processing chain is not really explicitly designed to handle these new features, it can be forced into collapse [01:11:30] by attackers. This is where we'd say it's not a bug, it's a feature and we'd mean it.

(01:11:38):
So CloudFlare continues. They said this zero day, which okay, it was a surprise, provided threat actors with a critical new tool in their Swiss Army knife of vulnerabilities to exploit and attack their victims at a magnitude that has never been seen before, [01:12:00] while at times complex and challenging to combat these attacks allowed cloud flare the opportunity, oh, it's an opportunity to develop purpose-built technology to mitigate the effects of the zero day vulnerability. In other words, whoops. As I said, they got caught with their DDoS down and needed to do up their technology in order to deal with this and as again, we're going to see, [01:12:30] and I'll shortly provide ample evidence to back it up, this is not really a vulnerability and I am a little disappointed in Cloudflare's approach to this. We all know that I'm a big fan of CloudFlare, but I believe they took the wrong tact on this one.

(01:12:45):
So they said in late August of this year, our team at CloudFlare noticed a new zero day vulnerability developed by an unknown threat actor that exploits the standard HTTP two protocol, [01:13:00] A fundamental protocol that's critical to how the internet and all websites work, meaning H T T P and now two is a lot better. This novel zero day vulnerability attack dubbed rapid reset leverages HTTP two's stream cancellation feature by sending a request, then immediately canceling it over and over by automating this trivial request [01:13:30] cancel, request cancel pattern at scale threat actors are able to create a denial of service and take down any server or application running the standard implementation of HTTP two. Furthermore, one crucial thing to note about the record breaking attack is that it involved a and pay attention here IT involved that is this attack that was bringing down cloud [01:14:00] flares. Very strong preexisting DDoS defenses, they said consisted of roughly 20 20,000 machines in a modestly sized botnet. They said CloudFlare regularly detects botnets that are orders of magnitude larger than this, comprising hundreds of thousands and even millions of machines [01:14:30] for a relatively small botnet to output such a large volume of requests with the potential to incapacitate nearly any server or application supporting HTTP two underscores how menacing this vulnerability is for unprotected networks.

Leo Laporte (01:14:53):
I had assumed that this was some sort of massive amplification attack because previous attacks have had to have massive [01:15:00] botnets in order to do this

Steve Gibson (01:15:03):
Right.

Leo Laporte (01:15:03):
Why is it that this HTT two canceled request, rapid reset request can't be more easily blocked if it's only 20,000 IP addresses it's coming from? How come you can't just mitigate it? I don't understand

Steve Gibson (01:15:19):
That actually is the strategy. That's the only strategy, and I actually I wind up saying exactly that. Good.

Leo Laporte (01:15:27):
The only

Steve Gibson (01:15:28):
Strategy that I think makes [01:15:30] sense yesterday

Leo Laporte (01:15:31):
I jump you jumped

Steve Gibson (01:15:32):
Ahead, is because thank God T C P cannot have its source address spoofed, right? You need to have a full communication link that is confirmed on each end, so you do know the IP address of every bot that's attacking you.

Leo Laporte (01:15:51):
So previous attacks were much simpler. They were often CAC floods where a bot would say hello [01:16:00] and then your server would have to say hello and you'd start to start the conversation at which point that client would go, nevermind. Hello.

Steve Gibson (01:16:12):
And that's actually a good analogy to this attack, but it's up to level where it's not just a sin, which is initiating A T C P connection, it's actually a request that is initiating backend work.

Leo Laporte (01:16:28):
So there's a lot of work because it's reset. [01:16:30] You're actually going to reset the thread. So it's more than just, hello? Hello, are you there?

Steve Gibson (01:16:36):
It's more than just a single packet being received.

Leo Laporte (01:16:38):
Yeah.

Steve Gibson (01:16:39):
Okay, so I'm going to make three points first. I hope it still astonishes us when CloudFlare writes, quote, CloudFlare regularly detects botnets that are orders of magnitude larger than 20,000 comprising hundreds of thousands or even millions [01:17:00] of machines. Just think about that. This kind of power in the hands of miscreants is fundamentally destabilizing because the internet is so powerful and because it mostly works so well, we've allowed ourselves to gradually grow to become utterly dependent upon its presence. But last week we noted Microsoft was observing around 1700 DDoS [01:17:30] attacks per day. So there are websites that are occasionally denied the sort of reliability that the internet has to offer because someone somewhere just says, no, we want you off the net. We're not happy with something you just did, so we're going to punish you.

Leo Laporte (01:17:50):
Well, people might remember back in the day, Steve had this happen to him.

Steve Gibson (01:17:54):
It

Leo Laporte (01:17:54):
Was going on during the show and it's just a vandalism [01:18:00] of a

Steve Gibson (01:18:00):
Well, it turned out it was because I had those DDoS attack reports on my site, and so they thought I was bragging in some way,

Leo Laporte (01:18:10):
But

Steve Gibson (01:18:10):
In fact I was not, and when I removed

Leo Laporte (01:18:13):
Those

Steve Gibson (01:18:14):
Pages, the attack stopped.

Leo Laporte (01:18:15):
Oh, that's very interesting.

Steve Gibson (01:18:17):
It wasn't me saying, nanny, nanny, nanny, you can't get me. It was like, ow, I just got blasted off the internet.

Leo Laporte (01:18:25):
Yeah.

Steve Gibson (01:18:26):
The second point I wanted to highlight was that CloudFlare witnessed [01:18:30] by far the largest attack they have ever experienced, which was produced by what is unfortunately now considered to be a relatively modest size botnet consisting of only around 20,000 individual clients. Since this sort of attack rides on T C P, the attacker source ips cannot be spoofed. Since T C P requires confirmation of packet round trips before the query can be sent, so CloudFlare has [01:19:00] the specific IP of every bot that was attacking it. That's how it knows how many because it can count.

(01:19:08):
The last point is that since this was a new form of abuse to which HTTP two is prone, every one of these bots needed to be updated with a specific module designed to implement this attack against its target. So someone [01:19:30] somewhere figured this out, wrote the code to do this, then either established a new botnet for this purpose or caused an existing modernized updateable botnet to be updated to add this new form of attack to its bag of tricks. They said, CloudFlare said, threat actors used botnets in tandem with the HTTP two vulnerability. Again, [01:20:00] I'll take issue with that characterization, but we'll get there in a second to amplify requests at rates we have never seen before. As a result, our team at CloudFlare experienced, I love this phrase here, some intermittent edge instability, intermittent edge instability, right? In other words, we're all about DDoS protection, but this one got us because we had never [01:20:30] seen anything like it and our backend architecture was ill-equipped to handle it at the time, and in fact, they did publish a chart where it was like up to 80% of their connections were having trouble because I mean this thing really clogged up their network.

Leo Laporte (01:20:50):
Well, that makes sense. Since you said 80% of all servers use HTTP two, it was able to hit 'em all basically.

Steve Gibson (01:20:57):
Well, so [01:21:00] their entire customer base, their whole front end is H http two equipped.

Leo Laporte (01:21:06):
If you're using CloudFlare, it is HTTP two.

Steve Gibson (01:21:09):
Yeah, you have

Leo Laporte (01:21:10):
HTTP two

Steve Gibson (01:21:12):
Front facing.

Leo Laporte (01:21:14):
So

Steve Gibson (01:21:14):
They said, while our systems were able to mitigate the overwhelming majority of incoming attacks, the volume overloaded some components of our network impacting a small number of customers' performance with intermittent [01:21:30] four xx and five XX errors, all of which were quickly resolved, right? Once we successfully mitigated these issues and halted potential attacks for all customers, our team immediately kicked off a responsible disclosure process, meaning for several months, there's been stuff going on behind the scenes that no one knew about until Tuesday. When this was officially revealed [01:22:00] in a coordinated disclosure, they said, we entered into conversations with industry peers to see how we could work together to help move our mission forward and safeguard the large percentage of the internet that relies on our network. Prior to releasing the news of this vulnerability to the general public, they wrote, there's no such thing as a perfect disclosure.

(01:22:27):
Thwarting attacks and responding to emerging incidents [01:22:30] requires organizations and security firms to live by an assume breach mindset because there will always be another zero day new evolving threat actor groups and never before seen novel attacks and techniques. This assume breach mindset is a key foundation towards information sharing and ensuring in instances such as this that the internet remains safe. While CloudFlare was experiencing [01:23:00] and mitigating these attacks, we were also working with industry partners to guarantee that industry at large could withstand this attack. During the process of mitigating this attack, our CloudFlare team developed and purpose-built new technology to stop these attacks and further improve our own mitigations for this and other future attacks at massive scale. In other words, they learned that there was something that was [01:23:30] able to get by their DDoS mitigations and they needed to develop new technology to deal with it.

Leo Laporte (01:23:38):
Boy, wouldn't you love to be a fly on the wall in the war room there when they get

Steve Gibson (01:23:42):
These

Leo Laporte (01:23:43):
Emergencies and they all leap into action, I have to say, I mean, whatever they did wrong here, you got to credit them with really talented people doing

Steve Gibson (01:23:54):
I do. I love CloudFlare.

Leo Laporte (01:23:57):
I

Steve Gibson (01:23:57):
Just wish they weren't, weren't calling [01:24:00] it a zero day vulnerability in HTTP two. It's not. It's a feature, but anyway, we'll get there in a second.

Leo Laporte (01:24:07):
It is a vulnerability, however, a feature rich vulnerability,

Steve Gibson (01:24:11):
It is an exploitable feature

Leo Laporte (01:24:15):
Feature there. That's a better phrase. Yeah, they should have said that

Steve Gibson (01:24:18):
Exploitable feature. Yeah, so they said these efforts that theirs have significantly increased our overall mitigation capabilities and resiliency, so yes, we're now better than ever. [01:24:30] They said, if you're using CloudFlare, we're confident that you are now protected. Our team also alerted web server software partners who are developing patches to ensure this vulnerability cannot be exploited. Then they said check their websites for more information. Disclosures they said are never one and done. The lifeblood of CloudFlare is to ensure a better internet, which stems from instances such as these. When we have the opportunity to work with our industry partners and governments to ensure [01:25:00] there are no widespread impacts on the internet, we're doing our part in increasing the cyber resiliency of every organization, no matter the size or the vertical. It may seem odd that CloudFlare was one of the first companies to witness these attacks.

(01:25:18):
Why would threat actors attack a company that is some of the most robust defenses against DDoS attacks in the world? Well, they've answered their own question and they're about to. The reality [01:25:30] is that CloudFlare often seize attacks before they're turned on more vulnerable targets. Threat actors need to develop and test their tools before they deploy them. In the wild threat actors who possess record shattering attack methods can have an extremely difficult time testing and understanding how large and effective they are because they don't have the infrastructure to absorb the attacks they're [01:26:00] launching. Because of the transparency we share on our network performance and the measurements of attacks they could glean from our public performance charts, this threat actor was likely targeting us to understand the capabilities of this new export,

Leo Laporte (01:26:19):
And

Steve Gibson (01:26:19):
I think that's exactly, exactly right.

Leo Laporte (01:26:22):
So we're just going to launch a small botnet against you just to see what you can do.

Steve Gibson (01:26:27):
Yeah, remember in the days when I [01:26:30] was under attack, I had two T ones, which were each 1.54 megabits, so a 3.4 megabit flood would swamp me. I mean, I was a gnat. I wasn't even worth swatting, so it took nothing to knock me off the net.

Leo Laporte (01:26:52):
But on the other hand, CloudFlare has terabytes of base.

Steve Gibson (01:26:56):
Yes, yes, it is very difficult to push them down,

Leo Laporte (01:27:00):
[01:27:00] But

Steve Gibson (01:27:01):
This attack did it initially,

Leo Laporte (01:27:03):
They

Steve Gibson (01:27:03):
Said, but that testing meaning the bad guy's testing and the ability to see the attack early helps us develop mitigations for the attack that benefit both our customers and the industry as a whole. That is, they shared what they learned. Even so both

Leo Laporte (01:27:17):
Sides get information, the bad guys and the good guys. Yeah.

Steve Gibson (01:27:20):
Yep. Even so whether it was log for J SolarWinds, eternal Blue, WannaCry, not Petya, Heartbleed or [01:27:30] shell Shock, all of these security incidents have a commonality, a tremendous explosion that ripples across the world and creates an opportunity to completely disrupt any internet dependent organization regardless of the industry or the size. While we wish we could say that rapid reset may be different this time, it is not. We're calling all CSOs. No matter if you've lived through the decades [01:28:00] of security incidents or this is your first day on the job, this is the time to ensure you're protected and stand up your cyber incident response team. We've kept the information restricted until today to give as many security vendors as possible the opportunity to react. However, at some point, the responsible thing becomes to publicly disclose zero day threats like this. Today is that day. [01:28:30] That means that after today, threat actors will largely be aware of the HTT largely,

Leo Laporte (01:28:37):
You

Steve Gibson (01:28:37):
Bet, largely be aware of the HTTP two vulnerability. Again, it's a feature that's exploitable and it will inevitably become trivial to exploit and kick off the race between defenders and attackers first to patch versus first to exploit organizations [01:29:00] should that systems will be tested and take proactive measures to ensure protection. The guy wrote to me, this is reminiscent of a vulnerability like log four J due to the many variants that are emerging daily and will continue to come to fruition in the weeks, months, and years to come as more researchers and threat actors experiment with the, and they're calling it again, a vulnerability. We may find different variants [01:29:30] with even shorter exploit cycles that contain even more advanced bypasses. I would argue against that, but we'll get to that in a minute. And they said, and just like log four J, managing incidents isn't as simple as run the patch.

(01:29:46):
Now you're done. You need to turn incident management, patching and evolving your security protections into an ongoing process because the patches for each variant of a vulnerability reduce your risk, but they don't [01:30:00] eliminate it. We don't mean to be alarmist, but will be direct. You must take this seriously, treat this as a full active incident to ensure nothing happens to your organization. Okay, so that's a little bit self-serving since CloudFlare has just finished explaining that while they were initially caught off guard by this unanticipated and massive zero day attack, they're now up to speed and are inviting everyone to [01:30:30] come and hide behind their perimeter. On the other hand, they're offering it at no charge. They do offer the service for free, so they wrap this up by writing. Cloudflare's mission is to help build a better internet if you're concerned with your current state of DDoS protection, and if you weren't before, you should be now where more than happy to provide you with our DDoS capabilities and resilience for free to mitigate any [01:31:00] attempts of a successful DDoS attack.

(01:31:03):
We know the stress that you're facing as we have fought off these attacks for the last 30 days and made our best already best in class systems even better. Okay, so thank you very much. Now, even though this first posting of theirs didn't give us any of the meaty technical details that this podcast never shies away from, I wanted to share it since is what CloudFlare is telling the entire world [01:31:30] and sharing it here is important because I believe, as I've said, that it mischaracterizes HTTP two as having surprising and unsuspected vulnerabilities, which is not the case. We're going to get into the meaty details. Leo, after you tell us who's paying for them,

Leo Laporte (01:31:52):
This is really fascinating and I mean the takeaway is if this was a test with a small [01:32:00] botnet, oh boy, imagine what the traffic could be if they really targeted somebody. And of course there's some events coming up that are very commonly DDoS including the Super Bowl. It's very common to DDoS betting groups or at least to extort them. That's the other reason. It's not just a test. It's also a note to companies that are getting ready for Black Friday Prime Day, the Super Bowl betting [01:32:30] that we have the capability to shut you down on those days, and I think that that's a very clear message and I'm sure that the security teams of those companies are sitting up and taking notice. It's hard to fix a feature. Exactly. Easier to fix a bug. Exactly. Wow. Hey, it's cybersecurity Awareness Month. You're doing your bit by listening to security.

(01:32:56):
Now you might want to do some more by telling everybody [01:33:00] I know you use a password manager, but for crying out loud, get your family, your friends, your loved ones, your acquaintances, the guy down at the Chevron station. Get them to start using a password manager because Passwordless is not here yet and you still need a password manager on almost every site you visit, and there is one and only one password manager that I recommend. Steve did his research. That's the one he moved to. I've [01:33:30] been using it for some time. I'm a big fan. First of all, it's open source and I really think that's important with any security program is that you can see the source code and maybe even more importantly, experts can see the source code and know what's going on. It's also completely cross platform. It runs on Windows, it runs on Mac, it runs on Linux, it runs on iOS, it runs on Android.

(01:33:50):
That's really important to me. I need to be able to use the same password manager everywhere and Bit Warden's the one plus [01:34:00] it works at home and at the office. We love Bit Warden. We're migrating over to Bit Warden at twit. I think almost all of our employees are already using Bit Warden personally. One of the reasons they do is because Bit Warden is free for personal use, free Forever. I like to cut 'em a little money for their premium account, 10 bucks a year because I want to support them and I've talked to them. I said, now we've had RUG pulls before with companies that said, oh, try our free [01:34:30] password service and then say, well, we're not going to give you quite as much as we used to or now you have to pay now. Bit Warden, they told me, look, we're open source.

(01:34:38):
We can't make it not free, so free forever for the personal account, bit Warden does so many great things. For instance, if you have a personal account, you can use Bit Warden to host your vault. I do. I trust them to keep it safe, but as many have pointed out, that's not trust, no one. That is a single point of failure. So host your own vault if you want, host it on a cloud server, encrypt it [01:35:00] and then put it up there. Host it on your nas, host it on one machine, let the other machine sink, it's up to you, and you can do that with Bit Warden. With Bit Warden. Every bit of data in your vault is password protected. Now that's something not every password manager does all the metadata. See, metadata is important. That metadata is almost as valuable knowing what sites you visit, what sites you have an account with, that's really valuable.

(01:35:25):
So Bit Warden Encrypts at all. It's not possible to see it. Bit Warden is. [01:35:30] I think the good news is the word's getting out in the summer 2023 G two Enterprise Grid report. Bit Warden was named the highest performing password manager for enterprise leaving competitors in the dust bit. Warden protects your data. They protect your privacy by adding strong randomly generated passwords for each account, but they can do more. You can go farther with a username generator, create unique usernames for each account, or use any of the six integrated email alias [01:36:00] services like FastMail, one of our sponsors, and you can have a unique address. This is actually what I do for every site you visit that makes it doubly harder for an attacker to figure out how to get into your account because not only is the password unique, not used anywhere else, not used anywhere else, that's really important.

(01:36:19):
But so is the email. So is the email. This is really great. You can log into Bit Warden and decrypt your vault after using SS s o on a registered trusted device. That means you don't have [01:36:30] to remember a master password anymore. Love that. This new solution makes it even easier for enterprise users to stay safe and secure with Bit Warden. Frankly, I think we should take credit for that because we told them before, we can move to Bit Warden at TWIT as an enterprise solution. You got to have this and now they do. Yay. Thank you. Bit Warden. You can see all of Bit Warden's code. When I say open source, it's all on GitHub so you can see it. But also Bit Warden has regular [01:37:00] professional third party audits performed every single year and they publish the results on the website. They don't keep it a secret.

(01:37:06):
So you can see and in the past, yeah, this is good. The third party odds have founded issues, bit warden's fixed 'em like that. Bit Warden also allows our listeners who's a listener, said, you know bit warden, P B K DF two is fine for your hash keyword hash, but what if we supported Scrt, which is memory hard and even better, [01:37:30] or Argon two. He wrote implementations after some conversation with Bit where he issued a pull request. Everybody decided, let's go. Let's work on Argon two. And within months, within months of this issue coming up, argon two was available to every bit Warden user. That's fantastic. I immediately switched to Argon two. That's one advantage. Open source can give you, right? You have a world of developers working for you, bit warden's great for the enterprise. [01:38:00] You can share private data securely with coworkers across departments or the entire company fully customizable adaptive plans.

(01:38:07):
There's bit warden's teams organization option that's $3 per user per month. We're going to do the enterprise organization five plan. That's $5 per user per month. There's a family plan gives up to six users. They don't even have to be blood relatives. It could be your chosen family. All those premium features for $3 and 33 cents a month for total. [01:38:30] The free accounts always free forever for unlimited passwords. I upgraded to, as I said to the premium account, 10 bucks a year because I wanted to support them as much as anything else. So bottom line, this cybersecurity awareness month, keep yourself. More importantly, I know you're safe. Keep your friends and family safe and secure. Teach 'em about Bit Warden. It's easy to use. It's got a fully featured free plan so they can't say, oh, it costs too much. It's free for everyone. By the way, [01:39:00] I really app praise them for this.

(01:39:02):
One of the reasons I decided to go premium is I want to use my UIC key as my second factor. It's now part of the free program. So not only are they not taking away features, they're actually adding features to the unlimited free program. You can use hardware security keys and pass keys, yay. As your form of two factor bit Warden was created with a vision of a world where no one is hacked. [01:39:30] It's the only major password manager offering passwordless either hardware, security keys or pass keys for your two-factor as a free forever option. So I love these people so much, I really appreciate what they've done and now they've just gone the extra step and that makes me really happy and I'm still going to give you my 10 bucks every month or every year I should say. It's nothing. It's less than a buck a month at twit [01:40:00] at security now, we believe password managers are still vital until the world moves to Squirrel.

(01:40:06):
Steve, we're going to have password managers get started with bit warden's free trial of a teams or enterprise plan right now or get started for free across all devices as an individual user. And yes, you get pass keys or YubiKey if you want, or a security key bit warden for free. Bit warden.com/twi. Bit warden.com/twit. I am [01:40:30] a big fan. I use Bit Warden every day and I thank my lucky stars when I pop up my iPhone and the face ID goes, yep, it's you and I use Bit Warden to log in and it's so fast and so easy. Just makes me really happy when I just got those two new phones. So easy to move over. The Pixel eight pro bit warden is just, it's beautiful now. It uses face ID on the pixel phones, which is really great as well as fingerprint. I can go on and on. I'll stop now. Steve has more to talk about, [01:41:00] but get it. Will you do us a favor bit warden.com/twit. Do the world a favor, be secure. Alright, I am just blown away by this HTTP two vulnerability that you say is a feature. This rapid recent is a feature,

Steve Gibson (01:41:20):
Not a bug.

(01:41:22):
Now we get to the meaty details. It is described in C V E 20 23, 4, 4, 4 8, 7. [01:41:30] The C V E disclosure is a bit more fair, but even it contains a bit of spin and a little bit of misdirection. The description on the C V E states, the HTTP two protocol allows a denial of service. It says server resource consumption because request cancellation can reset many streams quickly as exploited in the wild in August [01:42:00] through October of 2023. Okay, now Google's summary of the problem is, I think exactly correct. So here's what Google wrote, they said since late 2021. Okay, late 2021. So for the last two years, almost the majority of layer seven, meaning H T T P, high level application level DDoS attacks, we've observed [01:42:30] across Google first party services and Google Cloud projects protected by cloud armor have been based on HTTP two meaning.

(01:42:40):
So HTTP two has become the preferred DDoS attack protocol. They said both by number of attacks and by peak request rates. A primary design goal of HTTP two was efficiency and unfortunately, and here it is, the features [01:43:00] that make HTTP two more efficient for legitimate clients can also be used to make DDoS attacks more efficient. So that's what it is. It's an abuse of the efficiency that HTTP two offers. That is precisely the truth. The features that make HTTP two more efficient for legitimate clients can also be used to make DDoS attacks more efficient. So [01:43:30] not any surprise zero day vulnerability, just the clever abuse of a new feature of HTTP two that anyone designing HTTP two capable servers will need to take into account. Until now, CloudFlare hadn't. They'd been relying upon their conventional DDoS protections, which are doubtless very strong, but this new form of attack was too much even [01:44:00] for that.

(01:44:01):
But that said, I do agree with the creation of A C V E and the raising of alerts about this since indeed rapid HTTP two stream resets promised to cause some havoc with any HTTP two capable web server that's not capable of handling these resets efficiently to get some scope of the problem. Switching back to CloudFlare for a minute. In their [01:44:30] more technical writeup they wrote, starting on August 25th, 2023, we started to notice some unusually big H T T P attacks hitting many of our customers. These attacks were detected and mitigated by our automated DDoS system. It was not long, however, before they started to reach record breaking sizes and eventually peaked just above 200 and million, 200 and million [01:45:00] requests per second. This was nearly three times bigger than our previous biggest attack on record. Concerning is the fact that the attacker was able to generate such an attack with a botnet of merely, and unfortunately, we're now using the word merely when we

Leo Laporte (01:45:18):
Talk about

Steve Gibson (01:45:19):
20,000, 20,000 consumer routers and other things that have been compromised.

Leo Laporte (01:45:27):
I'll never forget, and this was 20 years ago [01:45:30] at Tech tv, we had, I can't remember who was one of our famous train hackers, come in and show us A I R C channel that was being used to for command and control of a botnet. And we're sitting here watching this I R C channel. I think this was right before they shut it down. This was law enforcement. But that just one after another, a compromised system announcing itself and effectively saying, what [01:46:00] is thy will my master? What do you want me to do? And they just all sit there in this I R c. I don't know if they still use I rrc for this, but they just sit there waiting to be commanded.

Steve Gibson (01:46:10):
Now maybe you're remembering my writeup. Maybe

Leo Laporte (01:46:14):
It was you. I don't know. I saw this happening though.

Steve Gibson (01:46:16):
Yeah,

Leo Laporte (01:46:17):
I think it was on the set of the TV show.

Steve Gibson (01:46:19):
Okay.

Leo Laporte (01:46:19):
And I believe law enforcement brought it in. Yeah,

Steve Gibson (01:46:21):
And I also saw the same thing. I infiltrated the attackers I R C channel and I captured a screen [01:46:30] of all these bots reporting for duty.

Leo Laporte (01:46:32):
Yeah, this was 20 years ago, so it's been going on for a long time. Did they still use that kind of command and control or do they have a more modern way?

Steve Gibson (01:46:41):
I have no idea. I could add that specific loop.

Leo Laporte (01:46:44):
Yeah,

Steve Gibson (01:46:44):
Curious. So the CloudFlare guy said there are botnets today that are made up of hundreds of thousands or millions, and remember, this is a 20,000 machine attack. There are botnets with millions of machines. So imagine when those millions of botnets [01:47:00] are updated with this attack. I mean, that's the meltdown event. I mean it's Whoa. So

Leo Laporte (01:47:07):
Yeah, we could see a world

Steve Gibson (01:47:09):
Error here given that the entire web, oh, here's another really interesting metric. So this was 201 million RRP S requests per second. They wrote, given that the entire web typically sees only between one and 3 billion requests [01:47:30] per second, it's not inconceivable that using this method could focus the entire global web's worth of requests onto a small number of targets.

Leo Laporte (01:47:44):
Holy, it's

Steve Gibson (01:47:45):
Like focusing the sun

Leo Laporte (01:47:47):
God

Steve Gibson (01:47:47):
Onto, yeah.

Leo Laporte (01:47:49):
Well, it's like that. A man of magnifying glass with all that power into a pinpoint, your server's going to melt.

Steve Gibson (01:47:57):
It's going to melt. So [01:48:00] just to be clear, since putting this into context is important, this attack by just 20,000 bots generated on its own a peak of 201 million requests per second and the internet across its entire expansive hole sees between one and 3 billion requests per second. And whereas this breathtaking attack was produced by only 20,000 bots working in concert botnets consisting of hundreds of thousands to millions of individual devices are known to exist. [01:48:30] And as CloudFlare wrote in their less technical overview as of last Tuesday, October 10th, every bad guy, bot operator, who may not have been in the loop until now, now knows about it and it's a trivial attack to create. What's more, as Wikipedia told us, HTTP two is offered as an available protocol by around 39% of the Internet's web [01:49:00] servers. We don't currently know what percentage of those web services poorly handle http two stream resets.

(01:49:07):
It could be that cloud providers inherently have a greater problem with this due to their more highly distributed architecture. A single standalone HTTP two server might now be much more affected by this than by an HTTP 1.1 request flood. [01:49:30] Okay? However, and less and until any vulnerable web servers are updated and or move behind some sort of perimeter protection, they're going to be at least in danger of exploitation from this novel server-side resource depletion attack. And what's is that? This is true even from the great many more lesser bot operators who run much smaller botnets once the code to implement this attack has [01:50:00] circulated throughout the underground, as I noted above, its trivial to do this attack and given how things are likely

Leo Laporte (01:50:08):
Out of curiosity, is it just a loop? Is it just a couple of lines of code that says, send this reset request, rapid reset request to this server and then go to 10?

Steve Gibson (01:50:21):
That's what it's,

Leo Laporte (01:50:22):
Is it basically that?

Steve Gibson (01:50:23):
That's it.

Leo Laporte (01:50:24):
I just implemented it.

Steve Gibson (01:50:26):
Yep.

Leo Laporte (01:50:27):
Good lord.

Steve Gibson (01:50:27):
Probably in lisp knowing you.

Leo Laporte (01:50:29):
Yeah, all [01:50:30] in lisp. What is the command for rapid reset? Is it a straightforward command?

Steve Gibson (01:50:35):
We're getting there.

Leo Laporte (01:50:36):
Okay. Alright. Sorry, I'm getting excited.

Steve Gibson (01:50:39):
Also dated last Tuesday the 10th was cloudflare's posting titled HTTP two rapid reset, deconstructing the record breaking attack. Their description begins, this attack was made possible by abusing some features of the HTTP two protocol and server implementation [01:51:00] details. Now see, and the techie guys are a little less hypey and they're getting it right. So this is CloudFlare saying this attack was made possible by abusing some features of the HTTP two protocol, not calling it a vulnerability

Leo Laporte (01:51:16):
And

Steve Gibson (01:51:17):
Server implementation details.

Leo Laporte (01:51:19):
I mean, to be fair, it's a feature, but it is also a vulnerability.

Steve Gibson (01:51:26):
It's not a goodie.

Leo Laporte (01:51:27):
If you had a soft spot at the top of your head, [01:51:30] it's a feature, but I could also knock it. It's still a

Steve Gibson (01:51:34):
Soft spot. It's also

Leo Laporte (01:51:35):
A vulner, still a soft spot.

Steve Gibson (01:51:37):
So they said, because the attack abuses an underlying weakness in the HTTP two protocol, we believe any vendor that has implemented HTTP two will be subject to the attack. This includes every modern web server. We, along with Google and a Ws have disclosed the attack method to web server vendors who we expect will [01:52:00] implement patches. We will see, I mean, there's not much you can do

Leo Laporte (01:52:04):
In the meantime. That's my question. Yeah,

Steve Gibson (01:52:05):
Yeah. The best defense is using, of course here they're going to sell themselves, although it is free a DDoS mitigation service like Cloudflare's in front of any web facing a p I or web server. Okay. Again, what exactly now is an HTTP two stream reset? I could write this up myself, [01:52:30] but in the interest of savings some time so that I have time to assemble other interesting things this week, because actually I did this first and then I did the user feedback. I'm going to switch to Google's explanation. I'm switching to Google because CloudFlare is pretty much unable to stop blaming this on HTTP two and calling it a zero day vulnerability. Google explains HTTP two uses streams. Bidirectional abstractions used to transmit [01:53:00] various messages or frames between the endpoints. Stream multiplexing is the core HTTP two feature, which allows higher utilization of each T C P connection streams are multiplexed in a way that can be tracked by both sides of the connection while only using one layer four, meaning T C P connection stream multiplexing [01:53:30] enables clients to have multiple in-flight requests without managing multiple individual connections.

(01:53:40):
Okay, now I'm going to switch back and note that we talked about this a million years ago on the podcast. Well, okay. 2015 with H T T P 1.1, a single T C P connection can be created to a server and a client. It's able to send multiple queries [01:54:00] to the server sequentially in a sort of pipeline, but the server must receive them in order and reply to each query in order so that the sequence of replies matches the sequence of queries. The big trouble with this is that many smaller and faster to answer queries could get held up behind an earlier query that takes a server more time [01:54:30] to reply to. Perhaps it needs to query something else to obtain an answer. Meanwhile, some queries like for simple images, for example, little icons and embellishments would be held up waiting behind the big slow query to be returned. When Google says that H T T P uses streams, they mean that ID tags are added to individual queries and ID tags are [01:55:00] returned with their matching replies. This creates a sort of parallel abstraction where even though there's still only one connection, a highly capable H T T P server could accept every incoming query. The moment it arrives, assign an independent task or thread to begin assembling each query's reply individually and then send back that query's reply with [01:55:30] its associated ID tag as soon as its reply is ready.

Leo Laporte (01:55:35):
That makes sense completely because modern machines are multi-threaded, so they're not doing a single process anymore. That's ancient history, so they could take advantage of that. A lot of programming languages have streams built in for this very reason,

Steve Gibson (01:55:49):
Right?

Leo Laporte (01:55:49):
Yeah.

Steve Gibson (01:55:50):
So Google continues. They said one of the main constraints when mounting a layer seven meaning an H T T P level annual application level [01:56:00] Doss attack is the number of concurrent transport connections. Each connection carries a cost including operating system memory for socket records and buffers C P U time for the t l s handshake as well as each connection needing a unique for tuple, that is the IP address and the port pair for each side of the connection, which constrains the number of concurrent connections between two IP [01:56:30] addresses in H T T P 1.1. Each request is processed serially. The server will read a request, process it, write a response, and only then read and process the next request. In practice, this means that the rate of requests that can be sent over a single connection is one request per round trip where a roundtrip includes the network latency, the proxy processing time, if there's a proxy [01:57:00] in front and backend request processing times while HTTP 1.1 pipelining is available in some clients and servers to increase the connections throughput.

(01:57:16):
It is not prevalent among legitimate clients. Now, I thought that was interesting. What Google just said was that while HTTP 1.1 does technically support allowing clients to send [01:57:30] ahead queries, as I originally mentioned in the process known as pipelining in practice, that has not been ever been widely adopted and now it won't be because HTTP two has come along now and all the browsers do that, and the reason was it's a little scary to send requests blind and just hope that the answers come back in the right sequence and you not get confused and counting them much cleaner [01:58:00] if you tag them with individual IDs so you're sure about which query matches which request. So anyway, it wasn't widely adopted. So the clients wait until they've received the reply to their single outstanding query before they're comfortable to submit the next query, even though technically they could.

(01:58:24):
Google says with HTTP two, the client can open multiple [01:58:30] concurrent streams. So again, get this that a stream is just an abstraction in the same way that packets are an abstraction of a connection on the internet. Two points we say we're connected, but it's actually a stream of packets. Similarly, the streams are an abstraction of multiple concurrent connections on a over a single T C P connection. So they said each stream [01:59:00] corresponding to one H T T P request, the maximum number of concurrent open streams is in theory controllable by the server. But in practice clients here it is may open 100 streams per connection and the servers process these requests in parallel, meaning that a client could connect using T C P, bring up A T L SS connection to go H [01:59:30] T T P S then generate. Then when the server has responded, it'll say that it is able to handle HTTP two.

(01:59:40):
So the client says, great, it then basically just pours as many queries for items on the server into this connection as possible. Giving each one a unique ID tag, knowing that as the server is [02:00:00] able to handle them and in any order that the server may choose, it will start sending the replies back tagged with the same IT ID as the query that it generated and a hundred of them can be outstanding at once. You can just pour a hundred in. So Google said, for example, the client can open 100 streams and send a request on each of them in a single round trip. The proxy, [02:00:30] which is the front end to a cloud service, we'll read and process each stream serially, but the requests to the backend servers can again be parallelized. The client can then open new streams as it receives responses to the previous ones, meaning it can always have a hundred of them outstanding, a hundred pieces of work to be done.

(02:00:54):
So as streams complete by the reply coming back that [02:01:00] says, oh, there's room for another stream. And so it just immediately emits another query. So Google says, this gives an effective throughput for single connection of 100 requests per round trip with similar round timing constants to HTP 1.1 requests. This will typically lead to almost 100 times higher utilization of each connection action. Okay, so this is [02:01:30] where, why and how HTTP two offers potentially far greater performance over HTTP 1.1, HTTP two solicits clients our web browsers to dump all their queries into a single connection at once. Since it's just a single T C P connection, they cannot actually move across the wire simultaneously, but they can be queued up and sent so they're packed very tightly [02:02:00] and so that fewer small and less efficient packets are sent. Remember we talked about this back then. If you send small queries, then you're only using a part of a packet, yet you're still having to switch and route an entire packet. If instead you're able to pack queries literally where the last bite of one starts with the first bite of the next, then all your packets are [02:02:30] full. And so all of your switching and your packet processing rate is also benefits. I mean, it's really the way that this should have been done, but it was simple back then. As I said a couple weeks ago. Nothing is simple anymore. Unfortunately, it's all getting complicated.

(02:02:51):
So anyway, assuming that the backend has sufficient parallel serving capability, all of the then outstanding replies can be assembled in any order. The moment [02:03:00] a rep reply is ready, it's queued up for return to the client with its ID tag identifying which query it's the reply for it is beautiful and elegant, and it is a future that is here. Unfortunately, it can be abused like nothing else ever has been. So note that the most modern preexisting high request rate attacks are already leveraging this parallel streaming capability. [02:03:30] As Google said since 2021, the largest attacks we've been seeing are HTTP two because they're using these streams in order to make this happen. So HTTP two already required or allowed much higher request rates than were possible under HTTP one or 1.1, [02:04:00] but now here's what you've been waiting for. The problem that's created by the new abuse of this fancy parallel architecture, Google writes, the HTTP two protocol allows clients to indicate to the server that a previous stream should be canceled by sending a reset stream frame. That's all you need to do at the client end. You just send [02:04:30] a what

Leo Laporte (02:04:30):
Wrong with that?

Steve Gibson (02:04:31):
Basically, nevermind, you send a nevermind stream.

Leo Laporte (02:04:36):
Nevermind.

(02:04:37):
Forget I asked.

Steve Gibson (02:04:38):
Yep. The protocol does not require the client and server to coordinate the cancellation in any way, meaning there's no need for it be confirmed. TCP automatically gives us reliable transport, so tcps got that covered. If the client sends a reset stream frame, [02:05:00] it can assume the server has and it is then able that frees up one of its 100 for another query. Thus the client can do this unilaterally. The client may also assume that the cancellation will take effect immediately when the server receives the reset stream frame before any other data from that T C P connection is processed. Google says this attack is called rapid [02:05:30] reset because it relies upon the ability for an endpoint to send a reset stream frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request. The request is canceled but leaves the HTTP two connection open for additional requests and cancellations.

(02:05:57):
Therefore, the HTTP [02:06:00] two rapid reset attack, which is enabled by this capability is simple, says Google the client opens a large number of streams at once as in the standard HTTP two DDoS attack. But rather than waiting for a response to each request stream from the server or the proxy, the client cancels each request immediately. And since HTTP two specifies [02:06:30] that the client may assume that any canceled stream is immediately available for another request, the H T T P abusing attacker may immediately follow a stream reset with another new request using the same stream. The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight by explicitly canceling the requests. The attacker never exceeds the limit on the number of concurrent [02:07:00] open streams, which is typically 100. The number of in-flight requests is no longer dependent on the roundtrip time, but only on the available network bandwidth.

(02:07:12):
In a typical HTP two server implementation, the server will still have to do significant amounts of work for canceled resets, such as allocating new stream data structures, parsing the query, and doing header decompression and mapping the U R L to a resource. Moreover, [02:07:30] in reverse proxy implementations, the request may be proxied to the backend server before the reset stream frame is processed. This requires the proxy to then forward the stream reset to the appropriate backend server. By comparison, the attacking client has almost no cost for sending requests. This creates an exploitable cost asymmetry between the server and the [02:08:00] client.

Leo Laporte (02:08:01):
Oh, I was just saying there's the problem clearly. Yep,

Steve Gibson (02:08:04):
Exactly

Leo Laporte (02:08:04):
Right. It's the problem with spam. It costs them nothing to send it. Exactly. Yeah,

Steve Gibson (02:08:10):
And so another advantage, the attacker gains is that the explicit cancellation requests immediately after creation means that a reverse proxy server, which is what CloudFlare has on their perimeter and all the big cloud providers do, won't send a response to any of the requests canceling [02:08:30] the requests before our response is returned, thus reduces the returning bandwidth to the attacker, meaning the upstream back to the attacker won't get clogged up with much bigger replies since they've been canceled. So all of the work is loaded onto the servers with very little traffic returning. Although it's diabolical, it's not a flaw in HTTP two. It's just an abuse of a deliberate [02:09:00] design feature. So what does Google recommend? They begin by explaining. They said, we don't expect that simply blocking individual stream requests is a viable mitigation against this class of attacks. Instead, the entire TCP connection needs to be closed when abuse is detected. Get this HTTP two they wrote provides built-in support for closing connections [02:09:30] using the go away frame.

(02:09:36):
You kind of love that the designers of the HTTP two protocol defined a go away frame that could be sent back up the link to the client telling it to, well go away. Google says, the R F C defines a process for gracefully closing a connection that involves first sending an informational go away [02:10:00] that does not set a limit on opening new streams and one round trip later sending another that forbids opening additional streams. So there is a process basically for allowing the client to gracefully shut down rather than just terminate all of its pending queries. However, they said that this graceful go away process is usually not implemented in a way which is robust against malicious clients.

Leo Laporte (02:10:30):
[02:10:30] Well, we got to work on that so

Steve Gibson (02:10:31):
That

Leo Laporte (02:10:32):
Maybe

Steve Gibson (02:10:33):
We're going to see about fixing

Leo Laporte (02:10:34):
That. We need a graceful go away for sure.

Steve Gibson (02:10:37):
This form of mitigation leaves the connection vulnerable to rapid reset attacks for too long and should not be used for building mitigations as it does not stop the inbound requests. Instead, the go away should be set up to limit stream creation immediately. So maybe we will see an HTTP 2.1 that will [02:11:00] tweak the definition of this. They wrote, this leaves the question of deciding which connections are abusive. A client, which cancels requests is not inherently abusive. The feature exists in the HTTP two protocol to help better manage request processing. Typical situation says Google are when a browser no longer needs resources it had requested due to the user, for example, [02:11:30] navigating away from the page or applications using a long polling approach with a client side timeout mitigations, or maybe when you've got multiple mouse pointers and they're in disagreement about

Leo Laporte (02:11:46):
What

Steve Gibson (02:11:47):
Could be done next. You see

Leo Laporte (02:11:47):
There's a good reason for that. Browser

Steve Gibson (02:11:51):
Mitigations for this attack, they said can take multiple forms, but mostly center around tracking connection, statistics, [02:12:00] basically detecting abuse and using various signals and business logic to determine how useful each connection is. For example, if a connection has more than 100 requests with more than 50% of the given requests canceled, it could be a candidate for a mitigation response. The magnitude and type of response depends on the risk to each platform, but responses can range from forceful go away frames as discussed before to [02:12:30] closing the TCP connection immediately. Okay, just hanging up to mitigate, basically go away then hang up. To mitigate against the non canceling variant of this attack, we recommend that HTTP two servers should close connections that exceed the concurrent stream limit. This could be either immediately or after some small number of repeat offenses.

Leo Laporte (02:12:53):
In other words, there's a fingerprint for this kind of attack.

Steve Gibson (02:12:56):
Yeah,

Leo Laporte (02:12:56):
You can

Steve Gibson (02:12:56):
Immediately, it's very obvious if [02:13:00] every request is immediately being canceled, I'd hang up on that jerk immediately.

Leo Laporte (02:13:06):
Yeah, well, that's good news. I mean,

Steve Gibson (02:13:10):
Oh yeah. So that's the story exactly. As Google initially characterized the problem, the deliberate design decision of HTTP two, which can be used to significantly increase its efficiency and does increase its efficiency, can also be used to significantly increase the [02:13:30] potential and the potency of DDoS attacks. Attackers figured out that rather than using HTTP two simply to simultaneously ask for tons of resources and then be flooded by their return, they could instead immediately cancel their request and reissue another

(02:13:55):
Against this threat. Today's servers, especially those in the cloud, which have distributed [02:14:00] their request handling among multiple backend components, which might make canceling those issued requests much more tricky because those also have to be distributed, make it more time consuming, would be seriously taxed by this new attack strategy. It'll be interesting to see whether anything could be done to change HTTP two to prevent or limit this abuse. At the moment, the various servers are testing themselves and modestly tweaking their request handling to [02:14:30] do a less bad job of dealing with this abuse of this HTTP two future. Since, as I mentioned above, it's not possible to spoof the IP addresses of anything that's writing on top of tcp. The best solution might be to dynamically blacklist or at least significantly throttle any IP that is found to be abusing HTTP two rapid reset, and that way the bots would be recognized and quickly ignored at the [02:15:00] perimeter of a large hosting provider like CloudFlare.

Leo Laporte (02:15:03):
We do something like that all the time with password mitigation, right? You don't let somebody log in, my light just went out 10 times. That's a sign, isn't it? You don't let somebody log in an infinite number of times, one after the other, and in many cases you just say, oh, six times you're out, or 10 times you're out.

Steve Gibson (02:15:23):
So

Leo Laporte (02:15:23):
This would be a simple mitigation. It doesn't change the protocol though, right? It [02:15:30] changes the way the server operates.

Steve Gibson (02:15:33):
Exactly.

Leo Laporte (02:15:34):
Yeah. I feel like this is a really good example of, and I'm sure these days when you're at an I E T F meeting or a W three C meeting and they propose these protocols, they try to think of ways people would take advantage of it, and somebody just forgot to say, oh, somebody could just keep sending rapid resets [02:16:00] and it would overwhelm us because we will start up a million threads per user. We should build into the protocol. There's a limit, and they just didn't, they missed it. It happens. It's not that the feature's a bad feature. I mean, the idea of these with a well-behaved client is fine.

Steve Gibson (02:16:22):
Well, and consider that this is eight years old. That is the protocol.

Leo Laporte (02:16:26):
It just took them a while.

Steve Gibson (02:16:27):
It took them this long. I mean, [02:16:30] it took the bad guys this long to go, Hey, so

Leo Laporte (02:16:33):
I'm not surprised that a bunch of engineers didn't figure it out in six

Steve Gibson (02:16:36):
Months, right? Our attack, we're only able to have a 100 concurrent outstanding requests, so we're having to wait for the replies to come back to free up a stream.

Leo Laporte (02:16:51):
So

Steve Gibson (02:16:52):
Why don't we just cancel that request then we don't have to wait for the answer.

Leo Laporte (02:16:56):
Yeah. Yeah. And I think it would be pretty [02:17:00] easy to spot somebody behaving badly that there would be, I guess the key is that there's enough distinction between a well-behaved user. This is the problem. If it's not distinct, you don't want to hang up on, well, this is why syn acts work, because you can't tell the difference between somebody who's sending you a reasonable sin and somebody who doesn't care. But if you could tell the difference between a bad actor and a normal user pretty readily, which it sounds like you could, I think mitigation might be doable.

Steve Gibson (02:17:28):
And here's where the term heuristic [02:17:30] comes to

Leo Laporte (02:17:31):
Our Oh yeah. That's a heuristic, isn't it?

Steve Gibson (02:17:33):
We would be using a heuristic, we would say, we've done stats on for the last month on all of the HTTP two connections. We've never seen more than 5%

Leo Laporte (02:17:45):
Rapid

Steve Gibson (02:17:46):
Resets, so if anybody does 25, they don't deserve our attention today.

Leo Laporte (02:17:53):
Right. You always want to have it so that legitimate, it's like spam fighting legitimate users. [02:18:00] You don't want false positives. You want the legitimate users who are doing things right, but maybe are demanding.

Steve Gibson (02:18:06):
Yeah. I mean, how many times is someone required to say, if you didn't get our email, check your spam folder

Leo Laporte (02:18:15):
Because

Steve Gibson (02:18:16):
You wanted, presumably you wanted that because you were talking to them, but it got routed into spam because unfortunately we've, again, you were too aggressive a heuristic that is not sharp enough.

Leo Laporte (02:18:28):
Yeah, it was too aggressive. This one seems like [02:18:30] it would be doable, but maybe, I don't know,

Steve Gibson (02:18:32):
Maybe

Leo Laporte (02:18:32):
It's

Steve Gibson (02:18:32):
Not and that's I'm sure what the server vendors are curring around now doing in anticipation of this, and I should mention that all of these servers have the ability to turn off this protocol, so if you were at your own HTTP two site and you were being attacked, just tell your server to turn off support for HTTP two, be a 1.1 server and [02:19:00] yes, your performance will drop, but it's better than being held off the internet by one pissed off bot.

Leo Laporte (02:19:06):
It really drops when you get 20,000 people doing hundreds requests.

Steve Gibson (02:19:10):
I mean, I wouldn't be at all surprised if one attacker could not hold, I mean not CloudFlare, but a standard website with no upfront protection. Well,

Leo Laporte (02:19:24):
And a static site like mine for instance, I'm not worried mine being DDoS, but a static site like mine for instance, I don't [02:19:30] need HTTP two.

Steve Gibson (02:19:31):
HT

Leo Laporte (02:19:32):
1.1 would be fine.

Steve Gibson (02:19:34):
Yep.

Leo Laporte (02:19:37):
I'm using Nginx. Is Nginx one of the server?

Steve Gibson (02:19:40):
Yes. Nginx is great and they are working on a mitigation.

Leo Laporte (02:19:48):
We use a Doss mitigation on our site, but I won't tell you by whom. Not CloudFlare, but there's a lot of companies that offer Doss mitigation, DDoS mitigation.

Steve Gibson (02:19:58):
Anybody who is curious could [02:20:00] follow your traffic. I guess

Leo Laporte (02:20:01):
It'd, you wouldn't be hard to out go

Steve Gibson (02:20:02):
Through your Doss mitigator.

Leo Laporte (02:20:07):
All right, Steve, what a great, great topic. Maybe I was more engaged because they weren't feeding me and asking me questions and saying, Leo, can you come down the hall and check on? So maybe I was more engaged, but what a good, I think a really good show. Really good.

Steve Gibson (02:20:23):
Yeah. Well, and I think it's important for us all to note that there is now a new DDoS [02:20:30] technology, a way of doing a DDoS that is effective against the latest HTTP two protocol, which nearly 40% of the web, 39% of the web is using,

Leo Laporte (02:20:48):
And maybe there's something with three too. We should probably look at all three of 'em.

Steve Gibson (02:20:52):
Yes, that was addressed and the presumption is that three won't be subject to the same [02:21:00] problem.

Leo Laporte (02:21:01):
Interesting. I imagine

Steve Gibson (02:21:03):
The hot screens, it's on top of Q U I C instead of on top of T C P.

Leo Laporte (02:21:08):
Okay. How interesting.

Steve Gibson (02:21:11):
Yeah, maybe

Leo Laporte (02:21:11):
That's this topic for another day.

Steve Gibson (02:21:13):
We will see,

Leo Laporte (02:21:14):
Will we get back to the top 10 mitigations next week?

Steve Gibson (02:21:18):
Assuming that nothing else melts down between now and next Tuesday, we will finish up with part two of the top 10 cybersecurity misconfigurations,

Leo Laporte (02:21:29):
Bad guys, [02:21:30] permitting, as always on this show, Steve Gibson's at grc.com. That's where you'll find valid drive. That's where you'll find the world's best smash storage, maintenance and recovery utility spin, right 6 6 1 on its way. And as Steve mentioned today, you can get a copy of it right now. If you're a 6.0 user, you can get a beta copy of it, so if you've got some need to run it, this is the time. grc.com. While you're there, of course [02:22:00] you could pick up a copy of this show. Steve has the usual 64 Kilobit audio, but he also has a couple of unique formats, the 16 Kilobit audio for the bandwidth impaired and really well done transcripts by Elaine Ferris. She writes out every erum and ow and probably got a few coughs in there a second ago, coughs. I'll have to look at the transcript later. That's all@grc.com. [02:22:30] You don't have to use X to message, Steve. You can do it at grc grc.com/feedback. It's a old school post and get HTP 1.1 form, but it'll work just fine. You're not using streams on your site, are you?

Steve Gibson (02:22:47):
Nope.

Leo Laporte (02:22:48):
No, of course not. He's a static site. It just sits there waiting. It's HTML baby. We also have the show 64 Kilobit audio at our website, [02:23:00] twit TV slash SS n. We do have video too though. If you want to see the video of my mom's house, you could see that. I don't know if you noticed it got dark as

Steve Gibson (02:23:11):
The show

Leo Laporte (02:23:11):
Progressed.

Steve Gibson (02:23:12):
Yeah,

Leo Laporte (02:23:13):
That's interesting, isn't it? Yeah. I opened the shutters for a while. You could see the houses and the trees, but now it's just dark out there. Video and all of that available and show notes too at both sites, TWIT tv slash ssn or grrc.com. He doesn't have the video. [02:23:30] We do. If you want to watch us do this live, we do security now every Tuesday, roundabout one 30 Pacific. That's four 30 Eastern time in the afternoon, and it's 2030 utc. The live streams audio and video are at live twit tv. If you're watching live chat, live with us. Of course, our club members have these special velvet rope gated access to the discord, the rest of the unwashed masses. You're welcome to use [02:24:00] I R C. Actually, we don't judge. We don't judge. Somebody in I R C said, and I have to assure you, this is not the case. I could just see security now going behind a paywall over my dead body. I think this is of all the shows, this is the last one that would ever go behind a paywall. It's not our intention to put a paywall up. I hate paywalls. I don't think they serve anybody, so the shows that have been free all along will continue to be free all along ad supported.

(02:24:29):
We have some paywall [02:24:30] shows in the club, but that's because the club pays for them in effect. And if enough people listen in the club and we can get some advertisers, we'll absolutely put it out as an ad supported free version. So I'm a strong believer in ad supported free. Don't worry. Security now is not going behind a paywall. Not anytime soon anyway. I don't think Steve would allow it, frankly. So forget what I say. What else should we say, Steve? [02:25:00] I guess, how

Steve Gibson (02:25:00):
About Goodbye?

Leo Laporte (02:25:01):
Goodbye would be a good one. Get the show on the website. There's a YouTube channel. Subscribe in your favorite podcast. Whatever you do, make sure you listen every single week. You do not want to miss an episode, and we'll see you next time on Security Now.

Steve Gibson (02:25:14):
And you're back in the shop next week. I'll be

Leo Laporte (02:25:16):
Back next week.

Steve Gibson (02:25:17):
Okay,

Leo Laporte (02:25:18):
Cool.

Lou Maresca (02:25:19):
Come join us on this week in enterprise Tech Expert Co-hosts and I talk about the enterprise world, and we're joined by industry professionals and trailblazers like CEOs, CIOs, CTOs, CCOs, [02:25:30] every acronym, role plus IT pros and marketeers, and we talk about technology, software plus services, security, you name it, everything under the sun. You know what? I learned something each and every week in Atually. You will too. So definitely join us and of course, check out the twit TV website and click on this weekend Enterprise Tech subscribe today.