Hands-On Windows 158 transcript
Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show.
Paul Thurrott [00:00:00]:
Coming up next on Hands on Windows, we're going to take a look at a feature called semantic search, which may or may not finally solve all of our search problems in Windows 11. Podcasts you love from people you trust. This is TWIT. Hello everybody and welcome back to Hands on Windows. I'm Paul Thurat and if you have been around Windows for a while like I have, then you probably remember things like Longhorn and WinFS and database back file systems and how we were going to solve all those file search problems and then didn't. Right? Yeah. This never happened. And so here we are, decades later, we're still using the same index based search functionality that we've always had.
Paul Thurrott [00:00:52]:
Windows added search directly to the start menu early 2000s. I don't even remember when that was. They added it to the taskbar, which is really the same as Start in Windows 10 and Windows 11. And it's not very good.
Speaker B [00:01:08]:
Right.
Paul Thurrott [00:01:08]:
If anything, they've made it worse. Right. There are these weird interfaces, like, let me bring up search here, like the search highlights screen which has ads on it. And you know, it's just been kind of a whole lousy thing. But in recent years AI has happened, and not just AI, but local AI.
Speaker B [00:01:27]:
Right.
Paul Thurrott [00:01:27]:
So we have these cloud based LLMs like OpenAI, ChatGPT and Copilot and Microsoft, like Google and Apple has also been working on local, smaller, smaller SMLs, I guess. SLMs, small language models, AI models that run on the device.
Speaker B [00:01:44]:
Right.
Paul Thurrott [00:01:45]:
And so in the PC space, this type of thing occurs today exclusively on what's called a Copilot plus PC that has an MPU of a certain type. I think this is going to expand eventually to other types of computers, but for now that's what we have. And it's kind of an interesting test bed for some of these new AI features that are coming down the pike, one of which is called semantic search. And if you do have a Copilot plus PC, you've probably seen this. So if I actually turned off my taskbar based search, let me put that back on so you can see it turn on the whole search box. So if you click in here and start typing, this will work like it has in Windows 11 kind of all along. But you can see here, it's got this purple, blue, pink kind of icon which helps to indicate that maybe we're doing something a little different here or could be doing something a little different. So I could just do something simple like search for the name of an app.
Paul Thurrott [00:02:43]:
And because that app is on My system, I can click on it and it runs. And that works the way it did before.
Speaker B [00:02:49]:
Right.
Paul Thurrott [00:02:49]:
The other way is just to go directly to the Start menu. So you can either bring up the Start menu like you would, you can see up at the top that the search box is selected. And what that means is if I start typing from here, it will give me that. It goes right to that same search experience. Right. I could also do a keyboard shortcut. So Windows key +s will bring up that search experience. Because I've got the search box down there.
Paul Thurrott [00:03:13]:
I'm typing into the taskbar. But if I didn't, there would be a search box up here in the top of the window. So far that works exactly the way it's always worked. But there are other places to search in Windows, right. Obviously you can go to the file system using File Explorer and same thing, see blue, pink, purple. Right. So we're playing around with what happens here, but the idea here is that you type for something. Typically what you would really do is go maybe to a particular folder and then maybe go into, you know, whatever, and then search from there.
Paul Thurrott [00:03:46]:
And because you know, this will kind of not ground the search, but filter the search down to what it is you're looking at, rather than the entire database of items it could search. Also, just on a side note, although this is not necessarily an exact topic for here, we also have search in the Settings app.
Speaker B [00:04:06]:
Right.
Paul Thurrott [00:04:06]:
And so one of the other features that's coming in windows 1125 H2 and 24 H2 is this notion of semantic search or natural language search really in the Settings app. And so they have a nice sample there. So my mouse pointer is too small is actually kind of cool because this is a plain or a normal English way of saying this thing. And instead of just matching it to an exact feature, it's using its natural language capabilities to understand what it is you're looking for. And it's even providing a way to kind of do it right there in line.
Speaker B [00:04:41]:
Right.
Paul Thurrott [00:04:42]:
Which is nice. So that's kind of a nice improvement. But the point of semantic search is to hope to hopefully solve this problem that we have with index based search, which is that at least in my experience, when I type a search for anything, I never find it. So if it's in OneDrive, for example, I could search in the file system and I never find it. But if I go onedrive. Com and search for it there, I find it pretty immediately. And so what this does is it works off of the index that your computer is already creating. And then it uses an on device AI model to basically create what is in essence like a secondary index that's based on natural language search.
Speaker B [00:05:22]:
Right.
Paul Thurrott [00:05:22]:
And so you can just type. It's searching basically a more constrained, grounded version of the index, if that makes any sense at all. So let's see what that looks like. And this, this will be terrible because I just turned this on in this computer this morning and it's not completely indexed. But let's give it a shot. So I mentioned Longhorn, so that maybe that's a good place to start because I have written a lot of content about Longhorn or did back in the day, so 20 years ago. Ish. And you can see here it's not quite done, but of course when I search For Longhorn, Windows 11 search by default is searching everywhere.
Paul Thurrott [00:05:59]:
So you're getting the best match is the Longhorn Steakhouse, which is absolutely not what I was looking for. But you can see over on the side they have. I guess these are. Yeah, these are reasonably close locations for this restaurant that I will never visit. It did find one folder that's kind of interesting, but I could go into Documents and just see this. And sure enough, I can tell just from looking at these things that these are articles mostly that I would have written, you know, a long time ago. Right. And so I'll give this one a shot.
Paul Thurrott [00:06:28]:
We'll just click on this, see what happens here if it ever loads. Loaded in a different window here, but yeah, here we go. So this is an article I wrote in 2005 that appeared in PC World. Yes, PC World magazine, about 64 bit versions of Windows, which at that time were incredibly rare. And. But the point is, this is somewhere sitting in my OneDrive, as you can see from the URL and in my archives. And even though this file is not live, you know, it's not synced to my desktop here, it still was able to find it.
Speaker B [00:07:05]:
Right.
Paul Thurrott [00:07:05]:
And so this is kind of one of those neat benefits of the AI in a way. It's not that index based search maybe wouldn't have found it, but you probably wouldn't have found it. So in my case, I find that I either have to navigate in and then search from a small, you know, like from a local location or from a location that's more confined or like I said, go to the web. So that's kind of interesting. It works from everywhere. So obviously if I do it from here, you know, I get the same exact result.
Speaker B [00:07:34]:
Right.
Paul Thurrott [00:07:34]:
And you can see the same thing. But you would typically do this type of thing or I would anyway, from File Explorer.
Speaker B [00:07:40]:
Right.
Paul Thurrott [00:07:40]:
And so to me this is a little more interesting. So again, I'll just keep. I'll stick with the Longhorn example and this one I actually really like because I use Synology drive and I can see this result here, some Synology. I do have my archive up in OneDrive, which you can see from the cloud icon here, is not. It's not local, it's. I mean you can access it locally, but it's not, you know, downloaded, but it still found it, which is kind of cool. See if I could go into any of these that makes any sense. Let me see an article or a folder I should say from 2005 that just has.
Paul Thurrott [00:08:15]:
Okay, it's just a web article, but let me go back actually, so. But you get the idea. So this came up. I mean I wrote a lot about long the rest of this thing found 374 results are still going actually. So there's going to be over 400 eventually and probably over 500 really. So in my, in this case it's everything from screenshots to PDFs to articles that I wrote, Word documents, HTML documents, whatever, it's like all kinds of stuff. So it's like this whole history of whatever that's tied to that particular search query. So that's pretty cool.
Paul Thurrott [00:08:51]:
So the problem with this to me is only that you have to have a copilot plus PC to get this. But like I said, I think it's going to come to everybody. If you're familiar with how the index works today, probably not most people aren't, but there is this old school indexing options control panel which you can find with Start Search and you can see it's actually, it's just going to town here trying to index the drive, which I'm sure it's not impacting the performance in any way, shape or form, but what you would typically do is click this modify button and then you could just see what all the locations were getting one of those user or Windows low authentication things and then you can just kind of dive in.
Speaker B [00:09:37]:
Right.
Paul Thurrott [00:09:37]:
And so in my case, like I know if I go into my Paul account, oops, I don't want to turn that off. But you can see all the stuff that's in there and you can just kind of go through and just, you would just select like I want you to index this, this part of it in Windows 11. There's actually a modern version of that UI in the Settings app. And so if you go into privacy and security and then search, this is some of the. It's actually all of the same options with other options.
Speaker B [00:10:03]:
Right.
Paul Thurrott [00:10:03]:
So in this case, you could turn off search highlights. And that was that UI that was so terrible. So you can see this is off, which is wonderful, which is what you actually want. And then for best results, I found, even though it has to do this initial index, it's set on Classic. By default, I turn this on to Enhanced and it's basically the entire PC.
Speaker B [00:10:26]:
Right?
Paul Thurrott [00:10:27]:
And what that means for me is that if I dive into my user account here in File Explorer, I have Google Drive, I have OneDrive, and I have Synology Drive and each of those things. Synology Drive is the thing I work off of. So that's my kind of current work. But all three of these locations have some archives. And because the indexing and the semantic search that are available now on Windows 11 with a copilot PC can work with cloud accounts, which is an option somewhere. Where is it somewhere in here, anyway? There's an option here somewhere that will say, maybe it's not tied to that. I hope that will say search through your cloud accounts as well. So right now, OneDrive is the only thing that's technically directly supported, but it's an open API that anyone could use.
Paul Thurrott [00:11:21]:
And in my experience, it already actually works. So you saw the results from Synology Drive, even though that thing is not part of that kind of new cloud system that they have. So does this kind of solve the problem? Yeah, asterisk. I mean, the problem with it, of course, is that it only solves the problem for those of you who have a copilot plus PC, which is probably less than 10% of you, right? So indexing still occurs. You still have to do that. You have to deal with that, so to speak. I mean, most people wouldn't touch it and it would just kind of work, but I found that it works better, like I said, when you turn on that enhanced indexing mode. And I think it's going to come, like I said, eventually to everybody has to.
Paul Thurrott [00:12:02]:
Right? I mean, this is such an obvious feature. It's something that we have been struggling with in Windows for so many years. And this is. I think this is going to kind of solve it. It's interesting. There was no one over the past 20 years who said, you know what? Once AI comes down the pike, we're going to solve that problem. They've been working on different ways to solve it, but when AI did arrive, they realized, wait, actually this is going to work really well and you know, it's tied into some of the features that are built into Windows 11, like click to do or recall where it's using AI behind the scenes to do pattern matching and understand what is in a document or is in an image or whatever, and creating its own index so that instead of just searching against say the file name or if it's a known file type like the file contents, it actually can search against AI's understanding of what is in those things. So it kind of broadens the amount of information that can come back.
Paul Thurrott [00:12:58]:
And in my experience it actually works faster and better. So don't often say that about AI, but that's the way it works today. So there you go. So nothing is changing from a UI perspective except for the cute little sparkly purple and pink stuff. Keyboard shortcuts are the same Windows key +s for Start Search Control F if you're in File Explorer, but if you do have a Copilot plus PC, definitely check this stuff out. Like I said, it just to me, it makes finding things that are in your file system or up in the cloud just a lot faster. So I hope you found this to be interesting and useful. We will have a new episode of Hands on Windows every Thursday.
Paul Thurrott [00:13:39]:
Can find out more@twit tv.h thank you so much for watching. Thank you as always to our Club Twit members. We love you and you can learn more about Club Twit if you're not a member at TWIT TV Club. Thanks. I'll see you next week.
