Hands-On Windows 140 transcript

Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show

0:00:00 - Paul Thurrott
I'm going to get up next on Hands and Windows. The wait is over. All those cool co-pilot plus PC features are finally coming. Podcasts you love From people you trust. This is Twit. Hello everybody, and welcome back to Hands and Windows.

For the past months years, we've been talking about all these new features coming down the pike, and three, four episodes ago I listed out several features of a dozen, or more, a couple of dozen that Microsoft has been testing in the Insider program but has never released to the public, and stable in the general availability channel, as we call it. But as of this recording, that stuff is finally happening. So the wait is over. We're going to get a really big Windows 11 update in May of this year. If you have a co-pilot plus PC, whether it's a Snapdragon X or X64, you know AMD, intel-based PC. Whatever this is going to be, this is the big one, as we say. So that's appropriate, right? May 2025 is one year since Microsoft announced Copilot plus PC, so that's how long this stuff takes. So we've talked about some of this, but some of this is new too. That's kind of interesting to me. So there's some new stuff coming as well. Some of this is new too. That's kind of interesting to me. So there's some new stuff coming as well. So the big ones, of course, are recall. I'm going to try to open this. I'll see how this works. It likes to. I'm going to do a. It has to do a pin. Okay, I'm on the other screen for this, but I'll bring it over here and this you can see my show notes for here and you can use. I just enabled it today, but you can go back in time here and see the things that I was working on today and hopefully nothing super embarrassing, but you get the idea. I played a little bit of Quake, you know. So it was a busy day. But recall, of course, is the marquee feature that Microsoft was advertising at the Copilot plus PC launch a year ago. They delayed it because of a lot of complaints about the security. They enabled it in preview in November on Snapdragon X and then, I think, in December on X64 PCs and now, as of this May, it will be generally available, still going to be in preview, still be marked preview, but anyone who wants it can get it if you have that kind of a computer, which isn't too many people, frankly. But that's part of the controversy, I guess. So that's happening. So I've got that going again. I had turned this off.

You may recall, last time we did a show the other big one is click to do. Now for this to work, I'm going to try to. Let's see if there's something in here that maybe is worth looking at. Probably not, but uh, yeah, no, I wanted to write about this topic but it's not there. So, yeah, I've started writing here.

So this is not very exciting, but the idea here is that you can hold down the windows key, sorry, and uh, get information about the stuff that's going on on screen. So the stuff that's going on screen here is text, and so I right click and I have text actions that I can work with. This is a list of things that are going to grow over time, but for now you know the copilot type stuff that everyone is familiar with summarize, rewrite, et cetera. You can ask copilot the app. You can search on the web with Bing, et cetera, et etc. So we've talked about this fairly recently. So nothing all that surprising there. And then same thing with images. So I'll just bring up this image, which is a backbone controller for a phone. Same deal, you can kind of click around and get know. This is looking for texture I'm the only one is the image actually, but you can find out information about the image etc. So very similar. So there are other co-pilot plus pc features that probably aren't worth going into. You know, co-creator and paint, for example, already available, I believe, in stable at this time on snapdragon, is coming to amd and that's great. Restyle Image and Image Creator in the Photos app those are actually pretty useful. Also coming to AMD and Intel Co-Pilot Plus PCs.

One of the features that we I couldn't show you last time when we talked about this is semantic search. So since that time, microsoft has enabled this. It doesn't work great in my experience, but this is another Copilot plus PC feature and you can see it from this little purple, blue, pink, whatever thing. I'll enable the search bar that I usually hide too, actually, so you can see it down here as well, and the idea here is that you can search from start or from file explorer. I'm using the old file explorer, as we talked about in a previous episode, but instead of just doing an index-based search of local files, which is how this works by default With semantic search on a Copilot plus PC, from search, meaning search, here this interface from start.

When you start typing here at the top, or from File Explorer, you can search for local files, documents, et cetera, images, whatever, cloud files, right now limited to OneDrive. And so if you have OneDrive, as you would by default, like I do here, it's not just searching in the file system, it's actually going out to OneDrive. So they're going to allow for integration with third party services in the future but not yet and then also with settings. So, for example, you might remember when Copilot first launched in Windows 11, about a year and a half ago, you could ask it questions like hey, I would like to change to light mode, right? So if I type in something like light mode here, it should bring me to that setting and then when you click on it, it goes to that setting. I could switch to light mode, which I won't do, but you get the idea there.

And then the way that semantic search is supposed to work for documents and this one will be kind of interesting, so I'll type in something like DuckDuckGo. Actually, let me just type in Duck. So this is looking across all of the stuff I have. Now I just wanted to look at documents. Now this one's easy because DuckDuckGo is in the file name and of course, these things here are files. These aren't actually on my local computer. They're linked to on my local computer. Those are in OneDrive, so that's actually finding those things out in the cloud as well. But the idea is that you could search for things that might be in a document. So maybe you have a PowerPoint presentation with a particular object or a color or whatever it is and you could look for that thing and it would find it. That way it doesn't have to just be in the file name, although the thing I'm looking for is in the file name.

The settings search is kind of an interesting take on this. It's natural language. The semantic bit refers just to its AI-powered running off the MPU in a Copilot plus PC, and it's not indexing technically, but it's looking for context inside of content. So in a way you could make the argument that it is creating some form of metadata and then searching across that. It's very similar to the way that recall works, right, except now it's working on something that maybe you weren't working on recently, wasn't on your screen. There's no screenshotting involved or anything like that, and this is something that will expand pretty dramatically over time. It's really cool, okay.

So most of that stuff we've at least talked about at one time or another. But now I want to talk about some things we haven't talked about, or in one case, we've talked about what we've never been able to see. So the first is kind of a secret. This is not coming out in May, but Microsoft is working on a new version of the start menu for Windows 11 that I think is going to address a lot of the complaints, so I have it enabled here. This is not available in the Insider program normally, at least not yet, but it will be soon.

And there are these different views now that you get. And so, first of all, it's bigger, so wider, taller. It takes up more of the space. You can change the view in real time. So instead of going into settings and changing how much I can see here, I can do it right here live. So we still have pinned, we still have recommended, but then there's this new section at the bottom.

The default is this category view, which to me looks like the iOS library view, if you're familiar with that system, and it's just a way to show all of the apps on the system. So they're bringing all apps forward to the screen. Before this was available off of a little more button up here at the top. And then there are these other views, so you can see the names in a grid, meaning you've got these letter-based things and you should be able to do this what we used to call semantic search but go through the list that way. So that's pretty cool. That's the one I use, and then just a name list and this is the old style all apps list. But I don't think this looks as good because the screen or the window here is so much bigger, so I leave it on this name grid. I think this is a good use to space and I've optionally turned on this most used thing. That's something that you can disable if you want. So that's pretty cool.

But the other thing you see here is this phone link bar here at the top. So this is something I've been talking about for a while. Microsoft has previewed it, released it, took it away, started rolling it out slowly, but now it is coming to stable and so I just got it fairly recently in the dev channel in the Insider program, which is what I'm running on this particular computer. But this gives you a front end to whatever the features that you get through PhoneLink with your phone. So in this case, messages, calls and photos, recent items, which in my case are a couple of text messages and then actually a group message, so they're all text messages. But this could be different things, right, and a little bit of a menu. Here we can access other parts, including just opening phone link, right. So this is the normal full screen or full window phone experience or phone link experience. It's the same basic things. There's a couple of additional things and if you have a Samsung phone or other high-end phone you might have additional features. But it's kind of cool that the core phone features here are available without running the app. I mean, it's running in the background, of course, but that's kind of a neat one.

A couple of more things. There's a feature called Copilot Vision that Microsoft announced fairly recently for Copilot. It's something you can access in the Copilot app that you can access. I can't it's not available here for me, but let me see what happens when I run the app. So I bring up Copilot. It's super big for some reason. Let me just bring that down a little bit there. Anyway, really really wants to manage the Windows. Okay, and the way you access this in the Copilot app, if it's available, is you click on this talk button and then it wants to talk to me, which is hilarious.

I don't have it yet, so I can't do this. I do have this reasoning button so you can do the think deeper or just do the normal AI search type functionality, but I don't have Copilot Vision. So part of the reason I don't is because I don't pay for it, right? Well, actually I do pay for it, so if you had a Copilot Plus or Pro subscription, you would get it. I actually do have a Microsoft 365 subscription, so I will get it here. I just don't have it yet.

The thing that they're changing is they're adding it to Copilot in Edge and they're adding it for everybody for free, and once again, I don't have it. So if I bring this thing up, you'll see that similar interface. It will eventually tell me it's listening. I guess I don't know. It doesn't matter, but I can't do that here. But what this does is it interacts with whatever is on the screen. So, for example, let me see I'll bring up a premium post. That isn't the thing about my health or whatever. We'll talk about smartphones here, so I can just use this feature here in Copilot to get a summary of this document. But with Copilot Vision, it actually just looks at what's on the screen and then provides you with feedback and you can talk to it. So I'm not going to do that here while I'm on the microphone, obviously, but this is sort of similar, but just in written form, and so obviously what they're doing here is transitioning Copilot over to this model where you can interact with it a little more naturally than just typing.

All right, so the final one, and this is a really cool one. So let me actually let me think about this. Let me bring Edge back. I need some text or something on screen, right? So let's bring this down so we can see some text here. So you're familiar with Snipping Tool. If you've been watching the show for a little while, you know that the Snipping Tool app has been updated repeatedly and it is being updated again. So this feature called Text Extractor has been available in Snipping Tool for six, seven, eight months, whatever. It's been there for a while.

It's cool OCR type function, right, but the idea, the way it's worked so far, is you take a screenshot or a screen snip or whatever they call it, and then you use this button to extract the text from the image and then you can copy to the clipboard, put it in Notepad, do whatever you want with it. This works with click to do and you don't have to take a screenshot. It will just do it with whatever is on screen at the time. So if you click this to start now, now I can select the text that I want. That's on screen. It does the little click to do AI animation thing and then I can select the text that I want. That's on screen. It does the little click to do AI animation thing and then I can copy the text and, again, just like the more convoluted way that this would work before, paste it into Notepad or whatever. That's cool. It's not doing it right now, but typically, once you do this one time, let me see what I get there. Oh, no, let me do it again. I'll do boo, boo, boo. Where's this thing? Yeah, usually it comes up with a click to do button in here, but maybe I can force it by. Yeah, it's not going to do it, but usually what happens here is you get a click to do from here and the reason that's oh there, it is Okay. So I had to do it three times to make that work.

Microsoft is trying to make click to do a little more discoverable. The ways you can initiate this normally are you can hold on the Windows key and click on screen. That will do that and you get that wavy AI effect. You can do Windows key plus Q, another way to bring that up. But how do you know that? How would someone know that? So I believe they are adding yes, so now there's an icon in start so you can go find it that way. But they're also adding it to places where you might actually want to do those actions right, and so if you're taking text from something that's on screen, as you would be doing here, it actually makes a little bit of sense that you might then want to do something with that text. So this is just another way to launch it. This interface is exactly the same as ClickToDo would be anywhere else in the system, but it's just a little front end to kind of give you a new way just to access that feature. So pretty cool.

So there's more. Most of what I showed you is Copilot plus PC specific, but phone link integration, copilot, vision for free and edge which is US only, I should say and the new snipping tool, integration, bringing text extractor to the front, and then the integration with click to do, are all new and all available to everybody. So if you have a Windows 11 PC, you run 24H2, you're just going all available to everybody. So if you have a Windows 11 PC, you run 24H2, you're just going to get this stuff. So that will start happening with Patch Tuesday in May.

So you should see it. You should see it by the time this recording is available. You should either see it or we'll soon see it. It's coming out that same week, so that was lucky timing on my part, but it's just good to know. Okay, thank you for watching. Hopefully you found this useful. We'll have a new episode of Hands on Windows every Thursday. You can find out more at twittv slash H-O-W. Thank you so much for watching. Thank you especially and always to our Club Twit members. If you're not a member, please think about joining. You can find out more about that at twittalktvcom. Thanks, see you next week.