Hands-On AI 7 Transcript
Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.
Mikah Sargent [00:00:00]:
When it comes to teaching people how to use AI, there's a sort of group of questions that I get a lot, and it all surrounds choice. Which model should I use for this specific task? Do I need to change the effort for this specific task? Is it enough to just crank everything up to max? Fair question. And it's something that I think bears digging into because of course you might wonder, should you just turn everything up? But the problem is higher settings can take a lot longer to get you the answer that you're looking for. And doing that can give you an answer that you wouldn't value more than an answer that you got a lot quicker. So in order to test this out, we will explore what each of these changes can mean and then also, as usual, test it out. A routine job and then a puzzle that we can check. I'm Micah Sargent and this is Hands-On AI.
TWiT.tv [00:00:56]:
Podcasts you love.
TWiT.tv [00:00:58]:
From people you trust.
TWiT.tv [00:00:59]:
This is TWiT.
Mikah Sargent [00:01:07]:
Welcome back to the show. Today we're going to be using Claude because Claude has several different models and several different efforts across those models. And Anthropic does a really good job of documenting the use cases, tools, et cetera, for each of these models. But this video will apply to any of the major AI offerings and give you some more insight into model choice and effort choice. So the first thing to understand is that in many an app, you will have a choice— I should say many an AI service— you'll have a choice between 2 dials, one being the model and the other being the effort. So let's kind of break that down as we find out how this works. So the model decides what does the work, which of Claude's models is actually doing the work. And the effort is, well, according to Anthropic, it controls how much thinking Claude applies.
Mikah Sargent [00:02:16]:
So higher effort is more thorough, but it takes longer and uses more of your usage. And the important thing to understand is that these are different terms among the different AI platforms, but they all boil down to, for the most part, the same basic understanding of how they work. You can, at any point in a conversation, change either the model or the effort, and when you do, it will then change the next response that comes in from the model. Doing so, you'll sometimes get a warning, but we will talk about that a little bit later. Just understand that changing either one of these options is going to have an effect on the conversation that you're having. Something else to be aware of is that many of the frontier models right now used to have the option to turn on or off thinking, and we'll talk more about thinking in a moment, but understand that now, uh, for most of the Sort of frontier models that you'll be using, be it Claude or ChatGPT or Gemini, you may or may not see an option to toggle on thinking because thinking is sort of built into what makes these models do what they can do. Effort is how you ask for more or ask for less. But understand that just because the effort changes, it doesn't mean that thinking goes away.
Mikah Sargent [00:03:53]:
Low doesn't mean In most cases, the thinking is off. Now, again, going back to kind of the overall understanding of how these different settings make a difference, you can kind of think about the model, the choice between Opus or Fable or Soul or a different model. That's the— it's who you hire. I don't want to say the person that you hire because obviously it's not a person, but we are doing a metaphor here. So it's the person that you hire. And then effort is how much work you're asking them to put into the answer. Okay? So it's like hiring someone to come up with a new logo for you. You hire a graphic designer and you pay one graphic designer very little and you say, And you give them very little information and you say, this is what I want.
Mikah Sargent [00:04:56]:
And what does that graphic designer do? They probably don't go out and do a bunch of research on your other competitors. They don't look at what's been done before. They don't take the time to understand the nuances of your brand. Then you hire that same person, but you pay them more money and You ask them for more and they go out and they do all of that work and they come back with a logo. The first logo may work. It may be that it's, you know, a clever use of the letters in your name. And so it's understandable, but it's not quite what you're after. Whereas the logo where you hired the same person, but you paid them more and you had them take more time, was able to create something even better.
Mikah Sargent [00:05:42]:
Of course, just like With humans, it can also be the case that sometimes the one that took a lot more thinking was not the one that you wanted in the end. And that's what we have to be aware of when it comes to testing out these models. So again, I am using Claude to do this, but it isn't just a Claude thing. ChatGPT has a model menu. It has a thinking level. It goes from, I think there's instant, there's pro, depending on your plan. Gemini has a model menu too. Its thinking levels, low, medium, high.
Mikah Sargent [00:06:11]:
And also Gemini does its best to pick for you so that you don't have to worry about the model so that you don't have to worry about the thinking level. And the names, again, almost like TV settings, the names change, but the idea behind it doesn't. So let's take a look at the models that we will be using today. The 4 models are Haiku, Sonnet, Opus, and Fable. Anthropic lines them up from— it describes it as quickest to most capable. But It doesn't mean that most capable means that it's best at the job. Haiku is great for quick work, high volume, quick jobs like sorting, like pulling out information, like routing. Sonnet is what Anthropic calls, quote, the best combination of speed and intelligence.
Mikah Sargent [00:07:04]:
Opus, which happens to be the model that I use the most, is built for long-running coding and knowledge work. Anthropic tells developers who are trying to use these tools, to start with Opus. And then Fable, which of course is the sort of magical model at the top, is the top of the line, demanding reasoning and long, many-step work. Now, its availability, of course, depends on your plan. And there is, for some people, a temporary launch offer going on right now as we record the show. So if you are tuning into this around November of 2026, then you might be able to check it out. But you'll need to check what your account includes specifically before choosing it. As far as Anthropic's developer guides, it says that tuning effort, again, not model, not who you hire, but for how long you hire them and how much you hire them for.
Mikah Sargent [00:07:58]:
It's the Anthropic itself says tuning effort is, quote, often a better lever than switching models. I wanna repeat that 'cause I want you to hold onto that. Anthropic's own developer guide says Tuning effort is often a better lever than switching models. So you may not need to switch models. You may just want to switch between effort levels. So we've got the models, the different people, so to speak, you can hire: Haiku, Sonnet, Opus, and Fable. And you can also think of these as people who you're hiring at different levels of understanding. And so Fable is, you know, the, the, the the deep expert who maybe is retired now and just has a lot of knowledge and some quirkiness over life that's been built in.
Mikah Sargent [00:08:51]:
Opus is the person in their prime. Sonnet is the person that has all of that fresh knowledge and energy that they just got after learning their skills. And Haiku, very much a beginner, but very good at doing Rote work. If it's all written out for them, by golly, Haiku can take care of it for you. Now, moving on to effort levels. There are 5 of them for your, for Claude, and that again changes depending on your account subscription and the tools that you use. So just bear that in mind. But the 5 levels are low, medium, high, extra high, and max by default.
Mikah Sargent [00:09:37]:
Okay. And understand too that low and medium work well for routine tasks. And doing that will also, because usage is a factor, spread out your usage and let it go, you know, further than you otherwise would. High, according to Anthropic, is the best overall balance of quality and speed. I usually, if you're curious, have my Opus 5.5 set to medium or high. Extra high is for long-running and coding and agent tasks. There's occasionally times when I'm doing coding work that I'll turn mine up to extra high. I don't remember the last time I regularly used max.
Mikah Sargent [00:10:22]:
I use max as just for testing. And so it's not something that is a regular part of my work, but max, it's the most thorough. It's got the deepest reasoning. Of course, that also means the longest waits and also means that it's going to use your usage faster. And this is the other thing. Don't consider effort to be a matter of accuracy because more thorough isn't the same as a guaranteed correct answer. Those aren't the same thing. Also, I should note that each model from, from Anthropic's Claude has a recommended level marked default, and it does differ by model.
Mikah Sargent [00:11:10]:
So currently, if we head over to macOS, I can see that for Opus 5.5, medium is the default and recommended value. Whereas for Sonnet, if we switch it to Sonnet, Also set to medium. Most of these are probably going to be medium for each of the tasks. And you'll also note that Anthropic makes it clear that max uses, as it says, 8 times or more usage than medium. So it does give you a warning of, hey, this is something that will be using up a lot of your usage credits. As for Anthropic's advice, the company says, hey, for everyday tasks, the default work— the defaults work well. Moving along, this is where we get into the explanation of everything, and this is the part that I always have fun. We gotta get behind the dials, right? Before we test anything, let's figure out what each of these actually is meant to do.
Mikah Sargent [00:12:15]:
What do the dials actually change? Remember that back in episode 2, we saw how a model writes out its responses. And in doing so, we talked about tokens, right? A model builds a reply a token at a time, and a token may or may not be a full word. It may be parts of a word or symbols or whatever. And then it uses, in order to write out its reply a token at a time, it uses the conversation so far. It scores the possible next pieces of text, it picks one, and then it does it again, over and over and over again, with slight adjustments based on temperature, the way that the dice are rolling. Now, one way that a reasoning model does more work is by generating extra sort of internal tokens before it gives you an answer. So while you are seeing the final answer or somewhere close to the final answer in your response. A lot of these models are doing a lot of that thinking behind the scenes.
Mikah Sargent [00:13:23]:
If you watched episode 2, you will remember that the one model kept saying things over and over and over again that were a little confusing, a little complex. Part of that has to do with the way that the model makes its choices. That even might have been episode 1. where we ran the local model. And so that is part of this more work, part of this effort that goes into it. Let's talk about these dials a little bit closer. That is actually what thinking refers to. Thinking and reasoning, it's internal scratch work.
Mikah Sargent [00:14:03]:
It's a little bit like us thinking inside of our brains, right? It's, it's the mental math of it all. The model can restate What's being asked. It can try different approaches. It can check intermediate results, and then it can also drop paths that don't hold up and then use that work in its answer. That doesn't mean, unfortunately, that it's not a promise that every answer gets every check. Okay? Again, this is all still a tool that is running To give the best possible answer that it can based on what has been said thus far, what has been generated thus far. But being able to do this internal work, right, of kind of thinking about— thinking, so to speak, here we go— thinking or reasoning about what is actually being asked and following different paths to their natural conclusion helps it to figure it out. So You probably are wondering, why is it that writing it out helps the model do this? Well, each— once again, this goes back to the token idea.
Mikah Sargent [00:15:16]:
Each step that the model does in doing this writing becomes something that the model can read back and then build on afterwards. So think of it like doing long division on a piece of paper. Instead of just doing that division in your head. In 2022, Google researchers showed that prompting large models to write out intermediate steps improved them on math, on common sense, and on logic problems. So in modern reasoning models, that work happens internally, and then you don't have to say, yes, please do the thinking thing. So Writing out its thoughts, its thoughts, so to speak, following those token paths over and over and over again can lead to a better answer. So remember in the first episode that we did when the model that was on my laptop spent, what was it, 89, 90 seconds almost thinking about a torque wrench? I remember that. That was an example of this extra work taking place.
Mikah Sargent [00:16:25]:
Before the answer. The thinking that you see, by the way, can vary. It can be a summary of the actual thinking that's going on behind the scenes. Some of the work stays hidden. And this is the thing that people might not realize. It's not free. Thinking isn't free. For developers who are paying by the token, hidden thinking still counts toward the bill as well.
Mikah Sargent [00:16:50]:
In Claude's app, then you have to understand on the website or in the app, higher effort can use your allowance faster. All of that thinking going on behind the scenes, it's all token usage. It's all making usage of your tokens. All right, let's keep on, keep it on with the dials, right? We've talked about thinking and internal, uh, as internal scratch work. What about the model? Well, switching models gives you a different trained system, which Might be able to handle a difficult task better. So it's important to understand that when it comes to choosing a model, it's a little bit difficult and a little bit nebulous for these closed models. Model size is only one ingredient in capability. It's not just the size of the model that makes a difference.
Mikah Sargent [00:17:40]:
Training matters, design matters. So Anthropic's comparison between the different models unfortunately doesn't give us parameter counts. So we don't really know for sure how those models differ on that in that way. And instead, we have to look at what Anthropic suggests for each of these models, how it talks about each of these models, and I think more importantly, how it prices each of these models to understand. So that's a little bit more about thinking. That toggle that gets turned on or off in earlier models. Now we get to effort because thinking and effort have kind of become combined in modern models and they need to be understood separately. Effort guides how much work the model puts into a specific request.
Mikah Sargent [00:18:43]:
So higher effort encourages more internal work. And if the model can search or use tools, then doing so with a higher effort means that it might call on those tools a lot more often. At lower effort, then simple requests may need little or no scratch work. And so that's why you would want to use a lower effort. And of course, more effort and more thinking doesn't necessarily mean that the answer itself is going to be longer. So if you thought that cranking up effort for a question would result in, you know, 6 paragraphs, and so you've chosen less effort so that you can keep the response down, those 2 are not the same thing. Lastly, let's talk about trade-off, because yes, you may think at first, oh yeah, let's crank that bad boy up and see what comes out of it. But the trade-off is that extra work can mean more time.
Mikah Sargent [00:19:42]:
that is yours that you could be doing other things with. And of course, more usage. But it's a balance because turning it down has a cost too. In fact, when it comes to Haiku, the least powerful model from Anthropic, Anthropic actually has a warning. It says Haiku, that model set at low effort, May skip searches or checks in long agent tasks. So when Anthropic or OpenAI or Google or any of these companies talk about long-running tasks specific to certain models, that's something to bear in mind— how you may end up having issues with models at low effort. Let's kind of— I know there's a lot there. Let's kind of gel this together.
Mikah Sargent [00:20:40]:
The short version of this is that more thinking, right, doesn't retrain the model. Sometimes people go, okay, if I crank up the effort, if I turn on thinking, then what it's doing is kind of like figuring it out in the moment, and it's training itself on how to do this, that, or the— No, no, no. No, no, no. More thinking doesn't retrain the model, but it can help it work out something new. It can help it use information from a document or a search that it may not have been able to do if the effort was set lower. That said, when it comes to figuring out what you should be doing, bear in mind what we've talked about so far, uh, sycophancy and the, uh, the guides I gave you there. If a fact is missing, then give it a reliable source. If it's still struggling with the task after you give it those reliable sources, that's maybe where effort should be tuned or model should be tuned.
Mikah Sargent [00:21:45]:
Now let's put this thing to the test. We are going to try out some different simple tests and then we will see how it goes. So, an easy job, which is just as simple as making a text sound friendlier. That is the first test that we're going to try here. So what is the, the sort of judging criteria? Well, before I see anything, I haven't seen anything yet, I'm going to ask, does it keep the meaning? Is it brief? Does it sound friendlier to me? Does it add anything the original didn't say? Okay, those are the kind of criteria that we're basing this off of. What is the test, the prompt? The prompt is, make this text sound friendlier and keep it short. The actual text is, can't make it tonight, something came up, talk later. First and foremost, we're going to try running this Haiku on low, and then we're gonna come back with a higher-powered model.
Mikah Sargent [00:23:00]:
So let's head over to macOS and take a look. All right, as promised, first of all, I'm in incognito mode, and I'm going to switch it to Haiku. I'm going to switch the effort level to low. Then I'm going to put in the prompt. Make this text sound friendlier and keep it short. Can't make it tonight. Something came up. Talk later.
Mikah Sargent [00:23:30]:
I'll hit return to start this. And here's what it said in response. Here's a friendlier version. Can't make it tonight, unfortunately. Something came up! Let's catch up soon. Okay. Yeah. Also, did you see how quickly it gave me that response? Incredibly quickly, I was able to get that response back.
Mikah Sargent [00:23:57]:
But I think it does sound a little bit friendlier. Can't make it tonight, unfortunately. Something came up! Let's catch up soon. There's a little bit more friendliness in there. We like that. All right, now we're going to do the stronger model. So we will test that out. All right, first and foremost, we're going to turn Opus up.
Mikah Sargent [00:24:23]:
And the reason I'm using Opus instead of Fable is that it is the model that many people are still going to be able to make use of, whereas Fable is one that you might not be able to. And so let's go with Opus 5.5 at max. And bear in mind, it says 4x or more usage than medium. And we'll do the same thing here. Make this text sound friendlier and keep it short. Can't make it tonight. Something came up. Talk later.
Mikah Sargent [00:24:58]:
Now, first and foremost, notice that it's already taking a longer time to respond to me. Right now, it's checking the tool description for the drafting approach. Choosing a drafting widget for the message. We're already at 13 seconds. Meanwhile, Haiku at low already gave us a response that was friendlier and was kept short. Now we're at 24 seconds, 25, finally drafting the message at 27 seconds. And here's what it has to say back. This is first and foremost pretty cool.
Mikah Sargent [00:25:31]:
It gave me 2 different options. Hey, exclamation point, so sorry, comma, but I can't make it tonight. Something came up, talk soon. It also uses my conventions for how I write, including the EM dash set off by 2 spaces on either side. And then offer a rain check is B. Hey, so sorry, but I can't make it tonight. Something came up. Rain check? I'll text you soon.
Mikah Sargent [00:25:56]:
It says below, if you already know when you're free next, naming a day in the rain check version makes it feel even warmer. It also gives me the option to copy this message or open it directly in the Messages app. So I'm curious to hear which of those responses you preferred, because for me, I would probably try to find a model that was somewhere in between, a model and an effort level that was somewhere in between. Haiku was able to give us that response that was friendlier in no time flat, while the other one took a lot longer, gave me 2 options, which adds more cognitive load, and then also costs a lot more money. Now, when it comes to, well, money usage, which becomes money, something to understand is that those dollar figures that you might see from AI companies, they come from developer tests. And so it doesn't quite tell us what these replies use in the app. We're not able to have those exact values. So in testing out these answers, it is a little bit subjective.
Mikah Sargent [00:27:02]:
But we're judging the answers, we're measuring the weight, and then getting an idea of how valuable it can be when you have more usage versus less. I want to do something that's a little bit more complicated that will give us a better understanding of how these 2 models might not be able to compare and where you might want a stronger effort or a stronger model in general. And in order to test that, not a routine writing task, let's do something that has a checkable answer. And we're going to do it in a complicated puzzle. First and foremost, we've got 5 friends: Ava, Ben, Cara, Dev, and Eli. And they sit in seats 1 through 5 at a concert that are numbered left to right. Each Brought a different snack. Popcorn, pretzels, nachos, candy, fruit.
Mikah Sargent [00:28:04]:
We know that Cara sits in an end seat. We know that Ava doesn't sit next to Cara. We know that Dev sits somewhere to the left of Eli. We know that Dev isn't in seat 1. We know the person in seat 3 brought fruit. We know the person immediately to the right of Ben brought nachos. We know that Eli brought candy. We know that the person with popcorn sits next to the person with fruit.
Mikah Sargent [00:28:32]:
We know that Ava didn't bring pretzels, and what we're going to ask is that the model give the seat order and each person's snack. Now I checked this by brute force, a second reviewer checked it by brute force, and there's only one seating That fits every clue that we can test this against. If there's some other response out there for you AI users who have access to powerful models, let me know. But based on what I understand and how I brute-forced it, these should be the only— this should be the only right answer, is, you know, the one solution. So I'm not going to reveal Just yet what the right answer is. We're going to do this with Haiku, and we're going to do this, uh, we're going to do the Haiku at low, and we're going to do it with Haiku at high for the same puzzle, and we'll see how long it takes. I also want to make something else clear. In doing these tests, I'm not, just like a research scientist should do, I'm not going into it hoping to prove any kind of point.
Mikah Sargent [00:29:48]:
I don't know what the outcome of these tests, these next tests that we're going to do, is going to be. We're going to see that live together. So there's every chance that the low effort for haiku 5-5 ends up being the one that gets it right, and the high effort ends up getting it wrong because of it doing too much or because of the— the tool calls that it's able to do getting in the way. There could be any number of results for this. So let's bear that in mind as we take a look at macOS. All right, here we are on macOS. We're going to start with low first, and here is the prompt. Now I'm going to hit send and we'll see how long it takes.
Mikah Sargent [00:30:36]:
Mulling, narrowing down where each snack and person can sit. Testing seating options against the snack constraints, ruling out settings in the second case. It took about 9 seconds to do so. And here is the— well, no, you know what? We're gonna do max and then we're going to see— or excuse me, high. And then we're going to see how long it takes. Now we'll hit enter. So that was, what was it? 9 seconds. At low effort.
Mikah Sargent [00:31:12]:
We'll see how long this takes. 8. So 8 seconds here for this version, 9 seconds for this version, and the correct answer is as follows: in seat 1 is Kara, and she's eating pretzels. Looks like low and high both got pretzels. In seat 2 is Dev. He's eating popcorn. Beautiful. And they both got that correct.
Mikah Sargent [00:31:50]:
In seat 3 is Ben eating fruit. That is the correct in both. 4 is Ava eating nachos, and 5 is Eli eating candy, and they both got that correct. So, One of them, the low version, was just a little bit slower to get us to our answer, which is interesting because again, less effort, but it was lower or it took, it took longer. Whereas the second model got there faster. And it appears that it's hard to tell what all it did in the background. But let's see, 1, 2, 3, 4. Sort of thinking processes that we can see.
Mikah Sargent [00:32:37]:
And we've got 4 here as well: setting up the logic puzzle's constraints, setting up a seating logic puzzle, narrowing down the clues on seat placement, narrowing down, testing seat— seating options, testing seating cases, and ruling out seatings in the second case, checking the seat 5 case against the clues. So It, in both of these cases where there's an actual answer, right? It has one right answer. They were both able to get that answer correctly. Yay. We love to see that, especially when it comes to models that we're trying to use for an objective answer. One took a little bit less time, one took a little bit more time, but it's nice that Haiku, Anthropic's Sort of least expensive model is the one, the one metric we can for sure go on, uh, is able to answer it in both cases. Does that mean that you should test it or that you should, uh, expect that it's going to get every answer correct? I think you can answer that by now, having watched the show for as long as you have. All right, moving along.
Mikah Sargent [00:33:47]:
The next question that's part of this, which model, which, um, effort Why not max out everything? If both dials are turned all the way up, then won't you get the best possible response? Not necessarily. And it also depends because first and foremost, it's probably going to take longer. No, it didn't take longer in the case of these— of haiku at 2 different effort levels. It was actually faster in the second one. But when it comes to real requests, not these simple requests that we're doing for this, but real requests, you saw earlier when it was just about asking for text, you saw the difference between the two and how much longer that took. More thinking before the answer takes time. And Anthropic says, hey, expect longer waits at max. And the other models, the other companies say the same thing about their models.
Mikah Sargent [00:34:53]:
Also, it often can use more of your allowance. Most Claude plans have a session limit that resets every 5 hours. They also have weekly limits. What you use, of course, depends on the model. It depends on the effort. It depends on how long and complex the chat is. It depends on higher effort, uh, or, and so therefore higher effort is going to use more and it's going to use more More the more you go. That's why compacting has become part of the process.
Mikah Sargent [00:35:23]:
But we'll talk about that later. I don't— I'm getting a little into the weeds there. Google, of course, says the same about Gemini. More advanced models and higher thinking levels use more of your usage. Also, the gains can be rather small. I mean, we saw that when we used Haiku at its low effort and then its high effort. They both got the right answer. So paying more for something that did not give us a better answer, since there was only the right answer, uh, was not a factor.
Mikah Sargent [00:35:56]:
There was one company that was able to do some developer tests, and so they're priced in dollars, which means that it's not usage case, uh, it's not usage based, but they did this test with Anthropic. Okay. And these were some coding tasks. Anthropic found that Opus, the model Opus, At medium effort, solved about 93% of the coding tasks provided to it. At high effort, it solved about 95%, 2% gain, and then at extra high, 97%, another 2% gain. You think that's— I mean, I would think that that's pretty good, but here's the problem. Each attempt cost about 2.5 times as much Upon doing so. So just that slight bump in effort, slight bump in accuracy due to effort only gave 2% better results.
Mikah Sargent [00:36:53]:
And it did so at a much higher cost. And lastly, and I used to see this a lot, particularly when it came to, to Fable, extra reasoning Can backfire. Extra reasoning, it's like extra seasoning. It can backfire. In a 2025 study, researchers working with Anthropic and university collaborators, they built tasks that exposed cases where longer reasoning went with worse answers. Now, older Claude models are more affected by this, and a lot of the newer models are being trained to deal with this, but older Claude models would get more distracted by irrelevant information, and OpenAI's reasoning models fixated on how a problem was framed. Of course, that doesn't tell us exactly how these newer models work or what we need to do to get these newer models working, but the point is, yes, extra reasoning can backfire and give us results that we don't want. On November 9th, as we record this on Tuesday, November 10th, on November 9th, developer Simon Wilson— sorry, Willison— reported trying Fable through the developer API and did so at all 5 effort levels, asking for code that simply draws a pelican riding a bicycle.
Mikah Sargent [00:38:22]:
And Simon said that his favorite came from extra high. Unfortunately, at max, he reported that it used the 128,000-token output allowance thinking— it used the full allowance just thinking, and it didn't produce a drawing. He had seen the same with Opus in September. So again, just because it can think so much doesn't mean that it's giving us what we're after. In other words, the top setting for a model, it isn't going to— it's not equivalent to saying, give me the best answer. Instead, it's a button that says, do more work. And depending on the task, doing more work actually does help. But sometimes, A task, once it's done, is just done and needs to just be done.
Mikah Sargent [00:39:23]:
Now the part that I hope will be helpful to everyone, because this is the question that comes up so much. Now we understand how these, uh, like what choosing a model means, how it makes a difference, what thinking and effort do and how they make a difference. So how do we choose which model and effort? Here's how I'd pick whether you're using Claude or ChatGPT or Gemini or something else. First and foremost, start at the default. Give it a try at the default and then adjust it for your task. So Anthropic says the defaults work well for everyday tasks. If you find yourself doing a lot of everyday tasks, then start there. If you've got a routine task, well, then try something lighter than the medium effort or, uh, instead of Opus, try Sonnet.
Mikah Sargent [00:40:18]:
What kind of routine tasks am I talking about? Rewriting a text like we showed Earlier, a quick summary of something, a simple question. Those don't need a lot of intermediate decisions made that the model does in the background. So you may not need it. If the model already produces a useful rewrite or a summary, then extra scratch work has not much help. It doesn't provide much help in adding value. You can try a lighter setting and then keep it at that if the answer still meets your need. In Claude, that might be Haiku, it might be Sonnet, it may be low, it may be medium effort, but I've been pleasantly surprised at the capabilities of Sonnet when it comes to routine tasks. Google's developer guide says the same thing.
Mikah Sargent [00:41:08]:
You know, this is not just coming from me. Minimal thinking for fact retrieval or classification, default thinking for comparing concepts or creative reasoning, and maximum thinking for advanced coding, Math, or multi-step planning. Of course, that is just one company's developer advice, and Google's maximum isn't the same as Claude's max, nor is it the same as OpenAI's options, but it is something to bear in mind. When it comes to linked steps, that is your— conceptually, if steps are linked, Multi-step, go higher. Don't use low for this. A puzzle, a budget, debugging, planning a trip that has a whole bunch of constraints tied to it. For work that has several steps that are dependent on steps that come before it, I'm having to use— we're doing this in reverse, but for things, steps that have kind of requirements that come before, before the next answer can be made. More effort can help with that planning, with that checking.
Mikah Sargent [00:42:22]:
Anthropic says that's math, that's project planning, that's detailed document analysis, and that's multi-step technical problems. If you still find that your answer is shallow after that, first, go ahead and check your own work. Check whether your request was clear. Check whether it was complete. If you did turn effort down, try switching it back to the default first. Then if it's still struggling with the task or it's missing some of that nuance that you were looking for, then compare a more capable model and keep what actually improves the answer. That is what you have to do. This requires a little bit of work, but it's helpful because you're starting to narrow in on your most typical tasks and what they require.
Mikah Sargent [00:43:14]:
All right. Now, 2 more before you trust the answer. First and foremost, missing facts. More thinking doesn't retrain a model. If it needs a fact, give it a reliable source or let it use search tools. Most of these models that you'll be using have that ability. And so if you are finding that it's not giving you the answer that you're looking for, it might be missing facts. We talked about that in a previous episode.
Mikah Sargent [00:43:45]:
Also, if mistakes matter, which I would argue is something that's very common in the work that you'll be doing, understand that a model's internal checking doesn't necessarily mean the same as checking against what you know to be true. Right? Because internal checking may be checking that it's doing itself for different responses and behaviors. It's not the same thing as checking the answer against evidence. Don't think of effort as, again, more likelihood that the answer is correct. A higher setting doesn't check it for you. There's not a setting that will reliably always check it for you. So what about max? When do we crank it up to max? Use max when it helps. I don't have too many instances where max has been something that I needed, right? I talked before about using it more as a test.
Mikah Sargent [00:44:51]:
Use it though when it helps long demanding work and when a lower setting actually did disappoint you. Regardless of anything though, going back to prompt engineering, whatever you pick, just be specific because clear prompts can almost be a cheat code when it comes to using these less powerful models. Clear prompts with plenty of information helps any chatbot use its effort properly and use it well. And then it doesn't have to expend what effort it has to do that context understanding for you. Let's round things out here. Wrapping it up, use the setting that gets the job done. Turn it up when the extra work helps. Consider turning it down as a test to see if you don't need the extra work.
Mikah Sargent [00:45:45]:
And if you'd like some homework, here's mine for you. Try one effort level lower on a routine task. Breathe. You can do it. Breathe. If the answer still meets your needs, consider keeping that setting. I did it today. I turned a couple of my tasks down an effort level and the responses still came out as I needed because most of them were checking spreadsheet cells or looking at something that already existed and didn't need a lot of dependent steps or analysis involved.
Mikah Sargent [00:46:16]:
Of course, you can always send your AI questions to handsonaiai@twit.tv and you just might hear yours answered. on the show. Hit subscribe or follow wherever you're watching or listening, or head to twit.tv/HOAI. If you want it ad-free, well, that's Club Twit, twit.tv/ClubTwit. I'm Micah Sargent. Don't just wonder about AI, get your hands on it. I'll see you next time.
