Greg Baugues: AI You’ve Been Doing It All Wrong

• Slides
• Transcript
• About the Speaker
• Join the BoS community

AI will take over the world according to some but take even a cursory look at, eg AI generated content, and you could be forgiven for concluding it’s filling the world with nonsense.

In this entertaining, insightful, and at times magical talk, Greg explains why understanding how AI works, its  strengths and limitations will help you harness its potential for your own benefit. Greg shares a series of  live and interactive demonstrations that offer you actionable, useful and powerful approaches that incorporate  AI tools in your day-to-day work to enhance your brain power and creativity.

Slides

Find out more about BoS

Get details about our next conference, subscribe to our newsletter, and watch more of the great BoS Talks you hear so much about.

Transcript

Hello everyone. It’s really nice to be back here. I’ve been thinking a lot about this quote recently, any sufficiently advanced technology is indistinguishable from magic. For the last two years, if I haven’t been hanging out with my family, I’ve been working on one of two things, either AI or magic. And so if you all will indulge me a little bit, I want to try an experiment that combines the two. I’ve never done this before, especially in front of this many people, but there’s such a friendly audience, a fun place to try new things. Could I have two volunteers who have chatGPT installed on their phone? If you use it already.

Yeah, please come on up. Could you open up chatGPT? Great. And could you in a new chat here? Could you ask chatGPT to pick a number between one and 50.

Awesome. You got it? All right. So what number did you get? 27. Fantastic. All right, here’s what I want you to do. Could you take your hands, so I have here books I borrowed from the Book Share. And so could you take your hands and just put them on two of these books? Here, two books. Great. And if you want to change your mind, go ahead and do so now. You’re good? Fantastic. So we’ll get rid of these two. Can you do it again? Put your hand on two books. These two, you want to keep these two? All right, fantastic. And then pick one of these two. Which one would you like for the end, which ones they? Either this, this one. All right, very good. All right, so you picked the entrepreneurs playbook from these from the folks at Quiet Light.

All right, and you picked a number here chatGPT. Could you open the book to that page. What’s wrong? It’s gone. It’s been ripped out. Actually, could you reach into this bag right over here, and could you unzip that? And could we check and see, does that page fit the book? All right, there we go. Thank you both very much.

So one of my favorite magicians is this man named Asi Wind. He’s based in New York, and Asi, if you may have seen him on Fool Us, he did this whole bit. So he has one quote he says, magicians don’t keep secrets from the audience. They keep secrets for the audience. Often with magic, once you know how it’s done, it’s just a trick, and it kind of loses the magic. But he also, when he performed on Fool Us, his whole routine was based on the premise that sometimes the method is actually more impressive than the effect. And so his routine on Fool Us, what he kind of described the method, because sometimes it’s actually for the audience, more gratifying to see how it’s done. So if it’s cool with you all, I would like to show you all how that one works. Is that okay? All right, fantastic.

So let’s go here onto a browser and let me see if I can make this bigger. There we go. All right, so I’m actually going to open up an incognito browser here. I’m going to go to chat.com and chat.com gets you to chatGPT. Funny story, Dharmesh Shah bought chat.com and sold it to chatGPT. He’s a long time BoS attendee and speaker, so I’m here in an incog, this is a fresh window. Those of you who are familiar with chatGPT, I could perhaps set like a system message or something. Obviously, over here, we’re using someone else’s phone.

So I’m going to ask chatGPT. I want to say, pick a number between one and 50. Oh, look, I got 27, that’s strange. Let’s ask it again. Let’s try again. Actually, let’s do this. Let’s go over to the API.

So I have a little bit of code here, and this code is how you interact with chatGPT via the API. I’m importing the helper library here. I’m creating a open AI client. I’m writing a prompt. My prompt is give me a number between one and 50, and then I make a call to the API using the GPT 4.0 model, which is the same one that I was just using there, I pass in the prompt, and then I print out the response.

So if I were to run this, go into my magic folder, and if I were to run this, pick a number, then you’ll see that, oh, man, we got something different. We got 28 this time. All right, so let’s try this multiple times. So I want to do what, and we’re going to get to this a little bit later, but I’m going to do a try a little bit of what the kids these days call vibe coding. Wrap this in a for loop and repeat these instructions 100 times. Oh my gosh, that was weird. Let’s try that one more time. Wrap this code block in a for loop, repeat it 100 times. There we go. You should promote super whisper. That was super whisper. We will chat about it here in just a little bit. All right, so you see that I have now wrapped this in a for loop, so I’m going to run this one more time and look, we get 27 27 27. Notice, actually, we get more variation in the text sometimes than we do in the number itself, right? So what’s going on here? Like, why? Why is this happening? And in order, I think understanding this effect is really crucial to understanding how to use LLM properly. And so let’s talk a little bit about why this is happening here.

Understanding How LLMs Work

So how do LLMs work? So LLMs sometimes pejoratively referred to as fancy autocomplete. Y’all are familiar with auto complete. Auto Complete completes the guesses what is the next word based on it was trained on a whole bunch of chats. I think auto complete on your phone uses mark off chains. And I don’t know why they talk about hallucinations. This seems pretty accurate to me. And so LLMs’ like fancy auto completing. People try to use that term to dismiss it. It’s actually a pretty good representation of how these things actually work.

By the way, this is how I ended up here. This is it like mark texts. We hop on a call, and here I am, fantastic. And so the difference is, LLMs are not guessing the next word. They’re guessing the next token. If you’ve been working with LLMs, you probably heard a bunch about tokens. What is a token? A token is a part of a word. So we have, for instance, here you can go to Open AI’s tokenizer, and you can see what the tokens are. So if you put in some text here from the BoS website, you can see here the highlighting of all the different tokens. So sometimes a token is a single character, such as a period, sometimes it’s the beginning of a word, sometimes it’s a full word. They trained neural networks on an incredibly large corpus of data, probably about 10 terabytes. You can conceptualize this as I was trying to actually chat with GPT to figure out, like, what’s a good mental model. They said 10 terabytes would probably be about every single book in university system.

So imagine every book in every library in Cambridge University, or another way that is probably just a good way to think about is everything that’s public on the internet. So like the transcripts of every YouTube video, everything that’s ever been posted on Reddit, on Hacker News, all the websites that are indexed by Google. Is not quite all of that, but you can imagine if you took all that data and you trained a neural network on it, and then this job is based on what it’s seen as if you give it some line of tokens to predict which token comes next. And it’s been incredible what we’ve been able to accomplish with LLMs based on this simple concept.

But what this also means, though, is that when you ask it to pick a number between one and 50. It is not doing what we are used to computers doing. So for instance, if I were to come over here, I have a Python script that I call really random. This is a Python script. This is four lines of code. And if I actually run this script. Then we actually get random numbers. It goes way faster. You can actually see numbers that are low and high. You’ll probably find in this list one and 50 of the boundaries. Humans don’t do that, though. Magicians actually know that if you ask a human to pick a number, often, that number is going to end in seven, because seven feels random, and if you pick between one and 50, you almost certainly are not going to get a single digit number. You’re almost certainly not going to get something on the high end either. And so what an LLM here is doing is based on all the text on the internet. It is emulating human behavior. And I think that this distinction is really, really important, because for folks who have spent their entire lives and careers working with computers. It’s easy to conceptualize an LLM as you’re just chatting with a computer, and it’s easy then to ascribe mental models of how this thing might work to working with the LLM. But it’s clearly not. It is something different, right? It is.

When to Use LLMs as the Right Tool

And so understanding what’s happening here will help you figure out how to use these better. And the way that I’ve been thinking about LLMs over the last two years is, when are these things the right tool for the job? We, my wife and I, recently became first time homeowners. So for the first time in my life, I own power tools, which is very exciting. I’m team yellow. I don’t know if team yellow is a thing here, that might just be a thing for us. But every, apparently, every homeowner has to decide which tool team they’re going to be on. So I got a bunch of DeWalt tools. And for a lot of the work for me over the last year of doing DIY stuff, is like, hey, when do I use this tool? Right?

So for instance, I needed to hang up shelves in my basement, and we have cinder block walls there, and I have a power drill. And that power drill is very powerful, like I squeeze it, it’s very satisfying, like I can feel there’s a lot of power in this. And I was trying to put a hole in the cinder block wall, and it would not work. And I was like, banging away on it. I was putting all my weight into it and everything, and I could not get a hole. I even had the right drill bit in there. Well, it, it turns out I needed a hammer drill, right? And so I have three different types of drills. I have the screwdriver, the power screwdriver, I have the driver. I don’t even know what these things are called, The Power Driver. And then I have the hammer drill that vibrates for putting stuff into cement. And it was really easy when I’m sitting there using the wrong tool for the job, to say, Oh, this thing is worthless. This thing doesn’t work at all.

And I think I see a lot of folks doing that with LLMs today. Like, I’m generally not super interested in the debates on the extremes. I generally feel like the folks who are saying LLMs are worthless, it’s just the next crypto hype phase, haven’t actually seriously used the tools. And I also feel like the folks who are saying LLMs are going to put all the developers out of business also haven’t seriously used the tools. I think the nuance is somewhere in the middle, in that there, these are like brand new power tools that have dropped in our laps over the last two years, and we are all now trying to experiment, to figure out what is the right shape of a job for which this is the right tool. So I’m going to pose some questions here. Some of these questions I have thoughts on. Some I don’t like, because I just we’re trying to figure this stuff out.

You know, the title of the talk you’re doing, AI wrong. I am too. Like this stuff is all new, and like, nobody’s figured all this stuff out. So throughout the talk, I’ll have some different questions here, more for like, conversation and discussion. But I think the main question for me has been, when are these the right tool for the job?

AI Demos and Real-World Applications

So for the rest of my talk here, what I want to do is show a number of demos, because I think there is so much hype in this space today that one of the best ways to combat that hype is to not talk about what might happen one day, but to show you, here’s what you’re capable of doing today with tools that you can go sign up for an account here today. And my inspiration for this also goes back to magic a little bit.

One of my favorite Magicians is David Blaine. And I had the opportunity to go to a show a year or two ago, and I got to see him perform. And David Blaine sort of made a name for himself. The way he puts it is, he likes to do things that are impossible that seem real, so like card tricks, right? And then he also likes to do things that are real but seem impossible. So, for instance, he has this trick where he puts an ice pick through his arm. And, you know, at the show, I can tell you the trick is, he just puts an ice pick through his arm. He has another trick where he holds his breath for 10 minutes. I was there in the audience, I can tell you the trick is, he holds his breath underwater for 10 minutes, right? And so for everything else here, there, these demos are real, but they are demos that two years ago would have felt like magic, like AI, to me, feels like sufficiently advanced technology. Some of these things that you can do with them today feel indistinguishable from magic.

Image Generation Demo

So let’s walk through a few of these, I’m going to start with design, because this goes all the way back to six days ago open. AI just announced that they released new image capabilities in chat GPT. So let’s go over to chat GPT and see what this is capable of. And I started a chat. Here I was, look, I was sitting in the room yesterday, and I noticed this poster right here, which who came up with a squirrel. I love the squirrel. It’s awesome, like so I was like, what are some different, variations on on the theme of the squirrel here? So I said, create a poster for the Business of Software conference. The top should say Business of Software. The bottom should say Europe 2025, the middle should be an emotive squirrel playing an electric guitar. So here was the first response that we got.

Now, if you’ve been playing with chatGPT the last couple years, you may have known that you’ve been able to create images with Dolly 3 and do images inside of chat GPT, but one of their downfalls was always lettering in the images. So this fixes that. Now you can do images that have letters on them. The other thing that you can do now is you can iterate. And so I said, this one didn’t go great, but I uploaded the BoS logo. I said, this is a good start, try putting this logo in there. It didn’t quite get that right. So I said, That’s not what I was looking for. Let’s just make the same poster. This one minimalistic, but in the style of heavy metal. So we came up with this, which is, yeah, pretty close, right? So I said, it’s great. Let’s make the text white.

And so what you can see here is that we’re actually able to maintain sort of the state between image to image, and that’s something that was difficult to do with the previous generations. And so now I said, remake it in the style of a Pixar movie. And we got this. And then I said, let’s go like a 1980s video game and and we came up with some pixel art like this. So I thought that was a good start. Then I also, another thing that we can do now is it works pretty well with images of people these days. And so I started with a picture of Mark from a BoS about 18 months ago. I said, Hey, can you make them in the style of a Pixar character? I thought that was pretty great. Actually, it actually picked up on some of the lettering. What this game? I don’t know what the but I didn’t even notice this in the original photo. But it actually does say game here. I’m not sure what this word is. Okay, yeah, and it picked up. It thought it said game day, but in fact, it picked up on lettering and then made that lettering legible there.

So then, okay, if you’ve been on Twitter at any point in the last week you’ve seen the Studio Ghibli. I’m not actually sure if I’m pronouncing that right. This is why you’ve seen, like your whole timeline has been filled with these. And so we have that Mark. So then I said, Okay, let’s go back to the original image. I want Mark, but I want to be on a heavy metal so it came up with this, and then, all right, now we need some taglines. All right, so what are some I was like, do you know about the Business of Software conference? And I said, Yes, I do. It said, I’m familiar with Business of Software conference is well regarded event that brings together et cetera, et cetera, known for its high quality talks, small size, strong community vibe. I said, Okay, let’s pretend that there was a heavy metal album based on what you know, I need 10 album covers. And so our title, so it came up with a whole bunch of titles for us. Death by demo, cherna, get in stakeholder, pain. Pivot or perish. Refactor, the abyss. Scrum of the demand. Damned lead, genocide funded and furious. Infinite beta BoS, lords of retention. I like that one.

So I want funded and furious. So I told it to update the title. So we got this there, and I said, now I need a squirrel. The squirrel needs to be playing an electric guitar. Mark should also be present. I did hit some rate limits here, but then we I asked it to make Mark really, really shredded as well. So then we ended up with this. And then I said, Okay, now we just need more pyrotechnics. And this was our final result. Here was all right. So, so much fun.

So I have a friend. He’s a Broadway set designer, and about 18 months or so ago, he was our next door neighbor. We were hanging out and we were just like, I showed him some of this stuff at the time. We made this with mid journey and he created this. And he said, let’s create a set for Death of a Salesman. And inspired by what he said. Frank Lloyd Wright, I think he said it put out this. So this is an image generated, like, 18 months ago, and he looked at and he just laughed. He goes, Well, first off, look, I would never actually use this for a set like, as someone who’s experienced there’s a lot, this looks good to an outsider, but there’s a lot about this set that wouldn’t actually work in like a Broadway production. But I have, many times in my career, spent two weeks working to produce mock ups and prototypes so that I could have a conversation with a client, and this here in like five minutes, just save me two weeks of those rough sketches in those mock ups, right?

And so I think a big question of these tools is that I don’t have an answer for is, what does the role of blank look like in five years? And so designer is the obvious one to put in here. But I think anyone who does any form of creative work is going to be wrestling with the same thing, right? So, like, I don’t know that the designs we just created, you can necessarily copy paste into whatever design or websites or whatnot. But I do know that these tools are going to rapidly change the way that people do the work of creating stuff, and it’s also going to make it possible for people like me, who have absolutely no design ability whatsoever to iterate and ship and to bring ideas to folks like I truly think about this a lot in the same way of photography.

I picked up photography over the last couple years. It’s a huge hobby of mine. Now, I have never processed I’ve never shot film, I guess, when I was a kid, a little point and shoot, but I’ve never developed film. But because of photography, because of digital photography, I can rapidly iterate right like I can get better faster. I can experiment with different settings. I can deliver photos that are much better than I probably never would have gotten into it if it wasn’t for the technology that’s available today. And so I think it’s going to make these tools accessible and these processes accessible to a lot of folks who wouldn’t otherwise be able to do it, but it’s also going to have pretty profound impacts on the folks who do this professionally.

Translation and Dubbing Demo

All right, let’s show another example here, translation plus dubbing. I think an outstanding question is, what is the content game going to look like in five years? I know, like the first kind of use case everyone goes to with these tools is like, I’m going to use chat GPT to write blog posts. I think it’s pretty easy. You know, there’s issues with that. I think folks still can tell when it’s AI generated. I also just if you play this out five years to an internet where it is completely full of text that was generated by chat GPT, I don’t know how much value there’s going to be there. Especially when people are increasingly spending more and more time in chat GPT, as opposed to like Google search, right? And so I personally, like my gambit here is I’ve started investing more in YouTube and shooting YouTube videos. I think the bar is higher, and I think people are going to have a desire for human to human connection. And when you’re seeing someone speak on YouTube, you know that it’s a real human right, and I think we’re going to have more desire for that, and that’s a way to, kind of like, verify the authenticity of the content.

There was an interview just a couple days ago with Mark Zuckerberg, and this is Jimmy Donaldson. Mr. Beast hopped on the interview. They asked him this question of, what would you do if you were CEO of Facebook for the day? So I think we’ll have audio here.

Video 

I was CEO. Yeah, that was the question. Basically, that’s the question, um, I mean, one thing I I think really sucks is, like on YouTube, you can do different audio tracks. So 70% of my audience on YouTube, they don’t speak English. So I upload videos, and then you just upload a bunch of different files. And so if you click on my video in Mexico, it auto plays in Spanish and we get like, 20 million views of video in Spanish. We get 10 million views of video in Portuguese, etc, all around the world. And what’s brutal is, when I post those same videos on Facebook or any of your platforms, I can’t import the dubs over. So my viewership is always just infinitely lower, because people from Mexico and people from Brazil and things like that, they can’t watch it with dubs. So, I mean, I know the teams are working on that kind of stuff, but if I was CEO, that’s one of the first things I would fast track, because.

Greg Baugues 

All right, so 70% of his viewers don’t speak English, right? So I make content about coding. Here’s a video that I made a few months ago about this magic trick. I just showed you all a quick clip from it. Magic trick, you can do a chat GPT, and then I’m going to explain how it works, because I think the method behind this magic trick demonstrates something that’s really important to understand about large language models. And here is a site called elevenlabs. So you can go here, elevenlabs specializes in AI generated audio. And so if you were to go here and sign up for elevenlabs, specifically, they have a lot of great text to speech models where you can generate AI audio. They also have the ability to dub videos.

So I went up here and I created a dub and I uploaded the video, and I chose my language here that I wanted to translate this video. They have a whole bunch of different options. I chose that I wanted to translate my video to Portuguese and uploaded. This is like just a quick little clip here, but I uploaded it, and then this is the result, card, comma, function, playing thing there. So this model is sitting Washington, volpia. So I want to know, how is it? It’s bad, really. So this is, like, what I think is really fascinating about AI tour to tools is there’s a thing called the gel man amnesia effect. Have you all heard of this before? It was coined by Michael Crichton.

So the idea here is that you read the newspaper and they’re writing an article about something that you know about, and you’re like, oh my god, that’s so wrong. That’s like, they’re idiots, right? And then you turn the page and you read about a world event that’s happening you know away from you. And you’re like, oh, that must be accurate, right? And it’s really interesting watching how that works with AI. I generally find, I think everyone kind of finds this, though, when you’re demoing AI tools, your first reaction, if it’s a realm you know nothing about, so, like, poetry, right? Like I asked her to write a poem. I’m like, Oh, my God, that’s a great poem. Well, I’m not a poet, and anyone who’s a poet, reason is like, No, this is terrible.

And in the same way, like, if you ask it to write code, which I’m going to do here in just a little bit, it’s really easy to be like, Wow, that’s incredible. And you see these quick five minute demos, and you’re like, these are incredible. And then, and then you actually get in there, and if you’re an expert, and you see, you’re like, oh, there’s some issues here, right? And so I wanted to ask you about this, because it’s like, to me, I’m like, Oh, my God, this is amazing. I can do these quick translations, right? And I don’t actually know if this would work for Portuguese speakers, but I think there’s incredible like opportunity here in that, because both on audio and for text, translation is something that these tools do really well, because it’s next token prediction, right? And they’re trained on incredible corpuses, not just English, but across all languages. And these models tend to keep getting bigger. And so one of the questions, just like, what happens in a world where as these models get better, and as the translation and the voices get better, when every business has the ability to do inexpensive, on demand, translation, like does every business all of a sudden be able to address like a much more global audience? Does that start to become more table stakes?

AI Coding Agents and Vibe Coding

All right, coding agents is this one place where I feel like we have unquestionably found product market fit with llms and LLM agents? I am a developer. Let me give you a brief history of coding. From my perspective, my first computer was a trs 80. My parents bought this when I was six or seven years old. We had the cassette drive. We didn’t even have floppy drives. When we very first started, I started coding in basic. Did anyone else hear code in basic? Oh, my God. I thought this would be like my crowd here. This is fantastic. All right, so yeah, 10 print. Hello World. 20. Go to 10, right? This was as accessible as coding has ever been in my life. The way that I learned how to code was that 321, contact would have programs in basic on the back cover that you would copy, not copy and paste, just copy character by character, and have little 40 line games that you would play.

Then at some point, I moved on to Pascal, and it was an integrated IDE and this is how you learned Pascal, and this is if you needed to debug or look up syntax for a thing. This is how you did it. It did also have help up here that was actually surprisingly good, but you were relying on books right then around, let’s say, like 98 or so. 99 I got started getting involved in the web. I moved on to Dreamweaver. This was the first time I had sort of a visual abstraction on I didn’t have to entirely write the code. I could, like us, a different sort of interface, and I could bold something, and it would write the code for me. And the really amazing part about learning coding that happened at the same time is that the documentation moved online, and so now we had PHP docs online.

We also ended up with Google right around, like, I think around the same time, 98 or so it came out. So now you can start searching, Googling for your problems. Stack Overflow came out, I think about 2008 this was an incredible place to find answers to all of your problems. And, of course, GitHub as well. And I would say that for about, let’s say 2000 like 2000 until like 2022, or so, let’s say 20 years. The way in which I coded stayed roughly the same. I would Google a question, I would copy and paste the code into my IDE I would iterate on it till I got stuck. Then I would Google, you know, maybe I google the error message, and then I would go find the answer. Then I copy and paste that, and I’d bring that in. And that was kind of the way it all works for like, 20 years. Like that method for me didn’t really change. Weird thing happened to Stack Overflows traffic a couple years ago. You can see the mark here is November 30, 2022 and not saying that this was entirely caused by but certainly correlated with the release of chat GPT. Chat GPT was released on November 30, 2022 and from basically that moment on for me, at least the way that I coded completely changed always before, when I let me walk you sort of through, like the way that, the way that this transformation happened for me is I started by just going to chat GPT and asking questions.

So for instance, I might say something like write a write a game of life, implementation, implementation in Python, right? And so now I could ask very specific questions, and I wasn’t copying and pasting generic code from other folks. I was getting a question that was really tailored to what I was interested in. And then I but I would still copy and paste this into my IDE and I would run it. Then Microsoft came out with copilot. This added some ability to do tab completion. So it was basically fancy auto complete. It was guessing, you know, what code you needed. But then this tool called cursor, came out, which I’ve already been demoing a little bit. So cursor was sort of the first, it was sort of make this smaller. It was sort of the first AI, first IDE. So their big change they made in the beginning was that they just brought the chat right here into the IDE, and so now you didn’t have to tab back and forth between a window, because it’s easy to get distracted when you do that. I didn’t have to copy and paste anymore. But also this now, because it was operating inside the same app, had access to the context of my file system, and so it could automatically pull in my code base and use that instead of meaning to copy and paste and move stuff in.

Over time, cursor got better and better at automating and guessing what needed to happen. And now recently, they rolled out their agent mode, which will do a lot of the work for you. And so this leads, I’m going to skip a little bit to the punch line here and then come back and show you what this looks like about six weeks ago. Andre karpathy. If you all are unfamiliar, he worked on AI at, I think Tesla, and then he was at open AI for quite a while. This guy is about as real and legit of a developer as it gets. And he had this tweet said, there’s a new kind of coding, I call vibe coding, where you fully give into the vibes, embrace exponentials and forget that the code even exists. It’s possible, because the llms, such as cursor, composer with sonnet, are getting too good. I also just talked to composer now called agent with super whisper, so I barely even touch the keyboard. I asked for the dumbest things like decrease the padding on the sidebar by half because I’m too lazy to find it. I accept all always. I don’t read the diffs anymore. When I get error messages, I just copy, paste them in with no comment. Usually, that fixes it. The code grows beyond my usual comprehension. I’d have to really read through it for a while. Sometimes the LLMs can’t fix a bug, so I just work around it for a little bit. It’s not too bad for throwaway weekend projects, but it’s still quite amusing.

I’m building a project or web app. It’s not really coding. I just see stuff, say, stuff, run stuff, Copy, Paste stuff, and it mostly works all right. And so this, let’s take a little look at what vibe coding actually looks like in action. I do have a little file here, or a folder here called store, and I have a blank index, dot HTML. I have the BoS logo. I have a picture of the notebook, and I have a picture of T shirt I put, oh, you know, I forgot to mention I was asking chat GPT to come up with names for names of the squirrel. And it named him shred nut, which I thought was, I liked it. And so we made some images of the squirrel here, with some good pyrotechnics behind here. So here’s what I want to do. I’m going to bring this over. Let me minimize my screen. I want to create a new chat window. I am using this tool called Super whisper, which is a dictation device. What super whisper will do is take my voice, it will then transcribe it using one of those texts to or actually, I guess, audio to text, LMS, and then it will run it through one of the elements, like, clean it up a little bit. And so let’s just, we’re not going to build something that works all the way. I want to keep kind of my iteration small here, but please build an HTML file here in index dot HTML that will use the notebook and T shirt images in the directory and create a prototype for an online store.

We’ll hit send. And you can see here that I have this in agent mode. This currently is set to auto model. Really the right way to do this is to set it for Claude 3 sonnet. Claude is a model put out by anthropic most developers would agree that Claude 3.5 sonnet is the it was the best model is beaten out chatGPT 4, for coding by quite a lot. And then they just recently released an update of three seven sonnet, which improved the base coding, but also introduced reasoning. So let’s see what’s going on here.

So it looks like it is doing stuff all right. So you can see it’s updated my code. It says your store page is ready. You can open it in the browser, so I’m going to just click Accept. I’m not going to bother reading the code, although we’ll talk about that in a minute. I’m. And let’s see. I don’t know, we’ll see if this works. So I’m going to come over to the store directory, and there we go. We have, we have a store, all right. So that’s crazy. I’ve been, I’ve been writing code my whole life, and I just, I can’t I’ve had multiple times over the last two years when, like, stuff like this happened, and I just had to close my laptop and go for a walk and, like, let my brain reconfigure what’s happening there. Also, I’ll mention here that there’s a variety of tools. So I really like cursor. Cursor’s coding agent is they updated it just a few weeks ago. It’s gotten pretty good. There’s a competitor to cursor called windsurf. I think cursor, don’t quote me on this, but it’s like one of the fastest companies ever, to $100 million and ARR and the it has amongst developers.

I worked at Twilio, doing Developer Relations and dev tools for nine years. I’ve never seen a developer tool like just capture the mind share of developers like her has, and it’s the one of the very few AI power tools that I use every single day. I will show you though another one. This was recently launched by Anthropic. Anthropic release Claude code. And what I think is really interesting about the distinction here is that cursor is built inside of an IDE. So the expectation this is actually just a Visual Studio Code fork. So the expectation here is that you’re still going to be manipulating files and everything on your own a little bit, but you’re still operating in this idea that you have, like, line level, granular control of the code itself, curse or anthropic release cursor code. The way you start it up is like, here, I’ll do it, I’ll show you all the video, because it goes a little slow. But the way you start it up is you just run cloud code in a directory, and then it just starts going.

So you typically, this is a, I did this last night. This is probably about a 20 minute coding session. I added the video down. I sped it up. Edit it down to like about 90 seconds or so. But you’re going to start it up. It’s going to you’re going to initialize it, give it some rules for what you want to do. It’s going to work on a code base. This was a this directory was completely empty when cloud code started this session. And what I wanted to do was have this generate some stripe test data. So I basically have a blank Stripe account. I wanted to do some work against it, so I’m having it create sample data in my Stripe account for a BoS store. So how can we sell, like, merch here, and so you’ll see what’s happens is, like, I don’t actually have the ability to edit code here. It does stuff and then asks me, What do you want me to do next? And it’s really interesting how this works. It has sort of this, like earned permission model, where it asks, Hey, do you want me to do this? And you can say yes, or you can say yes, and don’t ask me again for this command in the future or for the rest of this session. And so what tends to happen is increasingly, as this goes on, you’re giving it more and more permission, and it’s going longer and longer without you needing to intervene.

And so here I said, you know, you need to just reload the environment. It’s really fun watching this thing like run code. It will often run code and then run into errors, and then read the error message and then update the code on its own. And so here it has. It’s written all the code. It did all this. You can see we had a successful transaction. And then I think I told it to Okay, cool. We had one successful transaction. And I went over to my Stripe account, and I was able to see the successful transaction. I’ll just say here, maybe it’s been 10 years since I’ve worked with the stripe API. I did not read the docs. All I did was copy paste my stripe API key into an environment variable, and told Claude code to do this stuff. And then once it got it working, I said, go ahead and commit this, and we’ll talk more about commit message, commit and version control here in just a minute.

But so I gave it full access, and now I’ve told it also like, as you continue to make changes, please commit often. And that was the thing. And so now if we go on to my Stripe account, you can see here, under transactions, here’s all the different transactions that it has created for me. And I had it. I had it create products. So we have the the pixel art sleeve tattoo, we have the code sleeve tattoo, we have the tote bag, the water bottle, the USB drive, the coffee mug. And then it created a whole bunch of sample transactions. And again, I have no idea. I mean, I could probably figure it out. Probably figure it out, but I have no idea how to use the stripe API. I didn’t write anything. I didn’t touch the docs. It like from a blank file did all this. And here you can actually see, if I look in the directory here for under stripe data, all of this code here is code that it generated. One without my intervention.

So this is just like again, as someone who’s been writing code my whole life, it’s just kind of mind blowing to me. Now a few caveats here. Vibe coding has just been like blowing up on the internet. Simon Willison, he’s from the UK. He’s probably the most trusted voice amongst developers when it comes to this AI stuff right now, he had a great blog post a couple days ago. Not all AI assisted programming is vibe coding, but he’s also, I think you’re seeing a ton of pushback from especially senior engineers, and they’re not wrong that, like vibe coding has is dangerous, right? Like there’s going to be security issues, the code often grows to be unmaintainable. I had one instance where I was having it work on a legacy code base, and the it, I asked it to use the anthropic gem in a Rails app, and it tried three times and couldn’t figure out how to do it. So then it just wrote its own implementation using HTTP to like interact with the API. And it was like, Okay, this technically works, which was mind blowing to me, that it just like, wrote his own gym, but also, like, I don’t want to maintain that. And so there I’ve seen it make test pass by commenting them out before. Like, I’m like, hey, that’s what I do. Like, so there’s a lot of challenges here. And I do think it’s really important to distinguish between AI assisted development and vibe coding, right? Like, I think vibe coding makes a ton of sense for weekend projects.

And if you go back to the original tweet again, which is super important, he goes, it’s not too bad for throwaway weekend projects, but still quite amusing, right? And so I think it’s really important that, as we talk about vibe coding, realize there’s a time and a place for it, but don’t dismiss all of AI assisted coding just because, you know, vibe coding has its downfalls. So question here, what could you build if code was a lot cheaper, right? And it’s just incredible, I’m seeing a lot of my friends who are developers. My friend Harper Reed, he was talking to the other day. He’s like, I don’t have anything left on my two code to do list. He wrote this great blog post about it, and he’s basically saying, I’m able to bang out stuff while watching TV. Now that he would have taken me days before.

AI-Assisted Coding Best Practices

Let’s talk about a few AI assisted coding best practices, since this is a Business of Software, and all of y’all are either writing code or employing people who write code, so we’ll talk about both here, and I’ll go through these quickly. So one just for coding.

So one, write tests. Definitely, definitely write tests. You probably want to write them by hand. You’re probably already writing tests. But there, I think, when you’re doing AI assisted coding, it is so much more important to have these. And again, you really want to spend time on the test. And I’ve seen some folks for some projects, and I just want to say everything here, the demos are always going to be on greenfield projects. And then the reality is, you all are probably working on much larger code bases, and that have big teams and whatnot, so there’s going to be a lot of nuance, and you’re going nuance, and you’re going to need to experiment and figure out what works best for your particular situation. But generally speaking, you’re going to want to write tests and really understand what’s happening in those tests.

I have personally asked these things to write tests for me, and I feel great because there’s, like, 100 tests, but it’s not hitting the edge cases. It’s going it’s making things pass the easy way. But I do think that if you are writing tests and you’re comfortable with your tests, and then those tests are passing, you can have a lot more confidence in the code these things are writing, and it makes it easier just to trust use cursor rules. So every IDE has these, but they’re most sorry. All the AI powered IDEs have these. This is what it looks like.

So there’s a bunch of repositories for these, but cursor has this concept of a cursor rules file. You can think of this as basically a system prompt. So whatever type of project you’re working on, you can find cursor rules out on the internet, or you can write your own for instance, if you have certain styling or certain conventions inside your company, you can adopt these cursor rules, and then the llms will, for the most part, follow those rules. As you go work small you want to do small files, small changes. So much of what we’re dealing with here are the context windows and the llms work better when you have smaller amounts of context in there. And so I’ve actually found that working in a language like working in a framework like rails, where the files tend to be smaller, where you have more separation of concerns, tends to work a little bit better, because it’s not trying to do too much in memory at the same time.

And then, of course, just like regular coding, you want to be very have an iterative approach here, and take on small changes, and then you can have these things commit right like so a lot of times, the problem is they’ll make a bunch of changes and they’ll screw stuff up, and it’s hard to, like, undo, just like, really commit often, and what I find myself doing these things is reverting far more often than I ever have in my life, because I would be so, like, remiss to throw away code that I’ve written and, like, you know, really, like, sweat it over. But here, like, code is so cheap.

And so I often will have this. This happened to me the other day. I was working on this project, and I did a whole bunch of changes, and like, I had this, like, 15 minute session with Cloud code, and then, like, it just, it just wasn’t working. I was banging my head against the wall trying to figure this out, and I just threw it all away and just went back to where I was at 15 minutes ago. And there’s just more, you know, you have more options for starting over when the cost of code is so much cheaper on talent. So you want to make sure you hire and train for these tools. Analogy I’ve been using recently, you know, we have this home. We renovate. Had some renovations for our kitchen. We had to interview contractors, if we had had a GC, come to us and say, you know, I am a true craftsman. I only use hand tools. I do not believe in power tools. I would have been like, I’m really glad you have a hobby. But, like, we’re looking for a professional who can get this job done using, like the modern day tools. And I think I understand, I so much understand the skepticism and resistance from developers.

I’m 45 years old myself, like I get it, and these tools absolutely have problems, but we are moving towards a world if we’re not already in it we’re using these tools is an integral part of software development, and the challenge is not I’m not saying everyone becomes vibe coders, but I think that the developers who refuse to use these tools are going to have a very hard time getting employed. And I think that you should strongly encourage your folks to use these tools. One way to do that is internal hack a thons. It’s much easier to experiment with these things when you are working on greenfield projects, when there’s low risk associated with them.

I used to run it’s called the Enterprise hackathon program at Twilio. And we’d go into enterprises, and we would basically do this as before, all the AI stuff, but we basically have the stakeholders, the execs, identify what are three or four big problems that you have. We bring those problems to developers. We give developers a day or two to hack on stuff. On greenfield projects, say like, let them surprise us with how they would solve the problems. And internal hackathons just been a really great way for developers to play around with new tools for you to figure out, to build prototypes, to figure out what works what doesn’t, and it’s a great way then, for folks to take the lessons that they learn the hackathon and bring it into your everyday environment. Also, these tools are great for software development onboarding.

I’ve been talking to a lot of folks who are using these in the enterprise, and one trend that noticed is that they say that it actually can relieve some of the burden from senior developers when onboarding a junior, because you now have a infinitely patient tutor who understands the code base. And so, for instance, I asked Claude code on one of my apps. I just started up say, Hey, what is this code base? And it says, here, here, what’s going on. This is what’s going on. And so now you can have junior developers asking questions of the AI that they may have been hesitant to ask of a senior developer because they didn’t want to appear stupid, or they might maybe just would have been afraid to, and then now they’re taking some of that burden off the senior developer, and perhaps the senior developer can spend more time writing code. So these can be really, really useful for onboarding new developers. So I think a question here for me is like, how is this different than other programming abstractions that we’ve seen before? Right? Like nobody in this room is writing assembly, and so what we’ve increasingly been doing over the course of the last 40 years is inventing more high level ways to interact with the code. And so is this just another level of abstraction?

Model Context Protocol (MCP)

Similarly, 10 years ago, if you wanted to build an E commerce site, you probably need to spend $150,000 and hire a custom web shop maybe 15 years ago, and now we have Shopify. You know, same deal, like when WordPress came out, it enabled so many more folks who didn’t know how to write PHP or HTML to publish to the web. And so is this going to be a similar trend? Is it going to change? Like, it’s not clear to me, but it does feel to me, like as a developer, that I’m just working on one level up of abstraction. I still have to define a lot of the architectural concepts, but I’m able to do so using natural language, as opposed to having to write code. All right, last thing, and then we’re going to wrap up here, MCP model, context, protocol. Has anyone seen this yet? Played with this? All right, a little bit. I just wanted to mention it briefly, because it’s been so popular. So popular. It’s a little bit more technical. This came from anthropic.

Anthropic introduced this post called Building Effective agents. They introduced a concept, if you’re building anything agentic, you should check out this post. They introduced the concept of an augmented LLM. So this is an LLM that has retrieval tools and memory. And it’s basically just an LLM that has additional capabilities. It’s more than just a chat with an LLM. And then anthropic also, probably about six months ago, introduced model context protocol, which they would basically say is like the interoperability layer for everything below here. So it’s you can think of it as APIs for LLMs as a standard, or maybe as like rest for LLMs, it’s a standard way to equip your LLMs with retrieval tools and memory.

So I have a quick demo here. There’s tons, like literally 1000s, of NCP servers available, but I use Claude desktop. Is anyone here using Claude as like their daily driver? Awesome. All right, few folks, I personally prefer it better these days. I’ve started up my my Claude client. You can see this little hammer right here. So this is, these are the NC piece servers that I have installed. And so I have installed two. I’ve installed a Postgres one and I’ve installed a stripe one. So just in the interest of time, we’ll just do the stripe one. So I’m going to say, tell Oh boy, I don’t need to yell, actually. Let’s do it this way. Tell me about the products that I have set up in stripe.

So the MCP server is basically a wrapper for the stripe API. Oh, and it had a problem there. We’ll see if it tries again. OK, so that’s cool, all right, so this is actually a fun demo, right? So look, it tried the first time with a limit of 100 it ran into an error. It tried again with 10. And so it got him all right. So it was able to write custom code, basically to or use the tools available to it to interact with the Stripe account to pull out the products that we had in the Stripe account from the dashboard. And so now I am able I basically have an LM acting as a wrapper over the stripe API.

And if you go to modelcontext protocol.io, you can see a list of all of the servers that they have for they launched with just a few. And then there is from the community. There is an incredible, incredible, like, literally 1000s of different integrations that are available. So what an MCP server is going to allow you to do is basically make an LLM the interface for your API or for your product, right? And so I think this is going to be pretty profound, because people are increasingly going to be spending more and more time in chat, GPT and in Claude open. AI actually just announced that they were going to support NC p inside their desktop, which folks thought they wouldn’t, since it was launched by anthropic, but they are. So this is going to increasingly be the default way that we bring context into the models.

And so your homework, if you’re experimenting with this, if you are a business person who’s you’re not writing a lot of code anymore, I would strongly encourage you to try data exploration with MCP by adding MCP to one of your, you know, whatever, either the cloud app, or when they release it, the open AI app. And then second, if you have a software product, you should seriously consider writing an MCP server. These are it’s trivially easy to do. I went to a workshop by the folks who invented MCP. They said, basically the dozen or so that they launched with the MCP launch, they did in about 45 minutes using writing that code with cloud. So it’s a very simple protocol, but you should really be like, run the experiment of what does it look like to take your product, wrap an MCP server around it, so that now your customers can interact with your service using an LLM.

Closing Reflections

All right. Couple closing thoughts. This came up from a conversation yesterday, but if the computer was the bicycle of the mind, AI is electrical bike of the mind.

We used to live in Brooklyn. We just recently moved out. This is my motorcycle. This is like my midlife crisis mobile. And we, I got the electric bike. It’s called the urban arrow. I’ve loved urban biking for a long time. We have two girls, and I got this so that I could bring them around, and it’s incredible what happened once to our family, like once we had an electric bike, it wasn’t just that. It was the same as biking, but we could just go faster. It was there were new possibilities that we just straight up couldn’t do before. From a safety perspective, I could get off the line faster. I could take the lane and keep up with traffic so folks weren’t getting mad at me. Behind me trying to pass me. I made fewer fatigue related mistakes.

From the family perspective, I could take the girls out, and we could go to Trader Joe’s, and we could bring home, like, four bags of groceries and the girls at the same time, my wife could now throw both girls into the bike and we could ride across the Brooklyn Bridge into Manhattan. It’s again, it’s not, it was not the same as a bike, but just a little bit faster. It’s not like I was doing the same stuff, but I didn’t have to work as hard. It fundamentally changed the way that we interacted with the city, and I’m finding that LLM and AI assisted coding are doing the same for me with coding.

It is just fundamentally changing what I am capable of and the ambition that I have when pursuing projects. And I think it’s honestly as like someone who has been coding my entire life, the last two years have been the most fun I’ve had coding and the most excitement I’ve had coding my entire life. So thank you all so much.

Q&A

Mark Littlewood 

Well, that was eye opening for me. Thank you. I think couple of things. One, the squirrel is called Lemmy naturally, as is the BoS lobster. It’s called Lemmy for Boston, and is the Raleigh acorn called Lemmy. It’s like there’s a pattern, and it’s black because we have in Cambridge. This is natural history lesson for free. There is a knot of genetic knot of black squirrels, which are extremely aggressive and growing quite significantly, so eventually they will take over from the gray squirrels, which is a good thing, because the gray squirrels beat the red ones up. So I don’t know there’s something happening there with evolution, but that’s amazing. Thank you. Greg, questions, hands up question. Stephen, over there.

Audience Member 

Hi, Greg, that was, that was great. Really interesting. My nephews are both interested in the space industry, and one of the problems they’re finding in that industry is that the industry isn’t training mid level engineers. So you’ve got all these people who want to get into the industry. They’ve got no mentors above them. Yeah, it strikes I mean, maybe I’m wrong. I can see there’s going to be a lot of code generated, and people generating the code don’t actually understand how it works. Yeah, so you’re going to great create a lot of people who aren’t subject matter experts, and that’s going to create a problem, I think. What do you think about that?

Greg Baugues 

I agree to I was having a conversation with a friend who’s skeptical about this stuff, and a senior developer just two days ago in prep for this, and he said exactly the same thing. He said, I think it’s going to be easier for folks to get an analogy I would use.

So my 10 year old daughter wants to be a writer when she grows up, and she uses chatGPT a lot in her writing, and so she’ll write a few paragraphs and paste them into chatGPT and ask for edits. She’ll ask for, come up with 10 names for a character, for the villain, and she’ll ask for, like, settings and stuff. And so I, you know, she uses this stuff. My rule is I just get to see all the chat histories. And so I keep an eye on it. I can say with confidence that chatGPT has made her a better writer because she was motivated to be a better writer. If you, though, are a middle schooler and your motivation is to check the box because your teacher gave you assignment chatGPT will also just spit out 500 words for you that you can copy paste. And so I think that, like, if your motivation as a developer is I just want the thing to work, but I don’t care how it works, you’re going to experience a lot of pain.

I think if your motivation is I want to learn the code, there’s never been a better time to learn how to code, because you have this infinitely patient, like, sort of very, very knowledgeable with with caveats, private tutor available to you. So I think a lot of this is just going to come down to like, incentives and how we motivate developers to learn. And I suspect if it’s true that like vibe coding produces bad code, folks are going to experience enough pain in the process of trying to ship and scale stuff that there’s just going to be folks who are forced to learn how to code. But I do think this is going to very much change the way that we do apprenticeships and new folks entering the field, it’s going to be it’s going to have a profound impact on it.

Mark Littlewood 

Great question. Thank you. Steve.

Audience Member 

Greg, you didn’t finish explaining the magic trick, and it’s been bugging me. What if chatGPT hadn’t returned 27 because sometimes it returned 24 what was the next step?

Greg Baugues 

I got lucky there.

Audience Member 

Yeah, come on. You were prepared. What was the next step? Can you tell us?

Greg Baugues 

You want to know the magic part of it? Yeah, I’ll tell you, this is like the magic. So I asked for two folks. I think I’m sorry, I wouldn’t ruin the magic here, but it’s fine. You asked for two folks. So first person does, it comes up 24 Okay, great. Now, I know a lot of y’all are thinking that it always comes up 24 so could you? Could you try it out? And you almost always get on the second try.

Mark Littlewood 

Mick, yeah.

Audience Member 

Speaking as one of those people who is creating code and really doesn’t know what he’s doing, I’ve been used. In cursive for about three or four weeks, messing around, making some kind of website content, and I’ve had a few of the similar issues you found with is creating another code to run the code, to manage the code, and it’s really not sustainable. But after two or three times of finishing that process, getting it to work and then telling it to refactor the system back again, it started refactoring some of the fixes itself afterwards, which was great. Have you had any experience of that? Is it any good at actually streamlining the code after the event, when it’s going..

Greg Baugues 

With my rails code base, I’ll have it right stuff. It doesn’t not a great way. And then I’ll do another pass oftentimes, and say, Now, refactor this to adhere to, like, rails best practices. Make sure you’re adhering, like, single responsibility principle, like separation of concerns. And, yeah, it does. It does a pretty good job there, yeah. So again, very similar to me, right? Like, ywhat’s it called? Like, write, run, refactor, I think, is that the first time you write it, you just try to get it to work, and it’s messy, and then you do another passing refactor and try to make it maintainable. Thank you.

Audience Member 

Gareth, Greg, thank you. Love that a classic BoS tour de force from the cultural background to the takeaway point. So I really enjoyed the talk. Thank you. So I’ve been working a lot with chatGPT, but I’m a technologist, but not a coder, so a sort of different kind of level, and my sort of Thursday night existential angst was actually around Asimov, and the sort of the idea that we’d built the three laws of robotics in Asimov to kind of understand the messy consequences of the adoption of AI, and then the reflection there was, yeah, We made the positronic brain, but we didn’t have a Susan Calvin, right? We didn’t have the robo psychologist in the heart of the system helping to make it human. Now it speaks, seems to me this is kind of part of the role that you’re playing, right? Sort of get into the tool, really understand the tool, really understand the ups and the downs. But my final reflection was, I, too, spent Friday night in front of the sofa playing with the image generation stuff, literally doing Modigliani and Studio Ghibli versions of my cat, right? So about 20 and I got down to the bottom, and then I said, Okay, what was the carbon footprint of my use of chatGPT tonight compared to the last 18 months that I’ve been using chatGPT. And I thought for a bit, and I came back and said it was higher, yeah, all right, you’ve used more. You’ve done more carbon into the atmosphere this evening than over the last 18 months. Wow. Okay, so I mean, that took me away. I don’t fly anymore for those kinds of reasons, right? And that took me away. So I guess the question behind the ramble, the question is, when are you taking up, caught up short, when do you kind of think, hang on, there is a moral space that I’ve just entered here with the use of the with the use of the tool, and this is raising questions in my mind that are not associated with go more quickly.

Greg Baugues 

Yeah, there’s like, so much to process through and understand here. My general take has been focusing on the things I can control, and this sort of has, like, vague feeling of, this is not a great analogy, but it’s something like, we just invented fire. And some people are going to stay warm through the winter, and some people’s houses are going to burn down. And I don’t yet know, or want to pretend, like I can predict, which is more likely, like in the course of human history, generally, we’re inventing new technologies has worked out to our favor. Like, generally, before covid, more people were, you know, employed than ever before, and even though we had more technology. And so I’ll say that, and then, like, there’s another analogy that folks will use is like, if you look at the number of I’ll focus here on the economics things, because I think it’s like, but you’re you talk about, you talk about climate, and that’s value too.

But a thing that I think is more prescient with what I just demoed is like, is everyone going to lose their job, right? Like, are all the designers? Are all the translators? So, big issue, so, and I don’t know the answer, but I’ll share two different perspectives. So one is, I first heard this from Carl, who’s a BoS attendee. But if you look at the graph of number of employees employed by banks, and also the number of ATMs in existence in the world, like they both go like this, right? And so we have this machine that’s literally automated teller machine, like we took a job and automated it, and yet, as we got more of those banks employed more people, right? And so that is like this argument that as we have more technology, yes, some people need to retrain and whatnot, right? And I think it’s very callous way to put it.

So I was in San Francisco a few months ago. I was out there for open AI’s dev day. I took an Uber from the place I stay in to the conference. The Uber driver asked me, says, Hey, what are you here for? I’m here for conference. What conference? Open AI’s conference? Is like, oh, open AI, they’re gonna take away all the jobs. And I, like, shared that same story, you know, like, and, you know, even Uber, like, displaced cab drivers, but technology. And he’s like, Yeah, but Waymo. And I don’t know if any of you all been out in San Francisco, like, they have way more, like, fully self driving cars, and it’s awesome. Like, you get in and you’re by yourself, and the car is nice and it’s incredible. And, like, you don’t have to have conversations. And I had forgotten that these were a thing in San Francisco, and I did not get in an Uber for the rest of that trip. And so how do I sit there and tell the cab driver? Like, yeah, no, it’s all gonna be fine. Like, this would be retraining. And it’s like, oh no. Like, literally, I will take the self driving car every time now, right? And so I don’t know, like, what I personally feel responsibility for is probably, like, more my family and stuff.

And I feel like I don’t know how to have the big impact. And so right now, my sort of focus is like, how do I learn as much about these tools as possible? Because in both the negative and the positive outcomes, being familiar with the tools and what they really, truly can and can’t do is going to better equip you to navigate whatever the outcomes are. Because I don’t think it’s like, I don’t think we can stop it, like, we just tend to not slow down once we start doing these things.

Audience Member 

Final question, so you talk, you talked about the wide coding, which is one extreme of using AI, where AI makes all the decisions. And when talking about tests, you talked about coding by hand, which is another extreme in a practice I find that I need to go in the middle. And I’m not sure what is the best way to approach it, like I tried two things. One is to write just function signatures and then tell the AI to fill up the methods when I define them, so I can keep an eye on architecture. Another one is just to write a function comments or class comments, and then say, generate methods and classes from that. What is the best approach if you want to keep AI project manageable you?

Greg Baugues 

I think the fascinating thing here is whatever the best approach is today will probably not be the best approach in six months, like the models are just changing so fast. And the way I would do things with GPT three, five. Turbo versus GPT four, five versus cloud, three, five. Sonnet versus three. Seven sonnet versus like, oh, one mini has reasoning. Like, the models keep getting better and better and better. And I also think it depends entirely on the code base that you’re working on. So for instance, these tools work better on JavaScript and Python because it has more training data on those two languages. I heard a joke recently that was a joke or not, but like, in the future, there’s only going to be three programming languages. It’s going to be Python and TypeScript, because that’s what the tools are best at now. And so there’s going to be compounding there, where developers going to be incentivized to write more and more Python and TypeScript, and then everything else where they don’t perform well. So then, like, that’s going to feed back into the training data, it just compounds. And then there’s going to be a third language that’s invented by llms that humans can’t understand. It’s probably pretty close to assembly that just optimizes for tokens. And so again, I just say that to say, like, your performance is going to be very different depending on the language you’re using, your code base, your practices, and then what spot in time and what model you’re working with. Thank you.

Mark Littlewood 

Greg, thank you.

---

---

Upcoming Event

BoS Europe 2026 🇬🇧

13–14 April • Cambridge, UK

First speakers announced.

Get Tickets