The AI Sandwich: Where Humans Excel in an AI World

[00:00] Humans are the bread in the sandwich, and the AI is in the middle. The AI is whatever you put on your sandwich. If you ship something or do something, if you wanted [00:10] to be your own. You cannot fully automate everything. It's like art. [00:14] If you want it your own, it needs to be from you or somehow be connected. [00:19] So I believe it's so important to do things you enjoy and you love. [00:24] And it's very important to make it feel great because the bar is high, the bar will always get higher. The beginning and the end, the middles can be automated pretty well. And Trevin at some point said, oh, it's kind of like a sandwich, which was like very funny. [00:54] Kieran, welcome to the show. [00:56] Hello, Dan. Happy to be here. So for people who don't know, you are the GM of Quora. [01:04] And you are also the creator of Compend Engineering, the engineering framework and plugin that everyone inside of every uses and everyone who's really coding in with agents is at least aware of if not using. And so a pleasure to have you on the show. [01:20] Thank you. Yeah, it's always great. So I love getting to chat with you and getting to work with you because... [01:27] Every once in a while, you have a thing that you do or you figure out that I'm like, holy shit, that's definitely the future. And you just figured something out along with Trevin who also helps out on combat engineering. And I think it has massive implications for...

[01:44] how programming works. And then I think we can also translate that to [01:49] the rest of AI and its impact on work. And one of the things you've been doing, so you have this compound engineering plugin that [01:57] You've rebuilt the engineering workflow for how you should work with agents. And in thinking about that and thinking about where a human is used and where a human should not be present inside of that process, I think you've found something really interesting and deep about. [02:14] in general, how humans and AI are going to interact with work. So do you want to explain a little bit about [02:21] Combat engineering in that and the process that you've created, and then also explain this insight about where humans fit. [02:30] Yeah, absolutely. So... [02:33] compound engineering is like a philosophy of like doing engineering work but [02:39] We realize it applies to more than just engineering work. It's product work as well as design work. It could be knowledge work. It could be other things. But how I build it is while building Quora, [02:50] I had AI and I was like, how can I use AI to do better work more quickly? And, [02:57] The initial version of compound engineering really evolved around four steps, which is planning first. You make a great plan, so it's very clear what you need to build and do. [03:09] Then the work part where the agent does the work and implements it and actually writes the code, does the work.

[03:16] design work or whatever work needs to be done. [03:19] The third is review. So slop comes out or whatever you call it. Something beautiful comes out. One of the two, like something comes out, but how do you know it's good? And traditionally there's like a code review or like, [03:33] a PR that you review and see like, hey, this can be improved. And there's some iteration going on there. And then the most important step is the compound step, which is... [03:45] If anything comes up during that review or doing the planning, [03:49] that you think like, oh, this is a good learning. Probably we'll run into this again. You can compound that knowledge back into the system. And we store that as knowledge inside the repository. And agents next time when they go into planning or when they go into work or review, [04:09] they can see the mistakes they made before, so they won't make it the next time. And that's really the power, like that's by far the most powerful thing that is in this plugin. [04:21] But we start... [04:23] to realize more. Like, first of all, the work phase, [04:28] It's kind of done. It works. If you have a good plan, it does the work. [04:33] And it's pretty good. And then the review, it makes it a little bit better. And by that, you mean like... [04:38] having an entire phase dedicated to work in this whole system doesn't necessarily make it make that much sense when all that really means is run the model, let the model do the thing. Yeah, so there needs to be a step. But what I mean by done is,

[04:52] is I don't need to care. Or like, I don't need to think about it. I trust it. And this is not like, trust me, bro, it just works. But this is like... [05:03] I've seen if you put in a good plan, [05:07] Bye. [05:08] it does the plan, like it executes on the plan. LLMs are very good at just following steps, doing deep work, like working for hours, days even now. And that thing is kind of solved. And [05:21] The review... [05:23] starts to get there too. And the planning starts to get there too. And then there's this [05:28] Next step is like, okay, so if all these things work, where do I... [05:35] have to do anything because like... Yeah, did I automate my job? Yeah. Yeah. Did I automate myself out of a job? [05:42] If everything works, like where do I work? What is still the bottleneck? And... [05:47] There are two things we started to know. Like Trevin, he's a very, very great contributor to the Compact Engineering plugin. Like he is a product person. And he is like, I need more on the product side, which is like before the planning phase. So he added first a brainstorm step and an ideate step. And the ideate step is like really going wide. It's like, okay, let's come up with ideas in a room full of interesting people with angles. Brainstorm is more like, [06:16] I have a problem, but I don't really understand exactly what and how. So it's very much brainstorming with you around the problem. And the first thing we notice there is like the top is very important to be super well in the loop with a human.

[06:32] and really ask a lot of questions and really think hard. The human should think hard. The L1 should support the human. [06:40] But then after that, the planning phase, if you have a good brain from an idea of what problems you solve, [06:46] like it can create a very good plan and the human needs not to be in the loop. So that's the first realization where it's like, "Oh, hey, here's good to be in the loop versus not to be in the loop." And you can see [06:59] other like spec-driven development, for example, or other ways to do things, they assume that it's always good to have people in the loop. And I disagree. I think it's very important to know when to be in the loop versus when to hand it off, because that means we can think harder at the moments where we need to think harder. And that's the first one. [07:20] So, [07:21] The other one comes at the end. So like something comes out, how do you validate it's good? Well, it's already tested because we have browser automated testing, it clicks through all the [07:33] clearly specified and says, yeah, everything works. [07:37] But the beauty comes in when a human looks at it [07:40] clicks around and has a feel like, oh, this doesn't feel good. We can... [07:45] polish it even more. We can make it even better. We can [07:49] increase like, or like we can do something that's still missing or make it more beautiful, make the design better. And this is something I've learned from doing Pomodoros where, [08:00] Ideally, if you do Pomodoros, the old school way is like you start with a task,

[08:05] And if you finish after 15 minutes, [08:08] you have 10 more minutes to work on the same task. You cannot switch tasks. And sometimes in that space, [08:14] something beautiful happens because you will go deeper, you will go further than you would do. [08:20] And I think this is the other moment, which is all the way at the end when everything is done, where you can just elevate everything and make it even better. [08:30] better. And I think that's also what we need to do because if we don't do it, it will be all slop, all the same. [08:38] And it's very important to make it feel... [08:41] great because the bar is high, the bar will always get higher. So this is kind of what we realized, like the beginning and the end, and the middle is kind of solved and can be automated pretty well. And Trevin at some point said, oh, it's kind of like a sandwich, which was like very funny. And Dan is now referring to the AI sandwich, which I think is very cool. And I think the sandwich here [09:10] about what you do and really use your brain versus offload it to the LLM. We've all been there. You're sitting in an important meeting and you're trying to pay attention, you're trying to stay present, but you have this lingering underlying anxiety that you're going to forget everything, that you're going to miss the important detail, forget the decision, forget the action item, let something important slip through the cracks. That's why I love Granolay. It's an AI-powered notepad that works in the background while you're in your meetings. It takes notes on everything

[09:40] gets set transcribed action items and helps get rid of that feeling. You don't have to worry about whether you're going to miss something because granola has you covered. [09:48] And that lets you stay present in meetings. I've been using Granola for a long time, almost since they came out, and it's amazing for this. It doesn't join the meeting like some of those other clunky meeting note takers. The UI is really fast and well considered, and it feels like it's sort of just transcribing all the important moments in my work life. And that gives me the confidence to get great work done. And what's even cooler is you can chat with your notes afterwards. You can run detailed research reports on how your week was, how you act as a leader, [10:18] and how you can do better. It's really a power tool for anyone who cares about their meetings and also cares about how they show up in those meetings. It also has these things called recipes, which are pre-made prompts for common tasks like negotiating, coaching, or summarizing. I even have a recipe that I made that's in Granola that you should check out. Once you try it on one meeting, it's really, really hard to go back. The notes are always better than what you could do manually, and it helps me be much more present instead of frantically typing all the time. Head to [10:48] the code every, E-V-E-R-Y. That's for noah.ai slash every for three months free. And now, back to the episode. Humans are the sandwich, the bread in the sandwich, and the AI is in the middle. Yeah, the AI is whatever you put on your sandwich. Yeah, exactly. And I think that's really interesting and really cool because, A, it gives me a good...

[11:12] mental model for how I should be working with coding agents. But I think that also applies to the rest of knowledge work. And I think this is such an important question now because, you [11:22] We have all these questions about, oh, my God, like, what are agents going to do? And is everyone going to lose their job and all that kind of stuff? And I think software engineers are a little bit of the canary in the coal mine. [11:35] And so far, what we found internally at Every is we're absolutely not [11:40] like, [11:41] We still hire software engineers, we need software engineers. But, but the way that you're working and what you're doing looks a lot more like managing what you're doing it well, you're still involved, but you're involved in the beginning in the end as sort of this sandwich. [11:57] And I think the same is going to be true of every other kind of work, whether that's, you know, copywriting or strategy or design. And I think there are deep reasons why that is the case that I think will be interesting to talk about. And I want to start with... [12:17] Yeah. [12:18] an objection that I think people will have, which is like, OK, like for now, agents can't do the idea in the brainstorm. [12:27] But pretty soon they will. So then what happens? Now they're starting to do the beginning of that process. And I think that there's something interesting here where... [12:40] if you look within any given local frame of a problem, so to take a non coding example,

[12:48] The problem might be my knee hurts and I want to solve that problem. But you can say my knee hurts is the same as this feature is broken or customers are broken. [13:01] anxious about this part of the product or whatever, any problem, if you take that frame, [13:07] and you say, okay, well, the solution is maybe for your, if you're, if we're talking the knee hurting thing, the solution is take Advil. [13:15] Um, [13:17] Any part of that process, you know, getting to the store or whatever can be automated. Let's say DoorDash can go do it. [13:25] But there's always, even once you've solved it in that way, there's always a larger frame within which to think about the problem. So an example is if your knee hurts, you might need to stretch your IT band. [13:38] Or you might need to stop running on hard surfaces every day. And each one of these is sort of addressing the same problem at a different level of the stack, at a different, from a different frame. [13:49] And [13:50] Humans are very good at flipping and changing frames like that. [13:57] Our job is to set the frame or set the bounds within which we solve the problem. And I think it's very, it's going to be very, very hard for agents to do that well by themselves. And there are deep reasons for that. But does that, do you get that? Like, I know it's like a little bit hand wavy and the knee hurting thing is a little hard to understand, but does that resonate for you?

[14:18] Yeah, for sure. Yeah, it's like this all comes down to... [14:24] building an environment where the agent will thrive, [14:28] And you do not buy... [14:31] like picking the right things. And this is why it's so important to have humans with [14:38] experience and humans with [14:42] taste and humans that just [14:44] want to click around and like say this is shit or this is great and why it is shit or great. And I think it's similar to like the AdFill example, like, [14:55] If you keep doing that, it's probably... [14:58] your friend will say, yo, that's messed up. Just go, like, go fix the problem instead of, like, denying the problem. And maybe it will work for you for a little bit. You need someone to shake you up. And in that case, like, that's the human or that's the other person. But I do think, like, it will also, like, [15:18] be more automated. Like also the ideation, you can say, okay, let's have a persona of a hundred people and run simulations of how they think and how they behave. And clearly we're going there too, where we run simulations of like millions of people and see how things work. And probably you'll learn something from that. And there will be more automation. And maybe even that [15:43] step in the front will be fully automated. But I do think in the end,

[15:48] if you ship something or do something or make like a statement in the world, [15:55] If you want it to be your own, which you need to say yes or no at some point. You cannot fully automate everything. It's maybe a little bit like art, like making art. If you want it your own, you need to... [16:11] just it needs to be from you or somehow be connected. So I believe like having those moments where you decide [16:22] This is what I just enjoy. And that's why it's so important to do things you enjoy and you love. [16:30] are very important. Yeah, I agree. And I think, um, [16:34] Thank you. [16:36] Yeah, you can imagine it being like... [16:38] okay, yeah, we're going to simulate a bazillion people, and then we're going to make decisions based on what we think they would do. But that would still only cover a small set of the decisions that someone might make. [16:48] It will never be fully. It's a moving target. Like we always get something new. And then again, there is a layer up that we then can even make bigger impact on. [17:00] Especially because especially for a lot of these decisions, the feedback loops on these decisions, like the data is really rare. You may need you may only get a couple moments in your career where you gather the data. [17:10] that helps you decide about a particular thing. And that's very hard to get into language models, especially because it's hard to get and they need a lot of it. And so that sort of rare expertise that is encapsulated in an expert who has a personality and a worldview,

[17:27] is, um, [17:29] hard to hard to get. And you're right. It's also always moving. And I don't know that that makes me like very, [17:37] excited about this stuff because I feel like we've been wandering in the woods for a long time on like, okay, what is AI progress going to mean? [17:46] how are humans going to be involved and all that kind of stuff. And it just feels very much to me, like, [17:51] The simple answer is ride the model. [17:54] Um, or to mix the metaphor, be this, be this bread in the sandwich. And if you do that, you're going to be fine. It's going to be like really, really, really, really great. [18:04] Thank you. [18:05] Yeah, I agree. And it will be different for different people because... [18:10] Yeah, you need to change some things. Like you cannot keep doing what you're doing because if you like writing code only, [18:18] You need to find your way of writing code. Like, yes, you can write code, but maybe it's about... [18:26] beautiful code. And maybe you find also lots of value in just seeing beautiful codes. Like someone looks at the UI and says, "Oh, this is beautiful. This works great." Maybe you want that for code. Some people don't care about that, but they're like, "Oh, but the UI should feel great." And just really polish it, go [18:47] extra, like wherever you feel joy. But also, it's way more product focused. So as an engineer, you're going to become either more of a manager, but also more of a product person. So it's, I think, a product

[19:03] manager, products, engineer, like it's more of those things as well. And so, [19:10] there will be some changes, but lean into [19:14] making beautiful stuff. And whatever that means to you, that can mean beautiful code, beautiful abstractions, beautiful architecture, beautiful design, beautiful... [19:24] copy, I think it's very important to lean into what is beautiful to you because [19:30] then you will find a way to utilize an LLM, [19:34] to make something that gives you energy instead of drains you all the way. [20:04] we will see a rise of dictators around the world. More and more dictators are choosing monologue from Evry. It learns, transcribes and translates across different disciplines and languages, adjusting its output format to match your context, allowing you to stay in flow. Be a Dictator. An Idea by Evry. [20:27] every [20:28] the only subscription you need to stay at the edge of AI. Yeah, and I think there's a deep reason why

[20:35] language models are not going to be as good at that. There's, there's, there's one deep reason, which is it's just not going to be yours. [20:41] if you didn't decide it, if you didn't do it. But another deep reason is you can think of language models as being a super intelligence that has been kept in a box for the last year and has no idea of what's going on in the world, except for whatever it gets right when it pops out of the box. [20:59] And because of that, [21:01] it ends up being its outputs end up being a little bit more generic. [21:06] and less [21:07] personal to you and your situation. And you can see this in all of the stuff that's like, okay, all the AI writing that's like, it's X, not Y, or, you know, all that kind of stuff. It's just going to do all of that. [21:18] And to truly solve a problem well or to truly make art or to truly make a product that resonates with people, um, [21:27] It's going to have to be really well tuned to the exact problem that you're trying to solve or the exact form that you're trying to make. [21:37] And [21:38] Language models need a lot of help to get there. And that's why you have to be on either end of them to set the frame of the problem and then make sure the details are really right at the level of execution at the end. [21:51] And [21:53] I think that they will get better at doing this, but I actually think they're much further than we think they are from being able to do it all end to end. My general bar for AGI is...

[22:08] Yeah. [22:08] Whenever it is economically profitable, [22:11] or makes economic sense to run an agent 24/7. [22:15] where it never turns off. And OpenClaw is like pushing in this direction, but it's, it doesn't run 24 seven. It runs on a schedule. It has a heartbeat. [22:25] But it's not like you just say, hey, like OpenClaw, just go and just do a bunch of stuff and just work all the time, spend tokens all the time on stuff. And it's worthwhile. It's just we're not even close to that. And yes, we sometimes have well-specified tasks that we can send a model off to go for like 24 hours on. But again, it's not... [22:45] changing frames. It's not finishing the task and be like, cool, now I'm going to pick the next one. And that's going to take five minutes. And the next one, I'm going to spend four days on it. It's like, it's, we're not even close to that. And I think we're going to need some fundamental changes to language model architecture to like, let them learn better. [23:02] Um... [23:03] for them to get to a point where they're running 24/7. And I think that will, if they are running 24/7 like that, they'll be a lot closer to [23:14] I'm sensitive enough to context to like actually do interesting, creative things, but it's, we're not there yet. [23:19] Yeah, I agree. One other way to look at it. So I have a music background. I studied classical composition and [23:26] I think one of the [23:28] the beautiful things about music is like, yes, [23:32] Suno can create songs. But [23:35] It will never capture... [23:37] like a live performance or coming up with a melody. And it's something internally in the human, like as a composer or a musician, if you

[23:46] perform something and you deliver this to other people, that they feel that. Like it will not be like, sure, if you're a DJ, [23:57] It's maybe somewhere in the middle, but [23:59] Like there is something like performing, like you see something, you express something. And I think there is some of that element in these steps as well, where you... [24:11] see something and you're like, oh, it feels a little bit off here because I don't know why, but [24:17] I wanted to change it a little bit with a step at the end. And suddenly you're like kind of performing or iterating or you're making stuff. You're putting something in the world and [24:30] And I feel that special. Like practicing a piece for like playing it a hundred times is not very creative as a musician. And this is kind of the middle part. [24:41] But at the end, the performance is where you bring it out into the world to the people. So I think that's a special moment. And there is a little bit of a link for me with... [24:51] doing this polish step at the end. And at the start, it's maybe... [24:56] coming up with [24:57] a piece, like if you're a composer, like coming up with something out of nothing. And this is also a special moment. And normally everything in the middle is kind of boring, it's just work. [25:08] And I feel these moments are still special and it kind of works for [25:14] making software or other things with LMS as well for me. I think that's totally right. I love this art angle that you have and another

[25:23] Another way to say this is all the work exists on the spectrum from it being totally wrote to it being art. [25:30] And art itself has many tasks within it. Any kind of creative work has many tasks within it that are more rote or less rote. [25:39] And [25:40] if you're trying to map work on that spectrum, the stuff that is more rote is just [25:46] gonna be stuff that you're not gonna have to do anymore. [25:48] And [25:50] that is a big opportunity to move forward [25:55] a lot of the work that we do to the more creative [25:59] to us, probably more interesting parts of work. [26:02] and to recognize that that [26:06] frame is always changing or is always moving. So as as certain things get wrote, other things become things that humans start to do. And, and yes, those will get automated too. But like, we're going to also keep keep moving down along that spectrum. [26:19] and, and, [26:21] the [26:22] final thing that's not automatable is like art made by humans who [26:27] feel something. [26:29] And I think that's beautiful. [26:31] Yeah, it's still scary because what if you're in the middle and you want to move or if you want to figure out what that is to you, because this might sound very scary. [26:42] abstracts and weirds to some people. [26:45] If you're not an artist or haven't like, [26:49] really like [26:50] felt this in moments. Like it sounds maybe a little bit like, oh, but like, that's not me. But I do believe everyone has this. Like, think of it like what brings you joy? Like what, what,

[27:03] lights a fire in you, like what do you get excited about? Like I think [27:08] that thing you should like lean into, like whatever that is. And that can be beautiful writing or that can be very structured lists or whatever it is, like anything that just brings you [27:23] happiness. Like you should do more of that using LLMs in your work, Google's. [27:28] That's good. [27:30] I agree. Kieran, [27:32] Always a pleasure. [27:33] Thank you. Yeah. [27:35] Let's see where this goes. See you next time. [27:38] See you. Bye. [28:08] that will leave you on the edge of your seat. [28:10] craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the spaceship. [28:18] So do yourself a favor, hit like, smash subscribe, and strap in for the ride of your life. [28:24] And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.