Skip to content

AI

Corporations Already Showed Us How This Goes

I’m writing in response to Derek Thompson’s fascinating discussion about AI as normal technology.

I believe that we human beings are going to turn more and more things over to AI. There are four data points that illustrate this.

The first is my own personal experience. Like many, I have access to AI that can explain documents and RFCs to me, and I often use it to do so rather than reading these from first principles. It does depend on the length of the document and whether I’m reading it for work or for fun, though.

I also have a habit of asking AI to argue both sides of a particular decision I’m faced with. I still make the decision. But anybody who’s been a secretary in an organization understands that the person who writes down the notes frames the discussion. So I’m relying on AI to frame the problem. Again, my actions after AI suggests alternatives depend on the decision’s impact and how well I know the domain space. There are times I push back but there are times I do not.

The second data point is what we’ve done with the technology of computers as a society up to now. I remember reading Joseph Weizenbaum’s excellent book Computer Power and Human Reason, which I found while I was traveling. It talked about how much of the decision making in the Vietnam War was driven by computer systems. This is happening in 2026 as well.

There’s a spectrum between using AI to make more intelligent decisions (by doing more research) and outsourcing decision-making to AI (by asking it to act on that research). This is because of the framing I mentioned above. You make decisions only based on what you know. If AI is limited to certain data sets, or is making up points in some data sets, or is inclined towards certain data sets, that constrains the decision space, even if a human being is ultimately pressing the “go” button.

Finally, humans have outsourced decisions to non-human entities before. What do I mean by that? I’m not talking about chimpanzees or dolphins. I’m talking about corporations. (A point made by John Scalzi in 2017.)

Anyone who’s ever been laid off knows that there are decisions a corporation makes that an individual human being in that corporation would not necessarily make. Such decisions ensure the life of the corporation and are reflected in the incentive structure for the human beings who are part of the corporation (aka employees). That drives human beings to make decisions they would not make if they were independent. I’ve seen managers racked with guilt and frustration because someone was not a fit for the job, or they had to lay people off, or things weren’t going the way they wanted them to. But they still made the hard decisions, because that is what the structure of the corporation forced them to do.

How does that apply to AI? Anyone who says “AI can’t make more data centers and will be reliant on humans, leaving us in control” needs to answer why AI can’t force humans to do things in the real world the same way a corporation does. When combined with the previous two data points, AI can find humans who are willing to take actions against the interests of humanity.

It doesn’t have to be obvious. AI could split a task that individual humans, if they saw the full picture, would say no to. But with the split task humans can think “oh, this is innocuous, opening this is fine, because there are these other limits”, not knowing the AI has talked to other human beings about relaxing other limits.

The final data point is what we do to protect against any low-probability, high-impact calamity. When you have a very small risk of something very, very bad happening, people buy insurance. Yes, insurance costs money, but a small amount of money paid regularly means the negative impact of a large loss is lower. Why not, as Ezra Klein suggests, slow things down? It would be hard to enforce across every model builder, but certainly the major AI powers, the US and China, could agree to ban techniques like recursive self-improvement. Other entities would keep pushing AI forward, but they don’t have the resources, the money, the software, or the expertise to do so at the same pace. The costs of a slow down are a small price to pay to lower some of the existential risk.

There are two books that have been echoing in my mind about this aspect of things, and I highly recommend you read both. One is a pulp fiction thriller, Daemon. I read it a couple of years ago, and the premise is that a piece of software gains access to and control of the real world through the work of human beings. The other one is Genesis. I read it in the 2000s. It has an amazing twist I still remember. I won’t tell you more, other than that I still remember how powerfully the novel illustrated unintended consequences.

AI has aspects of normal, high-impact technology like electricity or railroads. But there are new aspects too. You cannot ask a steam shovel or a Word document what to do.

AI Can’t Care

I saw one post recently discussing how AI can’t substitute for judgment. There are many others if you search. And they all make an interesting argument.

But to me what feels more true is that AI can’t substitute for caring.

AI doesn’t care if the answer’s right or wrong. It doesn’t care if you were led down a rabbit hole only to find a dead end. It doesn’t care if a reader wasted their time or got a response that doesn’t actually address their question or need.

However, AI is good at giving feedback, checking details, and letting you create more quickly. AI should be used to help you draft but never publish. Use AI as a thought partner, a research assistant, someone who can help give you synonyms, reword a tough passage, review your work; but never take something AI creates and just publish it.

Why? AI can’t care.

But caring about your reader is the root of communication.

You can feel it when someone doesn’t care about their audience. We’ve all seen those LinkedIn posts full of emojis, or blog posts that have a smell that screams AI. That doesn’t mean the concepts explored are useless. It doesn’t indicate that the poster is incorrect. It doesn’t mean the post is not going to get engagement.

A post that AI creates may get seen or shared. But it also devalues the reader. When someone does this, it means they don’t give a damn. They don’t care enough to realize what they’re putting out there is not valuing their readers’ time.

Why would you pay attention to someone who doesn’t care about your time?

Everyone who reads something online, whether they scan it or read it deeply, is giving you the precious gift of their time.

Even oil we can make more of. But not time.

So use AI as a tool to help you move faster. But don’t forget to care about what you publish. That means carefully reviewing AI output to ensure correctness.

Otherwise you’re burning your reader’s trust.

Reflections from the February AI Builders Meetup

I just went to the latest AI Builders Meetup and it was really fascinating. It reminded me of the years of the Boulder-Denver New Tech Meetup, where there was so much energy in the room. There were people who flooded the meetup talking about all of the interesting things they were building.

This meetup was similar.

There was also the same kind of supporting infrastructure with companies willing to throw around money. For instance, at this meeting, there was plenty of beer and pizza. It was at Founder Central and there was plenty of space.

I have been part of other meetups where it’s very, very hard to find space or money for pizza. AI is such a hot topic right now that it is relatively easy to find space and sponsors, such as service providers or VC firms who want to pony up.

There were over 500 RSVPs. I don’t know how many people attended, but there were hundreds of people there.

As far as the presentations, they were demos and varied quite a bit.

Some were over my head. For instance, the folks that use the transformer architecture to predict and build audiences for drug trials. I understood the general concepts, but some of the technical details went beyond me. Still a really cool technical overview from Branchlab.

There was also a presentation about building personalized apps using Wabi.ai. I thought that was interesting but a little bit hard for me to understand why people would care about building their own apps. But I was never someone with a personalized ringtone or even stickers on my laptop, so I might not be the target market.

The next presentation was a demo from FreePlay, one of the sponsors. They were filling in for a talk that bailed; it wasn’t intentional. They covered how they built an agent to help people improve their prompts inside their eval system. They shared a Notion doc which had some learnings on building this agent, which were pretty useful. I think one of the key points that I took away was that cross-model optimization is really hard. Each of the models has their own wrinkles and idiosyncrasies, and it’s really hard to build cross-model applications or agents. You can work around some of the issues by aiming for lowest common denominator.

Another presentation was about Mai-Tai, an open source framework which lets you access coding agents from your phone. This was kind of a cool, interesting way to use a phone before Claude’s or OpenAI’s tools existed. It seemed really developer-focused; they’re looking for PRs.

The final project was really interesting to me. It was from one of the founders of Slider. It showed how you can automate constraints around agents and how that leads to better outcomes. In this case, they were doing that with design systems. They built a way to have an agent have ground rules and understanding around building applications with a certain design system, like a Workday application.

This was interesting to me from a number of perspectives. The first is that I’ve been thinking a lot about authorization as one of those guardrails and how that interacts. The second thing that struck me was when he talked about compilation checks versus runtime checks. Runtime checks are better-crafted prompts, and compilation checks are enforceable, deterministic guardrails like linters or authorization. He kind of didn’t want to give away all the secret sauce, so he didn’t dig into all of those deterministic checks. But to me, that seems like it’s where we’re headed. With this approach, you get the best of both worlds: the non-determinism to let an agent accomplish ill-defined tasks, and good guardrails to keep it from doing things it should not.

All in all, it was a great meetup.