Plain Inference
← Back

You keep using that word, "automated," I do not think it means what you think it means

Recently reacquainted with the masterpiece that is The Princess Bride, I feel myself quoting it constantly, in meetings, at my kids, at our dog (Chewie the Chihuahua), usually without meaning to. So please forgive the title, two articles in a row...

This week I tried to hand off keeping up with the world, one of those tasks that's pure tedium and a magnet for distraction but that quietly pays dividends across my whole life.

I run a daily radar for myself (patent filings, the steady drip of AI pricing-and-privacy news) that feeds the live feed on my site, sharpens me professionally, gives me plenty to chew on personally, and keeps my nerd status intact with the neighbors. It's repetitive, and it's a rabbit hole every single time, because the web is built to pull you off course. The handful of pages I actually need are surrounded by things engineered to grab my attention, and I'm as human as anyone, so they work on me. Half the effort is just dragging myself back to what I came for. It's part of why I get most of my news from podcasts, since my brain settles into listening far more easily than it does into reading. The proof is almost embarrassing. Even when I fast-forward through the ads, I remember them better than anything I scroll past online. Which makes this exactly the kind of task you're supposed to hand to a capable assistant now, not least because the assistant doesn't get distracted (at least not yet). So I built one, an agent that scans the sources each morning, pulls the few things that clear the bar, drafts them into a short brief, and pings me at 9am so I can decide what's worth fully digesting.

The first challenge wasn't wiring up the searches, it was trying to teach the thing to choose the way I choose. When I sat down to write out my own filter, I realized how much of it I'd never put into words, like which sources I trust on sight and which ones earn a raised eyebrow, or the difference between a headline that's merely loud and a development that actually moves something. I'd been running that filter on instinct for years. Handing it to an agent meant dragging all of it into the open and saying it plainly enough that something other than me could follow it.

When it works, it feels like cheating. The firehose arrives pre-filtered and I show up to judge, rather than gather information.

Unfortunately, it mostly didn't work. At least not at first, and not at all cleanly.

A run would get halfway through and die ("connection closed mid-response"), leaving me to guess whether it had finished the part I cared about. A folder I'd connected the day before simply wasn't there the next morning, same setup, different result, no explanation. And then the one that still makes me laugh. I pay for several news subscriptions, the actual sources this whole thing is supposed to read, and the agent couldn't see any of them. To get it into the paywalled pages I already pay for, I had to drive a browser extension through Chrome so it could borrow my own logged-in session, by hand, like passing a note under a door to myself.

So for a while my "automation" was me. Re-granting access, re-mounting the folder, re-running the job. Babysitting the thing that was supposed to babysit the work.

It does mostly work now, but still not cleanly, and not without a small stack of workarounds I have to keep an eye on, but the brief shows up most mornings, mostly right, and I do get to spend my time deciding instead of gathering. There's a light at the end of the tunnel. It just has more duct tape in it than anyone selling this future likes to admit.

Two sides, no trust

The AI at the center of all this was never the problem. The intelligence was abundant and the reasoning was good, for sure better than I'd get from a rushed half-hour of my own scanning. Everything that broke was around the edges, things like access, continuity, permission, and memory from one run to the next. The brain was fine, it just had no reliable hands, and no lasting way into the accounts and sources that are already mine.

I don't think it's my setup being janky. I think it's what the leading edge of an unfinished technology feels like up close with the missing pieces showing through the cracks.

We've spent almost four years getting models smart enough to do real work. The unglamorous layer underneath them is finally getting built, but it's fragmented and siloed, most of it enterprise plumbing assembled behind closed doors, and almost none of it pointed at a regular person who just wants an agent to act on her behalf using the things she already owns and already pays for. Your subscriptions, your files, your accounts, the context that makes you you to the tools you use. The ways I have to grant that access work fine for a person logging into an app, but they were never built for something acting on my behalf, over and over, while I'm not watching. There's no clean way to say "this agent is me, for this purpose, until I say otherwise" and have it hold.

The asymmetry here is hard to ignore. The same companies building these models, and plenty of the websites I pass through every day, are quietly learning an enormous amount about me through every interaction, every prompt, every half-formed question I type at midnight. That data flows one direction, toward them, smoothly and by default. None of that machinery is pointed the other way. Not one of them makes it easy to let my agent reach the subscriptions I already pay for or the accounts that are already mine. They are remarkably good at collecting from me, and far more guarded about letting anything move the other way, because whatever is asking for access on my behalf looks to them like one more model to defend against.

The lockdown is also rational, and I don't really blame the publishers for it. If you run a publication, why would you open a door to companies that have already shown they will take what they want and sort out permission later? A good deal of what these models know was scraped from those same publishers without consent in the first place, and nothing about how that unfolded suggests the data wouldn't eventually be used against the people who created it. Blocking the APIs and fighting the scrapers isn't paranoia. It's a reasonable read of how the last few years actually went. So the crack showing here isn't a simple failure of engineering, it's two sides that have good reason not to trust each other, and me standing in the middle re-granting access by hand.

A note for the tinkerers

The original version of this observation, the one I'd jotted down before the week went sideways, was about judgment. Once a model can monitor the firehose, the scarce thing stops being gathering and becomes deciding what counts as signal. That's still true, but I had the bottleneck in the wrong place. I assumed delegation would buy me back my attention so I could spend it on editorial judgment. Instead I traded gathering work for plumbing work. The promise of handing things off is that you get your judgment back. The reality, today, is that you become the integration layer holding fragile connections together with your own two hands.

There was a quieter surprise in it too. The act of describing my filter showed me how much of my judgment had been instinct I'd never examined. You can't ask an agent to find what matters until you can say what makes something matter, and most of us have never once had to write that down.

I'm not going to tell you which stack fixes this, because the stack doesn't really exist yet. What does exist, and what gives me some hope, are the workarounds popping up all over the place. They are clever and usually held together with tape, and there are more of them every week, because the same model advances that created this gap have also made it far easier for someone with an idea to ship working code. The distance between noticing a problem and building a rough fix for it has nearly collapsed. That's exactly why prescribing tools is pointless and naming needs isn't. Whatever exists today changes by the month. The need underneath is real, it's early, and the people living a few weeks ahead in how they use these systems are all bumping into the same wall at roughly the same time. The agent is capable, and it can't reliably get to your stuff.

So a small reframe, if you're starting to delegate real work to one of these things and it feels harder than the demos promised. When you find yourself re-granting access for the third time before coffee, that's not necessarily you doing it wrong. That's the gap telling you what it hasn't built yet, the part that lets something act for you, durably, with your permission, using what's already yours.

If you go build one of these to take the busy work off your plate, pay attention to every moment you have to step in by hand. That list is a map of what nobody has built yet.

I'll keep notes and will continue to share what I learn along the way.

What have you tried to hand to an agent, and where did it quietly hand the work back to you?

Get curious. Stay grounded. Keep testing.