Basically everyone has a sandbox of some sort to run agents inside. Everyone has a registry of some sort for tools. Everyone has a way of running agents inside a sandbox and giving it some tools.

Now the stuff people are coming up with is: how do you do authorization in this model? do you need a full sandbox all the time or can it be a workflow? how do you specify an agent is it a prompt or does it have some kind of control flow structure? How do you coordinate among many running agents?

I would say thats where we are now is there’s loads of people all solving the same problems a bit like when CoreOS, Kube etc. were all competing.

Note you don't need a sandbox if you are not doing code execution. There are a lot of applications where inference only is sufficient e.g. web scraping websites that don't change frequently or OCR on scanned documents.

Code execution (usually TS/JS or Python) is useful most when you are dealing with truly open ended problems. It's the opposite of the use cases of most enterprise SaaS.

You need it if your agent can access the internet and read files you don’t want public. That’s a relatively minimal case.

Do you actually need it, or do you just fear you need it?

What's the actual realistic threat model for median developer or median user here?

By realistic, I mean that leaking your grandma's recipes or your SSN or your million dollar idea to some pastebin is neither likely nor going to meaningfully make things worse for you, or be useful for any malicious actor. Surely this is not what everyone is worried about?

I spent today doing forensics on ten compromised WordPress sites sharing one hosting account.

I used two agents: One with network access to collect the evidence, and one with everything except the model endpoint cut off, which did the analysis.

The second agent's entire input was attacker-authored. So PHP droppers, obfuscated loaders, database rows, filenames, blah blah.

In this case I'm more worried about hostile input attacking the agent, and I need to contain the damage. My sandboxing solution does that by restricting access to the source data, making it read-only. The work dir can only transfer data via patch and apply (like a git workflow), so even my workspace can't be modified until I approve each change. And then restricted network means that any compromise ain't going noplace.

The second agent couldn't even install PHP or contact any CVE site to check if it was looking at a known attack, and that was by design. All it could do is write up a report about what it observed, not make assumptions about what it is. I could then take its (much smaller) clean output and pass that to a third agent with network access.

This is forensic work, so of course not your median dev's bread & butter. But the attack surface is only just starting to be plumbed. Compromising input can turn your agent into their agent, planting things as easily as planting worms was back in the early internet days when people connected without a firewall.

Security by obscurity is just a bet on weights, a belief that the economic motivations for attacking are insufficient. That worked before, but developments in ML calls to account all the debt we’ve accumulated through that practice.

It is really easy to restrict and contain an AI agent as long as you don't give it access to a terminal. If you only give it tools to read files and access the internet, then it is much easier to just restrict the tools themselves rather than setting up a whole isolated sandbox.

Access to read sensitive files and access to internet could easily lead to data exposures on the internet, no? Without any terminal or shell access.

Yes, but what I'm saying is that it is much easier to put a limit on the read_file tool or the http tool rather than sandboxing your whole environment.

You only really need to sandbox when you provide access to tools that are almost impossible to filter correctly, such as a bash tool or a tool for arbitrary code execution.

Limiting it to just read and http limits its usefulness. If you want something like “filter for this, count the number of matches, format like this” you have to make custom tools. And you have to make sure they don’t have any bugs that allow arbitrary code execution. You’re effectively building your own sandbox in this case. Using a standard one sounds easier unless you have a very focused use case.

I think I missed why is this faster? What I’m reading here is it’s similar to constrained decoding but I’m not seeing the explanation of why it’s able to get those results.

This thread should be on the first page of every book on microservices.

This or you just repeat the initial prompt every 200k tokens

Which gets you to the point where the whole thing is.. still unreliable. Generative text engines are going to generate. This calls for real enforcement in deterministic pre-edit hooks.

And here is where naive people will say something like "Why do I care if robots shit all over the codebase? Code is for machines, I don't expect to deal with it much now". But really externalized CoT like this confuses machines too, wastes tokens, and eventually wastes exponentially many tokens. Agents tend to think it's more real grounding than prompts are, even for comments-in-code. One bad comment poisons everything, then gets copied around as a ground-truth assumption everywhere. Hooks are more real to them than prompts or comments, and even then if you add enforced limits and tell them to externalize CoT ONLY in scratch task-tracking docs.. they will violate comment-enforcement hooks about 25% of the time. That tells you everything you need to know: even with constant reinforcement, they just really want to break this kind of rule.

Yup. If only there were a task completion hook that could be set to fire prior to rendering terminal output. That would more handily address all these issues, as we could simply enforce output style rules that way.

The current output style does work, but it’s a Sisyphean task to tweak it constantly only to find out that CC adhere’s to only 75% of it, no matter what…

My take on this is they are a tool to help speed up your work they are not meant to produce finished work. Humans produce finished work. LLM will never be deterministic cause their entire value is that they are generalizable.

I hear that, sort of, but here's the thing. Using AI at scale means AI needs to be nearly perfect about not shitting where they eat. That's the subject matter of the whole thread

So the options are a) being a really aggressive stickler for generative hygiene with deterministic rules, b) being massively wasteful about hiring a few machine janitors for every machine coder, or c) humans become the machine's janitor. If I haven't missed an option.. only the first option seems reasonable here.

"AI at scale" is just a euphemism for slop. Current gen AI can augment human engineers but not outright replace them.

This isn't really responsive to what I'm saying or what the discussion here is about, but if you insist. Would you describe lots of AI augmenting lots of human engineers as perhaps.. AI at scale?

I think Elder Scrolls Oblivion was like 80 person core team and 4 years of development and New Vegas was like 70 person dev team and 2 years of development.

But starfield, which was widely criticized by fans, was over 100 people and 8 years development. With the most common criticism from former devs being about the additional management structure and difficulty of communication.

I can’t help but feel there is a lesson there for tech companies where engineering orgs for products are 200+ people when a 20-30 person startup is your top competitor in the vertical.

You can also reduce team size by employing insane, life-destroying crunch time.

My favorite is 20th Century Food Court in Last Call BBS. Some of the other games remind me too much of work (I’ve bought all of them including the coincidence card games but I have only beaten this one), whereas this one reminds me of fun times I had making synths in Logic and VCV Rack for fun. Highly recommend!

I always wished they would make a management or simulation game, I think 90% of all programmers play Paradox games or Tycoon games etc. and I know their take on it would be amazing.

Not really if it takes you 15 minutes to write a 50 line function but it takes the AI 90 seconds then you already are at a 10x speedup just for this task.

This (non-yolo mode AI coding) is actually how we used to code in the old days (2023).

As someone who has used multiple vibe coded internal tools: you will care when you use these tools and encounter strange bugs and missing features.

The human touch is visible in the way your features work just like in vibe coded art and games it lacks intention.

Scale to zero is very useful.

When people say “you’re holding it wrong” it tells me they can’t even conceive of a better way of doing it.

The models produce the same slop for everybody, you don’t have a special way of doing it you lack taste and an opinion on your problem domain from lack of research and studying prior work.