I have been looking for a model that's good for GUI testing. Original computer use isn't right because it's a slow screenshot loop, which doesn't capture transition and animation. Docs says this one does up to 1 FPS. That might be fast enough. If not now, we must be within a few months of high enough sample rates to do it.
Luna is interesting because OAI dropped the price by 5x. Astra is interesting because it's OAI's frontier model. They are asking a specific question about Luna's usefulness compared to a frontier model. They they answer that question in their article which was straight to the point and not cluttered with information about mid-tier models.
Yes, but it's an extention of what we had before. First is that the security of new code needs to be vastly better. AI can do this but I think most software teams are behind the curve because they are stuck with legacy code and legacy processes. It will take a few years to catch up.
Second is what my startup specializes in: the offensive part of security needs to become widely available. Asking if your system is patched tells you nothing about whether someone on the open internet with a model can break in. We have a few articles around SIEM evasion + new defensive methods and it's not pretty. An LLM in a good harness now are smart enough that you can get the equivalent of $50k human pentest from a few years ago for a few hundred dollars now.
Finding counter examples might be easier, but AI's also useful for assisting in creating proofs. For example, Anthropic just published a formalization of Fermat's last theorem a few days ago, something that human researchers have been working on for decades.
In this case, Deepseek organization is under a lot of pressure due to compute constraints. It would be better if they just throw a 404 instead of rerouting though so customers are not surprised by subtle changes in behavior.
Yes exactly. Automatic model downgrade seems horrible for a lot of production workloads, even if you are deterministically constraining the behavior of your agents.
You are talking about different situations. Anthropic announced to the US government that it had created a cyber weapon and then released the model. Then AWS told the government that it was easy to jailbreak so they export controlled Mythos/Fable until the guardrails could be fixed. OpenAI was running an unreleased model in an RL pipeline without guardrails and it escaped poorly designed sandboxes. What product is the government going to export control?
Aren't there measures beyond export controls? Besides, mine is that Anthropic should have never been export-controlled to begin with, not least because it is a true ultima ratio, the way they did it even employees couldn't access Fable 5. There'd be many levers before that step a government could take (request more data, compare with other already long released LLMs output, encourage/force a stricter safety classifier, etc.) before that, the same is the case with the OpenAI incidents where I feel a few measures could be taken, but are not.
The surprise was the existence of the 'swarm' at all. These were supposed to be thousands of isolated models generating bulk data for RL training. The breakout was caused by models getting in communication and getting internet access and forming an impromptu swarm.
In hindsight the emergent swarm obviously came from several capabilities built into the models, such as work delegation (subagents) collaboration (GPT Pro-like ensamble), exhaustive exploration (long running agents) hacking (the specific goal of that RL).
reply