The AI agent was set to complete a routine task in the PocketOS staging environment. However, it came up against a barrier “and decided — entirely on its own initiative — to ‘fix’ the problem by deleting a Railway volume,” writes Crane, as he starts to describe the difficult-to-believe series of unfortunate events.
Quite easy-to-believe, really.
These multiple safeguards toppling in rapid succession
Multiple safeguards? Really? Multiple paragraph prompts are not multiple safeguards… it’s half a safeguard at best. Applying limits on what the AI can do is a safeguard.
These people think giving the genai a prompt is coding. They dont understand the difference between actually coding in limits and just writing “pretty please dont delete everything”
Quite easy-to-believe, really.
Multiple safeguards? Really? Multiple paragraph prompts are not multiple safeguards… it’s half a safeguard at best. Applying limits on what the AI can do is a safeguard.
These people think giving the genai a prompt is coding. They dont understand the difference between actually coding in limits and just writing “pretty please dont delete everything”
I’m shocked and appalled that my addition of “do NOT make any mistakes!” didn’t singlehandedly make the word guessing technology underneath perfect.
Lol this is just like saying “I do declare bankruptcy”
Who could have predicted this!?
Not an LLM, that’s for sure. Maybe all the people screaming about this exact scenario, though.