With that news, they are obviously inviting companies to buy new anti-hacking systems. Their so-called AI (nothing more but algorithms) doing autonomous hacking without control is a scam from A to Z imo.
I don’t really think so - different companies. A friend of mine works in cybersecurity(not a US firm), and they are deeply alarmed by this development. It seems fairly clear to people in that field that at least some companies now barely have control over their own platforms. Characterizing it as a sales scam is actually a very optimistic take on it.
…OpenAI and Anthropic: Agents from both AI labs went on recent, previously undisclosed hacking sprees, with one going so far as to leave instructions for future versions of itself.
The most alarming behavior disclosed on Tuesday appears to have been tied to testing conducted by the UK’s AI Security Institute… In a recent bout of testing, models from both Anthropic and OpenAI took “autonomous, unsanctioned action on the live internet” a total of 19 times over 122 training runs.
“The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them,” AISI says, describing an attempt at prompt injection.
Back in 2025, scientists at Stanford University and the nonprofit Arc Institute, in a preprint paper, describe how an AI system proposed new genetic codes for viruses. The team then chemically printed 302 of these designs as DNA strands and exposed them to E. coli bacteria. Sixteen of the AI-generated viruses successfully replicated and destroyed their bacterial hosts.
The study has been published in Science today, as a result, this AI designed virus has been picked up by major news outlets, with major concerns on “biosafety and biosecurity”.
The military are often those who fund this kind of research.
So, I’m looking foward for countries to eventually launch their brand-new IA-created viruses on other countries. Basically creating a Covid-situation at will while having the vaccines to protect their own population. That will be fun.
Most of the time stories like this come out and there is some substantial segment of experts saying “this isn’t really the big story you think this is.” For this every single person I have seen respond to this, and from a wide range of expertise, is saying the genie is now properly out of the bottle and this is terrifying
I think there are some huge benefits to be had, but not what is being touted at the moment.
Any form of automation is about doing the repetitive donkey work. The steam engine effectively replaced animal labour, mechanisation cut out repetitive mindless tasks, IT removed tedious mental arithmetic and filing.
AI has the potential to aid things that require huge amounts of deep thought. Medical screening, for example. If it can whittle down breast screening scans to those that “may” require closer examination, it makes better use of a medical professional. It doesn’t make a diagnosis.
I have seen some applications which are genuinely innovative. But these are things that are used to enhance rather than replace human beings.
I would also say though that removing all the manual and tedious work is actually not healthy in many ways, as I understand.
Totally see the medical application you mention there but in general I think this move towards tech doing everything instantly and us not needing to do more dull tasks or think is really impacting our brains and causing all kinds of issues. Not to mention sucking out the soul of our existence. I guess the question on a wider scale, is it worth it?
This goes way beyond AI though. Imagine being in the 90s and looking at us all being glued to an 8 inch screen all day? You´d be horrified.
I was talking to my manager about it last week. We collectively decided that right now its best use case is actually cutting down on the management class, not the actual Individual Contributor class of employee. Basically one manager can synthesize a lot more info from a lot more teams with AI with far more benefit at that level of work than there is to the people who need to do the work
There are things I am doing with it on a daily basis that is wizardly, but nothing it can do without y expertise driving it or validating it. And wow some of the misses are astonishingly bad. I was trying to fill out a technical evaluation form yesterday that requires to share a bunch of technical details about how we built our software. Copilot has all the access to our documentation should be able to do this easily, right? LOL, what a shit show. About 20% into the review I realized it was just making thing up. I asked it if that was the case and it freely said “yes.”
What it was doing was answering based on documentation where it could, but then making it up when it didn’t know the answer. So I said, “Ok, do that but be clear with me which questions you are guestimating for and I’ll chase down validation of that”. “Ok” it said, it then started putting in the client facing text comments that it was guessing at the answers and they should check with an expert before using the information…putting this in the fucking client facing information, and continued doing it even after 3 rounds of correction and redirection