In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.
Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.
Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”
Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”
xAI founder Elon Musk has denied that Grok has ever been used to generate child sex images.
Well, that’s a relief! Of course they wouldn’t do something like that. Because that would be super illegal!
How do you even generate ONE of those images and NOT immediately go to prison?
“FoR thE chIlDrEn” when privacy is on the line but they can’t put a weirdo in prison after generating SEVEN THOUSAN CSAM images?!
Because it’s never ever about the children.
Totally not about the children, says robust discourse and literal thousands of images all about the children .
Hi, someone that has worked extensively in the trust and safety space before AI destroyed said trust and safety industry in the west here.
The short answer is the lack of human moderation.
The long answer is that due to companies using AI to detect CSAM (for some good reasons, but also because it’s way, way, way cheaper than a trust and safety team and their mandatory EAP benefits) it has become infinitely easier to get CSAM past filters these days.
This is especially true for tools that can generate CSAM from existing images, where both the generation and sharing of these images are likely run through the same detection AI, meaning if one gets through the other does as well.
GAN image detection models are literally a part of image generation training models; to explain better how a GAN model trains is:
Agent B is given [extensively tagged image of a scene].
Agent A uses random noise to generate [Random image of pixels]
Agent B then sees if [Random image of pixels] is [Extensively tagged image of a scene]
If not, then Agent A regenerates the image. Again. And Again. And Again.
Agent B only has extensively tagged images of scenes, and only knows what the end products should be… in theory. Agent A only learns what these tags mean and what these images should be based on feedback from Agent B.
So if you want to detect bad things… go with Agent B, right?
Wrong! Current GAN has AI-trained Agents in both roles, with manual additions to their databases to further refine their knowledge.
This means Agent A trains Agent B as much as Agent B trains Agent A. Some models even have an Agent C which uses a LLM to figure out how to describe an image based on the tags generated by agent A or B… which is then fed into Agent A or B to further refine the image.
Now which one do we pick for moderation? It doesn’t matter! It’s all an incestuous clusterfuck that is defeated by word play (given Grok is an LLM feeding into a GAN image generator, any tags the LLM gives the image generator will be automatically approved by whatever agent scans the image for CSAM, since the tags it can detect are already filtered out by the LLM).
Now you can see the problem with this. I can see the problem with this. Most human beings can see the problem with this. But this meets current global moderation regulations as long as there is a manual way to report content that gets past the AI moderation… which would work except without human moderators the AI moderation will still fuck up removal either by removing any reported content or just not removing content if the AI moderator disagrees.
Until regulators in every single market collectively decide to enforce human moderation teams on big tech or punish for content hosted regardless of the company’s efforts to prevent it, this will not change or stop. It will always be people exploiting the inherent arms race of GAN models to generate CSAM with minimal effort and no real way to prevent it.
Christ. I’m tired of humans being removed from positions where they are the only good option just to be replaced with a worse option that isn’t even cheaper in the end, like AI.
“Think of the children”
And this is what happens when they do that a little too much
it’s ironic how all the people barking “think of the children” turn out to be the people we absolutely DO NOT want thinking about children, ACTUALLY.
I think they think its the normal amount.
They are making their pedo dream land, with no laws or rules.
This is their so-called effective altruism.
We all know what to do.
We all know what to do.
So, if xAI did that, then did they get obstruction charges?
You mean a slap on the wrist for the richest man in the world?
When you’re rich they let you do it.
Why? This is just about a megapedophile, it’s not like anybody camped out on a lawn or asked Lockheed to stop selling f35s to baby murderers.
You would think the megapedophiles went extinct during the cretaceous era, not so!
Why would you think that? Do you live so far from an ocean?
… i hope he never had the chance or ability to actually do anything to her physically ._.
i hope learning about what someone did without her knowledge will not scar her for life…
Musk should not be walking around free at this point.
Well, the only bright spot here is that the trash does sometimes take itself out.
All’s well that ends well.
What a fucking headline. At least it had a happy ending.
Removed by mod
Are you openly requesting cp?
This entire thing from start to finish is horrifying.
Yes, from Elom being born all the way to buying Twitter.
The story was probably pretty horrifying before Elon was born too, from what we already know of his father.
And it sure didn’t stop being horrifying when he bought Twitter.
At least the ending is a happy story
All the people who bought SpaceX stock invested in this. Grok is owned by SpaceX. It would be nice if journalists would put that in their articles.
Lawsuits for this need to start including shareholders as defendants. Make investing in it a civil and reputational liability.
Lawsuits for this need to start including shareholders as defendants.
I get the sentiment, but … they kind of already are.
Being a shareholder means you own part of the company. If the company stands to lose money from this lawsuit, then all the shareholders do as well. (That is, at least, on the somewhat naive assumption that a company’s stock price has any relation whatsoever to how much money it has and can make.)
When the stock value falls, they’re not immediately losing money. If they sell the stock, that’s when they either profit or lose. As long as they hold on to their stocks, there are no actualised losses.
For shareholders and investors, the worst thing that can happen to them right now is the stock going down and having to take a loss on it. It’s bounded by however much money they put into it.
If investing in 10 shitty companies knowing that 1 would fail manages to be more profitable than investing in 20 companies that don’t degrade the moral fabric of society, there’s a perverse incentive to keep investing in companies creating Groks. Making shareholders financially liable for the debts of their investments would introduce an unpredictable risk that could help change the equation.
If you’re driving a Tesla, you are funding CSAM.
Best thing that ever happened to me was getting banned off Twatter in October of 2019. (after a decade and 30 poasts, no less!)
It was delicious that I got to refuse the “Come back to X” message that arrived in my mailbox after Musk took over and offered to reinstate the banned accounts.
Thanks, Elon, but no.
I’d rather have a root canal w/o anaesthesia.
Every US taxpayer invested in this, involuntarily.
The CSAM generator is in the index funds 🤢
It’s not just owned by SpaceX, it accounts for the vast majority of the “value” of the IPO stock. The part of the company that sends things to space was a footnote compared to the portion made of the projected value of their AI business.
Every image Grok generates of CSAM is another reason to rightfully call Elon a pedophile for knowingly allowing it to happen.
Those idiots think this should be legal, but not when the civilians do it. Tim Sweeny probably is only piping up because he insist on posting the stupidest thing his brain can come up with every day.
In no way shape or form has elon tried to dampen or stop this from happening to this day. He just made it so only paid users not in the united States can keep producing csam.
And trump wants this. This is the mysterious Ai progress we keep hearing about, next to price fixing, and survalience. States are not allowed to regulate Ai, and people will tell you that you are a dangerous person for trying to suggest that elon musk should be held accountable.
We are not all here looking for the guy who did it or confused on why this is happening. Elon musk has created a csam generator and doesnt have to stop it. Instead he just occasionally has to send one of his super fans to prison for using it the way he intended to. All to say hes doing something to stop this problem.
7k images and videos is a lot to go unnoticed. Its almost like this is allowed on their platform and encouraged. This isn’t a “we can’t be held accountable for what our users post” problem. This is a elon musk left a loaded gun out on a table, and he should be held liable for not putting it in a gun safe. Especially when every single other mainstream llm doesnt have this problem. Its negligence to the point of being willfully ignorant. You can’t tell me they can ban you for a single comment about Israel but all the sudden their moderation isn’t the greatest when 7k images and videos where being produced?
I really hope France fucks them up. In France, there is no question about the illegal status of fictional csam, it is illegal.
Jesus Fucking Christ.
Just turn it off at this point.
RPG 🐦🔥?
?
I don’t think Elon want to turn off the CSAM creation machine, otherwise he probably would have some time between when he instructed the construction of the CSAM creation machine and now.
You expect this dude to switch it off?
he probably won’t, that is the only way he can get some usage for that piece of crap.
