X discussion: open-source AI builder Jun Song says he turned down an anonymous black hat group offering huge money to post-train open-weight models on exploit data
At 02:20 UTC on Nov 1, open-source AI builder Jun Song (@jun_song, who builds Supergemma) posted on X that an anonymous black hat group offered him an "astronomical" sum to post-train open-weight models with reinforcement learning on the exploit data it holds. He said the volume was so large he would have believed it was an undercover FBI dataset, that he turned it down, and that what scares him is that someone else will eventually take the deal. As of 17:10 SGT the post had about 2,029 likes, 222K views and 771 bookmarks, and it has been rewritten as a parody template. This is a personal account; the offer, the data volume and the group cannot be verified, so it is treated here as a discussion.
Key Takeaways
- Reach: the original post had about 2,029 likes, 127 reposts, 101 replies, 771 bookmarks and 222.2K views as of Nov 1, 17:10 SGT
- Claim: an anonymous black hat group offered an "astronomical" sum to post-train open-weight models on exploit data; he says he declined, and neither the amount nor the data volume is public or verifiable
- Comparison: he says a white-hat project of his built on GLM-5.3 used under 1% of that volume and already beat frontier models "by a huge margin"; this is self-reported with no public benchmark
- Reaction: accounts such as msuiche and Samuel Cardillo reused the opening lines as a parody template, so the thread splits between taking it seriously and joking about it
- Issue: open-weight models cannot be recalled once released and their safeguards can be trained away, so a request like this makes the open-weights versus misuse debate concrete
