from Hacker News

Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI

by WillieCubed on 6/3/25, 8:57 PM with 35 comments

• by Sytten on 6/3/25, 10:15 PM

  This guys annoys me a an entrepreneur because he gets a sh*t ton of government money and it starves the rest of the ecosystem in Montreal. The previous startup he made with that public money essentially failed. But he is some kind of hero of AI so it's an easy sell for politicians that need to demonstrate they are doing something about AI.

• by nemomarx on 6/3/25, 9:21 PM

  Is there any indication you can actually build hard safety rules into models? It seems like all current guard rails are basically just prompting it extra hard.

• by delichon on 6/3/25, 11:05 PM

  Asimov's Zeroth Law of robotics:

    A robot may not harm humanity, or, by inaction, allow humanity to come to harm.
  

  "Robots and Empire" is a nice discussion of the perils of LawZero. IMHO if successful it necessarily transfers human agency to bots, which we should be strenuously working to avoid, not accelerate.

• by Animats on 6/3/25, 9:28 PM

  This seems to be a funding proposal for "Scientist AI."[1] Start reading around page 21. They're arguing for "model-based AI", with a "world model". But they're vague about what form that "world model" takes.

  This is a good idea if you can do it. But people have been bashing their head against that problem for decades. That's what Cyc was all about - building a world model of some kind.

  Is there any indication there that they actually know how to build this thing?

  [1] https://arxiv.org/pdf/2502.15657

• by moralestapia on 6/3/25, 10:04 PM

  A nonprofit, just like OpenAI ...

  I don't get the "safe AI" crowd, it's all ghost and mirrors IMO.

  It's been almost a year to the date since Ilya got his first billion. Later, another two billion came in. Nothing to show. I'm honestly curious since I don't think Ilya is a scammer, but I can't imagine what kind of product they pretend to bring to the market.

• by didibus on 6/3/25, 9:28 PM

  Interesting thing to keep an eye on.

  Though personally, I'm not sure if I'm most scared of issues of safety with the models themselves, or more so in the impact these models will have on people's well being, lifestyles, and so on, which might fall under human law.