Meta reveals what kinds of AI even it would think too risky to release

LovabledanielsFebruary 4, 2025164 Views

Meta makes its Frontier AI Framework available to all
The company says it is concerned about AI-induced cybersecurity threats
Risk assessments and modeling will categorize AI models as critical, high, or moderate

Meta has revealed some concerns about the future of AI despite CEO Mark Zuckerberg’s well-publicized intentions to make artificial general intelligence (AGI) openly available to all.

The company’s newly-released Frontier AI Framework explores some “critical” risks that AI could pose, including its potential implications on cybersecurity and chemical and biological weapons.

By making its guidelines publicly available, Meta hopes to collaborate with other industry leaders to “anticipate and mitigate” such risks by identifying potential “catastrophic” outcomes and threat modeling to establish thresholds.

Stating, “open sourcing AI is not optional; it is essential,” Meta outlined in a blog post how sharing research helps organizations learn from each other’s assessments and encourages innovation.

Its framework works by proactively running periodic threat modeling exercises to complement its AI risk assessments – modeling will also be used if and when an AI model is identified to potentially “exceed current frontier capabilities,” where it becomes a threat.

These processes are informed by internal and external experts, resulting in one of three negative categories: ‘critical,’ where the development of the model must stop; ‘high,’ where the model in its current state must not be released; and ‘moderate,’ where further consideration is given to the release strategy.

Some threats include the discovery and exploitation of zero-day vulnerabilities, automated scams and frauds and the development of high-impact biological agents.

In the framework, Meta writes: “While the focus of this Framework is on our efforts to anticipate and mitigate risks of catastrophic outcomes, it is important to emphasize that the reason to develop advanced AI systems in the first place is because of the tremendous potential for benefits to society from those technologies.”

The company has committed to updating its framework with the help of academics, policymakers, civil society organizations, governments, and the wider AI community as the technology continues to develop.