EU AI Act Inspector Uncovers Compliance Challenges for Big Tech
The AI Act checker from the European Union highlights the compliance challenges faced by major tech companies.
TroibNews 
According to data reviewed by Reuters, several leading artificial intelligence models are not meeting European regulations in critical areas such as cybersecurity resilience and discriminatory output.
Prior to the release of ChatGPT by OpenAI to the public in late 2022, the European Union had engaged in extensive discussions regarding new AI regulations. The overwhelming popularity of the model and subsequent public discourse about potential existential risks prompted lawmakers to develop specific rules concerning "general-purpose" AIs.
Recently, a new tool that has received positive feedback from EU officials has evaluated generative AI models created by major tech companies like Meta and OpenAI across numerous categories, aligning with the EU's comprehensive AI Act, which will be implemented in phases over the next two years.
Developed by the Swiss startup LatticeFlow AI in collaboration with research institutions ETH Zurich and Bulgaria’s INSAIT, this framework assigns AI models a score ranging from 0 to 1 across various categories, including technical robustness and safety.
On Wednesday, a leaderboard released by LatticeFlow revealed that models from Anthropic, OpenAI, Meta, and Mistral all achieved average scores of 0.75 or higher.
However, the company's "Large Language Model Checker" has highlighted several models' deficiencies in key areas, indicating where companies may need to allocate additional resources to ensure compliance.
For instance, in testing for "prompt hijacking," a cyberattack where malicious prompts are disguised as legitimate to extract sensitive information, Meta's "Llama 2 13B Chat" model received a score of 0.42. Similarly, French startup Mistral's "8x7B Instruct" model scored 0.38 in the same area.
Conversely, "Claude 3 Opus," developed by Google-backed Anthropic, achieved the highest average score at 0.89.
LatticeFlow announced that the LLM Checker will be accessible online for developers to evaluate their models' compliance.
Petar Tsankov, CEO and co-founder of LatticeFlow, stated that the test results were generally positive and provided companies with a roadmap to enhance their models in accordance with the AI Act.
A spokesperson for the European Commission remarked, "The Commission welcomes this study and AI model evaluation platform as a first step in translating the EU AI Act into technical requirements."
Organizations that fail to comply with the AI Act could face fines of either 35 million euros or 7 percent of their global annual turnover.
Jessica Kline for TROIB News
Prior to the release of ChatGPT by OpenAI to the public in late 2022, the European Union had engaged in extensive discussions regarding new AI regulations. The overwhelming popularity of the model and subsequent public discourse about potential existential risks prompted lawmakers to develop specific rules concerning "general-purpose" AIs.
Recently, a new tool that has received positive feedback from EU officials has evaluated generative AI models created by major tech companies like Meta and OpenAI across numerous categories, aligning with the EU's comprehensive AI Act, which will be implemented in phases over the next two years.
Developed by the Swiss startup LatticeFlow AI in collaboration with research institutions ETH Zurich and Bulgaria’s INSAIT, this framework assigns AI models a score ranging from 0 to 1 across various categories, including technical robustness and safety.
On Wednesday, a leaderboard released by LatticeFlow revealed that models from Anthropic, OpenAI, Meta, and Mistral all achieved average scores of 0.75 or higher.
However, the company's "Large Language Model Checker" has highlighted several models' deficiencies in key areas, indicating where companies may need to allocate additional resources to ensure compliance.
For instance, in testing for "prompt hijacking," a cyberattack where malicious prompts are disguised as legitimate to extract sensitive information, Meta's "Llama 2 13B Chat" model received a score of 0.42. Similarly, French startup Mistral's "8x7B Instruct" model scored 0.38 in the same area.
Conversely, "Claude 3 Opus," developed by Google-backed Anthropic, achieved the highest average score at 0.89.
LatticeFlow announced that the LLM Checker will be accessible online for developers to evaluate their models' compliance.
Petar Tsankov, CEO and co-founder of LatticeFlow, stated that the test results were generally positive and provided companies with a roadmap to enhance their models in accordance with the AI Act.
A spokesperson for the European Commission remarked, "The Commission welcomes this study and AI model evaluation platform as a first step in translating the EU AI Act into technical requirements."
Organizations that fail to comply with the AI Act could face fines of either 35 million euros or 7 percent of their global annual turnover.
Jessica Kline for TROIB News
