Malicious AI Models Found on Hugging Face
Hugging Face, a prominent hub for artificial intelligence (AI) and machine learning (ML), recently faced a serious security threat. A cybersecurity research firm uncovered malicious ML models on the platform. These models contained code capable of packaging and distributing malware to users who downloaded them. Researchers identified a sophisticated method called Pickle file serialization that threat actors used to embed harmful software. After the discovery, the researchers promptly reported the issue, leading to the swift removal of the malicious models from Hugging Face.
Understanding the Threat: Pickle File Serialization
The cybersecurity firm ReversingLabs played a crucial role in uncovering the malicious ML models on Hugging Face. They detailed how threat actors exploited the platform, which is widely used by developers and companies to host open-source AI models. The exploit involved a technique known as Pickle file serialization. This method allows ML models to be stored in various data formats for easy sharing and reuse.
Pickle is a Python module designed for serializing and deserializing data. However, it is often regarded as an unsafe format. This is because executing Python code can occur during the deserialization process. In closed environments, Pickle files typically access limited data from trusted sources. Conversely, Hugging Face operates as an open-source platform, making it vulnerable to misuse. Attackers can exploit this openness to hide malware within these files.
During their investigation, ReversingLabs identified two specific models on Hugging Face that contained malicious code. Alarmingly, these models bypassed the platform’s security measures and went undetected. The researchers coined the term “nullifAI” to describe the technique used to insert malware, as it effectively evaded existing protections within the AI community.
How the Malicious Models Evaded Detection
The malicious models discovered by ReversingLabs were stored in PyTorch format, which is essentially a compressed Pickle file. The researchers noted that the models were compressed using the 7z format. This compression method prevented the models from being loaded using PyTorch’s “torch.load()” function. Consequently, Hugging Face’s Picklescan tool, designed to detect unsafe files, was unable to identify the malware.
This evasion technique poses a significant risk. Unsuspecting developers who download these models may unknowingly install malware on their devices. The implications of this are serious, as it could lead to data breaches or system compromises. The researchers reported their findings to the Hugging Face security team on January 20. Remarkably, the platform removed the malicious models within 24 hours of the report.
In response to this incident, Hugging Face has made improvements to its Picklescan tool. These enhancements aim to better identify threats in “broken” Pickle files, thereby increasing the platform’s security. The swift action taken by Hugging Face demonstrates the importance of vigilance in the ever-evolving landscape of cybersecurity.
The Importance of Vigilance in Open-Source Platforms
The discovery of malicious ML models on Hugging Face highlights the critical need for vigilance in open-source platforms. While these platforms foster innovation and collaboration, they also present unique security challenges. Developers and organizations must remain aware of the potential risks associated with downloading and using open-source models.
As the use of AI and ML continues to grow, so does the potential for exploitation by malicious actors. It is essential for users to exercise caution and conduct thorough checks before downloading any models. This includes verifying the source and ensuring that the models have been reviewed for security vulnerabilities.
Moreover, platforms like Hugging Face must continue to enhance their security measures. Regular audits and updates to detection tools are vital in keeping users safe from emerging threats. The incident serves as a reminder that cybersecurity is a shared responsibility. Developers, researchers, and platform providers must work together to create a safer environment for AI and ML innovation.
Observer Voice is the one stop site for National, International news, Editor’s Choice, Art/culture contents, Quotes and much more. We also cover historical contents. Historical contents includes World History, Indian History, and what happened today. The website also covers Entertainment across the India and World.