Cisco Releases Open Source Tool for AI Model Provenance

Summary

Cisco has unveiled the Model Provenance Kit, an open source Python-based toolkit designed to help organizations verify the provenance and integrity of third-party AI models sourced from repositories like HuggingFace. The tool generates unique fingerprints for models using metadata, tokenizer similarity, and weight-level signals, offering compare and scan modes to identify model lineage and detect potential poisoning, vulnerabilities, or bias. This addresses critical supply chain, compliance, and incident response challenges organizations face when deploying unverified AI models across their infrastructure.

Full text

Cisco on Thursday unveiled a new open source tool, named Model Provenance Kit, designed to help organizations address potential issues associated with the use of third-party AI models. Organizations often leverage AI models obtained from model repositories such as HuggingFace, where millions of models are available. While these models can offer many benefits, organizations often don’t track the changes made to them. In addition, although repositories provide guidance on the importance of model cards and metadata, the maintenance work performed by their developers can vary, affecting downstream users. Cisco also pointed out that claims made by model developers — including the model’s source, vulnerabilities, and training biases — are often not verified. These aspects can introduce various security, compliance, and liability issues. For instance, enterprises could end up using models that are poisoned or vulnerable to manipulation. “If unaccounted for, those vulnerabilities can continue to propagate, whether they affect an internal chatbot, an agent application, or a customer-facing tool,” Cisco explained. “Similarly, an enterprise could deploy a model that has biases in its training data that make it a poor choice for its use case or make it susceptible to manipulation.”Advertisement. Scroll to continue reading. “The vulnerabilities are inherited and would persist in generative and agentic applications. Without provenance, organizations have no easy way to trace an incident back to its root cause, and no way to determine which other models in their stack are also affected,” it added. In addition, in case of an incident involving an AI model, response and remediation become more difficult without insight into the model’s lineage. Other issues include licensing and regulatory risks related to government requirements for documenting the use of AI systems, and supply chain integrity risks stemming from organizations’ inability to verify claims made by model developers. The new Model Provenance Kit from Cisco, a Python-based toolkit and command-line interface (CLI), aims to address these issues by generating a ‘fingerprint’ for each model based on “metadata signals, tokenizer similarity, and weight-level identity signals such as embedding geometry, normalization layers, energy profiles, and direct weight comparisons”. The tool has two modes: compare, which enables users to compare two models to identify shared lineage; and scan, which attempts to find the closest lineage for a given model by comparing its fingerprint against a database of fingerprints compiled by Cisco. “As models are continuously fine-tuned, distilled, merged, and repackaged, model files have evolved past static assets. Lineage becomes harder to track and easier to obscure, and answering the question of ‘what are the origins of this model?’ requires more nuanced approaches. Our release of Model Provenance Kit is a step towards providing an evidence-based approach to model provenance,” Cisco said. The open source Model Provenance Kit is available on GitHub. Cisco’s dataset of base model fingerprints is on Hugging Face. Related: Hugging Face, ClawHub Abused for Malware Distribution Related: Why Cybersecurity Must Rethink Defense in the Age of Autonomous Agents Related: Anthropic Unveils Claude Security to Counter AI-Powered Exploit Surge Related: Fresh LiteLLM Vulnerability Exploited Shortly After Disclosure Written By Eduard Kovacs Eduard Kovacs (@EduardKovacs) is senior managing editor at SecurityWeek. He worked as a high school IT teacher before starting a career in journalism in 2011. Eduard holds a bachelor’s degree in industrial informatics and a master’s degree in computer techniques applied in electrical engineering. More from Eduard Kovacs Critical Gemini CLI Flaw Enabled Host Code Execution, Supply Chain AttacksEnOcean SmartServer Flaws Expose Buildings to Remote HackingSandhills Medical Says Ransomware Breach Affects 170,000Hundreds of Internet-Facing VNC Servers Expose ICS/OT38 Vulnerabilities Found in OpenEMR Medical SoftwareCritical GitHub Vulnerability Exposed Millions of RepositoriesVimeo Confirms User and Customer Data BreachRobinhood Vulnerability Exploited for Phishing Attacks Latest News In Other News: Scattered Spider Hacker Arrested, SOC Effectiveness Metrics, NSA Tool Vulnerability Google Adjusts Bug Bounties: Chrome Payouts Drop as Android Rewards Rise Amid AI SurgeTwo US Security Experts Sentenced to Prison for Helping Ransomware GangSophisticated Deep#Door Backdoor Enables Espionage, DisruptionHugging Face, ClawHub Abused for Malware DistributionFBI Warns of Surge in Hacker-Enabled Cargo Theft1,800 Hit in Mini Shai-Hulud Attack on SAP, Lightning, IntercomAnthropic Unveils Claude Security to Counter AI-Powered Exploit Surge Trending Daily Briefing Newsletter Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts. Webinar: A Step-by-Step Approach to AI Governance April 28, 2026 With "Shadow AI" usage becoming prevalent in organizations, learn how to balance the need for rapid experimentation with the rigorous controls required for enterprise-grade deployment. Register Virtual Event: Threat Detection and Incident Response Summit May 20, 2026 Delve into big-picture strategies to reduce attack surfaces, improve patch management, conduct post-incident forensics, and tools and tricks needed in a modern organization. Register People on the MoveChris Sistrunk has been promoted to Practice Leader for Mandiant's OT Security Consulting.Nudge Security has appointed Patrick Dillon as its Chief Revenue Officer.AutoNation has appointed Brian Fricke as Chief Information Security Officer.More People On The MoveExpert Insights The Mythos Moment: Enterprises Must Fight Agents with Agents Only with the right platform and an agentic, AI-driven defense, will enterprises be able to protect themselves in the agentic era. (Etay Maor) Why Cybersecurity Must Rethink Defense in the Age of Autonomous Agents From autonomous code generation to decision-making systems that initiate actions without human intervention, the industry is entering a new phase. (Torsten George) Government Can’t Win the Cyber War Without the Private Sector Securing national resilience now depends on faster, deeper partnerships with the private sector. (Steve Durbin) The Hidden ROI of Visibility: Better Decisions, Better Behavior, Better Security Beyond monitoring and compliance, visibility acts as a powerful deterrent, shaping user behavior, improving collaboration, and enabling more accurate, data-driven security decisions. (Joshua Goldfarb) The New Rules of Engagement: Matching Agentic Attack Speed The cybersecurity response to AI-enabled nation-state threats cannot be incremental. It must be architectural. (Nadir Izrael) Flipboard Reddit Whatsapp Whatsapp Email

Entities