AI-Driven Forensics at Scale: Lessons from Hugging Face’s Incident Response

Hugging Face’s subsequent investigation is another aspect of the incident that deserves close attention from the security industry.

The attacking agent generated more than 17,000 event records.

The security team had to reconstruct:

• The timeline;
• Indicators of compromise (IOCs);
• The credentials that were affected;
• Lateral-movement paths;
• The distinction between genuine impact and decoy activity;
• The affected clusters and nodes;
• The attacker’s intent;
• Any persistent footholds that remained.

Hugging Face did not rely entirely on humans to analyze the records one by one. Instead, it used an LLM-driven analysis agent to process the complete set of attack logs.

This enabled the team to complete, within hours, an analysis that would normally have taken several days.

This process can be abstracted as follows:

Hugging Face subsequently took several response measures:

• Disabled the abused dataset code-execution paths;
• Removed the attackers’ footholds from affected clusters;
• Rebuilt compromised nodes;
• Revoked and rotated the relevant credentials;
• Expanded the scope of secret rotation;
• Strengthened cluster admission controls;
• Improved high-severity alerting;
• Brought in an external forensics team;
• Reported the incident to law enforcement.

This was a textbook incident response.

Hugging Face initially attempted to use a frontier model through a commercial LLM API to analyze the logs. Unfortunately, the forensic materials contained numerous attack-related execution commands, anomalous payloads embedded in requests, and other similar content. The commercial model’s safety controls—LLM WAFs and guardrails—could not reliably determine whether the person submitting this material was an attacker or an incident responder, so the requests were blocked.

Ultimately, Hugging Face deployed and ran the open-weight GLM-5.2 model on its own infrastructure to complete the forensic analysis. This approach had a second advantage: the credentials contained in the attack data and logs never left the Hugging Face environment.