A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.

Simple pytorch implementation of FGSM and I-FGSM

Hack AI/ML applications — CTF challenges for model attacks, LLMs and AI Agent exploitation.

pretrained BERT model for cyber security text, learned CyberSecurity Knowledge

Code for our USENIX Security 2021 paper -- CADE: Detecting and Explaining Concept Drift Samples for Security Applications

Reading list for adversarial perspective and robustness in deep reinforcement learning.

Adversarial Machine Learning (AML) Capture the Flag (CTF)

The Anti-Virus for AI Artifacts & RAG Firewall. A static analysis tool scanning Models and Notebooks for RCE, Datasets and RAG docs for Data Poisoning, PII, and Prompt Injections. Secure your AI Supply Chain.

AI SBOM: AI Software Bill of Materials - The Supply Chain for Artificial Intelligence

Train AI (Keras + Tensorflow) to defend apps with Django REST Framework + Celery + Swagger + JWT - deploys to Kubernetes and OpenShift Container Platform

AI Firewall & LLM security toolkit - protect your AI applications from prompt injection, jailbreaks, PII leakage, and adversarial attacks

Do you want to learn AI Security but don't know where to start ? Take a look at this map.

Hands-on lessons for attacking and defending AI systems, starting with the OWASP Top 10 for LLM Applications.

Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs (ACM CCS'21)

Open-source prompt injection attack console. Test AI security by firing categorized attacks at any endpoint.

Test and evaluate Large Language Models against prompt injections, jailbreaks, and adversarial attacks with a web-based interactive lab.

Datasets for training deep neural networks to defend software applications

A curated bilingual list of cryptanalytic neural-network model extraction papers, code, taxonomies, and open problems.

Comprehensive, auto-updating literature review of GenAI & LLM security research, standards, tools, and resources. 100+ curated entries with interactive webapp.

Summary of the presentation on Real and Stealthy Attacks on State-of-the-Art Face Recognition Systems at the Seminar: Machine Learning in Cyber-security at FU Berlin