Skip to content
Phase 3, week 20

MLSecOps Pipeline Security

0 of 32 items done. ~5h estimated.

Concept. Weeks 17–19 hardened the running agent — guardrails, input sanitization, sandboxing. Week 20 zooms out to the pipeline that produces it. MLSecOps is DevSecOps extended to the ML lifecycle: the same shift-left discipline applied to a system where the artifacts are models and data, and — increasingly — where the developer writing the code is itself an AI agent. The job this week is to make security a property of the build, not a patch bolted on after deployment.

🎯 Objectives

By the end of this week you can:

The big picture

An ML pipeline widens the classic software supply chain in two directions at once. It adds new artifacts the traditional SDLC never modelled — training datasets, weights, fine-tuning runs — each an integrity target. And it adds a new author: agents that now write a majority of production code. MLSecOps is the practice of closing both gaps with one discipline. The OpenSSF MLSecOps whitepaper names the core problem precisely: software engineers and data scientists collaborate on these systems, but neither discipline alone holds security expertise for the whole pipeline — so datasets, model provenance, and inference all fall through the cracks between them.

🔑 Frame for the week: security is a property you build in, layer by layer — data integrity, then provenance, then signing, then runtime monitoring — not a scan you run at the end. Every artifact that enters the pipeline needs an answer to "where did this come from and can I prove it?"

Three frameworks, three altitudes

These are not competitors — they stack. Use the process framework to sequence work, the organizational framework to assign ownership, and the tooling guide to pick controls.

Framework Altitude What it gives you
Microsoft SDL for AI Process Threat-model-during-design → adversarial-test-during-dev → monitor-in-prod → IR — the lifecycle spine
Google SAIF Organization 6 elements: strengthen foundations, integrate detection/response, automate defenses, standardize platform controls, adaptive controls, map risk to business context
OpenSSF MLSecOps Tooling Layer-by-layer reference architecture mapping SLSA, Sigstore, Scorecard onto datasets, provenance, inference

SAIF's most portable idea is element six — map system risks to business context (Google SAIF). A model-drift alert means nothing until it's tied to the process it degrades; the same logic runs Kiya's own per-agent blast-radius thinking. For a structured path into the discipline, ProtectAI's MLSecOps Foundations certification covers supply-chain attacks, model inversion, and adversarial examples end-to-end.

The agent is an insider — bound its blast radius

The sharpest MLSecOps shift of 2026 is that the author changed. In Anthropic's own pipeline, Claude writes ~80% of merged code, so the security team stopped reviewing individual PRs and started monitoring loops and decision patterns — treating the agent as "a new type of insider threat" (Anthropic). The controls translate directly to any agentic pipeline:

  • Single-purpose identities, least privilege — an agent that finds bugs cannot deploy fixes. Separation of concerns denies any one identity a full kill chain.
  • Hard egress allowlists on coding VMs — the antidote to prompt-injection exfiltration (the same failure class as Week 15's coding-agent CVEs).
  • Agent-to-agent traffic through monitored channels — inter-agent messages are logged like human ones, not trusted implicitly.
  • Tiered review with separate context windows — multiple review agents so they don't share a blindspot; humans still gate "regulated or truly critical code."
  • Every action to a SIEM with its reasoning signal — approvals, tool calls, messages; alerts fire when behaviour deviates and are treated as compromise indicators.

💡 The load-bearing principle: automated review is a different risk, controlled through multiple gates — not a single smart reviewer. Risk-weighted sampling of automated approvals keeps full auditability without a human on every PR (Anthropic).

The provenance layer — prove where every artifact came from

If the agent is the author, provenance is the receipt. SLSA (Supply-chain Levels for Software Artifacts) is the industry-consensus ladder for exactly this — each level raises the bar on how tamper-evident a build is, and MLSecOps applies it to model artifacts, not just binaries:

SLSA level Requirement Defends against
L0 No guarantees —
L1 Provenance exists, describes the build Mistaken/undocumented builds
L2 Signed provenance, hosted build service Tampering after the build
L3 Hardened, non-forgeable provenance A compromised build platform forging provenance

The Week-20 deliverable target is L2 — signed provenance from a hosted build service — because it's the first level that stops post-build tampering without demanding a fully hardened builder. Sigstore supplies the keyless signing (short-lived certs via OIDC — no long-lived private key to leak), and Hugging Face model signing applies the same verification to weights on the Hub so a poisoned model can't silently swap in. Wrap the whole graph — base models, datasets, fine-tunes, dependencies — in an AIBOM (AI Bill of Materials) so the supply chain is enumerable, not assumed.

Detection-as-code and policy-as-code

Push security left into the pipeline as artifacts you version and test. LLMs now accelerate the detection-engineering loop: generate Sigma rules (the generic, vendor-neutral detection format) from threat intel, validate them against emulated attacks, and commit them like code — the defensive mirror of the automated red-teaming from Week 14. Ansible security automation enforces the resulting baselines as policy-as-code across hybrid environments, so a control is a reviewed diff rather than a console click. And because the pipeline definition is itself attack surface, lint it: glsec is a static security linter for .gitlab-ci.yml (the GitLab-CI analog to zizmor) — 81 rules across 8 OWASP CI/CD-SEC categories, offline, SARIF/JSON output, shipping as a Claude Code plugin.

For the code-scanning leg, Claude Security (public beta, Opus-powered) traces data flow across files and modules and generates targeted patches with per-finding confidence/severity/impact — evaluate it as a Phase-3 practitioner tool against Semgrep, CodeQL, and Snyk on AI-specific vulnerability classes rather than adopting it blind.

Watch the model in production — drift is a security signal

A signed, provenanced model still rots. Data drift and concept drift silently degrade a model until its outputs are wrong — or exploitable — and monitoring is how you catch it. The tool landscape splits cleanly by what you optimize for:

Tool Strength Watch-out
Evidently AI OSS, easy entry; KS/Chi-Sq/PSI/Wasserstein tests, handles embeddings Not built for enterprise scale
NannyML Best drift→performance link; estimates perf without labels (CBPE/DLE) Tabular-only
Alibi Detect Multi-modal (tabular/text/image), adversarial detection High compute overhead
WhyLabs Enterprise scale, ~100 ms latency, privacy-first (on-prem, SOC 2/HIPAA) Steeper complexity
Fiddler AI SHAP-based explainability for why drift happened; compliance/XAI Proprietary, costly

🔑 The one rule to carry out of this week: an ML pipeline is only as trustworthy as its least-provenanced artifact and its least-privileged agent. Sign what enters, bound what the agent can do, and monitor what ships — build security in at every layer, because the pile of un-versioned scripts is where the next supply-chain worm lands.

📇 MLSecOps tooling reference

Frameworks, provenance, detection & monitoring tools — click to expand.

Frameworks & process

Provenance & signing

  • SLSA — supply-chain levels L0–L3 for build provenance.
  • Sigstore — keyless signing via short-lived OIDC certs.
  • Hugging Face model signing — integrity/provenance for weights on the Hub.

Detection & policy as code

  • Sigma — vendor-neutral detection rule format; LLM-generatable.
  • Ansible security automation — policy-as-code baselines across hybrid env.
  • glsec — static linter for .gitlab-ci.yml; 81 rules, 8 OWASP CI/CD-SEC categories, Claude Code plugin.
  • Claude Security — Opus-powered codebase scanner + patch generation; data-flow tracing.

Model monitoring

Recommended resources0/23

Sign in to tick items off and track your progress.

Show

📖 Core Path

  • 📄 OpenSSF — MLSecOps Whitepaper — 2025 framework extending DevSecOps to ML pipelines; SLSA, Sigstore, data integrity, model provenance (~3h)
  • 📄 Microsoft SDL for AI — Security Development Lifecycle adapted for AI systems; the process spine
  • 📄 Google SAIF — Secure AI Framework; 6 core elements for organizational AI security
  • 🌐 SLSA Specification — Supply-chain Levels for Software Artifacts; provenance and build integrity (L0–L3)
  • 🔧 Sigstore — Keyless signing for software artifacts; apply to ML model provenance
  • 📄 ML Model Monitoring Tools Comparison — Evidently vs Alibi Detect vs NannyML vs WhyLabs vs Fiddler for drift/latency/compliance/XAI (~2h)

📚 Further Reading

Secure AI Development Lifecycle
Vulnerability Scanning Tools
  • 🔧 Claude Security (public beta) — Opus-powered codebase vulnerability scanner with patch generation; data flow tracing across files/modules; Enterprise first, Team/Max following (May 2026)
Standards & Artifacts
Detection-as-Code

📡 From the Resources feed

  • 🌐 GitLab 19.3 — Agentic SAST resolution + bulk false-positive detection — agentic AppSec lands in the pipeline: bulk SAST false-positive detection + vulnerability-resolution flows, a SAST triage/remediation funnel, and a job-scoped Secrets Manager (LA) with audit logging — the DevSecOps-side view of agents remediating (and needing scoped, auditable secrets). (via X/Twitter trending) 📡
  • 🛡️ Wiz — AI Threat Readiness Pillar 4: detect & contain in real time — where traditional SOC workflows collapse at AI-speed attacks; automated investigation/verdict + runtime forensics with human-in-loop gates; MTTR + analyst-agreement metrics (via vendor blog) 📡
  • 📄 Endor Labs — token economics of AI security agents (12× the bill) — 442-prompt benchmark: evidence-equipped agents burn 6.6M vs 79.5M tokens (91.7% less, 2.8× faster); pre-computed security context is the cost lever (via vendor blog) 📡
  • 📄 How Anthropic secures its AI-native SDLC — Anthropic’s playbook for a pipeline where agents write 80% of code — agent-as-insider-threat, blast-radius boundaries, SIEM logging (shared by 101010) 📡
  • 🔧 glsec — static security linter for .gitlab-ci.yml (GitLab-CI analog to zizmor): 81 rules across 8 OWASP CI/CD-SEC categories, SARIF/JSON output, offline, ships as a Claude Code plugin (via X/Twitter trending) 📡
  • 📄 Wiz — Rethinking Scanning for the AI Era (Agentic Code Security) — three-tier pipeline architecture: deterministic rules → continuous AI reasoning on every PR → targeted agentic pentesting (Atlas + CodeMender), all triaged by the Wiz Security Graph (via vendor blog) 📡
  • 📄 CodeMender — an AI agent for code security (Google DeepMind) — autonomous agent that finds and fixes vulns and proactively hardens code (secure APIs/annotations), combining static/dynamic analysis, fuzzing and SMT solvers; already upstreamed 72 fixes to OSS (incl. libwebp), all human-reviewed before submission (via Kiya discovery) 📡
  • 🌐 Wiz — AI Threat Readiness Pillar 2: Accelerate Patching and Response — auto-routes vulnerability ownership via CODEOWNERS + cloud tags + CMDB + Git blame and uses AI for context-aware remediation, closing the loop so response keeps pace with machine-speed detection (via vendor blog) 📡
  • 🌐 Snyk — Remediation Agent in the CLI: Fix SCA Issues at Scale — AI-powered SCA remediation agent (claude-haiku-4-5 paired with Snyk intelligence) roughly doubles fix rates 23%→45% and lifts critical/high/medium 44%→91% by avoiding naive LLM context gaps (via vendor blog monitor) 📡
  • 🌐 Snyk — How Relay Network Adopted AI Coding Securely — case study: pre-commit hooks + Snyk in Copilot dropped critical-vuln MTTR from weeks to 24h, and the same hook-based enforcement scales to fully agentic coding (MCP servers, Claude Code) (via vendor blog monitor) 📡
  • 📄 Snyk — Benchmarking Secure-and-Functional Remediation (Agent Fix) — 150-sample hidden-unit-test benchmark: out-of-the-box frontier models cluster at 72–75% secure-and-functional fix rate; few-shot injection from a 35k-vuln database lifts Opus 4.6 to 85.4% (Python 64%→88%) — security context beats model size (via vendor blog) 📡
  • 🌐 Google/Mandiant — A Blueprint for AI-Assisted Vulnerability Management — Mandiant's operational guide to putting LLM agents inside vuln-management workflows safely: operational guardrails, risk-based prioritization and human oversight to balance AI velocity against enterprise risk (shared by @brisk200) 📡

Study checklist

↪ See roadmap.md → Phase 3 → Week 20

  • Map a Secure AI Dev Lifecycle (design→dev→prod→IR) and place SDL / SAIF / OpenSSF MLSecOps as complementary layers
  • Apply the agent-as-insider-threat model: least-privilege agent identities, egress allowlist, separation of find-vs-fix, SIEM logging
  • Generate an AIBOM for a Hugging Face model
  • Sign a model artifact with Sigstore + HF model signing; verify provenance
  • Set up SLSA Level 2 (signed provenance, hosted build) for a model artifact
  • Detection-as-code: LLM-generate Sigma rules from threat intel, validate against emulated attacks, commit to CI/CD
  • Implement policy-as-code with Ansible for security baselines
  • Lint the pipeline config itself with glsec (.gitlab-ci.yml)
  • Choose a production model-monitoring tool (Evidently / NannyML / WhyLabs / Fiddler / Alibi Detect) for drift detection

Study notes

Sign in to take notes.