← All work

CI/CD governance for AI objects

CI/CD hardening owner, supported by the framework owner and mentors · Senior engineer's AI governance framework, for enterprise client adoption · 2026 · Updated September 2026

Hardened an AI governance framework for Snowflake semantic views and Cortex agents: a CI drift guard, a Snowsight-to-git capture step, and a human approval gate before irreversible production deploys.

Outcome Framework adopted by multiple organisations; hardening changes merged upstream.

  • MLOps
  • CI/CD
  • GitOps
  • Snowflake Cortex
  • Governance
  • Snowflake semantic views
  • Cortex agents
  • GitHub Actions
  • Python
  • Next.js / MUI dashboard

Context

A senior engineer’s governance framework managed Snowflake semantic views and Cortex agents, and it needed to be production-ready before enterprise client adoption. Two problems blocked that: the monitoring dashboard leaked raw database errors and lacked error handling, and live AI objects could silently drift from their version-controlled definitions, so an evaluation could be running against something that no longer matched git.

My role and ownership

I owned the hardening work on an existing framework, with support from its owner, colleagues and mentors. My changes were reviewed and merged upstream into the framework owner’s repository. I focused on the safety and governance gaps that stood between the framework and client adoption.

Technical approach

  • Built a CI drift guard that fails the build when a live Snowflake semantic view or Cortex agent diverges from its version-controlled definition, preventing stale evaluations and accidental overwrites of un-captured changes.
  • Closed the Snowsight-to-git loop with a capture step: a native YAML round-trip for semantic views and a spec export for agents, so changes made in the console could be brought back under version control.
  • Added a manual approval gate before irreversible production agent deploys, so a human signs off before a destructive change ships.
  • Hardened the monitoring dashboard (Next.js / MUI): sanitised leaked database errors, added loading and error boundaries, and corrected severity ranking ahead of client handover.

Tradeoffs

The key decision was how strict the deploy gate should be: a hard-fail CI gate, a warn-only check, or a manual approval. A hard fail maximises safety but can block legitimate releases; warn-only is friendly but easy to ignore. I evaluated the three and recommended a human approval gate before irreversible production deploys, which balances governance with release velocity: routine changes flow, but anything destructive needs a deliberate sign-off.

Outcome

The framework was adopted by multiple organisations. Git became the source of truth for AI objects across dev and prod, and the hardening changes were reviewed and merged upstream into the owner’s repository.

Reflection

Most “AI in production” failures I have seen are not model failures, they are governance failures: drift, silent overwrites, unclear deploy safety. Working on this sharpened how I think about treating AI objects like any other production artifact, with version control, evaluation, and deliberate deploy gates. If I extended it, I would add automated evaluation runs triggered by the drift guard so quality regressions are caught at the same moment drift is.