Most IT teams can name every AI tool officially approved for use in their organization. Fewer can name every AI tool actually in use. Employees have been testing chatbots, plugging sensitive data into public models, and building small automations for months, often without anyone in IT or legal knowing about it.
That gap in governance is where the real risk sits. Not in the technology itself, but in the absence of a structure to manage it. Before looking at what a governance framework should contain, it helps to understand why AI needs a different kind of oversight than the systems IT teams are used to managing.
Learn more about the potential risks of AI before adopting more tools: AI’s Double-Edged Sword: Innovation, Risk, and Responsibility
Why AI Governance Needs Its Own Framework
Traditional IT governance structures, or even data governance, was built for systems that behave predictably. A piece of software either works as designed or it doesn’t, and once it passes testing, it tends to stay that way. AI systems don’t follow that pattern.
An AI model can perform well during testing and then produce different results in production as inputs change. It can be updated by a vendor without any visible change on your end. It can also generate outputs that sound confident and are still wrong. None of this fits neatly into a change management process built around fixed software releases.
AI Risks
- Bias in outputs, often inherited from training data the organization has no visibility into
- Model drift, where performance changes over time without any code being touched
- Hallucination, where a system produces false information presented as fact
- Third-party model dependency, where core decisions rely on a vendor’s model that can change without notice
- Data exposure through prompts, where sensitive information is entered into tools without proper safeguards
Each of these needs its own set of controls. That’s the reason AI governance deserves a framework of its own, rather than being folded into existing IT policy as an afterthought.
Core Components of an AI Governance Framework
A workable framework rests on four pillars. Each one answers a different question: what’s allowed, what could go wrong, what’s required by law, and what’s technically enforced.
1. Policy
Policy sets the boundaries before anyone starts building or using an AI tool.
- An acceptable use policy that spells out which AI tools employees can use and for what purposes
- An approval workflow for any new AI use case, so nothing goes live without someone reviewing it first
- Data classification rules that define what information can and cannot be entered into an AI system, including public chatbots
2. Risk
Risk management gives the organization a consistent way to judge how much human oversight a given AI use case needs.
- A risk tiering method, such as low, medium, or high, based on the potential impact of a system and how much autonomy it has
- A risk assessment template that gets applied to every new AI project before deployment
- Monitoring triggers that flag when something needs a second look, such as a sudden drop in accuracy or unusual output patterns
3. Compliance
To ensure compliance, connect AI use back to the regulatory obligations the organization already carries.
- Mapping each AI use case to relevant regulations, such as GDPR, HIPAA, or sector-specific requirements
- Documentation that can hold up during an audit, including records of approvals and risk assessments
- Due diligence on vendors and third-party models, since regulatory responsibility often doesn’t disappear just because the model itself is outsourced
4. Technical Controls
Technical controls are what actually enforce the policy day to day.
- Access controls and logging so there’s a record of who used which AI system and when
- Validation and testing before any model is deployed, not just at launch
- Human review checkpoints for outputs that carry meaningful consequences, plus filtering to catch problematic content before it reaches an end user
Learn how SMS went about building security into the foundation of their internal knowledge base assistant, Adam: Unlocking Knowledge: Adopting AI with Security
How to Structure an AI Governance Committee
Who Should Be in the Room
AI risk doesn’t belong to any single department, so the committee shouldn’t either. A working group made up entirely of IT staff will miss legal exposure. One made up entirely of legal and compliance staff will miss technical risk that isn’t obvious from a policy document.
A functional AI governance committee typically includes:
- An IT or security lead who understands the technical risk of each system
- Legal or compliance staff who can assess regulatory exposure
- A data privacy officer, or equivalent, who understands how data moves through AI tools
- A representative from the business unit requesting or using the AI tool
- An executive sponsor who can make final calls and carry decisions to leadership
Committee Responsibilities
Once formed, the committee needs clear ownership of a defined set of tasks:
- Reviewing and approving new AI use cases before they go live
- Setting policy and updating it as tools and regulations change
- Reviewing incidents or near-misses to identify where controls fell short
- Reporting on the organization’s overall AI risk posture to leadership on a regular basis
Meeting Cadence and Decision Rights
Not every AI request needs a full committee review. A low-risk internal tool, like a writing assistant with no access to sensitive data, can move through a fast-track approval. A customer-facing system that makes decisions about people, such as loan approvals or hiring screens, should go through full committee review before deployment.
Whichever path a request takes, the decision and the reasoning behind it should be documented.
Aligning with the NIST AI Risk Management Framework
The NIST AI Risk Management Framework (NIST AI RMF) gives organizations a shared reference point for structuring AI oversight. It’s built around four functions, and each one maps to work the committee is already doing.
- Govern covers the policies, roles, and committee structure already outlined. It’s the foundation the other three functions sit on.
- Map means identifying every place AI is used across the organization and understanding what’s at stake with each use case.
- Measure involves testing AI systems against defined criteria and monitoring them after deployment, not just before.
- Manage is the response side: acting on identified risks, documenting what was decided, and adjusting controls as circumstances change.
Practical First Steps
Getting started with NIST AI RMF alignment doesn’t require adopting every element at once. A few initial moves cover most of the ground:
- Inventory every AI system currently in use, including tools individual employees have adopted without formal approval
- Assign an owner to each of the four functions so accountability doesn’t get lost between departments
- Build a risk register that tracks each AI use case, its tier, and its current status
- Set a recurring review schedule so the register stays current as tools and regulations change
NIST AI RMF is voluntary, but it’s the standard auditors and business partners are expecting to see referenced. Aligning with it early makes future compliance conversations considerably more straightforward.
Still not sure of the differences in various AI systems? LLM vs. AI: What IT Leaders Need to Know
Where Governance Frameworks Stall (and How to Avoid Them)
Most AI governance policies don’t fail because the framework was wrong; rather, they fail because of how they were built or maintained.
Pitfall: Treating governance as a one-time project.A framework approved once and never revisited falls behind the moment tools or regulations change.
Quick fix: Build a review date into the framework from the start, rather than waiting for a problem to force a revisit. A recurring slot on the committee’s calendar, even quarterly, keeps the framework tied to what’s actually happening across the organization.
Pitfall: Leaving IT or business teams out of the drafting process. Policy written in isolation tends to get worked around rather than followed.
Quick fix: Involve the people who will be using AI tools day to day before finalizing any policy, not after. A short review round with representatives from a few different departments will usually catch friction points that legal or compliance alone wouldn’t notice.
Pitfall: Ignoring AI tools already in use. A framework that only addresses future adoption misses the risk that already exists today.
Quick fix: Run the inventory step honestly and without assuming the answer will match what’s officially approved. Treat what you find as information to act on, not a compliance failure to punish, or employees will stop being forthcoming about what they’re using.
Pitfall: Overlooking vendor and third-party model risk. Responsibility for compliance doesn’t transfer just because the model runs on someone else’s infrastructure.
Quick fix: Ask vendors directly how they handle data retention, model training, and subprocessor access before signing anything. Build these questions into procurement so they’re asked consistently, rather than depending on whoever happens to be leading a given project.
Pitfall: No clear owner after approval. A framework without an accountable owner tends to quietly stop being enforced within a year.
Quick fix: Name a specific person or role responsible for keeping the framework current, separate from the committee as a whole. A committee can set direction, but someone still needs to own the day-to-day work of updating documentation and following up on outstanding actions.
Getting Started: A Simple Rollout Sequence
A governance framework doesn’t need to launch fully formed. A phased sequence keeps the work manageable and gives each step room to be done properly instead of rushed.
1. Inventory Current AI Tools and Use Cases Across Every Department
Start by asking department leads directly what AI tools their teams are using, rather than relying only on what IT has formally approved. Include browser extensions, embedded AI features in existing software, and any custom scripts or automations built in-house.
This step usually surfaces more activity than expected, and that’s the point. You can’t govern what you haven’t identified.
2. Form the Governance Committee and Assign an Executive Sponsor
Pull together the cross-functional group outlined earlier, including IT, legal, compliance, and a business unit representative. The executive sponsor matters here in particular, since they give the committee the authority to enforce decisions rather than just recommend them. Set an initial meeting to agree on scope and decide which use cases need attention first.
3. Draft the AI Acceptable Use Policy and Risk Tiering Model
Keep the first version of the policy simple. Define what data can and cannot be entered into AI tools, which tools are approved for general use, and what triggers a formal review. Pair this with a basic risk tiering model, sorting use cases into categories such as low, medium, and high risk based on their potential impact and level of autonomy.
This doesn’t need to be exhaustive on the first draft; it simply needs to be usable.
4. Run a Pilot Risk Assessment on One Acceptable AI Use Case
Choose a tool or system already in use, ideally one that touches sensitive data or makes decisions that affect people. Apply the risk assessment template to it from start to finish.
This does two things: it tests whether the template actually works in practice, and it gives the committee a real example to refer back to when reviewing future requests.
5. Map the Current State Against the Four NIST AI RMF Functions
Take stock of what the organization already has in place for Govern, Map, Measure, and Manage, and note where the gaps sit. Some organizations will find they already have informal versions of these functions scattered across different teams.
This step is about consolidating that work and identifying what still needs to be built.
6. Set a Review Cadence and Reporting Structure for Leadership
Decide how often the committee meets, how new AI requests get submitted, and how risk information gets reported to leadership. A quarterly review is a reasonable starting point for most organizations, with more frequent check-ins for high-risk use cases.
Without this step, even a well-built framework tends to quietly stop being used once the initial momentum fades.
Develop an AI Governance Framework with Clear Purpose
None of this requires solving every risk on day one. A framework built in stages, with clear ownership at each step, does more to protect an organization than one drafted all at once and left untouched.
What matters is that the work starts, and that someone is accountable for keeping it current as AI use grows.
SMS brings decades of federal-grade compliance and security experience to commercial organizations, helping IT leaders build governance frameworks that are practical to implement and built to hold up under audit.
Whether you’re starting with an inventory of existing AI use, or need a full framework aligned to NIST AI RMF, our team can help you get there without adding to your team’s workload.
