What Compliance Problems Arise when AI is Writing Your Policies?- Episode 231

On this episode of Compliance Unfiltered, AI can speed up policy drafting, but it can also create hidden compliance risk when no one validates the result. Listen, as Todd Coshow and Adam Goslin discuss why AI-generated policies often fail in audits and incident reviews, and how to use AI for drafting without losing accountability.

Episode Transcript:

Today, Adam, we’re gonna talk about what compliance problems arise when AI is writing your policy.

So if AI wrote your security policy and no one really reviewed it, whose policy is it?

Adam Goslin:
I love the compliance problems. That was good.

Before I go there, every time that you’re like, “Hey, tell a few friends that may be interested in what we’re doing here,” I always had that thing. I don’t remember what the frick commercial it was back in the day, but it’s like, “And they too tell two friends, and they tell two friends, and so on and so on.”

I forget what the hell the topic was. It was like some PSA or something.

Todd Coshow:
Sounds like a pyramid scheme. I don’t know.

Adam Goslin:
Nah. I’m gonna be forced to go figure that out now.

Anyway, if you just go jam it into AI and it spits out a policy and nobody’s taken a look at it, whose policy is it?

AI is becoming a good productive tool for compliance teams.

It isn’t a problem to have AI go take a whack at the first draft.

The problem is, and quite honestly, this is the same issue that organizations have had now for some time.

For some time you’ve been able to go out to whatever, I’m just gonna make it up, like www.writemycompliancepolicyforme.com, and you too can fill in blanks and get some steaming pile of garbage as a policy.

Some people literally do.

I’ve been doing engagements, and we’re going through the policy for the annual review, etc., and there’s literally placeholders left in the damn policy for where things that they should have filled in, obviously.

They didn’t do anything other than take a half-hearted whack at find and replace a couple of times and called it a policy.

It’s really no different.

When you’ve got AI blasting out the first draft and then you just go ahead and put your signature on the dotted line, it’s not terribly useful.

Quite honestly, one of the biggest problems that poses when you’re trying to go through your engagement, the relationship between people going through compliance and the folks that are gonna be assessing compliance, it’s one where there is a certain amount of built-up trust that happens between those parties.

There is absolutely no better way to erode any notion of trust building when you’re just serving up what usually is one of the earliest things.

Normally when you go and sit down, you go through the overview of the company and what is the scope, and the assessor’s asking all sorts of questions so they can get their arms around it.

But the very first thing that they’re actually looking at generally is the policies.

Do you wanna start it off on completely the wrong foot?

Go have AI write the initial draft and put your signature on the bottom line, or go grab a template policy and you haven’t really reviewed it.

Policies are more than just well-written language.

It’s literally a written commitment about how the organization is actually operating.

If leadership is signing off on policies that aren’t reflective of reality, aren’t reflective of coverage for the various standards that you’re ostensibly going up against, if it isn’t reflecting how you’re actually doing what you’re doing, etc., you’re creating compliance risk in advance of your assessor walking in to bat you over the head.

It’s really not a good look when you’re sitting there at your annual assessment or onsite and running square into that wall.