← Back to Insights
Andrew Coban··2 min read

What an AI Should Never Say About Your Strategic Plan

Getting an AI to answer questions about a strategic plan is easy. Teaching it what not to say is the actual work.

Getting an AI to answer questions about a strategic plan is easy. Teaching it what not to say is the actual work.

We recently added an assistant to BoardK12 dashboards, so a parent can ask "What is the district doing about literacy?" and get a plain-language answer drawn from the plan itself. We test it the way a skeptical board member would, with questions designed to get it to say something the plan doesn't. Most of the rules we built came out of those tests. Here are four things it's built never to say.

"The district is behind schedule." Many strategic plans set a multi-year range, not a pace. If the plan doesn't define what progress should look like by now, an AI has no basis to judge it. Ours reports the percentage and the plan's dates, and leaves the judgment to the people elected to make it.

"Nothing has happened on this goal." A strategy at 0 percent may be scheduled for year three. It may have started and just not been updated yet. Zero percent is a number on a dashboard. It isn't evidence that a meeting, a vote, or a purchase didn't happen.

Anything the dashboard doesn't record. Ask about a budget vote or last Tuesday's board meeting, and it should say plainly that the dashboard doesn't track that, then share what it does know. A confident guess is worse than an honest blank.

"The district remains committed to excellence." Ask for a paragraph for the newsletter and you should get facts from the dashboard, not promises. Commitments belong to the superintendent and the board. Software shouldn't make them on anyone's behalf.

The NIST AI Risk Management Framework (opens in a new tab) names "valid and reliable" and "accountable and transparent" among the characteristics of trustworthy AI. For a public strategic plan, those translate into one test we run on every answer: does it match what a parent would see if they clicked through to the page?

An AI that answers questions about public commitments is making a public statement. It should be as careful as the person who has to stand behind it at the next board meeting.