AI Data Governance: A Practical Guide for Australian Businesses

AI data governance is the set of rules, roles and records that decide what data your people and your AI tools may use, for what purpose, and who answers for it. For most Australian small and medium businesses the hard part is not writing a policy. It is knowing which AI tools staff already use and what data goes into them. This guide covers the Australian rules that matter, the controls worth setting up first, and a sequence you can finish in 90 days.

CalibreforumSee Where AI Is Already UsedCalibre Forum’s AI readiness assessment gathers employee insights and produces a scored report with a clear 90-day action plan.Explore the AI readiness assessment

TL;DR

AI data governance covers the data that trains, feeds and comes out of AI tools, plus the people and processes around it. It is wider than a security setting.
In Australia, the Voluntary AI Safety Standard and OAIC privacy guidance are the practical reference points.
The most common gap is shadow AI: staff pasting client or personal information into public tools nobody approved.
Start with visibility (who uses what), then classify data, check vendors, and write rules people can follow.
A 90-day plan beats a 40-page policy that nobody reads.

What AI data governance actually covers
The Australian rules and guidance to know
Why governance fails: shadow AI
Classify your data before you pick rules
Build governance in at procurement, not after
The data-use register: the one document worth having
Our perspective
A 90-day plan you can start this week
How Calibre Forum fits in
Common mistakes to avoid
FAQ
Sources

What AI data governance actually covers

Traditional data governance keeps data accurate, secure and trusted. AI adds three questions: what data went into the model, what data your staff are feeding it today, and what comes out. The Open Data Institute-backed briefing paper notes that AI governance and data governance are distinct but overlap, and that data governance needs to account for AI’s implications.

In a business that buys AI tools rather than builds them, the practical scope is narrower than the textbooks suggest. You are governing:

Inputs: what staff type, paste or upload into AI tools.
Storage and training: whether the vendor keeps that data or uses it to train models.
Outputs: whether results contain personal information, and who checks them.
Records: a list of tools, owners, permitted data and the rationale for each decision.

That last item matters more than it sounds. If a regulator, client or insurer asks how you handle AI, a current register is the quickest honest answer.

The Australian rules and guidance to know

Australia has no single AI statute for private businesses. What you have is existing privacy law applied to AI, plus voluntary standards. Check each of these against your own circumstances.

Privacy Act and OAIC guidance

The OAIC’s guidance on commercially available AI products says privacy obligations apply to personal information entered into an AI system and to AI output that contains personal information. It also advises doing due diligence on a product before adopting it and updating privacy policies and notices to describe your AI use.

On public tools, the OAIC restated its October 2024 advice that organisations covered by the Privacy Act should avoid entering personal information, particularly sensitive information, into publicly available AI tools. Its blog also points to a privacy impact assessment before using a generative AI tool.

There is a date to put in the diary. A law firm’s briefing says that from 10 December 2026, privacy policies must include extra information about automated decision-making where personal information is used by a computer program to make or substantially support decisions that could significantly affect an individual. Whether this applies to you depends on whether you are an APP entity and how you use AI, so confirm it with your privacy adviser.

The Voluntary AI Safety Standard

The Department of Industry, Science and Resources publishes a standard of 10 voluntary guardrails. Guardrail 1 is an accountability process covering governance, internal capability and a regulatory compliance strategy. The standard applies to organisations across the AI supply chain, which includes businesses that simply use third-party tools.

For data, the guardrail on data governance is the one to read closely: it asks you to manage data quality and provenance, and to document usage rights for each AI system, whether built in-house or supplied by a third party.

Three essentials of AI data governance

What Australians expect

The 2025 Australian Community Attitudes to Privacy Survey found that 84% of Australians want more control and choice over how their information is collected and used, and 83% believe companies should seek explicit consent before using their data to train AI models, as reported by the OAIC. Your customers hold these views, which is a commercial reason to govern well even where no law forces you.

Why governance fails: shadow AI

Privacy Commissioner Carly Kind has said that AI products should not be adopted simply because they are available, in the OAIC’s announcement of its AI guidance. The pattern many businesses fall into is the reverse: staff adopt whatever helps them finish work faster, and management finds out later.

This is shadow AI. A salesperson drops a client list into a chatbot to draft emails. A manager uploads a performance review to summarise it. An accountant pastes a spreadsheet to find errors. None of them think they are breaching anything, and software locks rarely catch it, because the data leaves through a browser tab that looks like normal work.

Three things follow:

You cannot govern what you cannot see. A policy written without knowing real usage will describe an imaginary workplace.
Blanket bans push use out of sight. People keep using the tool on a personal phone.
The control that works is a clear, short rule about which data may go into which tool, backed by an approved alternative.

If you want a deeper look at finding unapproved tools, our guide to an AI security assessment for business leaders walks through it.

Classify your data before you pick rules

Rules that say “no sensitive data in AI” fail when nobody agrees what counts as sensitive. Classification fixes that. Keep it to three or four tiers a busy person can remember:

Tier	Examples	Typical AI rule
Public	Published marketing copy, public price lists	Any approved tool
Internal	Meeting notes, draft procedures, internal templates	Approved business-grade tools only
Confidential	Client files, contracts, financials, staff records	Only tools with a vendor review and a named owner
Restricted	Health information, identity documents, other sensitive information	No AI use unless a privacy impact assessment signs it off

This table is an illustration of how to structure tiers, not a legal test. Under the Privacy Act, whether information is personal or sensitive depends on its definition in the Act, so map your tiers to that definition with advice for your sector. Our practical guide to AI data classification shows the tiers in more detail.

A note on de-identification

Teams often assume that removing names makes data safe to paste. The OAIC’s closure of its preliminary inquiries into I-MED, Annalise.ai and Harrison.ai is a reminder that de-identification gets scrutiny. If you rely on it, write down your method and your reasoning before you need to defend it, and do not treat it as a one-off task.

Build governance in at procurement, not after

The biggest structural mistake is treating governance as a downstream check once a tool is live. The Open Data Institute-linked briefing paper argues governance should be built in from design and procurement. The same logic applies to a small business buying software: the cheapest time to ask where data goes is before the contract is signed.

A useful side effect is cleaner data from the start. Even basic hygiene, such as normalizing job titles early before any heavy machine learning is applied to HR records, makes later AI use more reliable and easier to audit.

Before approving any AI tool, ask the vendor and record the answers:

Is our input stored, and for how long?
Is it used to train models, and can we opt out?
Where is data processed, and which subprocessors touch it?
Can we delete data and export logs?
Which security controls apply, and how do they map to our own?

For a one-page version, use our AI vendor risk checklist.

Connect AI to your security baseline

If you use the Essential Eight as your security baseline, connect AI tools to it. The Voluntary AI Safety Standard asks you to document security controls for each AI system, including third-party ones. An AI tool that sits outside your normal access and patching routines is just a new route to the same risks.

The data-use register: the one document worth having

If you build only one artifact, build a register. One row per AI tool, with these columns:

Tool name and vendor
Business owner
Purpose (what it is approved for)
Data tiers permitted and prohibited
Vendor review date and outcome
Whether personal information is involved, and whether a privacy impact assessment was done
Review date

A register tells staff what is allowed task by task, which is far more effective than trying to police behavior after the fact. It also gives you an answer when a client asks “do you use AI on our data?”

Our perspective

We built Calibre Forum because the first problem in nearly every Australian and New Zealand business is not a missing policy. It is missing visibility. Management usually believes AI use is low and limited to a few enthusiasts. Staff, asked privately, tell a different story.

Our perspective — overview diagram

That is why our assessment starts with an anonymous survey of every staff member rather than software installed on devices. People answer honestly when they know the answers are not tied to them, and honest answers are the raw material of any workable governance. We do not collect individual data, and we do not make compliance or legal claims: the report maps usage and risk, and your legal and privacy advisers decide what the law requires of you.

Our view is that governance should be a short, living loop: see usage, classify the data, set rules, review every quarter. Anything heavier tends to be filed and forgotten. You can read how we handle staff responses on our security and privacy page.

— Anthony

A 90-day plan you can start this week

Days 1 to 30: see

Ask every team what AI tools they use, anonymously if you can. Expect surprises.
List the tools, accounts (personal or business) and the data types involved.
Name one owner for AI data governance. In a small firm this is often the operations lead or IT manager.

Days 31 to 60: decide

Adopt a three or four tier data classification.
Run vendor checks on the tools in heavy use and approve, restrict or replace them.
Complete a privacy impact assessment for any tool touching personal information.
Update your privacy policy and collection notices to cover AI use.

Days 61 to 90: embed

Publish the register and a one-page rule set.
Train staff with real examples from your own workplace.
Set a quarterly review, and repeat the staff survey to measure change.

How Calibre Forum fits in

Our assessment turns the first 30 days into a single exercise. Staff complete a structured anonymous survey, and we combine the responses into a scored report that maps current tool use, strengths and risks, and gives a tailored 90-day action plan with concrete operational and security recommendations. The report costs A$1,990 one-off. If you want quarterly repeats to track progress, the annual membership is A$2,388 per year.

Sector detail matters too, since a medical clinic and a marketing agency hold very different data. We have tailored pages for law firms, medical clinics and recruitment agencies, among others.

Common mistakes to avoid

Buying security software and calling it governance. Tools help, but the most common failure is a person pasting data into the wrong place.
Writing a policy before measuring usage. You end up governing the workplace you imagine.
Banning everything. Staff route around bans.
Treating de-identification as automatic. Document method and reasoning.
One-off effort. Tools and staff habits change every quarter, so the register must too.
Ignoring data you already hold. Old, unclassified files are what staff will paste first.

FAQ

What is the difference between data governance and AI governance?

Data governance keeps data accurate, secure and properly used. AI governance covers how AI systems behave, including fairness, transparency and accountability. They overlap heavily, and for a business using third-party tools, the data side (what goes in and out) is usually where the first controls are needed.

Do Australian businesses have to follow the Voluntary AI Safety Standard?

No, it is voluntary. It is still a useful benchmark because it sets out 10 guardrails, including accountability and data governance, and it applies across the AI supply chain. Existing obligations such as the Privacy Act apply regardless, so check them with a privacy adviser.

Can staff use ChatGPT or similar tools with client data?

The OAIC advises organisations subject to the Privacy Act to avoid entering personal information, especially sensitive information, into publicly available AI tools. If a business-grade tool is approved after a vendor review and privacy impact assessment, your register should say exactly which data it may handle.

Where should a small business start?

Start with visibility: find out which AI tools people use and what data goes in. Then classify your data, review the main vendors and publish a short register and rule set. Repeating the exercise each quarter keeps it current.