An AI security assessment that focuses on people maps how your staff actually use AI, flags where sensitive data may be exposed, and ends in a scored report with a prioritized action plan. Professionals Australia’s 2025 survey found that privacy and data integrity were the top concern for 65% of respondents, and the National AI Centre says to start with your people. Our approach at Calibre Forum follows that logic.
TL;DR:
- Assessments map approved and unapproved tools, embedded AI features, sensitive data use, policy gaps, role specific training, monitoring, and incident response readiness.
- Invite every staff member to an anonymous survey, name an executive owner, and explain what is collected and how results will guide support.
- Use the first two weeks for access limits and approved tools, weeks three to six for training and policy, then monitoring through week twelve.
- A staff usage assessment does not replace a technical audit: it identifies real workplace practices, while the technical review tests systems and models.
CalibreforumSee How Your Team Uses AICalibre Forum’s AI readiness assessment gathers employee insights and provides a scored report with a 90-day action plan.Explore the AI readiness assessment
Table of Contents
- What does an organization-wide AI security assessment check?
- How do you run an assessment that captures real staff practice?
- How do you turn results into a 90-day action plan?
- What pitfalls undermine an AI assessment?
- Why start with people before technology?
- Our AI readiness assessment: what you get and how to start
- FAQ
- Sources
What does an organization-wide AI security assessment check?
This kind of assessment looks at how people use AI day to day, not at the internals of a model. It covers staff tool usage, shadow AI (tools used without approval), workflows, policies, training, monitoring and incident readiness. A model audit asks whether a system is technically sound. A staff-usage assessment asks what your team is really doing with it.
A solid assessment typically covers:
- An anonymous usage survey of all staff
- An inventory of AI tools and AI features inside existing software
- The kinds of sensitive data staff paste into those tools
- Alignment between daily practice and written policy
- Training gaps by role or team
- Logging and monitoring of AI activity
- Readiness to respond when an AI-related incident occurs
The Australian Cyber Security Centre’s guidance backs several of these items, including privacy checks, limited privileges, logging and including AI in incident response planning.
How do you run an assessment that captures real staff practice?
The sequence below works whether you run it yourself or commission it. Expect the first pass to take a handful of weeks, depending on your size.
- Plan and assign accountability. Name an owner before you start. The National AI Centre’s implementation guidance recommends clear accountability and an up-to-date AI inventory.
- Design the anonymous survey. Ask which tools people use, what data goes in, how often and for what purpose. Honesty depends on psychological safety, which the National AI Centre’s advice on unapproved AI use treats as the route from shadow AI to managed use.
- Inventory what is already installed. Scan existing software for embedded AI features and map them to workflows. OVIC’s guidance on enterprise generative AI stresses reviewing access controls and security maturity before rollout.
- Triage by impact and sensitivity. Rank findings against your organization’s risk tolerance, so a tool touching client records outranks one drafting social posts.
- Score and visualize. Combine survey answers and inventory results into one score and a heat map by team or workflow.
- Deliver the outputs. Expect a scored dashboard, prioritized actions with owners, a quick-wins list and monitoring recommendations.
A workable timeline depends on the organization’s size and complexity, including the time needed for the survey, inventory, analysis and report.
Pro Tip: Announce the survey with a short note from a senior leader explaining that the goal is to support productivity, not to catch anyone out.
How do you turn results into a 90-day action plan?
Read scores as signals for where to act first. A low score in a team means little visibility or weak controls. A high-risk flag, such as client data entered into a public chatbot or widespread shadow AI, deserves attention before anything else.
Prioritize in this order: immediate controls, targeted training, policy changes, then monitoring improvements. A sample plan could sequence actions like this:
- Weeks 0–2: Quick controls such as access limits and a short list of approved tools
- Weeks 3–6: Targeted training for the highest-risk teams and a plain-language policy rollout
- Weeks 6–12: Monitoring improvements and a repeat survey to measure change
- End of day 90: Reassess scores and set the next quarter’s priorities
Track a few measurable KPIs: the share of staff who completed training, the drop in risky tool entries between surveys, and the completion rate of high-priority actions. The Governance Institute’s survey report points to training gaps and trouble measuring return on AI, so these simple KPIs matter.
What pitfalls undermine an AI assessment?
Most failures are cultural, not technical. These are the ones we see most:
- Framing it as surveillance. Governance Institute guidance says to present assessments as productivity support, because staff who fear repercussions underreport.
- Skipping the explanation. Tell people what is collected, what is not, and how results will be used.
- No owners or deadlines. A report without named owners tends to sit in a drawer.
- One-and-done thinking. Updates to ordinary software can switch on AI features nobody noticed, so recheck your inventory regularly.
Pro Tip: Share one anonymous, positive finding with staff after the survey, such as a clever use case, to show that honesty gets rewarded.
Why start with people before technology?
In my view, most organizations audit the wrong thing first. Teams spend months reviewing vendors and models while the real exposure sits in what staff paste into a browser tab on a Tuesday afternoon. Visibility comes first, because you cannot control what you cannot see, and controls built without staff input push people toward workarounds. A people-first assessment lets you apply proportionate controls and train where it counts, which keeps both security and productivity intact.
— Anthony
Our AI readiness assessment: what you get and how to start
Calibre Forum’s AI readiness assessment surveys every staff member anonymously, then combines the answers into one scored report with a prioritized action plan. We install no software and collect no individual data, so people answer honestly and you get an organization-wide map of tools and workflows. It suits small and mid-sized teams that want visibility without intrusive tooling.
You can review what the report contains, look at a sample report, see how the process works, and check pricing. A single report costs A$1990 one-off, and quarterly assessments are available with a membership at A$2388 per year. Start with the pricing page to choose.
FAQ
How long does an AI security assessment take?
The time an organization-wide assessment takes depends on its size and complexity, from the survey through to the final report. Larger or more complex organizations may need longer for inventory and analysis.
Who should be involved in the assessment?
Every staff member should be invited to the anonymous survey, with a named executive owner and IT lead running it. National AI Centre guidance stresses involving staff and consulting stakeholders.
What is shadow AI and why does it matter?
Shadow AI is any AI tool staff use without approval. It matters because sensitive data can flow into public tools unseen, and banning it often drives it further out of sight.
Does an AI security assessment replace a technical audit?
No, they answer different questions. A staff-usage assessment shows real practice and gaps, while a technical audit tests systems and models, and the two work best together.
Sources
- Itbrief
- Bring your people along | National AI Centre
- Engaging with artificial intelligence | CGA
- Guidance for AI adoption: implementation guidance | National AI Centre
Recommended
- AI Readiness Assessment for Managed Service Providers
- AI Readiness Assessment for Manufacturers
- AI Readiness Assessment for Engineering Consultancies
- AI Readiness Assessment for Accounting Firms
Made with BabyLoveGrowth’s AI tools
General information only. Articles on this blog are written with the help of AI writing tools and published by Calibre Forum. They are not legal, insurance, tax or financial advice, so check anything important with a qualified adviser.