Choosing AI software and connecting it to your systems
Summary
This article shows you how to choose AI software and connect it to the systems you already run: a map of those systems and who owns each, a short intake, written questions for the vendor, and the rules before AI writes into a core system. It is one of the Putting AI to Work series for Australian businesses and organisations, and the other articles in the series are mapped below. It is written for small, medium and large organisations alike.
Skip to the AI Software Intake and Vendor Sheet ↓
Background
Choosing AI software well starts with a map of your systems: what each is the source of truth for, and who owns it. Then a short intake that tries what you already pay for first, written questions to the vendor, and a yearly review of every subscription. Before AI writes into your accounting system, your customer relationship management (CRM) system, your enterprise resource planning (ERP) system or your online store, six rules must be true. And when work moves out of a person’s role into a system, the named owner of the process signs off, never the system.
We put the six federal documents that tell an Australian business what to check before adopting AI or cloud software side by side. Between them they ask nine things of a vendor. Only one asks who owns your data. None asks, before you sign, whether you can get all of it back out. None mentions a System and Organization Controls 2 (SOC 2) report, the assurance report most software vendors hold. The Australian Signals Directorate (ASD) names ISO 27001 and, for cloud, an Information Security Registered Assessors Program (IRAP) assessment, and the vendor question below accepts any of the three. Where they name an accountable person, it is one per AI system, one for the whole organisation, or a person checking an output. None names the owner of the business process the AI now runs. Once the work has left anyone’s job, that owner is where accountability has to sit.
The staff AI policy template and How to write an AI policy cover the staff AI policy, which this piece sits under.
You do not need every document in place to begin. The staff AI policy, guide 2a, is the place to start, and the rest develops out from it at the pace AI itself moves: a document is written when the organisation reaches the question it answers, not before.
Checked against the OAIC, the Australian Signals Directorate’s Australian Cyber Security Centre, the National AI Centre, the Department of Industry, ASIC, the Federal Register of Legislation, ISO, the AICPA and one vendor’s own terms on 18 October 2026; the sign-off section was re-checked against the Office of Parliamentary Counsel, the Digital Transformation Agency, ASIC, APRA, the Federal Court, the Chief Justice of NSW and the OAIC on 5 November 2026 (notes 28 to 35). This covers Australian businesses and organisations of any size that buy or connect software, not businesses that build it for sale. It names the law and its section so you can check it; it is not legal advice. We re-check it quarterly.
Methodology
A long article made quickly with AI can be slop, and the risk is real. We would rather say how this one was made than have you guess.
- The questions came first. The issues this article addresses are ones the author has worked through with clients facing them, in his consulting practice.
- AI did the research and the first draft. Claude Code, running Claude Fable 5.1, read the primary sources and drafted the article to the house standard: every claim tied to a source, and every source opened by a person before the article leaves draft. Where an article says "we read" a document, or all 121 of something, the model read every one of them in full and did the counting; the author checks the method and a sample, not every page.
- The author did the first review. The author read the draft end to end, cut what did not hold, and edited the body.
- Other AI tools checked it. A second and third model read each article for errors and for one model's blind spots. This article has not had that check yet.
- Specialists review it last. Reviewers across technology, law and human resources read the final draft. This article has not had its specialist review yet, and no law firm has reviewed it.
Why use AI at all, given the risk? Because at the rate AI is moving it would be remiss not to use the top frontier models to synthesise the current state of play, so that a business decides from the most up-to-date information there is. Feedback and corrections are welcome, and so are volunteers, law firms among them, who want to make this the best resource available to Australian businesses: write to the editor.
Six federal documents ask nine questions, and none asks them all
Six documents published by federal agencies tell an Australian business what to check before it takes on AI or cloud software: the guidance from the Office of the Australian Information Commissioner (OAIC) on commercially available AI products,1 three pieces from the ASD’s Australian Cyber Security Centre (its guide to AI for small business,2 its small business cyber security handbook,34 and its cloud shared responsibility guidance for small and medium businesses5), the Guidance for AI Adoption from the National AI Centre (NAIC),67 and the Voluntary AI Safety Standard, which that guidance updated.8
We read each in full (all fourteen chapters of the handbook) and recorded whether it asks the vendor question in each row. “Partial” means the document raises the subject but not as a question to put to the vendor.
| The question for the vendor | OAIC, AI products (Oct 2024) | ASD, AI for small business (Jan 2026) | ASD, small business handbook (Oct 2026) | ASD, cloud responsibility (Oct 2025) | NAIC, Guidance for AI Adoption (Oct 2025) | Voluntary AI Safety Standard (2024) |
|---|---|---|---|---|---|---|
| 1. Will you train on, or be able to see, what we put in? | Yes | Yes | Yes | No | Partial | Partial |
| 2. Where is our data stored and processed? | Yes | Yes | Yes | Yes | No | No |
| 3. Who owns the data? | No | Yes | No | No | No | No |
| 4. Has it been tested for our use? Show us | Yes | No | Partial | No | Yes | Yes |
| 5. What independent security assurance do you hold? | Partial | Yes, names ISO 27001 | Yes, names ISO 27001 | Yes, names IRAP | No | No |
| 6. How and when will you tell us about a breach? | No | Yes | No | Partial | Yes | Partial |
| 7. What will it be able to reach and change in our systems? | Yes | No | Yes | Partial | No | No |
| 8. Who is responsible for what, you or us? | No | No | No | Yes | Yes | Yes |
| 9. Can we get all our data out when we leave? | No | No | No | No | Partial | No |
Three things stand out from the count.
Ownership of the data is asked once. The ASD’s checklist for small business is the only one with the line “I know who owns the data: my business or the AI vendor”, and it also tells you to “review the AI vendor’s terms and conditions related to data ownership, protection, usage and storage”.2
Getting your data out is never asked at the point of purchase. The NAIC asks you to “Determine a method to extract data and for the return or deletion of assets”, but only in its practice on decommissioning, when you are already leaving.7 The ASD’s cloud guidance asks about backups and restores, which is about losing data, not leaving with it.5 The exit question does appear in a seventh document, the OAIC’s guideline on cross-border disclosure, which asks “whether the personal information can be retrieved or permanently deleted by the entity when no longer required or at the end of the contract”.9
Certifications are named by the ASD alone. Its AI guide gives ISO 27001 as an example of “a recognised cyber security compliance framework”.2 Its handbook says “Select vendors with compliance certifications where possible”.4 Its cloud guidance says “ask for their IRAP assessment”.5 None of the six mentions SOC 2. The NAIC guidance and the safety standard say they align with ISO/IEC 42001, the AI management system standard,78 but neither tells you to ask a vendor for it.
No single federal document carries the full list, so the nine rows, below, are yours to use.
Start with a map: what each system is the source of truth for, and who owns it
You can connect a new tool safely only when you know what it connects to. The map is one table, kept where everyone can see it, with a row for each system: the accounting system, the CRM, the ERP or job management system, the online store, email and documents, payroll and HR, and the smaller tools around them. The map has its own guide, The systems map, which takes you through building and keeping one.
| Column | What goes in it |
|---|---|
| System | Its name and plan |
| Source of truth for | The records it is the official home of: invoices, customers, stock, prices, staff records. Each kind of record has one home |
| Owner | The person who knows when it is wrong and decides about it, not necessarily the person who logs in most |
| Administrator | Who adds and removes users and connections |
| Personal information | Whether it holds information about customers or staff, and what kind |
| Where the data is stored | The country, from the vendor’s terms |
| AI features | Which are switched on, and whether they read or write |
| How other tools reach it | Export, an application programming interface (API), a connector (a ready-made link between this system and another tool), or not at all |
| Renewal | The date, the notice period, and what it costs a year |
The first governance question the Australian Securities and Investments Commission (ASIC) puts to its licensees (the financial services and credit businesses it licenses) is whether you know where AI is used in your organisation.10 The NAIC’s AI register is meant to capture every AI system, including features embedded in ordinary software, and to cut down shadow use, meaning AI tools used without the organisation knowing.11 Its columns are in What AI reporting should come to the board. In a business of a few dozen people, a systems map with an AI-features column does that job without a second document.
One official number per measure. A headline number is any figure the organisation steers by and reports upward: a margin, a revenue total, a headcount, an occupancy rate, a waiting time, a case load, a donor count. Once AI makes analysis cheap, anyone can produce one in a minute from whichever system they have open, and two people will bring two different figures to the same meeting. The map records which report is the official source for each headline number, and who owns it.
The map shows where the Privacy Act bites. If the Privacy Act covers you (the test is turnover, and who is covered regardless of turnover is set out in AI regulation in Australia in 2026), the Act applies to every system in the personal information column. You “hold” personal information if you have “possession or control of a record” containing it,12 so records a vendor stores for you are still yours to protect.
The map shows which tools watch staff. An AI tool’s admin console that logs what staff type can be computer surveillance under NSW law. The rule and the state table are in How to write an AI policy.
The intake: five minutes of questions before anyone buys anything
New software, any renewal over a dollar threshold you set, and any AI tool that will see company data go through the same intake. Whoever wants the tool writes a few lines per question. The owner of the AI policy checks it, and the owner or a director decides above the threshold.
First, should you do anything at all? State the problem in one sentence and who has it, how often it happens and how long it takes today (“about four hours a month, across three people”), and what happens if you do nothing for six months. The OAIC: “AI products should not be used simply because they are available.”1
Then try it in this order: use what you have, then buy, then build.
- Use what you already have. Ask whether the accounting system, the CRM, the online store or the office suite already does it, and ask whether the general-purpose assistant on your business plan can do it with a connector to the system that holds the data. A growing share of what used to be a bought product, a quoting letter, the summary of a long thread, a reconciliation across two spreadsheets, the first draft of a report, is now a task for the assistant you already pay for. AI business process automation: where to start shows that much of the first gain is in features you already pay for. The NAIC wants staff to “check the register before deploying, buying or using any AI system to find out if this system (or something similar) is already in use”.11
- Buy. A product from a vendor who runs it. The ASD tells small and medium businesses to “use cloud services where the CSP has a high level of cyber security responsibility” (a CSP is a cloud service provider, the vendor that runs the software and the servers for you), such as a trusted software-as-a-service product “instead of managing a web server and operating system yourself”,5 and its handbook says “For many small businesses, reputable cloud services provide stronger security than self-hosted systems.”13 What you cannot hand over is the risk: “You can’t outsource that risk to a CSP.”5
- Build. A small automation of your own, on accounts the business owns, with a log. It is the right answer when nothing on the market fits, and it means you own the upkeep, the security and the person who knows how it works.
Price the two models apart. A general-purpose assistant on a business plan is a flat price per seat, whatever staff do with it. AI embedded in other software is often metered: the vendor pays its own model supplier per token (a token is a fragment of a word, the unit the model suppliers charge by) and passes the cost on as credits, per-action charges or a usage tier on top of the subscription, so the bill rises with use and is hard to forecast from the price list. Ask the vendor, in writing, which part of its AI is metered and what a month of your expected use costs, and put the answer in the systems map’s renewal column beside the subscription.
Be critical of thin layers. If the AI part of a product is something the assistant you already pay for does, you are paying for the screen. Price it that way.
Ask whether your assistant can get in. The question that decides more and more purchases is the reverse of question 7 below. Question 7 asks what the product can reach in your systems. This one asks whether your general-purpose assistant can reach the product. A vendor that offers an API or a connector of the kind the assistants now ship lets the assistant do the tasks your staff need inside that system, under the access rules below. A vendor that walls its data off leaves the work to its own embedded AI, metered, and to your staff copying between screens. Put it on the intake: can our assistant do the tasks we need in this product, through a connector the vendor supports?
Record the decision in the systems map: what you decided, why, the owner, and when you will look again. Record a decision not to buy too, so the question is not asked again in six months.
The nine questions to put to a vendor, and what a good answer looks like
Send these in writing and keep the answers with the contract. Where a law sits behind the question, it is named.
| Question | A good answer | Source |
|---|---|---|
| 1. Do you train on, or can your staff see, what we put in? | No training on our data, in the terms, not only an FAQ; staff access only for support, and logged | OAIC1 |
| 2. Where is our data stored and processed, and by which subcontractors? | Named countries and a published list of its subprocessors (the other companies it passes our data to) | Australian Privacy Principle (APP) 8, below |
| 3. Who owns the data and the outputs? | We do | ASD2 |
| 4. Has it been tested for our use? | Results for a use like ours, and a trial on our own data | NAIC6 |
| 5. What independent security assurance do you hold? | A current ISO 27001 certificate, a SOC 2 Type 2 report or an IRAP assessment, shown to us, not just claimed | ASD45 |
| 6. How and when will you tell us about a breach? | A contract clause with a time limit and a contact | The Notifiable Data Breaches (NDB) scheme, below |
| 7. What will it be able to reach and change in our systems? | The narrowest access that does the job, revocable by our administrator | ASD3 |
| 8. Who is responsible for what? | A shared-responsibility document for this product (which says what the vendor secures and what is left to us) | ASD5 |
| 9. Can we get all our data out, in a standard format, and have it deleted when we leave? | Yes, self-service, and we tried it during the trial | APP 8.14, below |
Ask two more for yourself: what it costs over two years, all in, against the hours it saves, and whether your own automations can drive it through a documented API or connector. A tool that only a person can click through cannot be connected to anything later.
When the vendor is overseas, APP 8 and s 16C make you answerable for it. APP 8 is the privacy principle on sending personal information overseas, and s 16C is the section of the Privacy Act that makes you liable for what the overseas recipient does with it. Before a business covered by the Privacy Act discloses personal information to a recipient outside Australia, it “must take such steps as are reasonable in the circumstances to ensure that the overseas recipient does not breach” the Australian Privacy Principles. If the recipient does, s 16C treats the act as “done, or engaged in, by the APP entity”, the Act’s name for a business or agency it covers.12
The OAIC says that liability can arise “even where” the business “has taken reasonable steps”, and that the usual reasonable step is “an enforceable contractual arrangement”, including “a data breach response plan which includes a mechanism for notifying the APP entity”.9
Storing data with an overseas cloud provider can instead count as a “use” rather than a disclosure, which puts it outside APP 8, where a binding contract limits the provider to storage and leaves you “effective control”, including the power to retrieve or delete the data at the end.9 Either way, question 9 matters, because the power to get the data back or deleted is what both routes rest on.
APP 11 covers what a vendor can change. A covered business must take reasonable steps to protect personal information “from unauthorised access, modification or disclosure”, and since 11 December 2024 those steps “include technical and organisational measures”.1214 An AI tool that can modify customer records is inside APP 11. The OAIC’s selection checklist asks whether the system “will be integrated into your entity’s systems and have access to its documents”.1
A breach at the vendor is your breach too. When a cloud provider stores your records and you keep contractual control, the OAIC says “both entities hold the information” and “Both will have obligations under the NDB scheme”. Only one needs to notify, but “If none of the entities notify, then all of the entities may be found to have breached the notification requirements”.15
The Act gives a business 30 days from suspecting a breach to assess whether it is one that must be notified (s 26WH(2)).12 So agree in the contract who tells whom, who assesses and who notifies, as the OAIC advises “when entering into service agreements”.15 The ASD: “Business contracts should include clear clauses on data protection, breach notification procedures, and service uptime guarantees.”2
A certificate describes the vendor’s management system, not your use of it. ISO/IEC 27001 “defines requirements an ISMS must meet” (an ISMS is an information security management system).16 ISO/IEC 42001 “specifies requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System”.17
SOC 2 is not a certificate but a CPA firm’s examination report on controls “Relevant to Security, Availability, Processing Integrity, Confidentiality, or Privacy”. The SOC page of the American Institute of Certified Public Accountants (AICPA), as at 18 October 2026, carries a notice that it is “looking into allegations” about one compliance vendor and that SOC services “should be thoroughly evaluated”.18 Read the certificate or report, check its date and scope, and do not treat it as an answer to questions 1, 3 or 9.
Training settings live inside ordinary software too. The ASD notes that whether a provider trains on what you submit “can depend on the configuration settings or the type of subscription”.2 One example, as at 18 October 2026: Slack says it does not train generative AI models on customer data without opt-in, but does use customer data to train its “global models” for features “such as emoji and channel recommendations”, and a workspace that wants out must have its owner email Slack’s customer experience team.19 The assistants’ settings are compared in Claude, ChatGPT, Copilot and Gemini on business plans.
Every subscription gets a yearly review, before the notice period
The OAIC says due diligence on an AI product does not end at purchase, a point AI regulation in Australia in 2026 quotes.1 The ASD tells small businesses to “Periodically check permissions and disable anything that is no longer needed”3 and to review access “especially when staff change roles, leave the business”.20 Once a year, before the renewal notice period opens, the owner of each row on the map answers five questions:
- Is it still the source of truth for anything, or has the business moved around it?
- Who would miss it, and what would they do instead? Could the job move to a system you already have?
- Have the terms changed? New AI features switched on by default, a new training setting, a new country in the subprocessor list.
- Who still has access, including connections from other tools and former staff?
- Can you still get your data out? Run the export.
The Australian Consumer Law covers one-sided renewal terms. A software subscription on the vendor’s standard form is a “small business contract” if, among other things, one party employs fewer than 100 people or has turnover under $10 million.21 An unfair term in it is void, and the Act’s examples of terms that “may be unfair” include one letting one party alone “vary the terms of the contract”, “renew or not renew the contract”, or “unilaterally to vary the characteristics of the goods or services”.21 Whether a term is unfair turns on the tests in s 24. The yearly review is when you read those clauses.
From 1 July 2027, subscriptions must be easier to leave. The Competition and Consumer Amendment (Unfair Trading Practices) Act 2026 requires a supplier offering a subscription to disclose its “renewal, extension or other continuation”, any notice needed to end it, and how to end it (new s 48D(4)).22 For a business subscriber on a standard form contract with fewer than 100 employees or turnover under $10 million, the way to end it must be “easy to find” and “straightforward”, and online if the contract was made online (new ss 48F and 48G(2)).22
The rules apply to contracts made, renewed or varied from that date, and regulations may add exceptions.22 Until then, the notice period in your current terms is the one that counts.
Before AI writes into a core system, six things must be true
A tool that reads your accounting system can give a wrong answer. A tool that writes into it can post two hundred wrong invoices before anyone notices. The actions an AI tool can take in a system form a ladder of risk, and the rules below get stricter with each rung.
| What the tool does | What can go wrong | Who checks, and when |
|---|---|---|
| Reads a system and answers a question | A wrong answer, caught when a person acts on it | The person who asked, before they act on it |
| Drafts something for a person to send or post | A wrong draft, caught before it leaves | The person whose name goes on it, every time |
| Writes an internal record (a contact, a note, a draft invoice, a category) | Wrong records that compound; two hundred of them before anyone looks | The process owner, under written rules, with a log, a preview and small batches |
| Sends outward (an email, an invoice, a quote) or changes what a customer sees (a price, a listing) | A wrong message or price in front of a customer, and a contract or a consumer-law problem behind it | A person confirms each one, or the owner signs rules and reviewers read every batch |
| Pays, deletes, or changes a record that cannot be restored | Money gone, data gone, no way back | A person, every time; the automation only drafts |
The ladder is ours. The rung a tool sits on is decided before it is switched on, written into the rules in point 1 below, and recorded in the systems map’s AI-features column as reads or writes.
Before anything it reads is relied on. The first rung has its own rules, because a wrong answer acted on is a wrong decision. Before a figure or a summary from a tool that reads your systems goes to a customer, the board or a decision, four things are true. It says where it came from, which system, which report and as at what date. It reconciles to the official source for that number on the systems map. A person who knows the data has checked it, under their own name. And the record says it was an AI output, as the OAIC asks.1 A tool that cannot say where a number came from is not relied on for it.
Two Australian primary sources address AI write access directly, both from the ASD. Its small business handbook: “Avoid giving it permission to make significant changes, such as approving transactions, sending communications or modifying business data, without your oversight.”3 Its guidance on agentic AI (AI that carries out a multi-step task on its own, acting inside systems rather than only answering), written with five other countries’ cyber agencies for government, critical infrastructure and industry teams: “Prevent agents from autonomously executing high‑impact actions or outputs without prior human approval”.23
Neither names an accounting system, CRM, ERP or online store. The rules below are ours. The notes show where a source says the same.
Before any AI tool or automation writes into the accounting system, the CRM, the ERP, the online store, a price list, a bank, or anything a customer or supplier sees:
- Written rules on what it may do. Which records, which fields, how many at a time, and what it must hand to a person. The ASD: “Check what apps, systems and data the agent can access before enabling it”, and give it “the minimum access required”.3
- A log a person can read, under the automation’s own identity. What changed, from what to what, when, and at whose request. An automation that acts for the business signs in as itself, not as the person who set it up, so the log says which automation did it and the action is attributed to it, never to a person who was not there. The ASD wants logs with “time, user, system, action, outcome”, protected “from modification or deletion”.24 If the system’s own audit trail cannot show which automation made a change, the automation keeps its own log.
- A person confirms anything outward or irreversible, under their own name. The automation drafts. It does not pay, send, delete or change a price on its own. The NAIC calls for “mandatory human review for high-stakes decisions”.6
- Preview, small batches, read-back. Every change is shown before it is written, first batches are small, and the result is read back and compared with what was intended. If one failed record can leave a batch half-written, the safeguard has to be yours.
- A way to stop it and put things back. The NAIC asks for points “where humans can pause, override, roll back or shut down AI systems if needed”.6 Use a test copy of the system first where one exists, and know whether you can restore a record’s earlier version.5
- The old way always stays open, and is written down as the fallback if the automation stops and as the way to verify what it did. It does not close when the new way is trusted. It is kept, tested at the yearly review, and named in the rules. The NAIC: “Make sure that all of your organisation’s critical functions can continue even if your AI systems malfunction or are being retired.”6
Mark what the machine wrote. For personal information, the OAIC asks: “Do your records clearly indicate where information is the product of an AI output and therefore a probabilistic assessment rather than fact?”1
Give the automation its own identity when it acts for the business. A connector that acts as the person who signed in is fine while one person uses it for their own work. An automation acting for the business, running unattended across several people’s data, or needing to answer “who did what” gets an identity the business owns, with its own log and limits. Every automation the business relies on lives on company accounts, so it keeps running, and stays visible, when the person who set it up leaves.
When the work leaves a role, the process owner signs off, not the system
Most AI policies, including our staff AI policy template, say a person approves anything an AI tool sends, pays or changes, and “the person who approves it is responsible for it”. That works while a person still does the step. Once supplier invoices are read, coded and posted overnight by an automation, nobody’s job includes looking at each one, and the policy’s human sign-off has no obvious signatory.
Our answer is the named owner of the process, never the system, and not by default the person who built the automation. The process is the business outcome (“supplier invoices are paid correctly and on time”, “customer records are accurate”, “online prices match the price list”). Its owner is the person who would answer for that outcome if no AI were involved, usually the manager of the team that used to do the work. The owner signs off in one of two ways:
- Decision by decision. The automation prepares. The owner, or someone they name, approves each item. This is the right mode at low volume, and the only mode for anything that pays money out, goes to a customer or supplier, or significantly affects a person.
- At volume: the rules, plus spot-checks. Where approving each item would become rubber-stamping, the owner signs off in writing the rules the automation follows, the thresholds above which it must stop and ask (a dollar amount, a number of records, a new supplier), and the limits it may never cross. The owner then runs or receives regular spot-checks on a schedule they set, reviews the exceptions, and answers for the outcomes as if they had made each decision.
Either way, the owner’s name and the signed rules go in the systems map beside the automation, with the date of the next review.
What the Australian sources say about who is accountable. None of them uses the phrase “process owner”, and none says what to do when work leaves a role. What they do say points the same way.
- The organisation is accountable, and a person must be named. The federal guidance asks for a senior leader to own AI governance and a named person for every AI system,6 and wants “documented deployment authorisation and rationale from the accountable person for the AI system”.7 Its register template’s “System owner” is “The person accountable for this system”.11 The Voluntary AI Safety Standard puts that accountability on leaders, not on the tool or the vendor.8 The guidance’s own words are in Who is accountable when AI makes the decision.
- Oversight is scaled to the stakes. The NAIC: “Make sure a person oversees your AI system in a way that matches how much autonomy the system has, and how high the stakes are.”6
- Both sign-off modes exist in practice. ASIC’s 2024 review of 624 AI use cases at 23 licensees found some had decided a human would be accountable for each decision where AI was involved, with documents saying so, and others ran periodic checks of models against established controls. It also found arrangements without enough human oversight, and asks licensees who is accountable for AI use and outcomes, at model level and overall.10
- The thresholds are a person’s decision, not the machine’s. The ASD’s agentic AI guidance describes the problem this section is about: when agents work together on a task “such as approving payments or updating records and an erroneous outcome occurs”, it “becomes difficult to determine which component or design choice caused the error”. Its answers: “Ensure decisions about when human approval is required are determined by system designers or operators, not delegated to the agentic AI system”, and “Clearly assign responsibility and accountability for errors or adverse outcomes caused by the system”.23
- When the government automates its own decisions, it names a person by position. The drafting note on automated decision-making from the Office of Parliamentary Counsel (the office that drafts federal legislation), updated 5 March 2026, says that where legislation lets a computer program make a decision, “a natural person must be nominated (by position or title) as having ultimate responsibility for the decision”, usually by providing that the automated decision “is taken to be made by the nominated person”.25 The Digital Transformation Agency’s policy for agencies, version 2.1 of 21 October 2026, requires “an accountable use case owner for each in-scope AI use case”, lets the owner’s actions be delegated “to other suitable staff”, and allows accountability to “be split between roles, for example, between business and technology areas”.26 Both bind Commonwealth agencies, not businesses. They are the closest Australian text to a process owner, and the second one draws the same line between the business role and the technology role that this section does.
- The regulators said it again in 2026. ASIC’s chair told directors on 10 March 2026 that “every board needs to have a conversation about AI use and determine their risk appetite and policies”.27 The letter of 30 April 2026 from the Australian Prudential Regulation Authority (APRA) to banks, insurers and super funds asks for “ownership and accountability across the AI lifecycle, from design and development through to deployment, monitoring and decommissioning” and for “human involvement for high-risk decisions and accountability”.28 Neither reaches an unregulated business of a few dozen people. Both describe the same shape.
Why the process owner and not the system owner. The federal documents attach accountability to “every AI system”. In a business of a few dozen people the two are often one person. When they differ, the system owner keeps the tool running and its access tidy, and the process owner knows when the output is wrong and decides what happens next. An automation that reads email, codes invoices in the accounting system and queues payments touches three systems and one process. The signature belongs to the person who answers for the payments.
A human signature does not change what you must disclose. From 10 December 2026, a business covered by the Privacy Act that arranges for a computer program to make, or substantially shape, a decision that significantly affects a person must say so in its privacy policy, under the new APP 1.7.14
The OAIC’s guidelines say a decision can be in scope “even where a computer program output does not replace the entire decision-making process or is subject to human review”. Its fact sheet of the same month adds that generative AI outputs used in significant decisions “would generally fall within scope of the transparency obligation unless subject to extensive human oversight and control”.2930
A process owner who signs off the refund rules at volume has not taken refunds outside APP 1.7. They have made sure someone answers for them. Decision-by-decision review may, on the fact sheet’s words, be the kind of oversight that takes a decision out of scope. The policy should be written on the stricter reading, which is that the decision stays in scope and is disclosed.
In a company, the directors sit above the process owner. Directors owe a duty of care and diligence under s 180 of the Corporations Act, and ASIC says it extends to the adoption and use of AI. Under s 190, a director who delegates a power stays answerable for it unless they reasonably believed the delegate reliable and competent. Under s 189, a director may rely on information and advice from others, and the people a director may rely on are people, not programs.31
The Federal Court said in March 2026, in a case where no AI was in issue, that “The use of technology may assist comprehension, but it cannot displace judgment. The statutory obligation imposed by s 180(1) remains personal, and it requires informed human judgment.”32
Two months later the Chief Justice of New South Wales, speaking outside court, said a director relying on a delegate under s 190(2) “would need to justify why their trust in a delegate who themselves was relying on AI was reasonable”, and that it “must be extremely doubtful” that a director who simply adopts an AI’s recommendation could rely on the business judgment rule (the defence in s 180(2) for a director who makes a business decision in good faith, having informed themselves about it).33
Neither is a ruling on an automated process. Both say which way a court would look. These duties belong to companies. A sole trader or partnership answers for its own decisions.
What the owner signs: one page per automation. What the process is and its owner’s name. What the automation may do, in which systems, and what it may never do. The thresholds at which it stops and asks a person. The spot-check: how many items, how often, by whom, and where the result is recorded. How to stop it, and who may. What happens if it is down: the old way, and who runs it. When the page is next reviewed. If the owner leaves or changes role, the page is re-signed by the new owner before the automation keeps running.
The AI Software Intake and Vendor Sheet
Copy the sheet below into a document and fill it in once for each tool you consider, including the ones you decide not to buy. It holds the intake questions, the nine questions for the vendor with what a good answer looks like, and the six things that must be true before the tool writes into a core system. Keep the filled-in sheet with the contract, and record the decision in the systems map.
[Business name]: AI software intake and vendor questions
Tool: [name and plan] · Vendor: [name] · Requested by: [name, role] · Checked by: [owner of the AI policy] · Decided by: [owner or director, above the threshold] · Date: [ ]
The intake (benefits and constraints)
- The problem, in one sentence, and who has it: [ ]
- How often it happens and how long it takes today: [about four hours a month, across three people]
- What happens if we do nothing for six months: [ ]
- Does something we already pay for do it? The accounting system, the CRM, the online store or the office suite: [ ]
- Can the general-purpose assistant on our plan (Claude, ChatGPT, Copilot or Gemini) do it instead of the proposed system, with a connector to the system that holds the data? [ ]
- Is this system, or something similar, already in use? [ ]
- Buy or build: [buy: a product from a vendor who runs it / build: our own automation, on accounts the business owns, with a log]
- Is the AI part something the assistant we already pay for does? [ ]
- What it costs over two years, all in, against the hours it saves: [ ]
- Which part of its AI is metered, and what a month of our expected use costs: [credits, per-action charges or a usage tier, in writing from the vendor]
- Can our general-purpose assistant (Claude, ChatGPT, Copilot or Gemini) connect into this product through an API or a connector the vendor supports, and are the actions we need (reading data, writing data) available to it? [ ]
- Can our own automations drive this product through a documented API or connector, with the same actions? [ ]
- The decision, and why: [ ] · Owner: [name, role] · Look again on: [date]
The nine questions for the vendor
Sent in writing on [date]. The answers are filed with the contract.
| # | Question | A good answer | Vendor’s answer | Date checked |
|---|---|---|---|---|
| 1 | Do you train on, or can your staff see, what we put in? | No training on our data, in the terms, not only an FAQ; staff access only for support, and logged | [ ] | [ ] |
| 2 | Where is our data stored and processed, and by which subcontractors? | Named countries and a published subprocessor list | [ ] | [ ] |
| 3 | Who owns the data and the outputs? | We do | [ ] | [ ] |
| 4 | Has it been tested for our use? | Results for a use like ours, and a trial on our own data | [ ] | [ ] |
| 5 | What independent security assurance do you hold? | A current ISO 27001 certificate, a SOC 2 Type 2 report or an IRAP assessment, shown to us, not just claimed | [ ] | [ ] |
| 6 | How and when will you tell us about a breach? | A contract clause with a time limit and a contact | [ ] | [ ] |
| 7 | What will it be able to reach and change in our systems? | The narrowest access that does the job, revocable by our administrator | [ ] | [ ] |
| 8 | Who is responsible for what? | A shared-responsibility document for this product | [ ] | [ ] |
| 9 | Can we get all our data out, in a standard format, and have it deleted when we leave? | Yes, self-service, and we tried it during the trial | [ ] | [ ] |
Before anything it reads is relied on
Before a figure or a summary from the tool goes to a customer, the board or a decision:
- It says where it came from: which system, which report, as at what date. [ ]
- It reconciles to the official source for that number on the systems map. [ ]
- A person who knows the data has checked it, under their own name. [ ]
- The record says it was an AI output. [ ]
Before it writes into a core system
Before the tool or automation writes into the accounting system, the CRM, the ERP, the online store, a price list, a bank, or anything a customer or supplier sees, each of these is true:
- Written rules on what it may do: which records, which fields, how many at a time, and what it must hand to a person. [ ]
- A log a person can read, under the automation’s own identity: what changed, from what to what, when, and at whose request. The automation signs in as itself, and the action is attributed to it, not to a person. [ ]
- A person confirms anything outward or irreversible, under their own name. The automation drafts, and does not pay, send, delete or change a price on its own. [ ]
- Preview, small batches, read-back: every change is shown before it is written, first batches are small, and the result is read back and compared with what was intended. [ ]
- A way to stop it and put things back. [ ]
- The old way always stays open, written down as the fallback if the automation stops and as the way to verify what it did, and tested at the yearly review. [ ]
Process owner: [name, role] · Rules signed on: [date] · Next review: [date]
What changed
This is version 1.0, first published 18 October 2026 and last updated 5 November 2026. The version moves at a major change, such as a law or a piece of guidance the guide rests on changing; a smaller fix moves the updated date only. Earlier versions are kept from the first major change on.
No changes since first publication.
What we could not confirm
- A rule for AI write access to an accounting system, CRM, ERP or online store in those terms. The two ASD documents speak of “business data” and “critical records” generally. The OAIC, NAIC, safety standard and ASIC documents we read say nothing specific. The six write rules are ours.
- Pending privacy law on vendors. The exposure draft Privacy Amendment (Personal Data Protection) Bill 2026 has a schedule headed “Exception for information processors”. It is an unintroduced draft and we do not describe it.
- An Australian source that uses “process owner” for AI accountability. None, as at 5 November 2026. The closest are the Digital Transformation Agency’s “accountable use case owner”, which applies to Commonwealth agencies,26 the federal register template’s “system owner”,11 and ASIC’s description of licensees with “ownership and accountability for AI at a model or business unit level”, which “did not always have a senior executive accountable overall”.10 ASIC reviewed financial services and credit licensees, not small and medium businesses.
- An Australian judgment applying s 189 or s 190 to reliance on an AI system’s output. We found none as at 5 November 2026. The Federal Court’s remarks on AI in ASIC v Bekier were made in a case where no AI was in issue. Section 189 was pleaded but “none of the directors advanced any submission at all concerning s 189”, and s 190 is not discussed.32 The penalty judgment of 17 June 2026 records that one defendant intended to appeal. We found no appeal judgment.
- Which reading of APP 1.7 the OAIC will apply to a human-reviewed decision. Its guidelines and its fact sheet, both of September 2026, read differently on whether review takes a decision out of scope. We quote both above and do not pick.2930
- A government Bill on automated decision-making. The Attorney-General’s Department consulted from November 2024 to January 2025 and says the feedback will “inform the implementation of recommendation 17.1” of the Robodebt Royal Commission. We found no government Bill or exposure draft as at 5 November 2026.34
- Whether any vendor’s renewal or variation clause is unfair. That turns on s 24 and the whole contract.
- The live ISO pages, which refused automated reading. We read Internet Archive copies of 1 June 2026 (ISO 27001) and 6 October 2026 (ISO 42001).
- The count in the table is ours. It records whether each of the six documents asks the question, not how well, as read on 18 October 2026. It leaves out the ASD’s agentic AI guidance, which is written for government, critical infrastructure and industry teams. The NAIC’s PDFs, dated October 2025 on their covers, are marked “Published 05 May 2026” on ai.gov.au. We found no change log.
We will re-check this piece every quarter, and on 10 December 2026 when APP 1.7 to 1.9 commence. When a source changes, the date at the top of the page changes with it.
Notes
-
OAIC, Guidance on privacy and the use of commercially available AI products, published 21 October 2024, updated 17 January 2025: “Top five takeaways”, “Selecting an AI product”, “What are the key privacy risks when using AI?”, and Checklists 1 and 2. Read 18 October 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Artificial intelligence for small business, published 14 January 2026, sections “Data leaks and privacy breaches”, “Supply chain vulnerabilities”, “Implementation example” and “AI cyber security checklist for small businesses”. Read 18 October 2026. ↩ ↩ ↩ ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Small business cyber security handbook: Using AI securely in your business, published 7 October 2026, including “Using AI agents securely in your business”. Read 18 October 2026. ↩ ↩ ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Small business cyber security handbook: Use Secure by Design products and services, “Choose trusted and reliable vendors”. Read 18 October 2026. ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Cloud shared responsibility model: Guidance for individuals and small and medium businesses, published 20 October 2025, sections “Understanding responsibilities”, “Foundational responsibilities”, “Coverage of responsibilities” and “Minimise your cyber security responsibilities”. Read 18 October 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
-
National AI Centre, Guidance for AI Adoption: Foundations, dated October 2025 on its cover and marked “Published 05 May 2026” on ai.gov.au, practices 1.1, 1.3, 1.5, 4.1, 5.1, 6.1, 6.2 and 6.4. Read 18 October 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩
-
National AI Centre, Guidance for AI Adoption: Implementation practices, October 2025, practices 1.1.1, 1.2.1, 3.4.4, 5.1.5 and 6.2.1, and “Our approach” (alignment with ISO/IEC 42001:2023). Read 18 October 2026. ↩ ↩ ↩ ↩
-
Department of Industry, Science and Resources, Voluntary AI Safety Standard: the 10 guardrails, published 5 September 2024, updated 2 December 2025: guardrail 1 (1.1, key concept), guardrail 4 (4.4.3), guardrail 5 (5.1.1), the procurement guidance boxes, and the statement that the guardrails “align with international standards including ISO/IEC 42001:2023”. The standard’s landing page says the Guidance for AI Adoption of 21 October 2025 “evolves the Voluntary AI Safety Standard”. Read 18 October 2026. ↩ ↩ ↩
-
OAIC, Australian Privacy Principles guidelines, Chapter 8: APP 8, updated 3 October 2025, paragraphs 8.14, 8.16 and 8.62. Read 18 October 2026. ↩ ↩ ↩
-
ASIC, Report 798: Beware the gap: Governance arrangements in the face of AI innovation, 29 October 2024: summary (23 licensees, 624 use cases), finding 5 (p 23, “The importance of meaningful human oversight”), finding 6 (p 25), “Licensees must consider their existing regulatory obligations” (p 34) and “AI governance: Questions for licensees” (p 35). Read 18 October 2026. ↩ ↩ ↩
-
National AI Centre, AI systems register, ai.gov.au, and the AI register template (Word version, published 22 April 2026), “Recommended fields”. Read 18 October 2026. ↩ ↩ ↩ ↩
-
Privacy Act 1988 (Cth), compilation 104, compilation date 4 June 2026 (authorised version C2026C00227): s 6(1) (definition of “holds”), s 16C, s 26WH(2), and Schedule 1, APP 8.1, APP 11.1 and APP 11.3. Read 18 October 2026. ↩ ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Small business cyber security handbook: Protecting devices, systems and networks, “Migrate to cloud services” and “Understand your cyber supply chain”. Read 18 October 2026. ↩
-
Privacy and Other Legislation Amendment Act 2024 (Cth), as made, Schedule 1, item 88 (inserting APP 1.7 to 1.9) and item 89 (application); Part 15 commences 10 December 2026. Read 18 October 2026. ↩ ↩
-
OAIC, Part 4: Notifiable Data Breach (NDB) Scheme, updated February 2025, section “Data breaches involving more than one entity”. Read 18 October 2026. ↩ ↩
-
ISO, ISO/IEC 27001:2022 Information security management systems: Requirements, read 18 October 2026 in the Internet Archive copy of 1 June 2026; iso.org refused automated reading of the live page. ↩
-
ISO, ISO/IEC 42001:2023 Information technology: Artificial intelligence: Management system, read 18 October 2026 in the Internet Archive copy of 6 October 2026; iso.org refused automated reading of the live page. ↩
-
AICPA & CIMA, System and Organization Controls: SOC Suite of Services, including the undated notice on the page and the title of the SOC 2 guide. Read 18 October 2026. ↩
-
Slack, Privacy principles: search, learning and artificial intelligence, slack.com, undated page. Read 18 October 2026. This is Slack’s own description of its practice; we cite it as an example of how an opt-out can work, not as a rating of the product. ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Small business cyber security handbook: Restrict and monitor access, “Implement access control”. Read 18 October 2026. ↩
-
Competition and Consumer Act 2010 (Cth), compilation 167, compilation date 16 September 2026 (authorised version C2026C00405, volume 4), Schedule 2 (Australian Consumer Law), ss 23(1) and (4) and 25(d), (e) and (g). Read 18 October 2026. ↩ ↩
-
Competition and Consumer Amendment (Unfair Trading Practices) Act 2026 (Cth), No. 64, 2026, as made, assented to 6 July 2026: s 2 (commencement, 1 July 2027) and Schedule 1, Part 3, inserting ss 48B, 48D, 48F and 48G into the Australian Consumer Law, and item 22 (application, new s 312). Read 18 October 2026. ↩ ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, with CISA, NSA, the Canadian Centre for Cyber Security, NCSC-NZ and NCSC-UK, Careful adoption of agentic AI services, published 1 May 2026, sections “Scope and audience”, “Accountability risks” and “Human in the loop”. Read 18 October 2026. ↩ ↩
-
Australian Signals Directorate, Australian Cyber Security Centre, Small business cyber security handbook: Event logging. Read 18 October 2026. ↩
-
Office of Parliamentary Counsel, Instructor Guidance Note: Automated decision-making, last updated 5 March 2026, paragraphs 4 and 12. Read 5 November 2026. ↩
-
Digital Transformation Agency, Policy for the responsible use of AI in government, Standard for accountability, version 2.1 of 21 October 2026 (version 2.0 in effect from 15 December 2025), “Accountable officials” and “Accountable use case owners”. Read 5 November 2026. ↩ ↩
-
ASIC, Joe Longo, It’s tough being a director, but that doesn’t mean you shouldn’t do it, keynote to the Australian Institute of Company Directors, 10 March 2026. Read 5 November 2026. ↩
-
APRA, Letter to industry: artificial intelligence (AI), 30 April 2026, the list of expectations in the summary. Read 5 November 2026. ↩
-
OAIC, Australian Privacy Principles guidelines, Chapter 1: APP 1, updated 30 September 2026, paragraph 1.41 and Example C (under paragraph 1.44). Read 18 October 2026. ↩ ↩
-
OAIC, APP 1.7 to 1.9 transparency obligation: fact sheet, September 2026, page 2 and Example 3, released with the 30 September 2026 guidelines. Read 5 November 2026. ↩ ↩
-
Corporations Act 2001 (Cth), compilation 149, compilation date 19 September 2026 (authorised version C2026C00432, volume 1): ss 180(1), 189, 190 and 198D(1) and its note 1. Read 18 October 2026. ↩
-
Federal Court of Australia, Australian Securities and Investments Commission v Bekier (Liability Judgment) [2026] FCA 196, Lee J, 5 March 2026, at [393], [394], [425] and [1956]; and ASIC v Bekier (Penalty Judgment) [2026] FCA 756, 17 June 2026. The court’s site refused automated readers on 5 November 2026; both were read in a headless browser and against an Internet Archive copy of the court’s page, and the quoted sentences match in both. ↩ ↩
-
Supreme Court of New South Wales, Chief Justice A S Bell, Harold Ford Memorial Lecture, 21 May 2026, at [62] and [69]. An extrajudicial speech, not a judgment. Read 5 November 2026. ↩
-
Attorney-General’s Department, Use of automated decision-making by government: consultation, “We did”, feedback updated 22 August 2025. Read 5 November 2026. ↩
Sources
- Guidance on privacy and the use of commercially available AI products · OAIC, 18 Oct 2026
- Artificial intelligence for small business · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Small business cyber security handbook: Using AI securely in your business · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Small business cyber security handbook: Use Secure by Design products and services · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Cloud shared responsibility model: Guidance for individuals and small and medium businesses · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Guidance for AI Adoption: Foundations · National AI Centre, 18 Oct 2026
- Guidance for AI Adoption: Implementation practices · National AI Centre, 18 Oct 2026
- Voluntary AI Safety Standard: the 10 guardrails · Department of Industry, Science and Resources, 18 Oct 2026
- AI systems register · National AI Centre, 18 Oct 2026
- Report 798: Beware the gap: Governance arrangements in the face of AI innovation · ASIC, 18 Oct 2026
- Corporations Act 2001, compilation 149 · Federal Register of Legislation, 18 Oct 2026
- Privacy Act 1988, compilation 104 · Federal Register of Legislation, 18 Oct 2026
- Australian Privacy Principles guidelines, Chapter 8: APP 8 Cross-border disclosure of personal information · OAIC, 18 Oct 2026
- Part 4: Notifiable Data Breach (NDB) Scheme · OAIC, 18 Oct 2026
- Privacy and Other Legislation Amendment Act 2024 (as made) · Federal Register of Legislation, 18 Oct 2026
- Australian Privacy Principles guidelines, Chapter 1: APP 1 Open and transparent management of personal information · OAIC, 18 Oct 2026
- Small business · OAIC, 18 Oct 2026
- Competition and Consumer Act 2010, compilation 167 (Schedule 2, the Australian Consumer Law) · Federal Register of Legislation, 18 Oct 2026
- Privacy principles: search, learning and artificial intelligence · Slack, 18 Oct 2026
- ISO/IEC 27001:2022 Information security management systems: Requirements · ISO, 1 June 2026
- ISO/IEC 42001:2023 Information technology: Artificial intelligence: Management system · ISO, 6 Oct 2026
- System and Organization Controls: SOC Suite of Services · AICPA & CIMA, 18 Oct 2026
- Small business cyber security handbook: Event logging · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Small business cyber security handbook: Restrict and monitor access · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Small business cyber security handbook: Protecting devices, systems and networks · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Careful adoption of agentic AI services · Australian Signals Directorate, Australian Cyber Security Centre, 18 Oct 2026
- Competition and Consumer Amendment (Unfair Trading Practices) Act 2026 (as made) · Federal Register of Legislation, 18 Oct 2026
- Instructor Guidance Note: Automated decision-making · Office of Parliamentary Counsel, 5 Mar 2026
- Policy for the responsible use of AI in government: Standard for accountability · Digital Transformation Agency, 21 Oct 2026
- It's tough being a director, but that doesn't mean you shouldn't do it · ASIC, 10 Mar 2026
- Letter to industry: artificial intelligence (AI) · APRA, 30 Apr 2026
- Australian Securities and Investments Commission v Bekier (Liability Judgment) [2026] FCA 196 · Federal Court of Australia, 5 Mar 2026
- Australian Securities and Investments Commission v Bekier (Penalty Judgment) [2026] FCA 756 · Federal Court of Australia, 17 June 2026
- Harold Ford Memorial Lecture · Supreme Court of New South Wales, Chief Justice A S Bell, 21 May 2026
- APP 1.7 to 1.9 transparency obligation: fact sheet · OAIC, 30 Sept 2026
- Use of automated decision-making by government: consultation · Attorney-General's Department, 22 Aug 2025