Pilot Programs
AI procurement and pilot programmes
Screen AI suppliers, run a bounded business pilot and use its evidence to make a purchase decision.
Buy an AI tool for a defined business use only after a bounded pilot shows the proposed workflow is useful and the purchase terms are workable. Base the decision on evidence from the intended account, the work staff will do, the review it requires and the conditions under which the tool may be used.
Define the purchase decision
Name the task, the current route to an approved result and the improvement sought. Before work begins, set the pilot's duration, permitted users, input boundary, spending limit and stop conditions. A demonstration may show that a feature exists; an adoption pilot also needs to examine corrections, exceptions, handover and operating effort.
| Gate | Question | Evidence |
|---|---|---|
| Candidate screen | Can the proposed plan perform the task under acceptable terms? | Current documentation, proposed terms and account entitlements |
| Pilot entry | Are the inputs, reviewers and stop conditions approved? | Task brief, data boundary and named decision owners |
| Pilot result | Did acceptable work reach its destination with manageable effort? | Attempts, failures, corrections and staff time |
| Purchase | Can the business operate the service and leave it if needed? | Approval record, support terms and exit arrangements |
Demonstration vs adoption pilot
- Demonstration
- Shows that a feature exists.
- Adoption pilot
- Examines corrections, exceptions, handover and operating effort.
Pilot decision gates
- Candidate screenCan the proposed plan perform the task under acceptable terms? Evidence: current documentation, proposed terms and account entitlements.
- Pilot entryAre the inputs, reviewers and stop conditions approved? Evidence: task brief, data boundary and named decision owners.
- Pilot resultDid acceptable work reach its destination with manageable effort? Evidence: attempts, failures, corrections and staff time.
- PurchaseCan the business operate the service and leave it if needed? Evidence: approval record, support terms and exit arrangements.
Screen the supplier and the intended account
Ask about the exact product, plan and feature. Check access controls, data handling, retention, use of submitted material for training, subcontractors, support and the records the business can retrieve. Confirm material answers against the applicable terms and account settings; a general marketing statement may describe a different service or plan.
The Office of the Australian Information Commissioner (OAIC) publishes guidance on privacy and the use of commercially available AI products. Before using personal information, assess whether the proposed use and the product's data handling meet the business's privacy obligations. Start an exercise with invented or otherwise authorised, non-sensitive material, and assess any proposed use of personal information separately.
Ask suppliers for evidence behind claims that matter to the purchase, including the conditions and methods of their tests. Test any proposed system for its intended use before deployment. For a business purchase, agree commitment stages, exit points and interoperability needs in the contract.
Supplier and account screening checks
- Confirm the exact product, plan and feature.
- Check access controls, data handling and retention.
- Clarify whether submitted material is used for training.
- Identify subcontractors and support arrangements.
- Confirm records the business can retrieve.
- Assess privacy obligations before using personal information.
- Start with invented or otherwise authorised, non-sensitive material.
- Test the proposed system for its intended use before deployment.
- Agree commitment stages, exit points and interoperability needs.
Use public-sector guidance as a prompt, not a rule
The Digital Transformation Agency's guidance for Australian Government agencies using publicly available generative AI tools sets out three principles: protect privacy and safeguard information, critically assess AI outputs, and take ownership of advice and decisions. A business can use these as prompts when deciding what its own adoption controls should cover.
The agency's materials include separate guidance for agencies and individual staff, plus examples of appropriate and inappropriate use. For a business, this highlights a decision gate beyond the tool itself: check that both the people setting controls and the staff permitted to use the service understand the approved-use boundary.
Public-sector principles to use as prompts
- Protect privacy and safeguard information
- Critically assess AI outputs
- Take ownership of advice and decisions
Judge the complete workflow
Where their interfaces permit, use equivalent approved inputs and the same acceptance rule for suitable candidates. Record any difference in preparation. Keep first attempts, retries, rejected outputs and manual repairs. Measure preparation, tool use, checking, correction, approval and handover against the current process for a comparable mix of work.
Keep supplier evidence separate from what the pilot showed. A non-sensitive pilot can establish how the workflow behaved on those cases. It cannot establish performance on restricted records or settle the privacy treatment of a later live-data use.
Judge the complete workflow
- Use equivalent approved inputsWhere interfaces permit, use the same acceptance rule for suitable candidates.
- Record preparation differencesNote any difference in preparation.
- Keep first attempts and repairsKeep first attempts, retries, rejected outputs and manual repairs.
- Measure the full workflowMeasure preparation, tool use, checking, correction, approval and handover against the current process.
- Separate supplier evidenceKeep supplier evidence separate from what the pilot showed.
Record the decision and its boundary
Choose a narrow approval, a specific change and retest, a hold for missing evidence or terms, or a decision not to buy. Do not let an overall score conceal a consequential error or unresolved data flow. Any approval should name the task, account, users, permitted inputs and human review route.
Before committing, check feature and usage entitlements, support, service-change terms, data return or deletion, and the practical exit route against what the pilot required. Name who will monitor incidents and changes after adoption. The pilot ends with a recorded decision, rather than continued trial-account use by default.
Record the decision and its boundary
- Choose a narrow approval, a specific change and retest, a hold, or a decision not to buy.
- Do not let an overall score conceal a consequential error or unresolved data flow.
- Name the task, account, users, permitted inputs and human review route.
- Check feature and usage entitlements, support and service-change terms.
- Confirm data return or deletion and the practical exit route.
- Name who will monitor incidents and changes after adoption.
- End the pilot with a recorded decision, not default trial-account use.
In this guide
- Running an AI tool pilot with non-sensitive test dataSet an input boundary, rehearse the full workflow and report what a non-sensitive AI pilot can establish.
- Choosing a pilot owner and approval criteriaAssign an AI pilot owner, record decision rights and set adoption gates before seeing results.
- Comparing vendor claims with observed resultsCheck AI supplier claims against the proposed plan, applicable terms and bounded pilot observations.
- Deciding when a tool is not ready for business adoptionUse AI pilot evidence to approve a narrow use, hold for a specific fix or decline adoption.
