Automated decision making in retail
Distinguish automated recommendations from decisions that affect a customer. Review retail examples, data use, explanations and routes to a person.
- Published
- Last updated
Automated decision making uses a computer program to choose an outcome or help determine it. In retail, the outcome might be a product ranking, an offer or a refused order. The review starts with what happens to the customer and who can change it.
A suggested jacket leaves the purchase choice with the shopper. A system that blocks checkout makes a different kind of decision. Both deserve a clear description, but their consequences call for different controls.
What counts as an automated decision
An automated decision happens when software determines an outcome, while decision support gives a person information or a recommendation to consider. Record both when mapping your store. Human involvement changes the process, but doesn’t automatically remove a decision from privacy rules.
AI is neither necessary nor sufficient. A fixed rule can automatically reject orders above a limit. An AI tool can draft a summary that a staff member checks against the original evidence before deciding independently.
The Office of the Australian Information Commissioner (OAIC) includes rule-based processes and AI within its automated-decision transparency guidance. Its scope can include software that substantially and directly contributes to a decision, even with a person involved.
Use these illustrative cases to separate the roles:
- A shopper chooses from a suggested shortlist. The system chooses what to display and the shopper chooses what to buy.
- A staff member reads a risk flag, checks the order and can approve it on the evidence. The software supports the decision.
- A rule blocks an account immediately. The system determines access, even if staff can review the block later.
Incorrect classification: “A person clicks approve, so the decision is human.” Correct classification: establish what the person actually assesses and whether they can reject the system’s recommendation.
Decisions in the customer journey
Catalogue ranking changes what a shopper sees, while a payment block changes what they can do. Assess the actual effect, including how long it lasts and whether another route is available. The following scenarios are fictional store processes.
| Retail process | What the system uses | Who determines the outcome | Effect to examine |
|---|---|---|---|
| Catalogue ranking | Search words, stock and browsing signals | Software orders products, then the shopper chooses | Whether relevant products disappear or remain available through other searches |
| Personalised offer | Purchase history and loyalty eligibility | A rule decides who sees or receives the offer | Whether a customer loses access to a price or receives inaccurate eligibility information |
| Fraud screening | Payment signals and order history | Software blocks the payment, or staff assess a flag | Whether the customer can complete a legitimate purchase and how quickly a false flag can be corrected |
| Account restriction | Account activity and recorded breaches | A rule suspends access, with later staff review | Whether the customer loses access to paid orders, records or support |
A recommendation can become consequential when another system treats it as authority. For example, a risk score used only to order a staff queue differs from a threshold that automatically cancels orders. Follow the score through to its final use.
Don’t classify every offer as harmless or every restriction as meeting the same legal threshold. The customer’s circumstances and the practical effect matter. A restriction on access to an important service can affect one customer more severely than another.
Find the data and decision owner
Map one outcome from its inputs to the person responsible for correcting it. Start with a real process, such as an order cancellation, and follow each handoff. A supplier’s model name alone won’t tell your team why the customer received that outcome.
- Name the decision. Write “cancel the order before dispatch” rather than “use fraud software”. State when it takes effect.
- Identify the inputs. Separate customer-provided details from transaction records and third-party information. Record where each comes from.
- Identify inferred attributes. A predicted risk level or likely interest is an inference, even when the system presents it as a field. Keep it distinct from observed facts.
- Describe the rule or model. Record the threshold or policy applied, the version used and what the output triggers downstream.
- Assign authority. Name the retailer role that can pause, reverse or remake the decision. Record which actions require the payment provider or another supplier.
- Identify the review record. Keep enough information to reconstruct the outcome, subject to access controls and an appropriate retention policy.
The OAIC’s AI privacy guidance addresses generated and inferred personal information. For covered organisations, existing privacy duties apply to that information too. Assess collection necessity and permitted use alongside accuracy and security.
Test human discretion with a fictional flagged order. Give the reviewer evidence that contradicts the flag. Can they see the relevant records and release the order before the customer suffers the consequence? If policy forces them to accept the score, record that restriction.
Ask the supplier for the inputs and retained records your staff need for that review. Agree who investigates missing records and who can change a configuration. Outsourcing the software doesn’t by itself settle the retailer’s responsibility.
Explain and review an outcome
Explain the action taken, the information that materially informed it and the route to a person who can reassess it. Give a supported explanation of the process. Don’t invent a precise reason from a model score when the underlying record cannot establish one.
Consider this fictional order hold. An address-comparison rule flags a mismatch between the order and the payment record. Staff can verify the details and release the hold. The customer-facing message could say:
We’ve paused this order because the delivery details need checking against the payment information. Our order support team can review the hold through your signed-in order page. Please use that page to confirm the delivery details.
That message describes a known rule and a working review route. It doesn’t accuse the shopper of fraud or ask them to send card details through an ordinary message. Use it only for a process that actually works that way.
For a review your team can act on:
- Confirm the requester’s authority using the store’s existing identity checks. Limit access to the customer’s own order and relevant account information.
- Retrieve the original decision record and the evidence the customer disputes. Check for stale details, incorrect inferences or a rule applied to the wrong transaction.
- Give an authorised staff member time and information to assess the case. If the supplier controls the restriction, route it to the supplier and track the response.
- Record the review result and tell the customer what changes. If the outcome remains, explain the supported reason and the next complaint route.
- Verify the result in the affected system. Closing a support conversation doesn’t prove that checkout or account access has been restored.
Keep other customers’ records out of the explanation. Restrict fraud-control details that would expose security measures, while still explaining the action and available remedy. A broad technical description of a model cannot substitute for reviewing the customer’s evidence.
Check privacy and consumer obligations
Australian privacy obligations depend on whether the retailer is covered by the Privacy Act and on the particular use of personal information. Automated-decision transparency has its own statutory test. It isn’t a universal right to demand human review of every computer-generated result.
Existing duties and the enacted transparency change
The Australian Privacy Principles (APPs) already govern covered entities’ handling of personal information. The OAIC’s small-business checklist explains coverage and exceptions. Most businesses with annual turnover of $3 million or less are exempt, but exceptions include some businesses trading in personal information and health service providers. Assess the business, including relevant related-company arrangements, before assuming an exemption.
The 2024 privacy amendments enact additional privacy-policy disclosures for certain automated decisions. These provisions have a separate commencement and have not yet commenced. The Act’s commencement table and automated-decision provisions set the boundary: the change applies to decisions made after commencement, including under earlier software arrangements.
Once applicable, the disclosure test requires personal information used by a computer program that makes, or substantially and directly contributes to, a decision. That decision must reasonably be expected to significantly affect an individual’s rights or interests. Beneficial effects can count too.
The policy must identify the kinds of personal information used and the kinds of decisions made solely by the program. It must also identify decisions to which the program substantially and directly contributes. This enacted disclosure duty doesn’t create a general right to an individual explanation or human review.
The OAIC’s APP 1 guidelines explain why human review alone doesn’t remove a decision from scope. They also address supplier arrangements and the effect on the individual. Apply that test to your recorded process, including any contractual rights affected by a restriction.
Keep broader reform proposals separate from enacted requirements. A consultation paper or proposed review right doesn’t establish a current entitlement. Distinguish your voluntary review service from a review process required by a law or contract.
Consumer law and overseas rules
The Australian Competition and Consumer Commission (ACCC) says business claims must be accurate and supported. That includes price and delivery information. Check that an automated offer’s conditions and the explanation of a restriction match what the store actually does.
European rules have a different structure. Article 22 of the European Union’s General Data Protection Regulation addresses solely automated decisions with legal or similarly significant effects, subject to exceptions and safeguards. It doesn’t establish an Australian-wide entitlement. A retailer operating across jurisdictions must assess which laws apply to its processing.
Take your decision map to the person responsible for privacy and customer complaints. Confirm the applicable disclosure duties and the review route the store can deliver. Then use the map to describe actual data use in your privacy policy template.
Give your store its own shopping agent
Customers ask in their own words. Your agent finds, compares and checks out from your catalogue, right on your site.