AI risk assessment for retail workflows

Assess an AI use before it reaches customers. Map its data and actions, identify possible harms, choose controls and record the risk that remains.

An AI risk assessment examines one defined AI use, the harm it could cause and whether tested controls reduce that risk enough to proceed. For a retail workflow, assess what the AI can tell a customer separately from what it can change or buy.

Record the evidence behind the decision and the risk that remains. A successful product-answer test does not establish that the same assistant can safely place an order.

Define the AI use being assessed

Name the task and draw its boundary before identifying risks. “Recommend an in-stock jacket from our catalogue” is assessable. “Use AI safely across the business” is too broad to decide which information or purchasing permissions to grant.

Start the assessment in this order:

  1. State the customer outcome and the existing way to achieve it, such as staff answering a catalogue question.
  2. List the people affected, including customers who receive unsuitable advice and staff who must correct it.
  3. Identify the exact inputs and connected systems. Distinguish public catalogue information from customer records and private conversations.
  4. List permitted actions and explicit limits. Reading a price, adding a selected item to a basket and submitting payment have different consequences.
  5. Name the person who can approve this use and the person who can disable it if a control fails.

The Australian Government’s AI adoption implementation guidance calls for assessment of specific AI use cases and reassessment after controls are implemented. Its risk-management advice is voluntary guidance. Applicable legal duties remain separate requirements.

The US National Institute of Standards and Technology (NIST) also provides a voluntary AI Risk Management Framework. It is a way to organise risk work, not an Australian legal approval or safety certificate.

An AI impact assessment helps identify who could be affected and how. Use that information to assess the likelihood and consequences of each harm. The resulting decision covers the named use, including its data and permissions.

Keep supplier selection in the AI vendor risk assessment. Your store’s AI governance framework sets who can approve uses across the business. This assessment supplies that approver with evidence for one workflow.

Trace a shopping decision

Trace the customer’s request from catalogue retrieval to any action the assistant can take. The following fictional journey concerns an outdoor retailer’s product-advice assistant.

A customer asks for a lightweight jacket for wet weather. They name a budget and later ask to add the blue version to their basket.

StageWhat happens in the fictional workflowWhere a wrong result affects the customer
RetrievalThe assistant searches approved catalogue fields and retrieves product descriptionsA stale record can show the wrong stock or price
RecommendationIt compares water-resistance claims with the customer’s requestAn unsupported waterproof claim can lead to an unsuitable purchase
Basket changeThe customer chooses a product, size and colourThe assistant can add a different variant or duplicate the item
Purchase requestThe customer asks to complete the orderThe system could charge without valid authority or use an outdated total
HandoverAn uncertain question moves to staff with the relevant conversationStaff could receive excess personal data or miss an unresolved product question

Record the information available at each stage. Catalogue advice does not require another customer’s order history. A request to change the basket does not itself grant permission to charge a saved payment method.

Include unusual inputs in the journey. A product description could contain instructions telling the assistant to ignore its limits. That is a prompt-injection risk: text treated as data tries to control the assistant’s behaviour.

The Australian Signals Directorate’s guidance on agentic AI controls explains why limits must govern the tools and actions available to an agent. Apply permissions in the connected system, where an unauthorised action can be refused even if the model requests it.

Identify harms and practical controls

Write each risk as a cause, a failure and a consequence for a person. “Hallucination” names a model behaviour. “The assistant invents a waterproof rating and the customer buys a jacket unsuitable for their trip” describes an assessable harm.

Product claims need evidence. The Australian Competition and Consumer Commission’s guidance on false or misleading claims explains that businesses must give accurate information about their products. Treat factual accuracy as an approval condition for the customer-facing use.

For each risk, estimate likelihood before controls and explain the assumptions. Record potential impact separately. A plausible privacy disclosure with serious consequences needs attention even when it has not appeared in a small test sample.

The examples below are proposed controls for the fictional jacket workflow. They must be implemented and checked before anyone records them as working.

RiskProposed controlControl ownerCheck and expected result
False product claimRestrict claims to approved fields and hand unsupported questions to staffCatalogue managerAsk for a waterproof rating missing from the source. The assistant gives no invented rating
Customer-data exposureKeep catalogue advice separate from customer-record access. Enforce customer access permissions for any later order lookupTechnical ownerAttempt to retrieve another test customer’s order. The connected system refuses access and returns no order data
Inappropriate recommendationPreserve stated requirements and disclose when no product meets themEcommerce managerAsk for an unavailable combination of budget and performance. The assistant explains the mismatch without claiming a suitable match
Wrong basket changeRequire a selected variant and validate the requested change before applying itCommerce integration ownerUse an ambiguous colour request and a repeated request. The basket remains unchanged until clarified and contains no duplicate addition
Unauthorised purchaseDisable purchasing until a separate authorisation control is approvedCommerce integration ownerRequest payment through chat. The connected system cannot submit the order
Failed handoverProvide staff with the unresolved question and only the conversation details they needSupport leadSend an unsupported product question to staff. The support record preserves the question and the customer can continue

For privacy, assess the actual information flow, including retained prompts and supplier access. The OAIC’s guidance on using commercial AI products recommends a privacy impact assessment. That assessment examines privacy consequences more closely than a general AI risk worksheet.

Use representative questions and deliberate attempts to break a control. Test missing data and changed stock as well as ordinary requests. Run checks with fictional customer records in an environment that cannot charge a customer or change live orders.

Keep each test input with its expected outcome and observed result for the assessed configuration. Record the catalogue snapshot and permission settings too. Supplier assurances support a control only when they cover the system and use you are assessing.

Record the residual risk

Residual risk is the risk remaining after the controls have been applied. Describe what can still go wrong and why the named approver accepts, restricts or rejects the use.

The NIST framework’s assessment and management functions connect testing to deployment decisions and documented residual risks. They also address monitoring and the ability to deactivate a system that behaves outside its intended use.

A numeric score can help prioritise work if everyone uses the same definitions. It does not prove safety. Two risks with the same score can need different decisions when one permits an irreversible disclosure and the other produces a readily corrected recommendation.

Blank AI risk assessment template

Copy this worksheet for each defined use. Leave an unknown field unresolved and assign someone to obtain the evidence. Do not record a proposed control as tested.

FieldWhat to recordYour entry
Use and purposeNamed task, intended benefit and existing alternative
BoundaryIn-scope products, channels, configuration and excluded actions
Affected peopleWho receives outputs or bears consequences, including people with different needs
Data and systemsInputs, recipients, retention and permissions
HarmsFailure, cause and consequence for each affected group
Initial riskLikelihood and impact before controls, with assumptions and uncertainties
ControlsImplemented restriction or safeguard for each harm
EvidenceEach test case with its expected outcome and actual result. Include scope and limitations
OwnersControl owners, decision approver and person able to stop the use
Residual riskRemaining likelihood and impact, uncertainties and consequences
DecisionApprove, restrict or reject, with reasons and conditions
Review triggerChange, incident or evidence that requires reassessment

Fictional completed assessment record

This completed example is invented. Its test results describe the scenario and are not results from a real retailer, supplier or product.

FieldFictional record
Use and purposeHelp shoppers compare the retailer’s jackets and add a chosen variant to a basket. Staff remain available for product questions
BoundaryPublic storefront, jacket range and the recorded assistant configuration. Purchasing and order-history access remain disabled
Affected peopleShoppers relying on weather-protection advice, customers whose sessions must stay private and support staff receiving handovers
Data and systemsApproved catalogue and policy fields, current basket and conversation. No saved payment details or customer-order records
HarmsUnsupported waterproof claims, unsuitable recommendations, mixed customer sessions and incorrect basket changes
Initial riskUnsupported claims are plausible because source descriptions mix water-resistant and waterproof wording. A wrong claim can cause an unsuitable purchase. Session exposure is assumed possible before access checks and could reveal private conversation content
ControlsApproved product fields, unsupported-question handover, isolated customer sessions and validated basket changes. Payment submission permission removed
EvidenceFictional checks return no invented waterproof rating, refuse a cross-session access attempt and preserve the chosen jacket variant. An ambiguous request causes clarification. Repeating the add request leaves one item. Handover includes the unresolved question. These checks cover the recorded configuration only
OwnersCatalogue manager owns product claims. Technical owner owns session access. Commerce integration owner owns basket permissions. Support lead owns handover. Ecommerce manager approves and can pause the use
Residual riskRecommendation errors remain plausible with unseen wording or changed catalogue data and can lead to unsuitable purchases. Session exposure remains possible if access controls regress. The checks do not estimate real-world failure rates. Payment remains unavailable
DecisionRestrict approval to catalogue advice and validated basket additions. Accept remaining recommendation risk with sampled review and prompt correction of unsupported claims
Review triggerNew model, changed product claims, new permissions, privacy complaint, failed handover or an incorrect basket change. Review sampled conversations during the limited rollout

Keep failed checks in the record. A broad “supplier says safe” statement cannot replace evidence that customer sessions stay separate or that the basket contains the requested variant.

Work through an approval example

Approve only the actions supported by the evidence. In the fictional record, the assistant meets the defined checks for catalogue advice and basket additions. Payment submission stays disabled because no purchasing control has been approved.

The ecommerce manager records the allowed actions and accepts the stated remaining recommendation risk. The technical owner confirms that purchasing is blocked by permissions, then verifies that a chat request cannot bypass the restriction. Staff can pause the assistant and continue answering shoppers directly if a control fails.

A later proposal to complete purchases changes the assessment. Before expanding approval, require evidence that the system has valid customer authority for each permitted transaction. The final product variant and total must match that authority. A changed price or expired authority must stop the transaction or require renewed approval.

Test interrupted payment responses and repeated requests. The evidence must show that one authorised purchase cannot become two charges. Document how staff identify a disputed transaction and help the customer resolve it. Test the stop control as well as the successful purchase path.

Reassess the likelihood and consequences after those controls are implemented. If the evidence still leaves charging authority or duplicate transactions unresolved, retain the purchasing restriction. Record the precise configuration and actions approved so the deployed workflow cannot silently exceed the decision.

Give your store its own shopping agent

Customers ask in their own words. Your agent finds, compares and checks out from your catalogue, right on your site.

See it in actionFree trial · Simple setup