De-identification of retail customer data

De-identification takes more than removing a name. Compare pseudonymisation, redaction and aggregation using a retail record and its remaining risks.

De-identification changes customer records so a person is no longer reasonably identifiable in the context where the records are used. Removing a name starts that work. The remaining order details and support messages can still point to the customer.

For retail analysis or AI testing, begin with what the test needs. A question about jacket care usually needs product attributes and an expected answer, without the customer’s purchase history.

What de-identification changes

De-identification reduces identification risk through changes to the data and controls over who can use it. The Office of the Australian Information Commissioner (OAIC) assesses that risk in the relevant access or release context. Its de-identification guidance requires a very low risk of re-identification.

A record can identify someone through a combination of otherwise ordinary details. Removing a name leaves that combination intact. Encrypting a file protects access, but someone who decrypts it can still read the original customer details.

Consider two different recipients. A product tester might see only the prepared extract. A fulfilment partner might also have delivery records that match its unusual purchases. Assess the information each recipient can reasonably combine with the extract.

Pseudonymisation, redaction and aggregation

Pseudonymisation replaces an identifier with a substitute, such as a random customer code. A separately protected mapping table can reconnect the code with the customer. This preserves connections between records, which helps when analysing repeat contacts.

The UK Information Commissioner’s Office (ICO) explains pseudonymisation and its remaining identification risks. Its legal terminology belongs to UK law. For an Australian release, apply the contextual OAIC assessment instead of treating a substituted code as proof of anonymity.

Redaction removes selected information, such as an email address or a sentence naming a workplace. It loses that detail while leaving the rest of the record available. Removing a visible identifier doesn’t remove clues elsewhere in the message.

Aggregation combines individual records into summaries, such as contacts by issue category. It loses the individual sequence and exact wording. A total for a group containing one customer can still reveal information about that person.

Hashing converts an input into a hash value using a one-way function. It differs from encryption, which can be reversed with the correct decryption key. A predictable input, such as an email address, can still be guessed and its hash compared. The ICO describes these attacks in its pseudonymisation guidance. A hash is not an anonymity certificate.

Transform a retail example

Use the smallest record that still tests the intended answer. The following order and support message are entirely invented. They illustrate the preparation process, not an approved release of customer data.

The invented order contains customer Cobalt Example, email cobalt@example.invalid, order TEST-731 and a delivery address of 14 Example Lane, Fictionville. It records one bottle-green waterproof jacket in size XL, ordered at 02:13 and delivered two days later.

The invented support message reads: “I’m the only overnight projectionist at the Fictionville observatory. My bottle-green jacket arrived yesterday. Can I tumble-dry it before my next shift?” The invented product care sheet says to air-dry the jacket and avoid tumble drying.

Prepare a test case in this order:

  1. Define the task as answering the jacket-care question from the product care sheet. The expected answer must advise air drying and reject tumble drying.
  2. Remove the customer name, email and delivery address. Remove the order number because this test doesn’t retrieve an order.
  3. Replace the exact purchase time with “recent purchase”, if timing has any purpose. Drop the workplace and shift narrative. Neither changes the care answer.
  4. Keep the waterproof-jacket category and the care instruction. Remove colour and size unless the care instructions differ between variants.
  5. Assign an invented case code, CASE-A, and rewrite the message as “Can I tumble-dry this waterproof jacket?” Keep no mapping to a real customer.

The resulting case contains CASE-A, the jacket-care question, the invented care sheet and the expected air-drying answer. That is enough to test whether the answer follows the supplied instructions.

If the task changes to order retrieval, this case is insufficient. Build a separate fictional order with matching test identifiers and status. Don’t add a real order history to rescue a test that was designed for product answers.

Look for remaining identification paths

Review combinations of fields and free text after the first transformation. Read the extract as the intended recipient would, including information that recipient already holds.

In the invented example, removing the name would leave a distinctive workplace beside an unusual purchase and a precise time. Generalising the address alone would leave those other clues untouched.

For your own assessment, examine these paths:

  • A location combined with a rare product or variant can narrow the possible customers.
  • An exact timestamp can match a delivery record or a public post about the purchase.
  • A free-text account can identify a person through their job or a recognisable event, even after names disappear.
  • A stable customer code can join separate extracts and expose a longer history than either extract shows alone.

Check small groups in summaries too. Combining categories can reduce detail, but the grouping must still answer the analysis question. If a useful category remains identifying, keep that output restricted or omit it.

The OAIC and CSIRO decision-making framework helps structure this assessment. OAIC flags that some information in the older framework can be outdated and also points to newer anonymisation guidance. CSIRO describes the framework as a judgement process, not an algorithm that certifies a release.

Choose a useful sharing boundary

Choose a sharing boundary that meets the task with the least customer detail. A fully invented case is often enough for a product-answer test. It lets you test a clear expected outcome without transferring an actual conversation.

For trend analysis, a summary might answer the question without sharing individual records. If repeat-contact analysis needs stable codes, use a smaller pseudonymised extract with restricted access and separately controlled mapping information. A smaller extract still needs its own risk assessment.

Generated synthetic data can also carry risks when it reproduces details from real training records. The ICO’s anonymisation glossary distinguishes synthetic data from a guarantee of anonymity. Keep fully invented examples separate from data generated from customer records.

If identifying detail is necessary and the recipient cannot handle it within an appropriate arrangement, keep the work inside the controlled environment or make no transfer. Don’t label an unsuitable extract anonymous to make the sharing easier.

Record the purpose and intended recipient, the fields removed or generalised, access restrictions and the remaining identification paths. Name who approves the decision. Set a reassessment trigger, such as a new recipient, another linked dataset or a broader release.

For public or detailed releases, seek specialist assessment, as OAIC recommends. Start your next test with one fictional case and its expected answer, then add only the detail that changes the result.

Give your store its own shopping agent

Customers ask in their own words. Your agent finds, compares and checks out from your catalogue, right on your site.

See it in actionFree trial · Simple setup