Context windows for AI shopping assistants in 2026
Understand an AI context window and what fits inside it. Compare token limits, long conversations and the difference between context and saved memory.
- Published
- Last updated
A context window is the amount of information an AI model can use while producing one response. It limits the material available for that answer, including the conversation context supplied by the application. A larger window gives an assistant more room, but it doesn’t make old product information current.
For a retailer, the useful question is what reaches the model when a shopper changes their mind. A customer’s budget must survive the conversation, while the price of their chosen variant must come from current catalogue data.
What a context window holds
A context window holds the information supplied for the current response, with output space governed by the model’s limits. Providers measure that capacity in tokens, the units a model processes. Tokens can contain parts of words, so a token limit isn’t a word count.
A shopping request can contain several kinds of material:
- System instructions tell the assistant how to behave, including which store policies it must follow.
- Retrieved product information supplies the relevant catalogue records and policy passages.
- Conversation history gives the shopper’s earlier requests and the assistant’s replies.
- The new customer message asks the next question or changes a requirement.
- The answer allowance leaves room for the generated response. Models that generate internal reasoning can use additional tokens within their documented budgets.
The application decides what to send. Its instructions take space, and it can impose a smaller limit than the underlying model supports. An advertised maximum therefore doesn’t tell you how much room remains for your product information.
Providers also describe their limits differently. Google’s long-context documentation distinguishes shared input-and-output capacity from models with separate input and output budgets. Compare those definitions before subtracting an answer allowance from a published number.
Saved memory is a separate application feature. An account can store a preference and supply it to a later request. That doesn’t mean every earlier conversation fits into the model’s current context. OpenAI’s explanation of ChatGPT memory says memory doesn’t retain every detail from every conversation.
Incorrect assumption: a saved chat means the model reads every word of that chat on each turn. Correct interpretation: the application supplies the history or summary used for the current response.
Why a larger window is not the whole catalogue
Selecting relevant product records can give an assistant the evidence it needs without sending every catalogue entry. If someone asks for navy raincoats under $180, unrelated lamps and furniture add no evidence for that choice.
A larger window can help when a comparison needs detailed specifications from several products. It also gives long policy passages more room. Capacity alone doesn’t select the correct records, resolve conflicting prices or identify which variant is available.
Retrieval-augmented generation, often shortened to RAG, selects outside material for a model to use in its answer. A retail application can retrieve the matching coats and the relevant delivery policy. The model then has a smaller set of facts to apply to the shopper’s request.
Retrieval still needs accurate product identifiers and current source data. If the application supplies yesterday’s stock record, a million-token window still contains yesterday’s stock record. Grounding an answer in supplied records helps only when those records support the claim being made.
Long conversations and product changes
A long shopping conversation needs both retained requirements and refreshed product facts. This hypothetical sequence shows what to look for when an assistant handles several refinements.
| Shopper message | Requirement to carry forward | Product information needed next |
|---|---|---|
| “Find a waterproof coat under $180.” | Waterproof coat, budget below $180 | Matching products and prices |
| “Show me those in navy.” | Waterproof coat and original budget | Navy variants and their availability |
| “Make it under $150.” | Navy and waterproof, with the lower budget replacing the old one | Products that meet the revised budget |
| “Compare the first two, then show the second in medium.” | The selected products and revised requirements | Comparison attributes, then the second product’s medium variant |
At the final step, “the second” must identify the second product in the comparison. A colour change must also select the correct variant. Repeating an earlier product description is insufficient when that variant has a different price or stock status.
Applications can manage growing history in different ways. Truncation removes older material. Summarisation replaces earlier turns with a shorter account. Fresh retrieval fetches product or policy information needed for the next answer. These approaches can work together, and they aren’t automatic consequences of a model’s context size.
For example, Anthropic’s compaction documentation describes replacing older turns with a summary. A summary can preserve “navy waterproof coat, under $150”, but a poor summary could omit the waterproof requirement. Removing old tool results also means the application needs another way to obtain a product fact when it becomes relevant again.
Questions to ask when comparing assistants
Compare assistants by the requirements they retain and the product facts they refresh. A model specification tells you the capacity available underneath that behaviour.
Anthropic’s context-window documentation lists a one-million-token window for both Claude Opus 4.6 and Claude Sonnet 4.6. It lists 200,000 tokens for the deprecated Claude Sonnet 4.5. These are model API limits. They don’t establish the usable capacity of a retail assistant or a consumer chat subscription.
Use the same catalogue and conversation sequence when comparing implementations:
- Start with a product requirement and a budget. Add a colour several turns later. Confirm that the answer still satisfies both earlier conditions.
- Replace the budget, then compare two products. Check that the assistant applies the new amount and identifies both products consistently.
- Change a variant. Compare the returned price and availability with that exact variant’s record.
- In a private test catalogue, change a product’s stock or price between turns. Confirm that the next relevant answer uses the updated record.
- Extend the conversation until history handling takes effect. Ask for an earlier requirement and a fresh product fact in the same message.
Ask the supplier what its application retains when it shortens history, and when it refreshes catalogue data. Request the model version and application limit as separate facts. Use the conversation’s results to judge whether the available context supports the shopping decisions your customers need to make.
Give your store its own shopping agent
Customers ask in their own words. Your agent finds, compares and checks out from your catalogue, right on your site.