Salesforce AI Research researchers have introduced Just-in-Time Memory, a new approach to long-term memory for AI agents that delays memory curation until an agent receives its next task. Instead of summarizing an agent’s experience immediately after each task, the system retains raw trajectories and uses the current task to determine which information should be extracted.
The research, presented in a paper submitted to arXiv on September 23, 2026, proposes a system called JitMem. Across three agent benchmarks ALFWorld, WebShop and τ²-bench the researchers report improvements of 16.2, 16.3 and 3.9 absolute success-rate points, respectively, over the strongest baseline evaluated in each benchmark.
The approach addresses a fundamental problem with conventional AI agent memory: deciding what information is worth preserving before the system knows what its future tasks will require.
Quick Summary
- Salesforce AI Research introduced the Just-in-Time Memory (JitMem) approach for LLM agents.
- Instead of summarizing experiences immediately after a task, JitMem preserves raw agent trajectories.
- When a new task arrives, the system retrieves relevant experiences and creates task-specific memory.
- The research reports improvements of 16.2 points on ALFWorld, 16.3 on WebShop and 3.9 on τ²-bench versus the strongest evaluated baselines.
- The approach also aims to reduce unnecessary information passed to the agent by generating compact memory payloads.
- The work is currently research, not evidence of a commercially deployed Salesforce product.
What Is Just-in-Time Memory?
Most agent memory systems process experience when a task ends. An agent completes a task, its trajectory is analyzed, and the system converts that experience into something more compact, such as a reflection, skill, workflow or reasoning strategy.
That memory can later be retrieved when a similar task appears.
The researchers argue that this creates an information-loss problem. A trajectory can contain several potentially useful pieces of information, but a write-time memory system has to decide what matters before the future task is known. Once unnecessary details have been discarded, they cannot be recovered from the stored summary.
JitMem reverses that process.
Instead of immediately distilling an experience, it stores the raw trajectory in a persistent memory bank. When a new task arrives, a retriever selects relevant past trajectories. A memory curator then reads those trajectories alongside the current task and creates a compact memory payload specifically for that task.
This means the same past experience can potentially produce different memories depending on what the agent is currently trying to accomplish.
How JitMem Works?
The JitMem architecture consists of four main components: a memory bank containing raw trajectories, a retriever, a trainable memory curator and a frozen agent executor. Only the curator is trained.
The process can be simplified into four stages:
- Retrieve: The system searches its memory bank for relevant past trajectories.
- Curate: The memory curator examines those trajectories together with the current task.
- Execute: The resulting task-specific memory payload is supplied to the agent executor.
- Store: Successful new trajectories can be added to the persistent memory bank.
The important difference is that curation occurs during the read stage, rather than when the original experience is first stored.
That also changes how the memory curator can be trained. Since the generated memory is immediately used for the current task, the system can directly observe whether that memory helped the agent succeed. The researchers describe this as avoiding the long-horizon credit-assignment problem associated with deciding whether a memory written today will be useful for an unknown task much later.
Just-in-Time Memory Results Across AI Agent Benchmarks
The researchers evaluated JitMem on three established agent environments: ALFWorld, WebShop and τ²-bench.
The paper reports the following improvements over the strongest baseline in each benchmark:
| Benchmark | Improvement in Success Rate |
|---|---|
| ALFWorld | +16.2 points |
| WebShop | +16.3 points |
| τ²-bench | +3.9 points |
These are absolute success-rate improvements, rather than percentage increases.
The results also extend beyond the trained version of the curator. The researchers report that an untrained read-time curator can already be competitive with, or outperform, write-time memory approaches.
For example, on WebShop, the paper reports an untrained JitMem configuration using Gemini-2.5-Pro as both curator and executor achieving a 61.0 success rate, compared with 41.0 for the corresponding SkillOS configuration.
The researchers interpret this as evidence that task-adaptive read-time curation itself contributes substantially to the performance improvement, rather than the gains coming exclusively from reinforcement-learning training of the curator.
Why Task-Aware Memory Matters for AI Agents?
The central idea behind Just-in-Time Memory is that the usefulness of an experience depends on the problem an agent is currently solving.
Consider an agent that previously interacted with a virtual environment and learned several useful behaviors. One future task might require remembering how an object changes state, while another could require remembering how objects are positioned. A fixed summary created after the original task may emphasize one lesson while omitting another.
JitMem instead keeps the underlying experience available and allows the current task to determine which information should be extracted.
This makes the memory system more similar to a retrieval-and-reconstruction process than a conventional database of prewritten summaries.
The paper also reports efficiency benefits. According to the researchers, JitMem’s compact task-specific payloads reduce input tokens by approximately 50.3% to 56.3% and executor steps by 28.4% to 31.4% compared with the write-time methods evaluated in the study.
JitMem Also Separates Memory From the Agent Executor
Another notable part of the research is the separation between the memory curator and the model that actually performs the task.
The curator can be trained independently while the executor remains frozen. The paper reports that trained curators can also transfer to stronger executors without requiring the same kind of retraining for each executor.
That design could be relevant to researchers building agent systems where the underlying model changes over time. However, the reported transfer results are still experimental findings from the benchmark evaluation rather than evidence of a production deployment.
What the Research Does Not Establish?
The results should be viewed in the context of the paper’s experimental setup.
JitMem has been evaluated on ALFWorld, WebShop and τ²-bench, rather than across a broad set of real-world production agents. The research therefore demonstrates the effectiveness of the approach within the tested environments, but it does not by itself establish how the architecture would perform across every type of long-running enterprise or consumer AI agent.
The work is also presented as an arXiv research paper rather than a commercial product announcement. There is no indication in the paper that JitMem itself is a generally available Salesforce product.
Still, the research highlights an important design question for AI agent memory: whether systems should decide what to remember immediately after an experience, or preserve more of the original experience and determine what matters only when the next task is known.
For JitMem, the researchers’ benchmark results suggest that delaying that decision can produce substantial gains across the tested agent environments, while also providing a more direct training signal for the memory curator.
Also Read –
SIFT Makes Self-Improving Coding Agents More Efficient
Source
Original arXiv paper – Just-in-Time Memory
DAIR.AI Academy – Just-in-Time Memory paper page
Salesforce AI Research on Hugging Face
