Persistent Memory for AI Agents: Mem0 vs. Vertex AI Memory Bank

Andrei Kopeikin · Mar 2, 2026

TL;DR (who should read this, and what to choose)

If your agents keep re-asking users for the same preferences or your support bots lose context between tickets, you likely need persistent memory, not just a bigger context window. This article is for platform engineers designing shared memory infrastructure and application teams wiring memory into specific agents.

In practice:

Choose Mem0 if you want an open-source, vendor-neutral memory layer you can self-host anywhere, customize aggressively, and reuse across multiple LLM providers.
Choose Vertex AI Memory Bank if you are already invested in Google Cloud / Gemini, want a fully managed service with strong integration into Vertex AI, and are willing to trade some flexibility for operational simplicity.

We compare both options as infrastructure building blocks: how they fail in production (drift, contradictions, privacy leakage), how they fit into your stack, and what tradeoffs they impose on teams.

1. Introduction

In many real systems today, AI agents forget what users told them yesterday. A scheduling assistant re-asks for your time zone. A customer-support bot ignores the last ticket and walks the user through the same troubleshooting steps again. This statelessness makes agents feel dumb, frustrates users, and increases operational costs.

Persistent memory addresses this gap by giving AI systems the ability to store, retrieve, and reason over information across sessions. Rather than relying solely on the context window, memory-augmented models can maintain a coherent understanding of users and tasks over time — much like a human colleague who remembers what was discussed last week.

Without a deliberate memory layer, teams often try to “just stuff more history into the prompt”, which fails in three ways: drift (outdated facts never get corrected), contradictions (the model sees inconsistent history and behaves unpredictably), and privacy leakage (sensitive details are replicated across logs, prompts, and tools). A dedicated memory system gives you a place to control how facts are stored, updated, and surfaced to the model.

Several frameworks have emerged to solve this problem. In this article we focus on Mem0 and Vertex AI Memory Bank, which we initially evaluated in response to direct client demand. Zep and other frameworks offer their own approaches to long-term memory and are worth exploring separately, but they are out of scope here.

2. Persistent Memory Frameworks

2.1. Mem0

Overview

Mem0 positions itself as a dedicated memory layer for AI applications — an infrastructure component that sits between the application and the LLM, giving agents the ability to persist and recall information across sessions. Rather than solving memory as a side effect of a larger framework, Mem0 treats it as a first-class concern with its own storage, retrieval, and lifecycle management.

The project is available both as a fully managed cloud service and as an open-source self-hosted library, allowing teams to choose between operational simplicity and deployment control.

Architecture

Mem0 is built around three integrated infrastructure components:

Vector store — encodes memories as embeddings for semantic similarity search.
Graph store — captures relationships between entities, enabling structured reasoning over linked facts (e.g., user → preference → tool).
Reranker — re-scores candidate memories after retrieval to improve relevance beyond raw vector similarity.

When a conversation is submitted, Mem0 runs it through an LLM-based extraction step that identifies facts worth persisting — user preferences, stated goals, decisions made — and stores them as discrete memory entries. On subsequent turns, relevant memories are retrieved and injected into the model's context, avoiding the need to replay full conversation history.

Memory Types

Mem0 organizes memories into three scopes:

Type	Scope	Typical content
User memory	Persistent across all sessions	Preferences, background, long-term goals
Session memory	Scoped to a single conversation	Transient context, in-session decisions
Agent memory	Associated with a specific agent	Agent state, task history, learned behaviors

This separation allows fine-grained control over what persists, for how long, and under which identity.

Memory Lifecycle

Mem0 exposes standard CRUD operations over its memory store:

Add — extract and store facts from a conversation turn.
Search — retrieve memories relevant to a query using vector + graph retrieval.
Update — revise an existing memory when new information supersedes it.
Delete — remove outdated or incorrect entries.

The LLM used during extraction also handles deduplication and contradiction resolution: if a user corrects a previously stated preference, Mem0 can update the stored fact rather than accumulating conflicting entries.

Multimodal Memory

Mem0 can extract and persist facts from non-text inputs — images, documents, and PDFs — using the same add() API. When a multimodal message is submitted, a vision-capable LLM processes the content and produces text facts that are stored and searchable in the same way as text-derived memories.

Supported input types

Type	Format
Image	JPEG, PNG, WebP, GIF
Text document	MDX, TXT, PDF

Retrieval works identically to text memories — search() and get_all() return multimodal-derived facts alongside text-derived ones.

The extraction model must support vision.

Custom Fact Extraction and Update Memory Prompts

By default, Mem0's extraction LLM decides freely what is worth remembering from a conversation. The custom_fact_extraction_prompt config key overrides this with an application-specific instruction that tells the model exactly which categories of facts to capture, how to format them, and what to ignore.

This is Mem0's equivalent of Vertex AI Memory Bank's memory topics: instead of a schema-driven config object, Mem0 uses a freeform prompt string, giving more flexibility at the cost of structure.

A companion key custom_update_memory_prompt is also available and follows the same pattern — it controls how the model resolves conflicts when new information contradicts an existing memory.

Retrieval and Context Injection

Retrieval combines semantic vector search with graph traversal, then applies reranking. The result is a ranked list of memories passed to the application, which injects them into the system prompt or augments the user message. This approach avoids prompt bloat by surfacing only the most relevant subset of stored knowledge rather than dumping the entire memory store into context.

According to Mem0's published benchmarks, this strategy yields a +26% accuracy improvement over full-context baselines, 91% faster response times, and 90% lower token usage — gains that compound as conversation history grows. These are vendor-reported results from Mem0's internal benchmarks on multi-turn conversational tasks, comparing Mem0-augmented agents against LLM-only baselines that receive the full dialogue history in the prompt. The Mem0 team reports task accuracy, end-to-end latency, and token usage on their research page, so these numbers should be treated as directional indicators rather than guarantees for all workloads.

Storage Options

Every component of the self-hosted stack is pluggable. Mem0 separates storage into four layers, each configured independently under its own key in the config dict.

Vector store

The vector store is the primary persistence layer — every memory is encoded as an embedding and written here. Qdrant is the default when no configuration is provided.

Graph store

The graph store is optional. When configured, Mem0 also writes entity relationships extracted from conversations into a property graph, enabling structured traversal queries that complement pure vector similarity.

LLM and embedder

Both the extraction LLM (used to identify facts from conversations and resolve conflicts) and the embedder (used to encode memories for vector storage) are configurable. The default LLM is OpenAI gpt-4.1-nano.

History store

Mem0 maintains a history of memory operations (add, update, delete) for audit and rollback purposes. The history store defaults to a local SQLite file.

Integrations and Deployment

Mem0 integrates with major agent frameworks including LangChain, LangGraph, CrewAI, and Google ADK. It also supports the Model Context Protocol (MCP), making it accessible to MCP-compatible hosts without framework-specific adapters.

For deployment, teams can use:

Managed platform (mem0.ai) — provides hosted infrastructure and scaling, and offers features intended to support common security and compliance needs (see the Mem0 documentation for current details).
Self-hosted (open-source) — configurable vector and graph backends for environments where data residency or customization is a priority.

Usage

Managed platform

The managed client authenticates via an API key and delegates all storage and retrieval to Mem0's cloud infrastructure:

from mem0 import MemoryClient

client = MemoryClient(api_key="m0-...")

# Store memories from a conversation turn
messages = [
    {"role": "user",      "content": "I'm a vegetarian and allergic to nuts."},
    {"role": "assistant", "content": "Got it, I'll keep that in mind."},
]
client.add(messages, user_id="alice")

# Retrieve relevant memories before the next turn
results = client.search("dietary restrictions", user_id="alice")
for m in results:
    print(m["memory"])   # e.g. "User is vegetarian and allergic to nuts"

Self-hosted configuration

The open-source Memory class accepts a config dict that wires together the LLM, embedder, vector store, and optional graph store. Each component can be swapped independently:

from mem0 import Memory

config = {
    "llm": {
        "provider": "openai",
        "config": {
            "model": "gpt-4o-mini",
            "temperature": 0.1,
            "max_tokens": 2000,
        },
    },
    "embedder": {
        "provider": "openai",
        "config": {"model": "text-embedding-3-small"},
    },
    "vector_store": {
        "provider": "qdrant",           # or pinecone, pgvector, chroma, …
        "config": {
            "collection_name": "mem0",
            "host": "localhost",
            "port": 6333,
        },
    },
    "graph_store": {                    # optional — enables relationship memory
        "provider": "neo4j",
        "config": {
            "url":      "bolt://localhost:7687",
            "username": "neo4j",
            "password": "<NEO4J_PASSWORD>",  # In production, load from env vars or a secret manager
        },
    },
}

memory = Memory(config=config)

Core operations:

user_id = "bob"

# Add — extract and persist facts from a conversation
memory.add(
    [{"role": "user", "content": "I prefer concise answers and dark-mode UIs."}],
    user_id=user_id,
)

# Search — retrieve top-k relevant memories for the next prompt
hits = memory.search("UI preferences", user_id=user_id, limit=5)
context = "\n".join(h["memory"] for h in hits)

# Get all — inspect the full memory store for a user
all_memories = memory.get_all(user_id=user_id)

# Delete — remove a specific entry by ID
memory.delete(memory_id=hits[0]["id"])

2.2. Vertex AI Agent Engine Memory Bank

Overview

Vertex AI Agent Engine Memory Bank is Google Cloud's managed memory service for AI agents, generally available since December 2025, as described in the official Vertex AI documentation. It is a component of Agent Engine — GCP's platform for deploying and operating agents in production — and is designed to work natively with Gemini models and the Agent Development Kit (ADK).

The service stores long-term, personalized memories extracted from user conversations and makes them available across sessions. Memory Bank is fully managed: there are no vector stores or graph databases to configure. All storage, indexing, and retrieval infrastructure is handled by Google Cloud.

Architecture

Memory Bank is built around a two-stage workflow: generation and retrieval.

During generation, a conversation (or a completed session) is submitted to Memory Bank. An LLM analyzes the exchange, identifies facts worth persisting, and writes them as discrete memory entries scoped to a user identity.

During retrieval, the agent queries Memory Bank before constructing its response. Memories can be fetched in two modes: listing all memories for a user, or performing a similarity search that returns the top-k entries most semantically relevant to the current query.

Both operations are mediated through the Agent Engine resource, which acts as a namespace that groups sessions, memories, and deployed agents under a single GCP project.

Memory Structure

Each memory entry is a simple fact with a scope:

Fact — a natural-language string extracted from the conversation, e.g. "User prefers concise answers".
Scope — a key–value map used for isolation and access control. At minimum, a user_id is provided; an agent_name can be added to further namespace memories per agent.
TTL — an optional time-to-live after which the memory expires automatically.
Revision history — Memory Bank tracks edits, so earlier versions of a fact can be inspected.

Memory Topics

By default, Memory Bank's extraction LLM decides freely what is worth remembering. Memory topics override this behavior by telling the extraction model exactly which categories of information to look for, how to reason about them, and what format the resulting facts should follow.

Each topic is a custom_memory_topic with two fields:

label — a short identifier used to tag and filter memories belonging to this topic.
description — a prompt-style instruction that guides the LLM during extraction: what signals to look for, what to ignore, and how to format the output fact.

Topics are combined with generate_memories_examples — few-shot pairs of a conversation snippet and the expected extracted memories — which ground the model's output in concrete examples.

Without topics, extraction is open-ended — the model may produce inconsistent fact formats, miss domain-relevant signals, or over-extract noise. Topics effectively give the extraction step a schema: memories are predictable in structure, queryable by label, and easier to validate. They are particularly useful in vertical applications where only a narrow slice of a conversation is worth persisting (purchase intent, skill level, compliance constraints, etc.).

Multimodal Memory

Memory Bank can process non-text content (images, video, and audio) in conversation events and extract textual facts from it. This follows the standard Gemini content format: each event's parts array can contain inline_data (base64-encoded bytes) or file_data (a GCS URI) alongside or instead of text parts.

Extraction behaves identically to text: the underlying Gemini model reads the visual content and writes natural-language facts into Memory Bank. Those facts are then stored, consolidated, and retrieved through the same call as any other memory.

Configuration and Usage

SDKs

Two separate libraries are involved when working with Memory Bank:

Vertex AI Agent Engine SDK (google-cloud-aiplatform, accessed via vertexai.Client) — the control-plane SDK. Use this when integrating Memory Bank with your own framework or calling the API directly.
Agent Development Kit (google-adk) — Google's open-source framework for building agent applications. It provides higher-level abstractions and pre-built Vertex AI adapters. ADK handles the session and memory plumbing automatically.

The two libraries can be used independently or together.

Generating memories from a conversation

The primary path: pass conversation events directly; Memory Bank extracts facts asynchronously.

client = vertexai.Client(project=PROJECT_ID, location=LOCATION)

operation = client.agent_engines.memories.generate(
    name=engine_name,
    direct_contents_source={
        "events": [
            {"content": {"role": "user",  "parts": [{"text": "I'm allergic to peanuts."}]}},
            {"content": {"role": "model", "parts": [{"text": "Noted — I'll always keep that in mind."}]}},
        ]
    },
    scope={"user_id": USER_ID},
    config={"wait_for_completion": True},   # set False for fire-and-forget
)

Retrieving memories

List all memories for a user:

memories = client.agent_engines.memories.retrieve(
    name=engine_name,
    scope={"user_id": USER_ID},
)
for m in memories:
    print(m.memory.fact)

Similarity search — returns the top-k entries most relevant to the current query:

memories = client.agent_engines.memories.retrieve(
    name=engine_name,
    scope={"user_id": USER_ID},
    similarity_search_params={
        "search_query": "dietary restrictions",
        "top_k": 3,
    },
)

Storage Options

Memory Bank is a fully managed service — there are no pluggable storage backends. Google Cloud handles the underlying vector index, extraction model, and data persistence transparently. Key operational knobs are:

Option	Description
Scope	`user_id` and optional `agent_name` keys isolate memories per user or per agent
TTL	Per-memory time-to-live for automatic expiration of short-lived facts
Revision tracking	Full history of memory edits; earlier versions can be retrieved
Multimodal input	Conversation events can include non-text content (images, etc.)
Region	Memories are stored within the GCP region specified at Agent Engine creation

Data isolation is enforced at the IAM level; condition-based policies can restrict which identities can read or write memories for a given scope.

Integrations

Memory Bank integrates with the three major agent frameworks through official notebooks and first-party adapters:

Framework	Integration
ADK	Native — `VertexAiMemoryBankService` and `PreloadMemoryTool` built into the ADK library
LangGraph	Via `client.agent_engines` API calls wrapped in LangGraph nodes
CrewAI	Via the same REST/SDK surface, passed as a memory backend to crew agents

The REST API also allows direct integration from any HTTP client or language without framework dependencies.

3. Memory Extraction Benchmarks

The choice of extraction LLM has a significant effect on memory volume and verbosity. To quantify this, we ran Mem0 with several extraction models and Memory Bank under two configurations over the same conversation dataset.

Methodology

To keep the comparison interpretable and reproducible, we used a simple, shared evaluation setup:

Dataset: 30 generated, English multi‑turn personal assistant conversations. Conversations ranged from 40–50 turns.
Extraction prompts: Mem0 used default extraction prompt templates across all tested models. Memory Bank was evaluated in two configurations: Default topics — the 4 built-in managed topics provided by Memory Bank out of the box; Custom topics — 1 managed topic retained alongside 3 custom topics added to refine memory categorization for the dataset. For both systems, memory generation was performed in batches of 6 dialogue turns, and we conducted a single pass over each conversation for each model/configuration.
Post‑processing: For counting, we used the text of each stored memory after the system’s own formatting/cleanup. Word totals were computed with a simple whitespace tokenizer and applied consistently to both Mem0 and Memory Bank outputs.

Mem0

Model	Memory count	Total words	Avg words/memory
`gemini-2.5-pro`	264	3,047	11
`gemini-2.5-flash`	231	2,551	11
`gemini-2.5-flash-lite`	1,503	17,824	11
`gpt-4.1`	1,542	17,457	11
`gpt-4.1-mini`	1,267	20,547	16

Vertex AI Memory Bank (gemini-2.5-flash)

Configuration	Memory count	Total words	Avg words/memory
Default topics	124	2,556	20
Custom topics	114	6,674	58

Observations:

The extraction model has an outsized effect on memory volume. Mem0 stores many short, atomic facts — the average stays consistently at ~11 words per entry regardless of model — so variation shows up in count rather than verbosity. The most selective models (gemini-2.5-flash, gemini-2.5-pro) produce 231–264 entries; the least selective (gpt-4.1, gemini-2.5-flash-lite) generate over 1,500 — roughly 6× more — suggesting weaker deduplication and less aggressive filtering. Larger models are measurably more conservative: gemini-2.5-pro extracts the fewest memories of all tested configurations, indicating that stronger reasoning leads to stricter filtering. Among tested configurations, gpt-4.1-mini offers the best practical balance for Mem0: 1,267 memories with a slightly higher average of 16 words per entry, combining reasonable selectivity with richer per-fact detail.

Memory Bank follows a different philosophy: consolidated aggregation over high-volume enumeration. With default topics it produces only 124 memories at 20 words each; with custom topics the count drops slightly to 114 but density rises sharply to 58 words per entry — because topics direct the model toward specific, richly structured information rather than short atomic statements. Both figures are far lower than most Mem0 configurations.

Practical notes:

Both systems support customization of what gets remembered. Memory Bank's memory topics give the extraction step a structured schema — each topic carries a label and a prompt-style description — which keeps output consistent and auditable. Mem0's custom_fact_extraction_prompt is more flexible: a single freeform instruction that shapes extraction without requiring a schema. Either mechanism, when carefully crafted, can substantially narrow what gets stored and improve retrieval quality.

Memory Bank is ecosystem-locked: it runs exclusively on Google Cloud and supports only Gemini models. Mem0, by contrast, works with any LLM provider and any compatible vector store — and optionally extends memory to a graph database (e.g. Neo4j) to capture entity relationships alongside factual entries, enabling structured traversal across linked facts. However, this breadth comes with rough edges: not all provider and model combinations are equally stable, and some may exhibit bugs. When reliability is a priority, sticking to well-supported configurations reduces friction.

4. Choosing a Framework

4.1. Feature comparison

	Mem0	Vertex AI Memory Bank
Deployment	Managed SaaS or self-hosted	Fully managed (GCP only)
LLM support	Any provider (OpenAI, Anthropic, Ollama, …)	Gemini only
Vector store	Set of pluggable backends	Managed by Google (not configurable)
Graph memory	Yes	No
Extraction control	Freeform prompt (`custom_fact_extraction_prompt`)	Structured topics + few-shot examples
Session integration	Manual (`add()` per turn)	Native ADK runner (`add_session_to_memory`)
Revision history	History store (SQLite / Supabase)	Built-in per-memory revision tracking
TTL	Not built-in	Yes — per-memory expiry
Multimodal	Images, documents, PDFs	Images, PDFs, text via Gemini `parts` format
TypeScript SDK	Yes	Via REST / google-adk
Data residency	Self-hosted option	GCP region selection
Compliance	SOC 2 Type II, GDPR (managed tier)	GCP IAM, audit logging

4.2. When to use Mem0

Non-GCP stack or multi-cloud — Mem0 is cloud-agnostic and works with any LLM provider or vector database.
Self-hosted requirement — when data must stay within your own infrastructure (on-prem, private cloud, custom VPC).
Non-Gemini models — using Anthropic, Ollama, Mistral, or any other provider as the extraction LLM.
Graph memory — when entity relationships between people, places, or objects need to be stored and traversed, not just individual facts.
TypeScript applications — Mem0 ships a first-class TS SDK; Memory Bank requires REST calls or ADK wrappers.
Flexible extraction schema — a single freeform prompt is easier to iterate on than a structured topic config, especially in early product stages.

4.3. When to use Vertex AI Memory Bank

GCP-native stack — if the application already runs on Vertex AI and uses Gemini models, Memory Bank adds zero infrastructure overhead.
ADK agents — Memory Bank integrates directly into the ADK Runner via VertexAiMemoryBankService and PreloadMemoryTool, making session-level memory automatic.
Structured, auditable extraction — memory topics enforce a consistent fact schema with few-shot grounding, and revision history provides a complete audit trail of every change.
TTL and automatic expiry — when some memories should expire after a fixed period (session summaries, time-bounded promotions, etc.) without manual deletion.
Multimodal conversations with Gemini — Memory Bank natively handles the Gemini parts format, so image and document content from an ADK session is ingested without format conversion.

5. Conclusion

Persistent memory is increasingly a prerequisite for production AI applications, not an optional enhancement. Mem0 and Vertex AI Memory Bank both solve the same problem — giving agents continuity across sessions — but from opposite ends of the design space. Mem0 is a flexible, infrastructure-agnostic layer that can be embedded in any stack and wired to almost any LLM or storage backend. Memory Bank is a tightly integrated managed service, optimized for GCP and Gemini, that trades configurability for operational simplicity.

The benchmarks surface a concrete difference in extraction philosophy. Mem0 produces many short, atomic facts; memory volume varies widely with the extraction model, ranging from ~230 to over 1,500 entries for the same dataset. Memory Bank produces far fewer but denser entries, synthesizing conversation context into structured, topic-anchored facts. Neither approach is strictly better — atomic storage offers finer retrieval granularity, while consolidated entries reduce noise and are more readable when injected into prompts.

For teams already on GCP building with Gemini and ADK, Memory Bank is the pragmatic choice: zero infrastructure to manage, native session integration, and built-in TTL and revision tracking. For everyone else — teams that need self-hosted deployment, non-Gemini models, graph memory, or simply more control over the extraction pipeline — Mem0 is the stronger foundation.

Technical insights from Grid Dynamics engineers

Persistent Memory for AI Agents: Mem0 vs. Vertex AI Memory Bank

1. Introduction

2. Persistent Memory Frameworks

2.1. Mem0

Overview

Architecture

Memory Types

Memory Lifecycle

Multimodal Memory

Custom Fact Extraction and Update Memory Prompts

Retrieval and Context Injection

Storage Options

Integrations and Deployment

Usage

Managed platform

Self-hosted configuration

2.2. Vertex AI Agent Engine Memory Bank

Overview

Architecture

Memory Structure

Memory Topics

Multimodal Memory

Configuration and Usage

Storage Options

Integrations

3. Memory Extraction Benchmarks

Methodology

4. Choosing a Framework

4.1. Feature comparison

4.2. When to use Mem0

4.3. When to use Vertex AI Memory Bank

5. Conclusion

Copyright © 2026 Grid Dynamics Holdings, Inc. All rights reserved.