Memory Embeddings: Semantic Search for Your Agent

Overview

Memory embeddings unlock semantic search capabilities for your OpenClaw agent, transforming it from a simple keyword matcher into an intelligent system that understands concepts and relationships. This guide explains how embeddings work, why they're essential for advanced workflows, and how to configure them properly.

What Are Memory Embeddings?

The Simple Explanation

Embeddings convert text into numbers so computers can understand meaning.

Example:

• "Pepsi" → [0.23, 0.87, 0.45, 0.12, ...] (vector of numbers)
• "soda" → [0.25, 0.85, 0.43, 0.14, ...] (similar numbers = similar meaning)
• "car" → [0.91, 0.15, 0.08, 0.73, ...] (different numbers = different meaning)

Why it matters:

• Computers are excellent with numbers, poor with words
• Similar concepts have similar vector representations
• Enables semantic search beyond exact keyword matching

The Technical Explanation

Vector embeddings are high-dimensional numerical representations of text that capture semantic meaning. Modern embedding models (like OpenAI's text-embedding-3-small) convert text into vectors of 1,536 dimensions, where:

• Distance between vectors indicates semantic similarity
• Clustering reveals related concepts
• Search finds content by meaning, not just keywords

Why Embeddings Matter

Without Embeddings (Keyword Search Only)

Query: "What's my soda preference?"

Search: Looks for exact word "soda"

Result: ❌ Nothing found (you wrote "Pepsi" not "soda")

With Embeddings (Semantic Search)

Query: "What's my soda preference?"

Search: Understands "soda" relates to "Pepsi", "Coca-Cola", "soft drink"

Result: ✅ Finds "Ron likes Pepsi diet with ice"

Real-World Impact

Scenario 1: Project recall

Query: "How did we handle user authentication?"
Without embeddings: Must use exact phrase "user authentication"
With embeddings: Finds "login system", "auth flow", "credential validation"

Scenario 2: Cross-session knowledge

Query: "What pricing model did we decide on?"
Without embeddings: Lost if you said "cost structure" instead
With embeddings: Finds related discussions regardless of exact wording

Scenario 3: Concept exploration

Query: "Show me everything about API integrations"
Without embeddings: Only finds exact phrase "API integrations"
With embeddings: Finds "REST endpoints", "webhook handlers", "third-party connections"

How OpenClaw Uses Embeddings

The Two-Layer Memory System

Layer 1: Daily memory files (keyword search)

• Basic text files in ~/.openclaw/memory/
• Fast but limited to exact matches
• Good for recent, explicit information

Layer 2: Vector database (semantic search)

• SQLite database with embeddings
• Slower but finds related concepts
• Essential for long-term, cross-session recall

Hybrid Search Strategy

OpenClaw combines both approaches:

1. Keyword search - Fast, exact matches
2. Semantic search - Slower, concept matches
3. Ranking - Best results from both methods

Result: Optimal balance of speed and accuracy

Enabling Memory Embeddings

Requirements

API Key from:

• OpenAI (recommended)
• Google Gemini (alternative)

Setup Process

Step 1: Check current status

How does your memory embedding system work?
Tell me about your vector database setup.

Expected response (if enabled):

I use a vector database stored in SQLite format at 
~/.openclaw/memory/memory.db. When you save information,
it's converted to embeddings using OpenAI's API and 
stored for semantic search.

Response if NOT enabled:

I don't currently have embeddings enabled. You'll need 
to configure an OpenAI or Gemini API key.

Step 2: Configure API key

For OpenAI:

Set up memory embeddings with my OpenAI API key: sk-...

For Gemini:

Configure memory embeddings using Gemini API key: AIza...

Step 3: Verify setup

Check for database file:

ls -la ~/.openclaw/memory/

Look for:

• memory.db or sessions.db (SQLite format)
• File size grows as you add memories
• Not human-readable (binary format)

Test semantic search:

Search my memory for anything related to "project deadlines"

Should find results even if you never used exact phrase "project deadlines"

How Embeddings Are Generated

The Embedding Pipeline

1. You save information
   "Ron prefers Pepsi diet with ice"
   ↓
2. Text is sent to embedding API
   OpenAI text-embedding-3-small
   ↓
3. API returns vector
   [0.23, 0.87, 0.45, 0.12, ..., 0.91]
   (1,536 dimensions)
   ↓
4. Vector stored in SQLite
   Indexed for fast similarity search
   ↓
5. Available for semantic queries
   "What's Ron's soda preference?"

Cost Considerations

OpenAI Pricing (text-embedding-3-small):

• $0.02 per 1M tokens
• Extremely cheap compared to LLM calls
• Typical memory entry: 100-500 tokens
• Cost per memory: ~$0.00001-0.00005

Example:

• 10,000 memory entries
• Average 300 tokens each
• Total: 3M tokens
• Cost: $0.06

Verdict: Negligible cost for massive capability boost

Memory Search Workflow

Two-Step Process

Step 1: Search (memory_search)

Agent: memory_search("pricing decision")
Result: [
  {file: "projects/saas-app.md", snippet: "...monthly subscription...", score: 0.89},
  {file: "memory/2026-04-15.md", snippet: "...tiered pricing model...", score: 0.82},
  {file: "memory/2026-03-20.md", snippet: "...cost structure analysis...", score: 0.76}
]

Step 2: Retrieve (memory_get)

Agent: memory_get("projects/saas-app.md")
Result: [Full file content with context]

Why two steps?

• Search returns snippets (fast, low token cost)
• Retrieve loads full context (slower, higher token cost)
• Agent only retrieves what's actually relevant

Search Parameters

Similarity threshold:

• 0.0 - 1.0 scale
• Higher = more strict matching
• Lower = more permissive matching

Typical thresholds:

• 0.8+ : Very similar (strict)
• 0.7-0.8 : Related concepts (moderate)
• 0.6-0.7 : Loosely related (permissive)
• <0.6 : Probably not relevant

Organizing Memory for Embeddings

Directory Structure

Recommended layout:

~/.openclaw/memory/
├── daily/
│   ├── 2026-05-01.md
│   ├── 2026-05-02.md
│   └── 2026-05-03.md
├── projects/
│   ├── project-alpha.md
│   ├── project-beta.md
│   └── project-gamma.md
├── knowledge/
│   ├── api-patterns.md
│   ├── deployment-procedures.md
│   └── troubleshooting-guides.md
├── preferences/
│   ├── coding-style.md
│   ├── communication.md
│   └── workflow.md
└── memory.db (SQLite - auto-generated)

What to Store Where

Daily memory (ephemeral):

• Today's tasks and decisions
• Temporary context
• Quick notes

Project memory (medium-term):

• Project-specific decisions
• Implementation details
• Lessons learned

Knowledge memory (evergreen):

• Reusable patterns
• Standard procedures
• Best practices

Preferences memory (permanent):

• Personal preferences
• Communication style
• Work habits

Keyword vs. Semantic Search

When Keyword Search Works

Good for:

• Exact names: "Project Apollo"
• Specific terms: "API key rotation"
• Recent information: "yesterday's meeting"
• Unique identifiers: "ticket-1234"

Example:

Query: "Find Project Apollo notes"
Keyword: ✅ Fast, exact match
Semantic: ⚠️ Overkill, slower

When Semantic Search Shines

Good for:

• Concept queries: "authentication approaches"
• Fuzzy recall: "that pricing thing we discussed"
• Cross-domain: "security best practices"
• Exploratory: "everything about deployments"

Example:

Query: "How do we handle user login?"
Keyword: ❌ Misses "authentication", "credentials", "auth flow"
Semantic: ✅ Finds all related concepts

Hybrid Strategy

OpenClaw automatically uses both:

1. Keyword search for exact matches (fast)
2. Semantic search for concept matches (thorough)
3. Merge and rank results by relevance
4. Return top N results

Best of both worlds: Speed + Intelligence

Advanced Configuration

Embedding Model Selection

OpenAI options:

• text-embedding-3-small (default, 1536 dimensions)
• text-embedding-3-large (3072 dimensions, more accurate, more expensive)

Gemini options:

• text-embedding-004 (768 dimensions)

Configuration:

Configure embeddings to use text-embedding-3-large for 
higher accuracy on technical content

Batch Embedding

For large memory imports:

# Import 100 files at once
openclaw memory import --batch ./knowledge-base/

Benefits:

• Faster than one-by-one
• More efficient API usage
• Progress tracking

Re-indexing

When to re-index:

• Changed embedding model
• Corrupted database
• Major memory reorganization

How to re-index:

Rebuild my memory embeddings from scratch

Warning: May take time for large memory directories

Integration with External Storage

Obsidian + GitHub Pattern

Architecture:

OpenClaw Memory (local)
    ↓
Obsidian (editing interface)
    ↓
GitHub (backup + version control)
    ↓
Retrieval Index (searchable)

Workflow:

1. Agent saves to memory directory
2. Obsidian syncs and displays (human-readable)
3. GitHub backs up (version control)
4. Embeddings index (searchable)

Benefits:

• Edit memories in Obsidian (better UX)
• Version control via GitHub
• Backup and sync across devices
• Semantic search via OpenClaw

Setup Guide

Step 1: Configure Obsidian vault

# Point Obsidian to OpenClaw memory directory
Vault location: ~/.openclaw/memory/

Step 2: Initialize Git

cd ~/.openclaw/memory/
git init
git remote add origin git@github.com:username/openclaw-memory.git

Step 3: Auto-sync script

#!/bin/bash
# ~/.openclaw/hooks/memory-sync.sh

cd ~/.openclaw/memory/
git add .
git commit -m "Memory update: $(date)"
git push origin main

Step 4: Configure OpenClaw hook

{
  "hooks": {
    "memory_write": "~/.openclaw/hooks/memory-sync.sh"
  }
}

Use Cases and Examples

Use Case 1: Long-Term Project Memory

Scenario: Working on multiple projects over months

Setup:

memory/
├── projects/
│   ├── project-a.md (started 3 months ago)
│   ├── project-b.md (started 1 month ago)
│   └── project-c.md (started this week)

Query:

What authentication approach did we use in Project A?

Result:

Found in projects/project-a.md (similarity: 0.91):
"Implemented OAuth 2.0 with JWT tokens. Refresh tokens 
stored in httpOnly cookies. Access tokens in memory only."

Without embeddings: Would need to remember exact file and search manually

Use Case 2: Cross-Session Learning

Scenario: Agent learns from past mistakes

Memory entry (2 months ago):

# Lesson Learned: Database Migrations

Never run migrations directly on production. Always:
1. Test on staging first
2. Create backup
3. Run during low-traffic window
4. Have rollback plan ready

Context: Production outage on 2026-03-15 due to 
untested migration breaking user authentication.

Query (today):

How should I handle this database migration?

Result:

Found relevant lesson from 2 months ago about migration 
best practices. Recommending staging test first...

Impact: Agent applies lessons learned without explicit reminders

Use Case 3: News Research Archive

Scenario: Daily news scraping with searchable archive

Setup:

memory/
├── news/
│   ├── 2026-05/
│   │   ├── 2026-05-01-ai-developments.md
│   │   ├── 2026-05-02-ai-developments.md
│   │   └── 2026-05-03-ai-developments.md

Query:

What AI model releases happened last week?

Result:

Found 7 relevant entries:
- GPT-5 announcement (2026-04-29)
- Claude 4.8 beta (2026-05-01)
- Gemini Pro 2.0 (2026-05-02)
...

Without embeddings: Would need to read all files manually

Troubleshooting

"Embeddings not working"

Symptoms:

• Semantic search returns no results
• Only exact keyword matches work
• No SQLite database file

Diagnosis:

# Check for database
ls -la ~/.openclaw/memory/*.db

# Check API key
echo $OPENAI_API_KEY

# Test embedding generation
curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "test", "model": "text-embedding-3-small"}'

Solutions:

1. Verify API key is set correctly
2. Check API key has embeddings permission
3. Ensure sufficient API credits
4. Restart OpenClaw to reload configuration

"Search returns irrelevant results"

Cause: Similarity threshold too low

Solution:

Adjust memory search threshold to 0.8 for stricter matching

Or:

Use more specific query terms to improve relevance

"Database file is huge"

Cause: Too many embeddings stored

Solutions:

Option 1: Archive old memories

# Move old daily memories to archive
mkdir ~/.openclaw/memory/archive/
mv ~/.openclaw/memory/daily/2025-* ~/.openclaw/memory/archive/

Option 2: Selective indexing

Only index files in projects/ and knowledge/ directories,
skip daily/ memories older than 30 days

Option 3: Database cleanup

Rebuild memory database, excluding archived content

"Embeddings are expensive"

Reality check:

• Embeddings cost ~$0.02 per 1M tokens
• LLM calls cost $3-15 per 1M tokens
• Embeddings are 150-750x cheaper

If still concerned:

• Use text-embedding-3-small (cheapest)
• Batch embed instead of real-time
• Archive old memories to reduce index size

Best Practices

1. Enable Embeddings Early

Don't wait - Set up embeddings from day one

Why:

• Retroactive indexing is slower
• Lose semantic search benefits during setup
• Harder to organize memory later

2. Write Descriptive Memory Entries

Bad:

# Meeting Notes
Discussed stuff. Made decisions.

Good:

# Product Roadmap Meeting - 2026-05-06

Decided to prioritize API v2 launch over mobile app.
Reasoning: 60% of users access via API, only 20% via mobile.

Key decisions:
- API v2 launch: June 2026
- Mobile app: Delayed to Q3 2026
- Focus: GraphQL endpoint for better flexibility

Why: More context = better semantic search results

3. Use Consistent Terminology

Inconsistent:

• "user login" (file 1)
• "authentication" (file 2)
• "sign-in flow" (file 3)

Consistent:

• "authentication" (primary term)
• "Also known as: login, sign-in" (aliases)

Why: Helps embeddings cluster related concepts

4. Regular Memory Maintenance

Monthly tasks:

• Archive old daily memories
• Consolidate related entries
• Remove outdated information
• Update evergreen knowledge

Why: Keeps search results relevant and database size manageable

5. Test Semantic Search

After adding important information:

Search my memory for [concept] to verify it's indexed

Ensures:

• Embeddings are working
• Information is findable
• Search quality is good

Related Resources

• Memory Management (4-Layer System)
• Context Window Management
• Skills Optimization

---

Duration: 11 minutes
Difficulty: Intermediate
Video Reference: OpenClaw Memory Embeddings EXPLAINED