# Moss — Complete Technical Reference for AI Agents > Moss is a real-time semantic search runtime for AI agents, voice agents, copilots, and multimodal apps. Sub-10ms lookups, zero infrastructure. Built in Rust and WebAssembly. Backed by Y Combinator. > This document contains everything an AI coding agent (like Claude Code, Cursor, GitHub Copilot, etc.) needs to help a developer integrate Moss into their codebase. --- ## 1. What is Moss? Moss is the runtime for real-time semantic search in conversational apps. It delivers sub-10ms lookups and instant index updates without extra infrastructure. It runs in the browser, on-device, or in the cloud — wherever your agent lives — so search feels native. Connect your data once; Moss packages, distributes, and keeps indexes fresh. Key properties: - Sub-10ms lookups with instant updates - No infra to run; local-first with optional cloud sync - Browser, device, or cloud — same API - Privacy by architecture: data stays on-device by default - Built in Rust and WebAssembly for maximum performance - Trusted by 500+ teams including Microsoft, EPAM, UC Berkeley, Carnegie Mellon, Podium, and Stanford ### The Problem Moss Solves Agents make dozens of lookups per task. At 100–500ms each, that's seconds of dead time on every turn, every user. With a remote database, your context is only as fresh as your last sync job. Moss runs search inside your agent runtime to eliminate this latency. ### Where Moss Shines - Sub-10ms answers for docs/FAQ/search - Ground agents with your data without centralizing user info - Local or hybrid embeddings; minimal infra - Voice agents & copilots: sub-10ms context retrieval for real-time conversation - Docs & knowledge search: instant semantic search inside help centers and knowledge bases - On-device & edge apps: lightweight runtime for browsers, mobile apps, desktop tools. Works offline, syncs when connected - AI-native platforms: drop into any agent framework with built-in A/B testing for embeddings --- ## 2. Installation ### JavaScript / TypeScript ```bash npm install @moss-dev/moss ``` The npm package name is `@moss-dev/moss`. ### Python ```bash pip install moss ``` The PyPI package name is `moss`. The import name is `moss`. ### Prerequisites - Node.js 16+ (for JavaScript/TypeScript) - Python 3.8+ (for Python) - Valid Moss project credentials (MOSS_PROJECT_ID, MOSS_PROJECT_KEY) --- ## 3. Authentication Moss uses project-based authentication with two credentials: - `MOSS_PROJECT_ID`: Your project identifier - `MOSS_PROJECT_KEY`: Your project authentication key ### Getting Credentials 1. Sign up at https://portal.usemoss.dev 2. Confirm your email and sign in 3. From the portal, click "Create Index" and copy your Project ID and Project Key ### Setting Environment Variables ```bash export MOSS_PROJECT_ID="your_project_id" export MOSS_PROJECT_KEY="your_project_key" ``` ### Initializing the Client **TypeScript:** ```typescript import { MossClient } from '@moss-dev/moss'; const client = new MossClient( process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY! ); ``` **Python:** ```python import os from moss import MossClient client = MossClient( os.getenv("MOSS_PROJECT_ID"), os.getenv("MOSS_PROJECT_KEY") ) ``` Authentication is used for optional cloud features like syncing or hosted embedding models. Local mode requires no network access after initial index creation. --- ## 4. Core Concepts ### Index A searchable structure that powers fast, local search. You add documents, Moss builds an efficient index for sub-10ms queries. Supports multiple indexes per project. ### Document Schema Each document has: - `id` (string): Unique identifier. Upserts replace matching ids; keep ids stable for updates. - `text` (string): The searchable content. - `metadata` (optional string map): Key-value pairs for filtering (e.g., category, lang, tags). ### Embeddings Semantic vector representations of text. Can be generated locally or via a remote model, then stored alongside your index. Available models: - `moss-minilm` (default): Fast, lightweight, ideal for edge/offline use - `moss-mediumlm`: Higher accuracy with reasonable performance Use `moss-minilm` for speed-first, edge/offline use. Use `moss-mediumlm` when you need higher recall/precision. ### Chunking Best Practices - Aim for ~200–500 tokens per chunk - Overlap 10–20% to preserve context continuity - Normalize whitespace and strip boilerplate - Smaller chunks improve recall; overlap preserves context ### Retrieval How results are fetched. Options include: - Vector similarity (pure semantic) - Keyword/BM25 (pure keyword) - Hybrid (best of both, controlled via alpha parameter) Retrieval knobs: - `top_k`: Number of results to return - `alpha`: Blend semantic (1.0) vs keyword (0.0); defaults to semantic-heavy (~0.8) - Filters: Constrain by metadata (e.g., category, lang) - Rerank: Reorder top-k for precision ### Storage & Sync Indexes are stored on-device. Optionally enable background sync to cloud for backup and sharing. Export/import indexes periodically for disaster recovery. ### Client Lifecycle Create index → upsert docs → load → query → delete when done. ### Performance Expectations - Sub-10ms local queries (hardware-dependent) - Sync is optional; compute stays on-device - Without loadIndex(): queries go to cloud API (~100-500ms) - With loadIndex(): queries run entirely in-memory (~1-10ms) --- ## 5. Quickstart — Complete Working Examples ### TypeScript Quickstart ```typescript import { MossClient, DocumentInfo } from '@moss-dev/moss'; const client = new MossClient( process.env.MOSS_PROJECT_ID!, process.env.MOSS_PROJECT_KEY! ); const indexName = "faqs"; const documents: DocumentInfo[] = [ { id: 'doc1', text: 'How do I track my order? You can track your order by logging into your account.', metadata: { category: 'shipping' } }, { id: 'doc2', text: 'What is your return policy? We offer a 30-day return policy for most items.', metadata: { category: 'returns' } }, { id: 'doc3', text: 'How can I change my shipping address? Contact our customer service team.', metadata: { category: 'support' } } ]; // Create the index (uses moss-minilm by default) await client.createIndex(indexName, documents); // Load index into memory for fast local queries await client.loadIndex(indexName); // Query const results = await client.query(indexName, 'How do I return a damaged product?', { topK: 3 }); console.log(results.docs[0].id); // "doc2" console.log(results.docs[0].text); // "What is your return policy?..." console.log(results.docs[0].score); // ~0.88 ``` ### Python Quickstart ```python import os import asyncio from moss import MossClient, DocumentInfo, QueryOptions client = MossClient( os.getenv("MOSS_PROJECT_ID"), os.getenv("MOSS_PROJECT_KEY") ) index_name = "faqs" documents = [ DocumentInfo( id="doc1", text="How do I track my order? You can track your order by logging into your account.", metadata={"category": "shipping"} ), DocumentInfo( id="doc2", text="What is your return policy? We offer a 30-day return policy for most items.", metadata={"category": "returns"} ), DocumentInfo( id="doc3", text="How can I change my shipping address? Contact our customer service team.", metadata={"category": "support"} ), ] async def main(): # Create index with moss-minilm (default); use "moss-mediumlm" for higher accuracy await client.create_index(index_name, documents, "moss-minilm") # Load index into memory for fast local queries await client.load_index(index_name) # Query with hybrid search (alpha=0.6 means 60% semantic, 40% keyword) results = await client.query(index_name, "How do I return a damaged product?", QueryOptions(top_k=3, alpha=0.6)) print(f" ID: {results.docs[0].id}") print(f" Text: {results.docs[0].text}") print(f" Score: {results.docs[0].score}") asyncio.run(main()) ``` ### Expected Output ```json { "id": "doc2", "score": 0.88, "text": "What is your return policy? We offer a 30-day return policy for most items." } ``` --- ## 6. JavaScript/TypeScript SDK — Full API Reference Package: `@moss-dev/moss` ### MossClient Class ```typescript import { MossClient } from '@moss-dev/moss'; const client = new MossClient(projectId: string, projectKey: string); ``` ### Methods #### createIndex(indexName, docs, options?) Creates a new index with the provided documents. Handles the full flow: init → upload → build → poll until complete. ```typescript const result = await client.createIndex('knowledge-base', [ { id: 'doc1', text: 'Introduction to AI' }, { id: 'doc2', text: 'Machine learning basics' } ], { onProgress: (p) => console.log(`${p.status} ${p.progress}%`), }); ``` Parameters: - indexName (string): Name of the index to create - docs (DocumentInfo[]): Documents, optionally with pre-computed embeddings - options? (CreateIndexOptions): Optional model ID and progress callback Returns: Promise Throws: If the index already exists or creation fails. #### loadIndex(indexName, options?) Downloads an index from the cloud into memory for fast local querying (~1-10ms instead of ~100-500ms). ```typescript // Simple load await client.loadIndex('my-index'); // Load with auto-refresh to keep index up-to-date await client.loadIndex('my-index', { autoRefresh: true, pollingIntervalInSeconds: 300, // Check cloud every 5 minutes }); ``` Parameters: - indexName (string): Name of the index to load - options? (LoadIndexOptions): Optional auto-refresh settings Returns: Promise #### query(indexName, query, options?) Performs semantic similarity search. If index is loaded via loadIndex(), runs in-memory. Otherwise falls back to cloud. ```typescript const results = await client.query('knowledge-base', 'machine learning'); results.docs.forEach(doc => { console.log(`${doc.id}: ${doc.text} (score: ${doc.score})`); }); ``` Parameters: - indexName (string): Name of the index to search - query (string): The search query text - options? (QueryOptions): topK (default: 5), embedding overrides Returns: Promise #### addDocs(indexName, docs, options?) Adds or updates documents in an index asynchronously. Polls until rebuild is complete. ```typescript const result = await client.addDocs('knowledge-base', [ { id: 'new-doc', text: 'New content to index' } ], { upsert: true }); ``` Parameters: - indexName (string): Name of the target index - docs (DocumentInfo[]): Documents to add or update - options? (MutationOptions): upsert boolean, onProgress callback Returns: Promise #### deleteDocs(indexName, docIds, options?) Deletes documents from an index by their IDs asynchronously. ```typescript const result = await client.deleteDocs('knowledge-base', ['doc1', 'doc2']); ``` #### getDocs(indexName, options?) Retrieves documents from an index. ```typescript // Get all documents const allDocs = await client.getDocs('knowledge-base'); // Get specific documents const specificDocs = await client.getDocs('knowledge-base', { docIds: ['doc1', 'doc2'] }); ``` #### getIndex(indexName) Gets information about a specific index. ```typescript const info = await client.getIndex('knowledge-base'); console.log(`Index has ${info.docCount} documents`); ``` Returns: Promise #### listIndexes() Lists all available indexes. ```typescript const indexes = await client.listIndexes(); indexes.forEach(index => { console.log(`${index.name}: ${index.docCount} docs`); }); ``` Returns: Promise #### deleteIndex(indexName) Deletes an index and all its data. ```typescript const deleted = await client.deleteIndex('old-index'); ``` Returns: Promise #### getJobStatus(jobId) Gets the current status of an async job. ```typescript const status = await client.getJobStatus(jobId); console.log(`${status.status} — ${status.progress}%`); ``` Returns: Promise ### TypeScript Interfaces ```typescript interface DocumentInfo { id: string; text: string; metadata?: Record; } interface QueryOptions { topK?: number; // default: 5 embedding?: number[]; // optional pre-computed embedding } interface SearchResult { docs: QueryResultDocumentInfo[]; } interface QueryResultDocumentInfo { id: string; text: string; score: number; metadata?: Record; } interface IndexInfo { name: string; docCount: number; // additional fields may be present } interface MutationResult { jobId: string; } interface MutationOptions { upsert?: boolean; onProgress?: (progress: JobProgress) => void; } interface CreateIndexOptions { modelId?: string; // "moss-minilm" (default) or "moss-mediumlm" onProgress?: (progress: JobProgress) => void; } interface LoadIndexOptions { autoRefresh?: boolean; pollingIntervalInSeconds?: number; } interface GetDocumentsOptions { docIds?: string[]; } interface JobProgress { status: string; progress: number; } interface JobStatusResponse { status: string; progress: number; } ``` --- ## 7. Python SDK Reference Package: `moss` (pip install moss) Import: `from moss import MossClient, DocumentInfo, QueryOptions, AddDocumentsOptions, GetDocumentsOptions` ### MossClient Class ```python from moss import MossClient client = MossClient(project_id: str, project_key: str) ``` ### Methods (all async) #### create_index(index_name, documents, model_id="moss-minilm") ```python await client.create_index("faqs", documents, "moss-minilm") ``` #### load_index(index_name) ```python await client.load_index("faqs") ``` #### query(index_name, query_text, options?) ```python from moss import QueryOptions # Hybrid search (60% semantic, 40% keyword) results = await client.query("faqs", "return policy", QueryOptions(top_k=3, alpha=0.6)) # Pure keyword results = await client.query("faqs", "return policy", QueryOptions(top_k=3, alpha=0.0)) # Pure semantic results = await client.query("faqs", "return policy", QueryOptions(top_k=3, alpha=1.0)) # With pre-computed embedding (BYOE) results = await client.query("faqs", "return policy", QueryOptions(embedding=my_embedding, top_k=3)) ``` The alpha parameter controls hybrid weighting: - alpha=1.0: pure semantic (embeddings only) - alpha=0.0: pure keyword (BM25 only) - Values between 0 and 1 blend both (default is semantic-heavy, ~0.8) #### add_docs(index_name, documents, options?) ```python from moss import AddDocumentsOptions await client.add_docs("my-index", [ {"id": "doc-2", "text": "updated text"}, {"id": "doc-3", "text": "new text"} ], AddDocumentsOptions(upsert=True)) ``` #### get_docs(index_name, options?) ```python from moss import GetDocumentsOptions subset = await client.get_docs("my-index", GetDocumentsOptions(doc_ids=["doc-1", "doc-3"])) ``` #### delete_docs(index_name, doc_ids) ```python await client.delete_docs("my-index", ["doc-3"]) ``` #### delete_index(index_name) ```python await client.delete_index("my-index") ``` ### Python Types ```python from moss import DocumentInfo, QueryOptions, AddDocumentsOptions, GetDocumentsOptions # DocumentInfo DocumentInfo(id="doc1", text="some text", metadata={"category": "faq"}) # QueryOptions QueryOptions(top_k=3, alpha=0.6, filters={"category": "faq"}) # AddDocumentsOptions AddDocumentsOptions(upsert=True) # GetDocumentsOptions GetDocumentsOptions(doc_ids=["doc-1", "doc-3"]) ``` ### Metadata Filtering (Python) ```python # Filter by metadata during query filtered = await client.query("my-index", "refund policy", QueryOptions(top_k=5, filters={"category": "faq", "lang": "en"})) ``` --- ## 8. REST API Reference Base URL: https://service.usemoss.dev/v1 The Moss Control Plane API powers all index lifecycle operations. It exposes a single authenticated control plane endpoint for managing indexes. ### Endpoints #### Index Management - Init Upload: Initialize an upload for index creation - Start Build: Trigger index building after upload - Get Job Status: Check async job progress - Get Index: Retrieve index information - List Indexes: List all indexes in the project - Delete Index: Remove an index and all its data #### Document Operations - Add Documents: Add or upsert documents to an existing index - Get Documents: Retrieve documents from an index - Delete Documents: Remove documents by ID Full REST API documentation: https://docs.moss.dev/docs/api-reference/v1/getting-started/introduction --- ## 9. Integration Guide — Voice Agent with LiveKit Build a voice AI agent with sub-10ms semantic retrieval using Moss + LiveKit + OpenAI + Deepgram. ### Required Tools - Moss (semantic search) - LiveKit (voice infrastructure) - OpenAI (LLM) - Deepgram (speech-to-text) ### Step 1: Install Dependencies ```bash pip install moss python-dotenv ``` ### Step 2: Environment Setup Create a .env file: ``` # LiveKit Credentials (for local dev) LIVEKIT_URL=ws://localhost:7880 LIVEKIT_API_KEY=devkey LIVEKIT_API_SECRET=secret # Moss Credentials MOSS_PROJECT_ID=your-moss-id MOSS_PROJECT_KEY=your-moss-key # AI Provider Keys OPENAI_API_KEY=sk-... DEEPGRAM_API_KEY=your-deepgram-key ``` ### Step 3: Build the Knowledge Base ```python import asyncio import os from dotenv import load_dotenv from moss import MossClient, DocumentInfo load_dotenv() async def main(): client = MossClient( project_id=os.environ["MOSS_PROJECT_ID"], project_key=os.environ["MOSS_PROJECT_KEY"] ) index_name = os.getenv("MOSS_INDEX_NAME", "product-knowledge") docs = [ DocumentInfo(id="1", text="Our return policy allows returns within 30 days of purchase with a receipt."), DocumentInfo(id="2", text="Standard shipping takes 3-5 business days. Express shipping takes 1-2 days."), DocumentInfo(id="3", text="Technical support is available 24/7 via email at support@example.com."), ] print(f"Creating index '{index_name}'...") await client.create_index(index_name, docs, model_id="moss-minilm") print("Index created successfully.") if __name__ == "__main__": asyncio.run(main()) ``` ### Step 4: Build the Voice Agent Uses a Context Injection pattern — Moss is queried automatically on every user message and results are injected into the LLM context. This is faster than tool calling because there's no LLM "thinking" step to decide whether to search. ```python import asyncio import logging import os from dotenv import load_dotenv from livekit.plugins import openai, deepgram, silero from livekit.plugins.turn_detector.english import EnglishModel from livekit.agents import ( JobContext, WorkerOptions, cli, ChatContext, ChatMessage, Agent, AgentSession, ) from moss import MossClient load_dotenv() MOSS_PROJECT_ID = os.getenv("MOSS_PROJECT_ID") MOSS_PROJECT_KEY = os.getenv("MOSS_PROJECT_KEY") INDEX_NAME = os.getenv("MOSS_INDEX_NAME", "product-knowledge") logger = logging.getLogger("moss-agent") class MossSemanticRetrievalAgent(Agent): def __init__(self, moss_client: MossClient): super().__init__( instructions="""You are a helpful customer support voice assistant. You have access to a knowledge base which will be provided to you as context. Always answer the user's question based on the provided context. If the context doesn't contain the answer, politely say you don't know.""" ) self.moss = moss_client async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage) -> None: user_query = new_message.text_content try: results = await self.moss.query(INDEX_NAME, user_query, top_k=5, alpha=0.8) if results.docs: context_str = "\n".join([f"- {d.text}" for d in results.docs]) injection = f"Relevant context from knowledge base:\n{context_str}\n\nUse this to answer the user." turn_ctx.add_message(role="system", content=injection) except Exception as e: logger.error(f"Moss search failed: {e}", exc_info=True) await super().on_user_turn_completed(turn_ctx, new_message) async def entrypoint(ctx: JobContext): await ctx.connect() moss_client = MossClient(project_id=MOSS_PROJECT_ID, project_key=MOSS_PROJECT_KEY) await moss_client.load_index(INDEX_NAME) session = AgentSession( stt=deepgram.STT(), llm=openai.LLM(model="gpt-4o"), tts=openai.TTS(), turn_detection=EnglishModel(), vad=silero.VAD.load(), ) await session.start(agent=MossSemanticRetrievalAgent(moss_client), room=ctx.room) if __name__ == "__main__": cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint)) ``` ### Step 5: Run ```bash # Terminal 1: Start LiveKit server livekit-server --dev # Terminal 2: Start agent python agent.py download-files python agent.py console ``` --- ## 10. Deployment / Production Checklist - Configure API keys via environment variables (never hardcode) - Persist indexes to a durable path - Monitor index size and query latency (track p50/p95) - Enable sync (optional) and test offline mode - Add health checks for embedding/runtime services - Keep data local whenever possible - Encrypt synced data at rest/in transit - Log index lifecycle events (create/load/delete, rebuilds) - Establish backup/export cadence for disaster recovery --- ## 11. Common Patterns ### Upsert Documents (Add or Update) ```python await client.add_docs("my-index", [ {"id": "doc-2", "text": "updated text"}, {"id": "doc-3", "text": "new text"} ], AddDocumentsOptions(upsert=True)) ``` ### Metadata Filtering ```python filtered = await client.query("my-index", "refund policy", 5, { "filters": {"category": "faq", "lang": "en"} }) ``` ### Auto-Refresh Index (JS) ```typescript await client.loadIndex('my-index', { autoRefresh: true, pollingIntervalInSeconds: 300, }); ``` ### Multiple Indexes ```python await client.create_index("faq-index", faq_docs, "moss-minilm") await client.create_index("product-index", product_docs, "moss-mediumlm") await client.load_index("faq-index") await client.load_index("product-index") ``` --- ## 12. Bring Your Own Embeddings (BYOE) Moss supports custom embeddings — generate vectors with any external model (OpenAI, Cohere, a local model, etc.) and pass them directly. Moss stores and searches against your embeddings instead of generating its own. ### When to Use BYOE - You already have an embedding pipeline and want consistency across your stack - You need a domain-specific or fine-tuned embedding model (e.g., for aviation, medical, legal) - You want to use a high-dimensional model like `text-embedding-3-large` for maximum recall ### Install ```bash pip install moss openai python-dotenv ``` ### Python — Full BYOE Example ```python import os import asyncio from dotenv import load_dotenv from moss import MossClient, DocumentInfo, QueryOptions from openai import OpenAI load_dotenv() moss = MossClient( os.getenv("MOSS_PROJECT_ID"), os.getenv("MOSS_PROJECT_KEY") ) openai_client = OpenAI(api_key=os.getenv("OPENAI_API_KEY")) def get_embedding(text: str) -> list[float]: response = openai_client.embeddings.create( model="text-embedding-3-small", input=text ) return response.data[0].embedding async def main(): index_name = "pilot-docs" # Index documents with your own pre-computed embeddings docs = [ DocumentInfo( id="doc-1", text="Standard instrument departure procedures require...", embedding=get_embedding("Standard instrument departure procedures require...") ), DocumentInfo( id="doc-2", text="Weather minimums for IFR flight are defined as...", embedding=get_embedding("Weather minimums for IFR flight are defined as...") ), ] await moss.create_index(index_name, docs) # Load index into memory for sub-10ms queries await moss.load_index(index_name) # Query using your own embedding for the query too query_text = "what are the IFR weather minimums?" results = await moss.query( index_name, query_text, QueryOptions(embedding=get_embedding(query_text)) ) for doc in results.docs: print(f"[{doc.score:.3f}] {doc.id}: {doc.text[:80]}") asyncio.run(main()) ``` Full working example: https://github.com/usemoss/moss-samples/blob/main/python/custom_embedding_sample.py ### Key Points - Pass `embedding` on each `DocumentInfo` at index creation time to use your vectors - Pass `QueryOptions(embedding=...)` at query time so Moss compares against your embedding space - You must use the same model for both indexing and querying — mixing models will degrade results - The `model_id` parameter in `create_index` is ignored when all documents carry pre-computed embeddings --- ## 13. Pricing Summary - Developer (Free): $5/month free credits, unlimited local queries, shared infra, community support - Hobbyist ($30/mo): Continuous sync engine, unlimited projects & indexes, session replays (7 days), file uploads - Start-up ($200/mo): Hot path cloud search, 150 concurrent sessions, session replays (30 days), priority ingest, email support - Enterprise (Custom): Custom scale, white glove onboarding, 99.9% SLA, SSO, 24/7 Slack support, SOC2 & HIPAA compliance Full pricing details: https://www.moss.dev/pricing --- ## 14. Links & Resources - Website: https://www.moss.dev - Documentation: https://docs.moss.dev/docs - Moss Portal (sign up / dashboard): https://portal.usemoss.dev - GitHub: https://github.com/usemoss - Sample Code: https://github.com/usemoss/moss-samples - Discord Community: https://moss.link/discord - Release Notes: https://docs.moss.dev/docs/changelog - Privacy Policy: https://docs.moss.dev/docs/privacy - Terms of Service: https://docs.moss.dev/docs/tos --- ## 15. Company Moss is built by InferEdge Inc., based in San Francisco, CA. Backed by Y Combinator.