Aside is building an AI browser that remembers what users do, turning browsing history and past tasks into persistent context that agents can use to take action. The product is growing 25% week over week, with its memory infrastructure now spanning more than 80,000 devices across 150+ countries. Every month, Aside processes approximately 113M documents and 24.8B tokens directly on device, giving its agents access to a growing body of private, searchable context.
Making that work at this scale meant building memory infrastructure that could embed and retrieve large amounts of information directly on users’ devices, without sending sensitive data to the cloud. Moss provides that retrieval layer, including the local semantic indexing, embedding infrastructure, and retrieval engine that sits underneath Aside’s memory system.
I spoke with Jun Kim, Co-Founder and CEO of Aside, about why persistent memory is becoming an important part of AI agents, how Aside built its memory architecture, and why local retrieval became a critical part of the product.
“The kind of product picking, I believe, is not achievable without a good memory layer. That's why we invest a lot in memory.”
Aside’s mission: Give AI agents the memory to understand how people work
AI agents are becoming increasingly capable at completing tasks, but they still need context to understand the work behind those tasks. A user might spend an hour working with an agent on a project, only to have to explain the same context the next time they open it. The agent may know how to complete the task, but it does not necessarily know what the user was working on yesterday, who they were talking to, or what happened the last time they asked it to do something.
Aside is tackling that problem through the browser. Because so much of modern work already happens in the browser, it gives Aside access to a rich source of context without requiring users to connect every tool they use. The browser can learn from browsing history and past tasks, building a persistent understanding of the websites, projects, people, conversations, and work that matter to each user.
As Jun explains:
“Browser is our daily driver, so we use browser every day, and we do 80% of work on the browser. But at the same time, browser is the most richest context source.”
The goal is an AI browser that can carry context from one task into the next, rather than treating every interaction as a blank slate.
113M documents and 24.8B tokens every month: Building memory on device
Aside’s memory system extracts information from individual interactions and turns it into episodic memory. It organizes that information into different types of context, including people, projects, websites, and user-specific information. When information does not fit neatly into an existing category, the system can evolve its own taxonomy, creating what Jun describes as a “self organizing memory.”
The challenge is figuring out what information is useful, organizing it so an agent can understand it, and retrieving the right context when it matters. Aside built its own Markdown semantic chunker to break individual memory entries into retrievable chunks, then uses Moss for the local embedding, semantic indexing, and retrieval. The team was specifically looking for three things from its memory infrastructure: retrieval quality, local execution, and speed.
Local execution was particularly important because Aside’s memory can contain browsing history, work context, and other sensitive information. The team did not want that data sent to a cloud memory service simply so it could be searched.
“It has to be local. I didn't want to use cloud back memory at the moment because it is sensitive data.”
With Moss, Aside keeps memory on the user’s device while giving its agents semantic search across a growing amount of context. Today, that means processing around 113M documents and 24.8B tokens each month across more than 80,000 devices in 150+ countries. The volume is growing 25% week over week as more users rely on Aside to remember their browsing history and past work.
80,000+ devices across 150+ countries: Scaling local retrieval
Running retrieval locally creates a different set of infrastructure requirements. Aside needs its memory system to work across a distributed fleet of devices while maintaining the performance and reliability expected from an AI agent that is constantly accessing context. Embedding and semantic retrieval also need to happen locally rather than relying on a centralized cloud database.
Aside considered building its own retrieval infrastructure, but the question was less about whether the team could build it and more about where its engineering resources were best spent. Retrieval infrastructure is a specialized area that spans embedding models, semantic indexing, local execution, and fast retrieval across a wide range of devices. Building and maintaining that infrastructure internally would also mean taking on the ongoing work of optimizing it as Aside’s device footprint and memory volume continued to grow.
The team wanted to focus its engineering effort on the parts of Aside that make the product what it is: the AI browser, agent harness, memory extraction and organization system, and password manager. Moss handles the underlying retrieval infrastructure, while Aside controls how memory is created, organized, and ultimately used by its agents.
<10ms query latency: Keeping retrieval out of the way
For an agent, memory is only useful if retrieving it does not slow down the experience. As Aside’s memory system grows, retrieval needs to happen quickly enough that accessing historical context feels like a natural part of using the agent rather than a separate step. Moss provides query latency of less than 10 milliseconds, giving Aside a fast retrieval layer for its growing on-device memory system.
For Aside, the goal is for users to stop thinking about whether the agent remembers something or whether they need to explain it again. The relevant context should simply be available when the agent needs it.
Turning persistent memory into action
The value of persistent memory becomes more apparent when it changes what the user has to tell the agent. During our conversation, Jun showed me a scenario involving an investor who had asked for a financial and investor update. Instead of explaining who the person was, finding the relevant email thread, and describing what needed to happen, Jun simply told Aside to handle what the investor had sent.
Aside remembered who the person was, found the relevant email thread, understood the surrounding context, and began coordinating the work.
“I was really surprised, because Aside handled the exact same thing I wanted with only four words.”
That is what makes persistent memory useful in practice. The agent is not just storing more information; it is reducing the amount of context the user has to provide every time they want something done. Because Aside is a browser, that memory can also connect directly to action. Users do not necessarily need to connect Gmail, Google Drive, GitHub, Google Calendar, Slack, and other individual tools before asking the agent to work across them. The browser already has access to the websites and credentials needed to carry out the work.
“You don't have to connect anything, and you don't have to type a long prompt thanks to memory layer and our password manager credential system.”
Four words instead of a paragraph
Aside does not treat memory as the only source of truth. If a memory retrieval does not return exactly what the agent needs, it can continue searching for context by opening the relevant website and verifying the information directly. Memory gives the agent a starting point: it helps it understand the user’s history, identify where relevant information may exist, and decide what to look for next.
That means memory does not have to contain every piece of information forever. It needs to give the agent enough context to understand what the user means and take the next step without making the user start from scratch. In practice, that moves the interaction away from lengthy prompts and toward simply telling the agent what needs to get done.
A local memory layer for AI agents
Memory is becoming a core part of how Aside works. The company has built its own browser and agent harness, memory extraction and organization system, and password manager, while using Moss for the infrastructure underneath memory retrieval.
That separation lets Aside focus its engineering effort on the parts of the product that are unique to its approach, while relying on specialized infrastructure for embedding, indexing, and local retrieval across a growing number of devices. Keeping the memory on device also means sensitive user information does not need to be sent to a cloud memory service simply to make it searchable. Jun said this has opened up interest from companies operating in areas with stricter security requirements, including finance and law.
For Aside, memory is becoming an increasingly important part of what makes an AI browser useful.
“The kind of product picking, I believe, is not achievable without a good memory layer. That's why we invest a lot in memory.”
Today, Moss provides the infrastructure underneath that memory system, helping Aside embed and retrieve 113M documents and 24.8B tokens every month across more than 80,000 devices while keeping memory local to the user.
Built with Moss
Aside is building an AI browser that remembers users’ work, turning browsing history and past tasks into persistent context that agents can use across future tasks.
Moss provides the local memory retrieval infrastructure behind Aside, including high-performance embedding, semantic indexing, and retrieval running directly on device.
Building an AI agent that needs fast, private, scalable memory? Get in touch
