How ItzCrazyKns/Perplexica Works

Core Flow: Answering a User's Query

This flow describes the primary user journey of asking a question and receiving a comprehensive, cited answer. The user enters a query, and the system initiates a sophisticated AI agent pipeline. First, it classifies the query's intent to decide whether a web search is needed and what sources to use. It then executes research tasks (like web searches) and helpful widgets (like a calculator) in parallel to optimize for speed. The user sees real-time updates as research steps are completed and widgets provide instant results. Once enough information is gathered, the system generates a final answer, tightly grounded in the collected sources with inline citations, and streams it token-by-token to the user. Finally, the entire interaction, including all research steps and the final answer, is persisted to a local database for future reference.

1. User submits a new query.
2. System classifies the query's intent to determine the required search scope and tools.
3. System begins the iterative research loop, dynamically selecting and using search tools (web, academic, files).
4. System streams intermediate UI blocks for research steps and widget results in real-time.
5. System synthesizes all gathered information and streams the final, cited answer to the user.
6. System provides contextual follow-up suggestions to continue the conversation.
7. System persists the full conversation and its generated answer blocks to the local database.

RAG Flow: Asking a Question About an Uploaded Document

This flow enables users to leverage Perplexica as a personal knowledge base. The user begins by uploading one or more documents (PDF, DOCX, etc.). The system processes these files locally, extracting text, chunking it, and generating embeddings, all without external dependencies. When the user subsequently asks a question related to the document's content, the system's classifier recognizes the intent for a 'personal search'. The AI agent then uses a specialized semantic search tool to query the indexed document chunks directly. This retrieves the most relevant passages, which are then used as the primary source material to generate a precise answer, effectively allowing the user to have a conversation with their own documents.

1. User uploads a document (e.g., a PDF) via the chat interface.
2. System ingests the document, extracting text, chunking it, and generating embeddings locally.
3. User asks a question related to the content of the uploaded document.
4. System classifies the query as a 'personal search', enabling the file search tool.
5. AI agent uses the semantic search tool to find relevant chunks within the document.
6. System generates an answer based on the retrieved document chunks, citing the file as the source.

Admin Flow: Configuring a New AI Provider

This flow details the process for an administrator or power user to customize the application's AI backend. The user navigates to the settings panel, where they can manage AI model providers. They can add a new provider by selecting its type (e.g., 'Ollama' for a local model or 'OpenAI' for a cloud service) and entering the necessary configuration, such as the API URL or secret key. Upon saving, the system securely validates the configuration and attempts to connect to the provider to fetch a list of available models. If successful, the new provider and its models immediately become available for use in the chat interface. This flow is critical for enabling the product's core value of customizability and control over the AI stack.

1. User navigates to the Settings page and selects the 'Model Providers' section.
2. User clicks 'Add Provider' and fills in the form with the provider type and credentials.
3. System receives the configuration, validates it, and attempts to initialize the new provider.
4. System fetches the list of available chat and embedding models from the new provider.
5. The new provider and its models appear in the selection dropdown in the main chat interface, ready for use.

Engagement Flow: Using the Discover Feed

This flow is designed for content discovery and user engagement. The user navigates to the 'Discover' page and selects a topic of interest, such as 'Technology' or 'Finance'. The system then performs a series of targeted searches using pre-defined keywords and trusted websites related to that topic. It collects the results, removes duplicates, and presents a randomly shuffled feed of relevant articles. When the user clicks on an article they find interesting, a new tab is opened, automatically launching Perplexica's core answering engine with a pre-filled query to 'Summarize' the article's URL, seamlessly connecting content discovery to the app's primary function.

1. User navigates to the 'Discover' page and selects a topic.
2. System performs targeted searches on news engines for that topic, deduplicates, and shuffles the results.
3. The user is presented with a feed of article cards.
4. User clicks an article card, which opens a new tab with a pre-filled summary query for that article's URL.

AI Search & Answering Engine

This is the core orchestration module that manages the entire process of turning a user's question into a cited, well-reasoned answer. It employs a multi-step AI agent pipeline that first classifies the user's intent to form a plan, then executes research and other tasks in parallel to gather information, and finally synthesizes the findings into a comprehensive, streaming response. The design prioritizes both response quality and perceived speed by showing users intermediate steps while the full answer is being composed.

• Query Intent Classification — 【User Value】Prevents an expensive and slow web search for simple questions that don't require it (e.g., 'hello'), and ensures the right research tools are used for complex questions (e.g., academic papers for a scientific query). 【Design Strategy】Before any action is taken, the system uses an AI model to analyze the user's query in the context of the conversation. This classification step acts as a smart dispatcher, creating a plan for the rest of the system to follow. 【Business Logic】 - Step 1: The user's query and conversation history are sent to an LLM with a specific prompt designed for classification. - Step 2: The LLM is forced to return a structured JSON object that contains a series of boolean flags. - Step 3: These flags determine the subsequent workflow: - `skipSearch`: Decides if a web search is necessary at all. - `academicSearch` / `discussionSearch` / `personalSearch`: Determines which specific information sources to prioritize. - `showWeatherWidget` / `showStockWidget`: Decides if a specialized UI widget should be activated. - `standaloneFollowUp`: The query is rewritten to be context-independent for better search results.
• Adaptive Iterative Research — 【User Value】Provides a flexible tradeoff between speed and answer quality. Users can opt for a quick answer for simple queries or a deep, comprehensive report for research-heavy topics, directly controlling cost and wait time. 【Design Strategy】The system employs an iterative research loop where an AI agent repeatedly uses search tools to gather information until it decides it has enough context to form a high-quality answer. The depth of this research is directly tied to a user-selectable mode. 【Business Logic】 - Step 1: The user selects a search mode: 'Speed', 'Balanced', or 'Quality'. - Step 2: The system starts a research loop with a maximum number of iterations based on the selected mode: - Speed Mode: 2 iterations (for quick, direct answers). - Balanced Mode: 6 iterations (for standard, well-rounded answers). - Quality Mode: 25 iterations (for in-depth research). - Step 3: In each iteration, the AI agent can use available search tools. The loop continues until one of three conditions is met: - The agent explicitly calls a 'done' tool, signaling it has sufficient information. - The maximum number of iterations for the mode is reached. - The agent produces no further actions, indicating it has concluded its research.
• Grounded and Cited Answer Generation — 【User Value】Builds trust and allows for verification by ensuring that the AI-generated answer is transparently linked back to the source information it used. This transforms the AI from a black box into an auditable research assistant. 【Design Strategy】After the research phase is complete, the system compiles all gathered information into a structured context for a final 'writer' AI. This AI is given strict instructions to generate an answer based *only* on the provided sources and to include inline citations for every statement. 【Business Logic】 - Step 1: All findings from the research phase are formatted into a context block, with each piece of information numbered and linked to its source URL and title. - Step 2: A separate context block is created for information from widgets (like a calculator result), with explicit instructions not to cite these as research sources. - Step 3: This combined context is sent to an LLM with a system prompt that enforces strict rules: - The answer must include inline citations in the format `[number]`, corresponding to the numbered sources. - Every sentence must be backed by at least one citation. - If no relevant information is found, the AI must output a specific apology rather than hallucinating. - Step 4 (Quality Mode): If the user selected 'Quality' mode, an additional instruction is added, requiring the response to be a comprehensive report of at least 2000 words.

Multi-Source Research

This module contains the set of tools the AI agent uses to gather information from the outside world. It is designed to be dynamic and targeted, allowing the agent to select the most appropriate source for a given query, from general web search to specialized academic databases or discussion forums. All external searches are routed through the SearxNG metasearch engine to preserve user privacy.

• Dynamic Research Action System — 【User Value】Ensures the AI uses the most relevant and efficient tools for the job, preventing it from searching academic papers for a question about the weather, or vice-versa. This improves both the speed and relevance of the information gathering process. 【Design Strategy】A central registry maintains a list of all available research tools (actions). Before the research loop begins, this registry filters the list to activate only the tools that are relevant to the user's current query and settings. 【Business Logic】 - Step 1: The system maintains a registry of all possible research actions (e.g., general web search, academic search, discussion search, URL scraping). - Step 2: When a user query is received, the registry dynamically determines which actions should be made available to the AI agent based on a set of conditions: - **User Configuration:** Is the 'academic' source enabled in the user's settings? - **Query Classification:** Did the initial classification step flag this query as needing academic or discussion-based sources? - **Search Mode:** Certain actions, like initial planning, are disabled in 'Speed' mode to save time. - **User Input:** Is there a local file attached to the query? If so, the file search action is enabled. - Step 3: Only the enabled actions are presented to the AI agent as callable tools during the research loop.
• Targeted Source Search — 【User Value】Improves the quality and relevance of search results by querying the right type of source for the user's intent, such as finding scientific papers for research questions or community opinions for product reviews. 【Design Strategy】Instead of a single, generic web search, the system provides specialized search actions that query specific categories of search engines through the underlying SearxNG integration. 【Business Logic】 - **General Web Search:** This is the default action for general-purpose queries. It calls the SearxNG API without any engine restrictions. - **Academic Search:** This action is used for research-intensive queries. It specifically instructs SearxNG to query only a pre-defined list of academic engines (e.g., 'arxiv', 'google scholar', 'pubmed'). This action is disabled in 'Speed' mode. - **Discussion Search:** This action is used to find opinions and community discussions. It queries SearxNG targeting only discussion forums like 'reddit'.
• Direct URL Content Extraction — 【User Value】Allows users to ask detailed questions about a specific webpage by providing its URL, enabling the AI to analyze content that might not be fully captured in a search engine's summary. 【Design Strategy】A dedicated tool is provided to scrape the full content of a given URL and convert it into a clean, analyzable format for the AI. 【Business Logic】 - Step 1: The AI agent can decide to use the `scrape_url` tool with one or more URLs. - Step 2: The system fetches the full HTML content of each URL. - Step 3: The HTML is converted into clean Markdown format, stripping away unnecessary layout and script elements. - Step 4: The resulting Markdown text is returned to the agent as a numbered source, ready to be used for generating the final answer.

Personal Knowledge Base (File Uploads)

This module provides a complete Retrieval-Augmented Generation (RAG) pipeline that allows users to upload their own documents (PDFs, DOCX, TXT) and ask questions about them. The entire process, from file ingestion to semantic search, is handled locally on the file system, reinforcing the product's privacy-first commitment without requiring an external vector database.

• Multi-Format Document Ingestion and Embedding — 【User Value】Users can easily create a personal knowledge base from their existing documents in common formats, without needing to manually copy-paste text or use conversion tools. 【Design Strategy】An automated pipeline processes uploaded files by extracting their text content, breaking it into semantically meaningful chunks, and generating vector embeddings for each chunk to enable semantic search. 【Business Logic】 - Step 1: The user uploads one or more files (PDF, DOCX, or TXT). - Step 2: For each file, the system extracts the raw text content using the appropriate parser. - Step 3: The extracted text is split into smaller chunks. The chunking strategy is token-aware, aiming for a maximum of 512 tokens per chunk with a 64-token overlap between adjacent chunks. This overlap helps preserve context across chunk boundaries. - Step 4: The system uses the configured embedding model to generate a vector embedding for each text chunk. - Step 5: The original file is saved, and a companion JSON file is created alongside it, containing all the text chunks and their corresponding embeddings.
• Semantic Search over Uploaded Files — 【User Value】Enables a natural language Q&A experience over personal documents. Users can ask questions conceptually instead of just searching for keywords, and the system will find the most relevant passages. 【Design Strategy】When a search is performed, the system embeds the user's query and computes the cosine similarity against all pre-computed document chunk embeddings. The most similar chunks are returned as context for the answer. 【Business Logic】 - Step 1: The AI agent initiates a search on the uploaded files with one or more natural language queries. - Step 2: The system loads all the chunks and their embeddings from the relevant document's JSON file into memory. - Step 3: The user's search queries are embedded using the same model. - Step 4: The system calculates the cosine similarity score between each query embedding and all the document chunk embeddings. - Step 5: Results from multiple queries are merged using a weighted scoring formula that prioritizes chunks relevant to more queries. - Step 6: The top-ranked, deduplicated chunks are returned to the agent as citable sources, providing the context needed to answer the user's question.

Content Discovery & Conversation Aids

This module enhances the user experience beyond the core question-answering flow. It provides a 'Discover' feed for browsing trending articles and offers contextual follow-up suggestions to help users continue their conversations, promoting engagement and reducing the friction of formulating new queries.

• Topic-Based 'Discover' Feed — 【User Value】Allows users to stay informed and discover interesting content on topics they care about, without having to actively search for it. 【Design Strategy】The system pre-defines a set of topics (e.g., Tech, Finance, Sports) with associated keywords and trusted site domains. When a topic is selected, it performs targeted searches to generate a fresh, relevant feed of articles. 【Business Logic】 - Step 1: The user selects a topic from a list (e.g., 'Tech', 'Finance'). - Step 2: The system retrieves a list of pre-defined search queries and website domains associated with that topic. - Step 3: It performs a series of targeted searches on the 'bing news' engine (e.g., searching for 'AI developments' on `site:techcrunch.com`). - Step 4: The results from all searches are collected, and duplicate articles (based on URL) are removed. - Step 5: The final list is randomly shuffled to provide a different experience on each visit and presented to the user. - Step 6: Clicking an article card opens a new tab with a pre-filled query to summarize that article's URL, seamlessly integrating discovery with the core search functionality.
• Contextual Follow-up Suggestions — 【User Value】Reduces user effort and helps guide the conversation by providing relevant, pre-formulated follow-up questions based on the current chat history. 【Design Strategy】After an answer is generated, the entire conversation history is sent to an AI model with a specific prompt instructing it to generate a list of 4-5 helpful and relevant follow-up questions. 【Business Logic】 - Step 1: After a user receives an answer, the client sends the full chat history to the suggestions API. - Step 2: The server sends this history to an LLM, using a system prompt that asks it to act as a helpful assistant and generate 4 to 5 potential follow-up questions. - Step 3: To ensure reliability, the LLM is required to return its output in a structured JSON format: an object containing a simple array of suggestion strings. - Step 4: This JSON is parsed and the suggestions are displayed to the user as clickable buttons.

System Configuration & Provider Management

This module provides the administrative backbone of Perplexica, allowing users to configure the application and manage connections to various AI model providers. It's designed for flexibility, supporting configuration through both environment variables (for automated deployments) and a graphical user interface. This enables deep customization of the AI stack, from choosing different LLMs for different tasks to managing API keys.

• Unified Configuration System — 【User Value】Enables flexible deployment and configuration. System administrators can use environment variables for secure, automated setups (e.g., in Docker), while end-users can easily adjust settings through a UI without editing files. 【Design Strategy】The system uses a layered configuration approach. It first loads a baseline configuration from a JSON file, and then overrides these settings with any environment variables that have been set. This provides a clear order of precedence. 【Business Logic】 - Step 1: On application startup, the system loads its configuration from a local `config.json` file. - Step 2: It then scans for a predefined set of environment variables (e.g., `OPENAI_API_KEY`, `SEARXNG_API_URL`). - Step 3: If an environment variable is present and a corresponding value in the `config.json` file is empty, the value from the environment variable is used. - Step 4: The final, merged configuration is held in memory and used by the application. Changes made via the API/UI are saved back to the `config.json` file.
• Dynamic AI Provider Registry — 【User Value】Allows users to connect Perplexica to a wide range of AI models from different providers, including privately hosted local models. It also makes the system resilient to misconfigurations. 【Design Strategy】A central `ModelRegistry` is responsible for the entire lifecycle of AI providers. It uses a provider pattern, where each supported service (OpenAI, Ollama, etc.) has its own implementation class. The registry instantiates and manages these providers based on the user's configuration. 【Business Logic】 - Step 1: At startup, the registry reads the list of configured providers from the configuration system. - Step 2: For each provider, it attempts to create an instance. If a provider is misconfigured (e.g., wrong API key format), the instantiation fails, but the error is caught and logged. - Step 3: Instead of crashing, the system continues to initialize other providers. The misconfigured provider is still shown in the UI, but with a special 'error' model that displays the error message, making debugging easy for the user. - Step 4: For successfully initialized providers, the registry fetches the list of available chat and embedding models, which are then exposed to the rest of the application.
• Provider and Model Management API — 【User Value】Gives administrators full control over the AI models connected to the system at runtime, allowing them to add new providers, update credentials, or remove services through a simple UI without needing to restart the application. 【Design Strategy】A set of REST API endpoints exposes full Create, Read, Update, and Delete (CRUD) functionality for managing AI providers and their associated models. 【Business Logic】 - **Add Provider:** A user can send a POST request with the provider type (e.g., 'openai') and configuration (e.g., API key). The system adds it to the configuration and attempts to initialize it. - **List Providers:** A GET request returns a list of all currently active providers and the models they offer. - **Update Provider:** A PATCH request allows updating a provider's name or configuration. - **Remove Provider:** A DELETE request removes a provider from the configuration. - **Manage Models:** Additional endpoints allow for manually adding or removing specific models for a provider, which is useful for services that don't automatically list all available models.

Chat History & Real-Time UI

This module handles the stateful part of the user experience. It persists every conversation to a local SQLite database, allowing users to browse and revisit their chat history. It also powers the real-time, streaming user interface, delivering updates from the AI agent to the user's browser as they happen. The design uses Server-Sent Events (SSE) and a clever patching mechanism to create a highly responsive and dynamic UI without sending redundant data.

• SQLite-based Chat Persistence — 【User Value】Users never lose their work. All conversations are automatically saved locally, allowing them to close the browser and resume their research or review past findings at any time. 【Design Strategy】The system uses a local SQLite database with a two-table schema managed by the Drizzle ORM. One table stores metadata for each conversation, and the other stores every individual message and the complete set of UI blocks that formed its response. 【Business Logic】 - **`chats` table:** Stores one record per conversation, containing its unique ID, title (auto-generated from the first query), creation date, and lists of sources and files used. - **`messages` table:** Stores every message within a conversation. Crucially, it includes a `responseBlocks` column. This JSON column stores the complete array of UI blocks (text, sources, widgets, etc.) that were generated as the answer, serving as a perfect reconstruction of what the user saw.
• Real-Time UI Streaming via JSON Patch — 【User Value】Creates a modern, dynamic user experience where the user can see the AI 'thinking' in real-time. Research steps appear as they are executed, and the final answer streams in token-by-token, improving perceived performance. 【Design Strategy】The system uses Server-Sent Events (SSE) to maintain a persistent connection from the server to the client. Instead of re-sending the entire UI state with each update, the server sends small, specific instructions on how to change the UI, using the standard RFC6902 JSON Patch format. 【Business Logic】 - Step 1: When a user sends a message, the browser opens an SSE connection to the server. - Step 2: As the AI agent works, it generates events. For a completely new UI element (like a source list), the server sends a `'block'` event containing the full JSON for that UI block. - Step 3: For an existing element that needs updating (like a text answer being streamed), the server sends an `'updateBlock'` event. This event doesn't contain the full text, but rather a tiny JSON patch instruction, like `[{ "op": "replace", "path": "/data", "value": "new accumulated text" }]`. - Step 4: The client-side code receives these simple instructions and applies the patches to its local state, resulting in efficient, incremental updates to the UI.
• Idempotent Message State Management — 【User Value】Increases the reliability of the chat experience. If a network error causes a message submission to be retried, the system won't create duplicate conversations or messages. 【Design Strategy】When a message is submitted, the system performs a safe 'upsert' (update or insert) operation. It uses the client-generated message and chat IDs to check if the message already exists before creating a new one, ensuring each message is processed exactly once. 【Business Logic】 - Step 1: The client generates unique IDs for the chat and the message before sending the request. - Step 2: The server's first action is to look for a message with that Chat ID and Message ID in the database. - Step 3: If the message does not exist, it inserts a new record with a status of 'answering'. - Step 4: If the message *does* exist (indicating a retry), it updates the existing record, resetting its status to 'answering' and clearing out any old response data. It also cleans up any subsequent messages in the same chat to ensure a clean state for the retry. - Step 5: Once the answer is complete, the message record's status is updated to 'completed' and the final UI blocks are saved.
• In-Memory Session Management with Reconnect — 【User Value】Improves the robustness of the streaming connection. If the user's connection drops and reconnects mid-answer, they can resume streaming from where they left off. 【Design Strategy】The server maintains a short-term, in-memory session for each active chat. This session not only manages the live event stream but also buffers all events that have already been sent. A dedicated reconnect API allows a client to connect to an existing session and get a replay of all past events before joining the live stream. 【Business Logic】 - Step 1: When a chat request starts, a session is created in memory with a 30-minute time-to-live (TTL). - Step 2: As UI events are generated, they are sent to the client and also stored in an event buffer within the session object. - Step 3: If a client disconnects and then hits the `/api/reconnect/[session_id]` endpoint, the server finds the active session. - Step 4: The server first replays the entire event buffer to the reconnected client, instantly bringing their UI up to the current state. - Step 5: The client then starts receiving live events as normal. If the session has expired (older than 30 mins), the reconnect will fail.

Real-Time UI Streaming via Incremental Patching

Problem: How to deliver a rich, real-time user experience where users see the AI 'thinking' (e.g., research steps, streaming text) without overwhelming the client with data or requiring complex state management?

Solution: The system uses a persistent Server-Sent Events (SSE) connection to stream newline-delimited JSON objects to the client. Instead of re-sending the entire UI state on every update, it sends small, atomic instructions. - Step 1: For new UI elements (like a list of sources), the server emits a `'block'` event with the full JSON object for that element. - Step 2: To update an existing element (like appending text to an answer), the server emits an `'updateBlock'` event. This event contains a standard RFC6902 JSON Patch payload (e.g., `[{ "op": "replace", "path": "/data", "value": "new text" }]`). - Step 3: The client receives these events, adds new blocks to its state, and applies the lightweight patches to existing blocks. This approach makes streaming text extremely efficient, as only the new content is sent over the wire, not the entire accumulated answer.

Technologies: Server-Sent Events (SSE), RFC6902 JSON Patch, TransformStream

Boundaries & Risks: This capability relies on an in-memory session manager on the server. This makes it very fast but means it does not scale horizontally across multiple server instances without an external shared session store (like Redis). Sessions also have a 30-minute time-to-live, so very long-running queries could lose their streaming connection. If a patch is malformed or applied to the wrong state, it can break the client-side UI.

Pluggable, Multi-Provider AI Model Management

Problem: How to create a flexible system that can support a growing list of AI providers (like OpenAI, Groq, local Ollama models) with different capabilities and configurations, without locking the user into a single vendor or requiring code changes to add new ones?

Solution: The architecture is built around a central `ModelRegistry` and a `BaseModelProvider` interface. - Step 1: Each AI service is implemented as a separate 'Provider' class that conforms to the common interface, handling its specific API client and authentication logic. - Step 2: At startup, the `ModelRegistry` reads the user's configuration and acts as a factory, instantiating only the providers the user has configured. - Step 3: The registry includes robust error handling. If a provider is misconfigured, it logs the error but does not crash the application. Instead, it creates a special 'error model' that appears in the UI, informing the user of the specific problem. - Step 4: The rest of the application interacts with a single, unified interface (`registry.loadChatModel()`) to get an AI model, completely abstracted from the underlying provider's implementation details.

Technologies: Provider Pattern, Factory Pattern, TypeScript Polymorphism

Boundaries & Risks: The design provides excellent extensibility for adding new providers. However, managing the in-memory state of these providers can be complex; for example, updating a provider's configuration at runtime currently involves creating a new instance rather than modifying the existing one, which can lead to stale instances if not handled carefully. All provider credentials are read from a plaintext JSON file, posing a security risk in production environments without external secret management.

Adaptive, Multi-Stage Research Orchestration

Problem: How to build an AI agent that is more intelligent than a simple RAG pipeline? It needs to be efficient for simple queries, thorough for complex ones, and capable of using different tools based on the user's intent.

Solution: The system implements a multi-stage agentic pipeline that mimics a human research process. - Step 1 (Classify): An initial LLM call analyzes the user's query and generates a structured JSON plan, deciding whether to search, what sources to use (web, academic, files), and whether to show UI widgets. - Step 2 (Select Tools): A dynamic `ActionRegistry` makes specific search tools available to the agent based on the plan from the classification step. - Step 3 (Iterate): A `Researcher` agent enters a tool-calling loop. The number of iterations is controlled by the user's chosen mode (Speed: 2, Balanced: 6, Quality: 25). The agent uses the available tools to gather information until it decides it has enough context and calls a `'done'` tool to exit the loop. - Step 4 (Synthesize): The final 'Writer' agent receives all the gathered evidence, organized and ready for synthesis into a cited answer.

Technologies: Agentic AI, LLM Tool Calling, Zod (for structured output)

Boundaries & Risks: This advanced agentic workflow produces significantly higher-quality and more relevant answers than simple LLM calls. The main risk is complexity; the multi-step nature means there are more potential points of failure. The quality of the entire process is highly dependent on the reliability and instruction-following capabilities of the underlying LLM used for classification and tool use.

Self-Contained Document RAG on the Local File System

Problem: How to implement a 'talk to your documents' feature in a privacy-first application that aims for simple deployment, without forcing users to set up and manage a separate vector database?

Solution: The solution is an entirely file-system-based RAG pipeline. - Step 1 (Ingestion): When a user uploads a document (e.g., a PDF), the system extracts its text content. It then uses a token-aware algorithm to split the text into chunks of a specific size with overlap. - Step 2 (Embedding & Storage): For each chunk, it generates a vector embedding and stores the text chunk *and* its corresponding embedding together in a single companion `.content.json` file. This file sits right next to the original uploaded document in the `data/uploads` directory. - Step 3 (Querying): At query time, the system loads the contents of these JSON files for the relevant documents into memory. It then performs an in-memory cosine similarity calculation between the user's query embedding and all the loaded chunk embeddings to find the most relevant results.

Technologies: Text Chunking (js-tiktoken), On-the-fly Cosine Similarity, File-based Persistence

Boundaries & Risks: The key value is deployment simplicity—it 'just works' without any external dependencies like a vector DB, which is a major advantage for a self-hosted application. The primary trade-off is scalability. The current design loads all document chunks into memory for every search, which will lead to high memory usage and poor performance for users with very large document libraries or in high-concurrency environments.

Self-Hosted Private Search with Cited Answers

A full private “AI search engine” experience you can run locally, with citations for auditability.

User Benefit: Users can run a Perplexity-like answering engine entirely on their own hardware, combining web research with local or cloud AI models while returning answers with explicit source citations. This supports privacy-sensitive research workflows where sending queries or browsing behavior to a third-party search product is unacceptable.

Competitive Moat: Delivering a complete end-to-end experience (UI, streaming answers, multi-source retrieval, citations, persistence, and local deployment) is substantially more work than a demo chatbot and typically takes sustained product engineering. The integration surface across search, model providers, streaming UX, and local persistence creates meaningful replication effort for a competitor.

Transparent Research-in-Progress User Experience

Users can watch research happen live and keep the UI updated without full page reloads.

User Benefit: Users can see the system’s research steps unfold in real time (search sub-steps, tool outputs, and a streamed final answer) and reconnect to an in-progress session. This reduces perceived latency and improves trust because the product shows what it is doing rather than only returning a final response.

Competitive Moat: A robust streaming experience with incremental updates and reconnection requires coordinated server state, client patch application, and persistence of message outcomes. Many LLM apps stop at basic token streaming; building a coherent “live research” UX typically takes significant iteration.

Multi-Source Research That Adapts to the Question

One query can automatically pull from web, papers, discussions, specific URLs, and your uploaded files.

User Benefit: The system can dynamically choose between general web search, academic sources, discussions, URL scraping, and personal document retrieval based on query classification and user-selected sources. This improves answer relevance by using the right information channels for different question types.

Competitive Moat: The value comes from the integrated action registry, tool descriptions, and normalized citation output format across heterogeneous sources. While not scientifically novel, the product-level integration and orchestration across multiple sources is non-trivial to implement well.

Local Document Question Answering Without a Database Stack

Upload files and get answers grounded in your own documents, stored locally.

User Benefit: Users can upload documents (such as PDFs and DOCX files) and ask questions that are answered using semantically relevant passages from those files. This enables a personal knowledge base experience while keeping data local and avoiding the operational overhead of running a dedicated vector database.

Competitive Moat: It includes a complete ingestion pipeline (content extraction, token-aware chunking with overlap, embedding generation, and persistent storage) plus relevance ranking at query time. Competitors can replicate it, but delivering it as a cohesive feature still requires meaningful engineering effort.

Flexible Model Choice Across Local and Cloud Providers

Mix local and cloud AI models under one system, choosing what fits your cost and privacy needs.

User Benefit: Teams can switch between local models for privacy/cost control and cloud models for peak quality, without rewriting the application. This reduces vendor lock-in and allows practical optimization based on performance, cost, and compliance needs.

Competitive Moat: The provider abstraction and runtime model loading reduce integration friction across multiple AI vendors. This is not unique in the market, but it is still valuable product infrastructure that saves time for adopters.

Built-In Cost and Latency Controls via Search Modes

A simple user control that changes how deep the system researches, trading speed for completeness.

User Benefit: Users can select speed, balanced, or quality modes that directly control research depth and response behavior. This makes the product usable in both quick “lookup” moments and deep research workflows, with an explicit cost-time tradeoff.

Competitive Moat: Mode-driven orchestration across research iterations and writing requirements requires careful system-level design to avoid inconsistent behavior. While competitors can implement similar toggles, integrating them end-to-end with tool execution and prompting is meaningful product work.

Administrative Settings and Provider Credentials Can Be Read or Changed Without Access Control (Commercial Blocker)

Configuration and provider management endpoints lack authentication and authorization checks, including setup completion marking and provider CRUD. This exposes the ability to read/modify model provider settings and potentially disrupt service or exfiltrate secrets through the API surface.

Business Impact: Any user who can reach the service could change which AI providers are used, sabotage availability, or capture credentials and run up third-party API bills. This blocks any multi-user or internet-exposed deployment.

Sensitive Credentials Are Stored in Plaintext on Disk (Commercial Blocker)

Provider API keys and other secrets are persisted as plaintext JSON in the local configuration file. There is no encryption at rest or integration with an external secret manager in the evidenced implementation.

Business Impact: If a server, container, or volume is compromised (or accessed by an insider), all AI provider credentials can be extracted. This is likely to fail enterprise security reviews and increases blast radius of operational incidents.

Configuration Changes Are Not Safely Versioned or Validated (Commercial Blocker)

Configuration migration is marked as a TODO and there is no robust schema validation after loading configuration. Version tracking exists but is not enforced, meaning upgrades can silently break existing installs or accept invalid settings until runtime failures occur.

Business Impact: Upgrades can cause unexpected outages or misbehavior, increasing support costs and making it risky to roll out changes in production environments.

User Chats and Live Sessions Can Be Accessed Without Authentication (Commercial Blocker)

Streaming chat endpoints and session reconnection endpoints do not show explicit authentication checks in the route handlers. Additional routes that trigger external calls (search, discover, suggestions) also show no access control in the evidenced code.

Business Impact: If the service is reachable by untrusted users, conversations, queries, and research results could be accessed or abused. This is a severe privacy and cost risk for any shared deployment.

Uploaded Documents Are Not Isolated Between Users (Commercial Blocker)

Uploads are stored in a shared local directory with shared metadata tracking. There is no user or tenant boundary in storage layout or access checks in the evidenced upload storage manager, meaning knowledge of an identifier can enable cross-user access in a multi-user deployment.

Business Impact: This breaks the privacy promise of “personal documents” in any shared environment and prevents safe deployment for teams or organizations without redesigning identity and access control.

Running Multiple Server Instances Will Break Streaming and Reconnect Behavior (Commercial Blocker)

Live session state is held in process memory and is not shared across nodes. Reconnect depends on finding the session in the same process, which fails under horizontal scaling or after restarts.

Business Impact: A standard production deployment pattern (multiple pods/instances) can cause users to lose in-progress responses or be unable to reconnect, making the product unreliable at scale unless constrained to single-instance or sticky sessions.

Provider Updates Can Create Duplicate or Stale Runtime State (Scale Blocker)

Provider update logic appends updated provider instances to the in-memory active provider list rather than clearly replacing existing entries, which can leave duplicates and stale configurations in long-running processes.

Business Impact: Operations teams may find that changing provider settings does not reliably take effect, causing inconsistent behavior and avoidable downtime during configuration changes.

External Search and Chat APIs Lack Rate Limiting, Retries, and Backoff (Scale Blocker)

External search requests (via the SearxNG wrapper) are made without built-in retries/backoff, and multiple actions can execute in parallel. Across the API surface, there is no demonstrated rate limiting or abuse protection for creating streaming sessions or invoking external calls.

Business Impact: Under load or attack, the service can become slow or fail due to upstream rate limits, resource exhaustion, or an overwhelmed SearxNG instance. This creates reliability risk and unpredictable operating costs.

Discover Feed Can Trigger Many External Calls per Page Load Without Caching (Scale Blocker)

The Discover endpoint can issue multiple SearxNG searches per request (multiplying topic links by queries) and does not show caching, pagination, or throttling in the evidenced implementation.

Business Impact: Users may experience slow page loads, and production deployments may face higher infrastructure and upstream search costs than expected. At scale this becomes a reliability and unit economics problem.

Uploaded-File Search Can Exhaust Memory with Large Document Libraries (Scale Blocker)

Semantic search over uploads initializes by loading all chunks from the specified files into memory. This design does not show paging/streaming and can grow unbounded with large uploads.

Business Impact: Users with many or large documents may see slowdowns or crashes, limiting adoption for serious personal knowledge base use cases.

If the Search Backend Fails, Web Research Becomes Unavailable (Scale Blocker)

External research relies on a single configured SearxNG endpoint without an evidenced fallback provider strategy or caching layer. If that endpoint is misconfigured or unavailable, web research actions cannot function.

Business Impact: Reliability depends heavily on one service. Outages or misconfiguration lead to a degraded or non-functional core product experience.

Configuration Writes Can Block the Server Under Load (Scale Blocker)

Configuration persistence uses synchronous file writes, and every config update triggers an immediate write. This can block the server event loop and risks race conditions under concurrent updates.

Business Impact: In busy environments, configuration changes can cause latency spikes or even corrupt configuration state, increasing operational risk.

Search Results Can Contain Redundant Sources and Waste Context (Notable)

Deduplication exists within uploaded-file results, but there is no demonstrated cross-action deduplication across web, academic, and discussion searches, allowing repeated URLs to enter the citation pool.

Business Impact: Answers may cite the same source multiple times and waste limited model context, reducing answer quality and increasing cost.

Time-Sensitive Questions May Return Outdated Sources (Notable)

The SearxNG wrapper shows support for language and pagination but does not evidence date-range constraints or recency prioritization across the research actions.

Business Impact: Users researching current events may receive mixed or stale results, harming trust in the product’s accuracy.

Weak Handling of Low-Quality or Failed Source Extraction (Notable)

URL scraping failures can be returned as plain text in the content field without structured error metadata, and empty/low-content results can still flow through as citable chunks.

Business Impact: The system can cite sources that do not meaningfully support the answer, reducing perceived reliability and increasing support burden.