Raga Documentation

English

Indonesia

English

Indonesia

Appearance

Chatbot Service ​

Chatbot Service is the Python + FastAPI service that acts as RAGA's inference engine — this is the concrete implementation of the concept described on the Raga Engine page. It accepts a user question along with a list of relevant knowledge sources, gathers context from Elasticsearch and supporting services in parallel, dynamically assembles a system prompt, then calls one of several LLM providers to generate an answer.

Chatbot Service is called by API Tarantula (env var CHATBOT_URL) whenever a chat message comes in for a workspace.

Tech Stack ​

ComponentTechnology
RuntimePython (FastAPI + uvicorn)
Search / Knowledge IndexElasticsearch
LLM Clientopenai SDK (OpenAI-compatible, used for every provider)
ObservabilityOpenTelemetry (traces + FastAPI/requests/logging instrumentation)
Secret Managementinfisical_sdk (Python)

Full environment variable, endpoint, and prompt-assembly flow details are on the Technical page.

How It Works ​

  1. The caller sends POST /chatbot with the user's question, a list of relevant knowledge sources (km), the selected LLM provider, and various flags (chain, image, files, source, think, etc.).
  2. If the provider is a local model (local/local_v2), the request waits (polling) until the VLLM queue has an available slot before proceeding.
  3. All five knowledge source types (basic, document/OCR, audio, database, API) are fetched in parallel — one source failing does not fail the others (degraded_sources is reported back to the caller).
  4. For database and API sources, Chatbot Service doesn't query directly: it asks the LLM to compose the right query/call, then executes it through Database Connect and API Connect.
  5. All context (knowledge sources, chat history, uploaded files, image descriptions, personalization) is merged into a single system prompt, sent to the LLM, and the answer is returned — either all at once or as a stream.

Summary ​

Chatbot Service is RAGA's brain: it orchestrates five types of knowledge sources, delegates Text-to-SQL and Text-to-API to other services, then weaves everything into one coherent conversation through any of six supported LLM providers.