Technical — API Tarantula

The main RAGA backend service, built on NestJS. It serves as the core API layer — managing workspaces, knowledge (documents, audio, database, API), chat, system prompts, user access, and orchestrating calls to external engines (chatbot, OCR, summarize, speech).

Repository

Key	Value
Git Remote	`https://git.tlab.co.id/tarantula/tarantula-v2/service/api-tarantula.git`
Active Branch	`dev`
Other Branches	`main`, `staging`, `refactor/code-integrations`

bash

git clone https://git.tlab.co.id/tarantula/tarantula-v2/service/api-tarantula.git
cd api-tarantula
git checkout dev

Tech Stack

Layer	Technology
Framework	NestJS 10 (TypeScript)
Database	PostgreSQL 16 via TypeORM 0.3
Cache / Queue	Redis 7 (ioredis + Bull)
Object Storage	MinIO
Search / Index	Elasticsearch 8.17
Secret Management	Infisical SDK v4
Logging	nest-winston (Winston 3.17)
Testing	Jest 29 + Supertest
Runtime	Node.js (Docker)

Environment Variables

`.env` File (Infisical Bootstrap)

The .env file only holds bootstrap variables for authenticating to Infisical. All application secrets are fetched from Infisical at startup.

bash

INFISICAL_ENV=dev
INFISICAL_PATH=/
INFISICAL_SITE_URL=http://infisical-backend:8080
INFISICAL_CLIENT_ID=<client-id>
INFISICAL_CLIENT_SECRET=<client-secret>
INFISICAL_PROJECT_ID=<project-id>

Variable	Description
`INFISICAL_ENV`	Target environment in Infisical (`dev` / `staging` / `prod`)
`INFISICAL_PATH`	Secret path in Infisical (default `/`)
`INFISICAL_SITE_URL`	Infisical server URL (self-hosted or `https://app.infisical.com`)
`INFISICAL_CLIENT_ID`	Client ID for Universal Auth
`INFISICAL_CLIENT_SECRET`	Client Secret for Universal Auth
`INFISICAL_PROJECT_ID`	Infisical project ID holding this service's secrets

Secrets via Infisical

All variables below are managed in Infisical and injected into process.env at runtime by InfisicalService.

Database (PostgreSQL)

Variable	Description
`DB_HOST`	PostgreSQL host
`DB_PORT`	PostgreSQL port (default `5432`)
`DB_USER`	PostgreSQL username
`DB_PASSWORD`	PostgreSQL password
`DB_NAME`	Database name

Redis

Variable	Default	Description
`REDIS_HOST`	`localhost`	Redis host
`REDIS_PORT`	`6379`	Redis port
`REDIS_PASSWORD`	—	Redis password (optional)

MinIO (Object Storage)

Variable	Default	Description
`MINIO_ENDPOINT`	`localhost`	MinIO endpoint
`MINIO_PORT`	—	MinIO port (optional)
`MINIO_SSL`	`false`	Use SSL (`true`/`false`)
`MINIO_ACCESS_KEY`	—	MinIO access key
`MINIO_SECRET_KEY`	—	MinIO secret key
`MINIO_BUCKET`	`tarantula`	Default bucket name
`MINIO_DOMAIN`	`https://s3.ziwardingai.xyz`	Public domain used to build URLs for uploaded files (documents, chat images, etc.)

Elasticsearch

Variable	Default	Description
`ELASTICSEARCH_HOST`	`http://localhost:9200`	Elasticsearch node URL
`ELASTICSEARCH_USERNAME`	—	Elasticsearch username
`ELASTICSEARCH_PASSWORD`	—	Elasticsearch password
`MAXIMUM_CHAT_PER_ROOM`	`2`	Maximum indexed messages per room
`INDEX_USER_ACTIVITY_LOG`	`user-activity-log`	Elasticsearch index name `activity-log` writes user activity to

Chatbot Engine (opa-data / VLLM)

Variable	Default	Description
`CHATBOT_URL`	`http://192.168.0.25:8034`	Chatbot engine base URL
`MAX_CHAR_GENERATE_REPORT`	`100000`	Character limit for report generation

RDBMS Engine (database-connect)

Variable	Default	Description
`DATABASE_CONNECT_URL`	`http://192.168.0.25:8105`	Base URL of the `database-connect` service for external RDBMS connections & Text-to-SQL
`DATABASE_CONNECT_TIMEOUT`	`86400000`	Timeout (ms) for requests to `database-connect`

OCR / PDF Engine

Variable	Default	Description
`PDF_URL`	`http://192.168.0.25:8090/api/v1`	Tarantula PDF/OCR service URL
`ENGINE_SOURCE_LOCAL`	`true`	Use local file source for OCR

Summarize Engine

Variable	Default	Description
`SUMMARIZE_URL`	`http://10.1.102.14:8014`	Summarize service URL
`SUMMARIZE_TIMEOUT`	`86400000`	Timeout (ms) for the summarize process
`SUMMARIZE_ENABLE_NER`	`true`	Enable Named Entity Recognition
`SUMMARIZE_CHUNK_SIZE`	`1000`	Text chunk size for summarization
`SUMMARIZE_TARGET_CONTEXT`	`128000`	Target LLM context window

Speaches (Speech-to-Text)

Variable	Default	Description
`SPEACHES_URL`	`https://speaches.ziwardingai.xyz/v1`	Transcription service URL
`SPEACHES_MODEL`	`turbo`	Whisper model used
`SPEACHES_DIARIZE`	`true`	Enable speaker diarization
`SPEACHES_SUPPRESS_NUMERALS`	`false`	Suppress numerals in output

WhatsApp Integration

Variable	Default	Description
`WHATSAPP_URL`	`http://192.168.0.27:18004`	WhatsApp Go service URL
`WHATSAPP_TIMEOUT`	`86400000`	WhatsApp connection timeout (ms)

License Validation

Variable	Default	Description
`LICENSE_API_URL`	`http://10.1.102.15:8003`	License validation service URL
`LICENSE_API_EMAIL`	`admin@mail.com`	Basic Auth email for the license service
`LICENSE_API_PASSWORD`	`123456`	Basic Auth password for the license service

Folder Structure

api-tarantula/
├── src/
│   ├── app.module.ts               # Root module, bootstraps all modules
│   ├── main.ts                     # NestJS entry point
│   │
│   ├── common/                     # Shared utilities, config, integrations
│   │   ├── config/                 # Config class per external service
│   │   │   ├── chatbot.config.ts
│   │   │   ├── elasticsearch.config.ts
│   │   │   ├── minio.config.ts
│   │   │   ├── pdf-tarantula.config.ts
│   │   │   ├── rdbms.config.ts
│   │   │   ├── redis.config.ts
│   │   │   ├── speaches.config.ts
│   │   │   ├── summarize.config.ts
│   │   │   ├── typeorm.config.ts
│   │   │   └── whatsapp-go.config.ts
│   │   ├── decorator/               # Custom decorators
│   │   ├── dto/                     # Shared DTOs (PaginationDto, ParamDto)
│   │   ├── exception/                # Custom exceptions
│   │   ├── filter/                   # Global exception filter
│   │   ├── helper/                   # PDF helper, safe JSON parse
│   │   ├── integrations/             # HTTP client per external service
│   │   │   ├── chatbot/
│   │   │   ├── elasticsearch/
│   │   │   ├── minio/
│   │   │   ├── pdf-tarantula/
│   │   │   ├── rdbms/
│   │   │   ├── redis/
│   │   │   ├── speaches/
│   │   │   ├── summarize/
│   │   │   └── whatsapp-go/
│   │   ├── interceptor/              # Response interceptor (standard format)
│   │   ├── logger/                   # Telegram transport for Winston
│   │   ├── middleware/                # API key middleware, dynamic CSP
│   │   └── services/
│   │       └── document/              # Document chunk service
│   │
│   ├── db/
│   │   ├── migrations/               # TypeORM migrations (70+ files)
│   │   └── seeds/                     # Data seeder
│   │
│   ├── infisical/                    # Infisical secret loader & service
│   │
│   ├── activity-log/                 # User activity logging
│   ├── apis/                         # External API management within a workspace
│   ├── api-users/                    # ACL: user access to an API
│   ├── audio-document-chunks/        # Audio transcription chunks
│   ├── audio-document-summaries/     # Audio document summaries
│   ├── audio-document-users/         # ACL: user access to an audio document
│   ├── audio-documents/              # Audio transcription result documents
│   ├── audio-users/                  # ACL: user access to audio
│   ├── audios/                       # Audio file management
│   ├── canvas/                       # Endpoint for testing knowledge & LLM combinations
│   ├── chat-histories/               # Chat history per room
│   ├── dashboard/                    # Dashboard summary data
│   ├── database-users/               # ACL: user access to a database
│   ├── databases/                    # External database (RDBMS) connections
│   ├── document-folder/              # Document folder management
│   ├── document-folder-users/        # ACL: user access to a folder
│   ├── document-ocr/                 # Per-document OCR results
│   ├── document-summaries/           # Document summaries
│   ├── document-users/               # ACL: user access to a document
│   ├── documents/                    # Document management (upload, metadata)
│   ├── endpoints/                    # Endpoints of an external API
│   ├── health/                       # Health check endpoint
│   ├── log/                          # HTTP request logging middleware
│   ├── mail/                         # Email service (Bull queue + templates)
│   ├── model-management/             # LLM model management per workspace
│   ├── open-api/                     # Public API surface (app_key)
│   ├── openai-compat/                # OpenAI-compatible chat completion endpoint
│   ├── room-chats/                   # Room chat management
│   ├── settings/                     # Global application settings
│   ├── shared/                       # Shared service (JWT, auth helper)
│   ├── system-prompt/                # Workspace system prompt management
│   ├── system-prompt-users/          # ACL: user access to a system prompt
│   ├── topic-document-users/         # ACL: user access to a topic document
│   ├── topic-documents/              # Topic ↔ document relation
│   ├── topic-users/                  # ACL: user access to a topic
│   ├── topics/                       # Knowledge topic management
│   ├── utils/                        # Misc utility endpoints
│   ├── whatsapp/                     # WhatsApp integration
│   ├── workspace-iframes/            # Workspace iframe embed configuration
│   ├── workspace-integrations/       # External integrations per workspace
│   ├── workspace-roles/              # Role management within a workspace
│   ├── workspace-users/              # Workspace user membership
│   └── workspaces/                   # Workspace management (core entity)
│
├── test/                             # E2E tests
├── scripts/                          # Infisical CLI helper script
├── docker-compose.dev.yml            # Docker for local development
├── docker-compose.yml                # Docker for production/staging
├── Dockerfile.dev                    # Development image
├── Dockerfile                        # Production image
├── Dockerfile.stag                   # Staging image
├── nest-cli.json
├── tsconfig.json
└── package.json

Module Architecture

Core domains are grouped into NestJS modules following a controller → service → entity pattern:

Workspace Core — workspaces, workspace-users, workspace-roles, workspace-integrations, workspace-iframes
Knowledge: Documents — documents, document-ocr, document-folder, document-summaries, topics, topic-documents
Knowledge: Audio — audios, audio-documents, audio-document-chunks, audio-document-summaries
Knowledge: Database & API — databases, apis, endpoints
Chat & Intelligence — room-chats, chat-histories, system-prompt, model-management, canvas, openai-compat
Auth & Access — shared (JWT), open-api, activity-log
Support — mail, health, settings, dashboard, whatsapp, infisical

Every knowledge source also has a matching *-users module (document-users, audio-users, topic-users, database-users, api-users, etc.) for per-resource access control — see Architecture Overview.

Key Modules

Module	Responsibility
`workspaces`	RAGA's core entity; configures LLM, chain mode, streaming, personalization, and relations to all knowledge sources
`documents`	Upload, metadata, OCR status, progress tracking for PDF/file documents
`document-ocr`	Stores OCR results per chunk; relates to the PDF Tarantula service
`audios`	Audio file upload; triggers transcription to the Speaches service
`audio-documents`	Per-segment transcription results; `is_publish_transcribe` status
`topics`	Groups documents into a knowledge topic within a workspace
`databases`	External RDBMS (PostgreSQL/MySQL) connections for Text-to-SQL
`apis`	External API definitions callable by the chatbot (Text-to-API)
`system-prompt`	Workspace system prompt templates; supports report and persistent prompt types
`model-management`	LLM model configuration available per workspace (QWEN, GLM, etc.)
`chat-histories`	Stores conversation history per room; used as context by the engine
`openai-compat`	OpenAI-compatible endpoints (`POST /open-api/chat/completions`, `GET /open-api/models`) for third-party integration — not under `/v1/...`, but sharing the `open-api` namespace
`infisical`	Infisical secret loader; initializes `process.env` before other modules run
`activity-log`	Interceptor + decorator that logs every user action to an Elasticsearch index (`INDEX_USER_ACTIVITY_LOG`), not a database table

Infrastructure (Development)

Run via docker-compose.dev.yml:

bash

docker compose -f docker-compose.dev.yml up -d

Container	Image	Port	Description
`api-jabarin`	`Dockerfile.dev` (NestJS)	3000	Application with hot-reload
`db-api-jabarin`	`postgres:16-alpine`	5432	PostgreSQL
`cached-api-jabarin`	`redis:7.4.5-alpine3.21`	6379	Redis for cache & Bull queue
`minio-jabarin`	`quay.io/minio/minio`	9000 (API), 9001 (Console)	Object storage for file uploads
`es-jabarin`	`elasticsearch-wolfi:8.17.0`	9200, 9300	Single-node Elasticsearch for indexing

All containers share the network-jabarin network.

External Integrations

Service	Env Var	Description
Chatbot Engine (opa-data)	`CHATBOT_URL`	RAGA chatbot engine; called for LLM inference
PDF Tarantula (OCR)	`PDF_URL`	Document OCR service; accepts PDF files and returns text per chunk
Speaches (STT)	`SPEACHES_URL`	Whisper-based speech-to-text; used for audio transcription
Summarize Engine	`SUMMARIZE_URL`	Summarize + NER service for documents and audio
WhatsApp Go	`WHATSAPP_URL`	WhatsApp bridge for chatbot integration via WA
Infisical	`INFISICAL_SITE_URL`	Secret management; all sensitive env vars are pulled from here

Development Commands

bash

# Install dependencies
npm install

# Run development (hot-reload)
npm run start:dev

# Build production
npm run build

# Run migrations
npm run migration:run

# Create a new migration
npm run migration:create --name=MigrationName

# Rollback migration
npm run migration:revert

# Run tests
npm run test

# Run tests + coverage
npm run test:cov

# Sync new secrets to Infisical
npm run sync:infisical

API Tarantula

User Service

OPA

OPA Auth

OPA Config

OPA Data

Summarizer

OCR

OCR V3

Minio Uploader

Database Connect

API Connect

Chatbot Service

Frontend

Raga License Env

Documentation Site

Technical — API Tarantula

Repository

Tech Stack

Environment Variables

`.env` File (Infisical Bootstrap)

Secrets via Infisical

Folder Structure

Module Architecture

Key Modules

Infrastructure (Development)

External Integrations

Development Commands

OPA Auth

OPA Config

OPA Data

OCR V3

Technical — API Tarantula ​

Repository ​

Tech Stack ​

Environment Variables ​

.env File (Infisical Bootstrap) ​

Secrets via Infisical ​

Folder Structure ​

Module Architecture ​

Key Modules ​

Infrastructure (Development) ​

External Integrations ​

Development Commands ​

Technical — API Tarantula

Repository

Tech Stack

Environment Variables

`.env` File (Infisical Bootstrap)

Secrets via Infisical

Folder Structure

Module Architecture

Key Modules

Infrastructure (Development)

External Integrations

Development Commands