Technical — Summarizer

RAGA's hierarchical summarization & Named Entity Recognition (NER) service, built on Python + FastAPI. Accepts a text file, splits it into chunks, calls the LLM in parallel for each chunk, then merges the results into a final summary along with named entities. Used by api-tarantula to process documents and audio transcripts.

Repository

Key	Value
Git Remote	`https://git.tlab.co.id/tarantula/tarantula-v2/service/summarizer.git`
Active Branch	`main`

bash

git clone https://git.tlab.co.id/tarantula/tarantula-v2/service/summarizer.git
cd summarizer

Tech Stack

Layer	Technology
Runtime	Python 3.11
Framework	FastAPI + uvicorn
LLM Client	`openai` Python SDK (OpenAI-compatible API)
HTTP Client	`httpx` (async), `requests` (sync health probe)
Secret Management	`infisical_sdk` (Python)
Async	`asyncio` (semaphore-controlled concurrency)
JSON Repair	`json-repair` (repairs malformed JSON from LLM output)
Containerization	Docker (`python:3.11-slim`)

Folder Structure

summarizer/
├── app/
│   ├── main.py              # FastAPI app, pipeline, endpoints
│   └── infisical_config.py  # Infisical client + env fallback
├── tests/
│   ├── conftest.py
│   ├── test_call_json.py
│   ├── test_endpoints.py
│   ├── test_infisical_config.py
│   ├── test_summarize_and_ner.py
│   └── test_utils.py
├── requirements.txt
├── env.example
├── Dockerfile
└── Dockerfile.stag

Environment Variables

`.env` File / Container Env

bash

# Infisical (required)
INFISICAL_PROJECT_ID=
INFISICAL_ENVIRONMENT=
INFISICAL_SECRET_PATH=/summarizer
INFISICAL_CLIENT_ID=
INFISICAL_CLIENT_SECRET=
INFISICAL_HOST=

# LLM tuning (optional, not via Infisical)
LLM_TIMEOUT=600
LLM_MAX_RETRIES=2
LLM_MAX_CONCURRENCY=2

# Disable Infisical (fall back to plain env vars)
DISABLE_INFISICAL=0

Variable	Default	Description
`INFISICAL_PROJECT_ID`	—	Infisical project ID
`INFISICAL_ENVIRONMENT`	`staging`	Target environment
`INFISICAL_SECRET_PATH`	`/ocrproxy` (code default) / `/summarizer` (in `env.example`)	Secrets path in Infisical — see note below
`INFISICAL_CLIENT_ID`	—	Universal Auth Client ID
`INFISICAL_CLIENT_SECRET`	—	Universal Auth Client Secret
`INFISICAL_HOST`	`http://10.1.102.15:8002`	Infisical server URL
`LLM_TIMEOUT`	`600`	Timeout per LLM call (seconds) — large chunks can take a while
`LLM_MAX_RETRIES`	`2`	Maximum retries when an LLM call fails
`LLM_MAX_CONCURRENCY`	`2`	Maximum concurrent chunk calls to the LLM (semaphore)
`DISABLE_INFISICAL`	`0`	Set to `1` to skip Infisical and use plain env vars

Note: the code default for INFISICAL_SECRET_PATH (app/infisical_config.py) is /ocrproxy — likely leftover from whichever other service this was originally based on. env.example correctly points to /summarizer, but the code default should be reconciled so it doesn't resolve to the wrong path if the env var isn't set explicitly.

Security Warning: INFISICAL_PROJECT_ID, INFISICAL_CLIENT_ID, and INFISICAL_CLIENT_SECRET in app/infisical_config.py have hardcoded default values in the code (not empty strings) that are used whenever the env var isn't set. Always set all three explicitly in every environment (dev/staging/prod) — don't rely on the code's built-in defaults, since they may point at the wrong Infisical project or credentials.

Secrets via Infisical

Variable	Default	Description
`API_KEY_LOCAL`	`EMPTY`	API key for the LLM server (OpenAI-compatible)
`BASE_URL_LOCAL`	`http://192.168.0.27/qwen/v1`	LLM server base URL (`/v1` endpoint)
`MODEL_LOCAL`	`TLab/LLM-VL-Models`	Model name used for inference

InfisicalConfig has a layered fallback: if Infisical is unreachable (network down, auth failure, or DISABLE_INFISICAL=1), the service still runs and reads values from os.getenv() plus defaults.

Endpoints

Method	Path	Description
`GET`	`/health`	Health check + probes the upstream LLM server's `/models`
`POST`	`/summarize`	Hierarchical summarization + NER; returns a JSON file
`POST`	`/extract_entities`	NER only (no summarization); returns a JSON file

`POST /summarize`

Content-Type: multipart/form-data

file           (UploadFile) — text file to summarize
chunk_size     (int, default 10000) — chunk size in words
target_context (int, default 128000) — target context window (words); if the merged summary still exceeds this, the pipeline runs an additional merge pass
enable_ner     (bool, default true) — enable NER

Response: summary_YYYYMMDD_HHMMSS.json file (downloaded directly, temp file auto-deleted via BackgroundTask).

`POST /extract_entities`

Content-Type: multipart/form-data

file  (UploadFile) — text file for entity extraction

Response: entities_YYYYMMDD_HHMMSS.json file with the structure:

json

{
  "named_entities": {
    "PER": ["Budi Santoso"],
    "ORG": ["Kementerian Keuangan"],
    "LOC": [], "GPE": ["Jakarta"],
    "DATE": ["12 Maret 2024"],
    "TIME": [], "EVENT": [], "FAC": []
  },
  "input_tokens": 542,
  "timestamp": "2026-08-02T10:15:00.123456"
}

Hierarchical Summarization Pipeline

Pipeline Stages

Stage	Description
Chunking	Text is split per sentence (`[.!?]`) with a `chunk_size` word cap. Sentences are never cut in half.
Stage 1	Each chunk is processed in parallel via `asyncio.gather`. A semaphore caps concurrent calls at `LLM_MAX_CONCURRENCY`. One LLM call produces `summary + key_points + topics + named_entities` (merge-on-raw).
Stage 2+	If the combined summaries still exceed `target_context` words, the pipeline rechunks and re-summarizes iteratively. NER is disabled in intermediate passes to save tokens.
Final merge	A single LLM call over the combined text of all chunk summaries; produces the final summary + final NER.
NER aggregation	Entities are merged from 3 sources: chunk-level (highest recall), final summary, and key points.

The "Merge-on-Raw" Strategy

NER runs on the raw chunk text, not on the summary. This yields higher entity recall since summaries tend to drop specific mentions (names, dates, locations). With this approach, a single LLM call per chunk produces both the summary and entities — saving half the round-trips compared to the older approach.

Failure Resilience

asyncio.gather(return_exceptions=True) — one chunk failing does not cancel the others
A failed chunk is replaced with an empty placeholder (summary "", entities [])
LLM calls are retried LLM_MAX_RETRIES times at the transport/rate-limit level, with the openai SDK's built-in backoff
Separately, _call_json() (parsing the LLM result into JSON) has its own retry loop — 3 attempts by default per call; each failed attempt is logged with a snippet of the raw response for debugging
<think>...</think> blocks from reasoning models are stripped before JSON parsing
JSON responses wrapped in a code fence (```json) are cleaned before parsing
If json.loads() still fails (e.g. the upstream vLLM server returns malformed JSON — doubled braces, unescaped quotes), json_repair.repair_json() heuristically attempts a fix before the attempt is counted as failed

`/summarize` Output Structure

json

{
  "total_initial_chunks": 5,
  "summarize_chunk": [
    {
      "chunk_index": 1,
      "original_tokens": 987,
      "summary_tokens": 145,
      "original_text": "...",
      "summary": "...",
      "key_points": ["...", "..."],
      "topics": ["...", "..."],
      "named_entities": {
        "PER": ["Budi Santoso"],
        "ORG": ["Kementerian Keuangan"],
        "LOC": [], "GPE": ["Jakarta"],
        "DATE": ["12 Maret 2024"],
        "TIME": [], "EVENT": [], "FAC": []
      }
    }
  ],
  "final_summary": {
    "summary": "Long, comprehensive summary...",
    "key_points_model": ["..."],
    "key_points_aggregated": ["..."],
    "topics_aggregated": ["..."],
    "named_entities": {
      "final_summary_entities": { "PER": [], "ORG": [] },
      "aggregated_key_points_entities": { "PER": [] },
      "chunk_entities_merged": { "PER": [] },
      "all_entities_merged": { "PER": [] }
    }
  },
  "final_summary_tokens": 312
}

NER Entity Types

Category	Description	Example
`PER`	Person name	`"Joko Widodo"`
`ORG`	Organization / company	`"PT Telkom"`
`GPE`	Geo-political entity (country, city, region)	`"Jakarta"`, `"Indonesia"`
`LOC`	Location / general place	`"Gedung Sate"`
`DATE`	Date	`"12 Maret 2024"`
`TIME`	Time	`"pukul 09.00 WIB"`
`EVENT`	Event name	`"Pemilu 2024"`
`FAC`	Facility (building, airport, bridge)	`"Bandara Soekarno-Hatta"`

Health Check

GET /health checks:

Service status ("ok")
Active configuration (model_base_url, configured_model)
Connectivity to the upstream LLM (/models endpoint) — lists available models

json

{
  "status": "ok",
  "model_base_url": "http://192.168.0.27/qwen/v1",
  "configured_model": "TLab/LLM-VL-Models",
  "upstream": {
    "reachable": true,
    "models": ["TLab/LLM-VL-Models"],
    "error": null
  }
}

Startup: Chunk Budget Probe

On application start (lifespan handler), the service calls the upstream LLM's /models endpoint once and compares the model's max_model_len against the estimated token count of the default chunk_size (10,000 words, converted to tokens with a 1.4 word-to-token ratio for Indonesian text). The result is only logged — it never blocks startup:

If the default chunk's estimated token count exceeds 60% of the upstream max_model_len, a warning-level log is emitted suggesting a lower chunk_size.
If the probe fails (upstream unreachable at startup), the service still starts normally — only a warning is logged.

Build & Run

bash

# Install dependencies
pip install -r requirements.txt

# Run development
uvicorn app.main:app --reload --port 8000

# Docker build & run
docker build -t summarizer .
docker run -p 8000:8000 \
  -e INFISICAL_PROJECT_ID=... \
  -e INFISICAL_ENVIRONMENT=dev \
  -e INFISICAL_SECRET_PATH=/summarizer \
  -e INFISICAL_CLIENT_ID=... \
  -e INFISICAL_CLIENT_SECRET=... \
  -e INFISICAL_HOST=http://10.1.102.15:8002 \
  summarizer

# Run tests
pytest tests/

API Tarantula

User Service

OPA

OPA Auth

OPA Config

OPA Data

Summarizer

OCR

OCR V3

Minio Uploader

Database Connect

API Connect

Chatbot Service

Frontend

Raga License Env

Documentation Site

Technical — Summarizer

Repository

Tech Stack

Folder Structure

Environment Variables

`.env` File / Container Env

Secrets via Infisical

Endpoints

`POST /summarize`

`POST /extract_entities`

Hierarchical Summarization Pipeline

Pipeline Stages

The "Merge-on-Raw" Strategy

Failure Resilience

`/summarize` Output Structure

NER Entity Types

Health Check

Startup: Chunk Budget Probe

Build & Run

OPA Auth

OPA Config

OPA Data

OCR V3

Technical — Summarizer ​

Repository ​

Tech Stack ​

Folder Structure ​

Environment Variables ​

.env File / Container Env ​

Secrets via Infisical ​

Endpoints ​

POST /summarize ​

POST /extract_entities ​

Hierarchical Summarization Pipeline ​

Pipeline Stages ​

The "Merge-on-Raw" Strategy ​

Failure Resilience ​

/summarize Output Structure ​

NER Entity Types ​

Health Check ​

Startup: Chunk Budget Probe ​

Build & Run ​

Technical — Summarizer

Repository

Tech Stack

Folder Structure

Environment Variables

`.env` File / Container Env

Secrets via Infisical

Endpoints

`POST /summarize`

`POST /extract_entities`

Hierarchical Summarization Pipeline

Pipeline Stages

The "Merge-on-Raw" Strategy

Failure Resilience

`/summarize` Output Structure

NER Entity Types

Health Check

Startup: Chunk Budget Probe

Build & Run