No description
  • Python 98%
  • Dockerfile 2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Julian Imhof d489996594
All checks were successful
build-image / docker (push) Successful in 30s
docs: document the CI pipeline and running from the registry
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 23:00:48 +02:00
.forgejo/workflows ci: verify the published tag by pulling it back from the registry 2026-09-03 22:59:40 +02:00
app feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
.dockerignore feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
.env.example feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
.gitignore feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
docker-compose.yml feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
Dockerfile feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
healthcheck.py feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00
README.md docs: document the CI pipeline and running from the registry 2026-09-03 23:00:48 +02:00
requirements.txt feat: dockerized Wikipedia MCP server backed by a local Kiwix ZIM 2026-09-03 22:48:14 +02:00

Wikipedia MCP Server

A small, self-contained MCP server that answers Wikipedia questions from a local Kiwix ZIM snapshot — no internet access at query time. Built for Home Assistant's Model Context Protocol integration, so a local or cloud LLM conversation agent can look things up by voice.

  • Downloads a Wikipedia ZIM from the Kiwix mirror and keeps it up to date on a schedule
  • Full-text search plus article/summary retrieval, straight out of the ZIM (libzim)
  • Atomic updates: queries keep working during a refresh, including the moment the new snapshot goes live
  • Everything lives in one Docker volume; the container is stateless
  • Four dependencies, no sidecar service, no database

Contents

Dockerfile             # python:3.12-slim, non-root, healthcheck
docker-compose.yml     # service + named volume
.env.example           # every setting, with defaults
healthcheck.py         # container healthcheck (GET /health)
app/config.py          # environment configuration
app/zimstore.py        # holds the live archive, swaps it atomically
app/updater.py         # discovery, resumable download, checksum, activation
app/wiki.py            # search, article fetch, HTML to plain text
app/server.py          # MCP tools + HTTP endpoints

Quick start

cp .env.example .env       # optional; defaults work as-is
docker compose up -d
docker compose logs -f     # watch the first download

The default snapshot is Simple English Wikipedia, no images (~0.9 GB, ~400k articles), which is a good match for voice answers. The server starts serving as soon as the download finishes:

curl -s http://localhost:8808/health | jq
{
  "status": "ok",
  "ready": true,
  "zim": { "file": "wikipedia_en_simple_all_nopic_2026-05.zim", "article_count": 396632,
           "has_fulltext_index": true, "size_bytes": 982777856 },
  "updater": { "state": "idle", "last_check": 1788467272.8, "last_error": null }
}

MCP endpoint for clients: http://<host>:8808/sse

Choosing a snapshot

Set ZIM_NAME_PREFIX to any file prefix from https://download.kiwix.org/zim/wikipedia/. The server appends the newest _YYYY-MM.zim it finds there.

ZIM_NAME_PREFIX Content Size
wikipedia_en_simple_all_nopic Simple English, all articles, no images (default) ~0.9 GB
wikipedia_en_top_nopic Top ~50k English articles, no images ~2 GB
wikipedia_en_top_maxi Top ~50k English articles, with images ~8 GB
wikipedia_en_all_nopic Full English Wikipedia, no images ~50 GB
wikipedia_de_all_nopic, wikipedia_fr_all_nopic, ... Other languages varies

Use a _nopic or _mini build unless you need images: the MCP tools return text only, so images are pure disk cost. Sizing rule: plan for about 2x the ZIM size — the new file is downloaded alongside the old one and the old one is deleted only after the swap.

_mini builds hold lead paragraphs only. If a snapshot ships without a full-text index, the server detects that and falls back to title search automatically; get_wikipedia_status reports which mode is in use.

Configuration

All configuration is environment variables (see .env.example).

Variable Default Meaning
ZIM_NAME_PREFIX wikipedia_en_simple_all_nopic Which ZIM family to mirror
ZIM_BASE_URL https://download.kiwix.org/zim/wikipedia/ Mirror directory to scan
ZIM_URL (unset) Pin one exact file URL; disables discovery
UPDATE_INTERVAL_HOURS 168 (weekly) Check interval; 0 disables periodic checks
UPDATE_ON_START true Also check right after startup
VERIFY_CHECKSUM true Verify the mirror's published .sha256 before activating
KEEP_OLD_ZIM false Keep superseded snapshots on disk
MIN_FREE_MB 512 Refuse to download if less free space would remain
MCP_PORT 8808 HTTP port
MCP_TRANSPORT sse sse (Home Assistant) or streamable-http
MAX_RESULTS 5 Default number of search hits
MAX_ARTICLE_CHARS 6000 Default article truncation length
SNIPPET_CHARS 320 Snippet length in search results
DATA_DIR /data Volume mount point
LOG_LEVEL INFO Python log level

Kiwix publishes new snapshots roughly monthly, so weekly checks are plenty. Checks are cheap when nothing changed: one directory listing, then a no-op.

MCP tools

Tool Purpose
search_wikipedia(query, limit) Full-text search; returns titles, snippets and article ids
get_wikipedia_summary(title, max_chars) Lead section only — best for spoken answers
get_wikipedia_article(title, max_chars, offset) Full article text, paged via offset
get_wikipedia_status() Which snapshot is loaded, its date, and updater state

Article HTML is converted to plain text with navboxes, infoboxes, reference lists, edit links and site chrome stripped, so the LLM sees prose instead of markup.

HTTP endpoints

Endpoint Purpose
GET /sse MCP over SSE (this is the URL Home Assistant needs)
POST /messages/ MCP client-to-server channel (used by the SSE transport)
GET /health Health and status JSON; also used by the container healthcheck
POST /update Trigger an update check immediately

GET /mcp replaces /sse when MCP_TRANSPORT=streamable-http.

How updates stay atomic

  1. Fetch the mirror's directory listing and pick the newest PREFIX_YYYY-MM.zim.
  2. Download to /data/tmp/<name>.part. Interrupted downloads resume with an HTTP range request on the next attempt or the next container start.
  3. Verify the mirror's published SHA-256.
  4. os.replace() the finished file into /data/zim/<name> — atomic, same filesystem.
  5. Open the new archive, then swap the in-memory reference in a single assignment.
  6. Delete the superseded file.

The live archive is replaced only in step 5, after the new file is complete, verified and successfully opened. Requests already in flight keep reading the old archive object; on Linux the old file's inode survives the unlink in step 6, so those reads finish normally. A failed download, bad checksum or corrupt file leaves the current snapshot serving and is reported in /health under updater.last_error.

Force a check without waiting for the timer:

curl -X POST http://localhost:8808/update
docker compose logs -f    # watch progress

Home Assistant setup

Requires Home Assistant 2025.2 or newer (when the Model Context Protocol integration landed) and an LLM-backed conversation agent. The built-in "Home Assistant" Assist agent matches sentence templates and cannot call tools — you need an agent such as Anthropic, OpenAI, Google Generative AI, or Ollama.

1. Make the container reachable from Home Assistant

Home Assistant needs an address it can actually route to:

  • HA OS / supervised / a separate machine — use the server's LAN IP: http://192.168.1.50:8808/sse. Do not use localhost; from HA that means the HA host.

  • HA in Docker on the same host — either use the host IP as above, or put both containers on the same Docker network and use the container name:

    # docker-compose.yml
    services:
      wikipedia-mcp:
        networks: [ha]
    networks:
      ha:
        external: true
        name: <the network your HA container uses>
    

    Then the URL is http://wikipedia-mcp:8808/sse.

Verify from the HA host before continuing:

curl -s http://192.168.1.50:8808/health

2. Add the MCP server integration

  1. Settings → Devices & Services → Add Integration
  2. Search for Model Context Protocol and select it.
  3. Enter the SSE URL: http://192.168.1.50:8808/sse
  4. Submit. Home Assistant connects, reads the tool list, and creates an entry named after the server (wikipedia).

If the config flow errors, check /health from the HA host first — nearly every failure here is networking or a URL missing the /sse suffix.

3. Expose the tools to your conversation agent

The MCP integration adds its tools to Home Assistant's Assist LLM API. Any conversation agent using that API can call them.

  1. Settings → Devices & Services, open your LLM integration (Anthropic, OpenAI, Google Generative AI, Ollama, ...).
  2. Configure on the conversation agent entry.
  3. Set Control Home Assistant (the LLM API selector) to Assist.
  4. Save. The Wikipedia tools are now offered to that agent alongside the device-control tools.

Optionally add a line to the agent's prompt so it reaches for the tools naturally:

When asked a factual or general-knowledge question, use the Wikipedia tools. Prefer get_wikipedia_summary and answer in one or two sentences.

4. Put the agent in your Assist pipeline

  1. Settings → Voice assistants → Assist (or + Add assistant).
  2. Set Conversation agent to the LLM agent you configured above.
  3. Pick your speech-to-text and text-to-speech engines, then Update.
  4. To make it the default for voice hardware (Voice PE, ESPHome satellites, the mobile app), mark this pipeline as preferred, or select it on the device page.

5. Test it

Type into Assist first (the chat icon, top right), then try voice:

  • "Search Wikipedia for the Ryder Cup."
  • "What does Wikipedia say about black holes?"
  • "Who was Ada Lovelace?"
  • "Give me a short summary of the Apollo 11 mission."
  • "Which Wikipedia snapshot are you using?" — exercises get_wikipedia_status.

Expect a couple of seconds for the first query after a container start while the ZIM index warms.

If the agent answers from its own knowledge instead of calling a tool, it usually still has the tools available — LLMs skip lookups for facts they are confident about. Force the path with "Look it up on Wikipedia" and check the container logs, or ask a question the model cannot know ("What does Wikipedia say about the 2026 Masters Tournament?").

Troubleshooting

Symptom Check
Container unhealthy right after start It is still downloading; docker compose logs -f. The healthcheck allows a 30 minute start period, so raise start-period in the Dockerfile for very large ZIMs on slow links.
Tools return "still being downloaded" /health shows ready: false and updater.progress_percent.
No remote ZIM matched prefix Typo in ZIM_NAME_PREFIX; confirm the exact prefix at https://download.kiwix.org/zim/wikipedia/.
Search returns nothing but the article exists The ZIM has no full-text index (common for _mini builds); get_wikipedia_status shows Full-text search: no. Switch to an _all or _top build.
Not enough free space The volume needs room for the old and the new snapshot at once.
HA integration will not connect curl http://<ip>:8808/health from the HA host; verify the URL ends in /sse.
Checksum mismatch A truncated mirror download; the file is discarded and retried on the next check. The old snapshot keeps serving.

Operations

docker compose logs -f                          # progress and update activity
curl -s localhost:8808/health | jq .updater      # updater state
curl -X POST localhost:8808/update               # check for a new snapshot now
docker volume inspect wikipedia-mcp_wikipedia_data
docker compose down && docker compose up -d --build   # upgrade, snapshot is preserved

Volume layout:

/data/zim/<name>.zim     # the active snapshot
/data/tmp/<name>.part    # in-progress download (resumable)
/data/state.json         # last check / last update bookkeeping

Continuous integration

.forgejo/workflows/build.yml builds the image on every push to main (and on manual dispatch) and publishes it to the Forgejo container registry:

Tag Meaning
git.imhof.cloud/julian/wikipedia-mcp:latest Newest build from main
git.imhof.cloud/julian/wikipedia-mcp:<short-sha> Immutable per-commit tag

The job runs on the ubuntu-latest runner label. That runner's image ships without a docker CLI, so the workflow installs the static CLI and talks to the daemon exposed at DOCKER_HOST. After pushing, it deletes the locally built layers and pulls :latest back from the registry before running an import smoke test, so a green run means the tag is genuinely retrievable.

Registry authentication uses the PACKAGE_TOKEN secret. The automatic Actions token is not enough — Forgejo rejects the push with unauthorized: reqPackageAccess — so PACKAGE_TOKEN must be a token with write:package scope.

Running the published image

docker login git.imhof.cloud            # needs a token with read:package
docker pull git.imhof.cloud/julian/wikipedia-mcp:latest

To deploy from the registry instead of building locally, replace build: . in docker-compose.yml with:

    image: git.imhof.cloud/julian/wikipedia-mcp:latest

Notes and limits

  • No authentication on the HTTP endpoint. Keep it on a trusted LAN, or put a reverse proxy in front of it. Do not expose port 8808 to the internet.
  • Answers are only as fresh as the snapshot — typically a few weeks behind live Wikipedia. get_wikipedia_status reports the snapshot date.
  • Tables and images are omitted from article text; prose, headings and lists are kept.
  • Content is licensed CC BY-SA by Wikipedia; the license notice is stripped from tool output, so attribute accordingly if you republish answers.