Arr Stack + Local AI

A dockerized media automation stack (VPN-locked torrents + Jellyfin) plus a local AI stack built around Ternary-Bonsai-27B for the Beelink SER5 PRO (Ryzen 7 7735HS + Radeon 680M).

Services

Service	URL / ports	Purpose
Landing	http://host/ · https://devopshomelab.com/	Dark map of services (Caddy). Public site has basic auth (LANDING_BASIC_* in .env)
Jellyfin	http://host:8096 · https://jellyfin.devopshomelab.com/	Media server / playback
qBittorrent	http://host:8080 · https://qbittorrent… / https://qbit…	Torrents VPN-locked via gluetun/WireGuard (no ISP leak)
Prowlarr	http://host:9696 · https://prowlarr.devopshomelab.com/	Indexer manager
FlareSolverr	(internal :8191)	Cloudflare proxy for indexers
Sonarr	http://host:8989 · https://sonarr.devopshomelab.com/	TV shows
Radarr	http://host:7878 · https://radarr.devopshomelab.com/	Movies
Bazarr	http://host:6767 · https://bazarr.devopshomelab.com/	Subtitles
Lidarr	http://host:8686 · https://lidarr.devopshomelab.com/	Music
LazyLibrarian	http://host:5299 · https://lazylibrarian.devopshomelab.com/	Books (ebooks / audiobooks)
SABnzbd	http://host:8085 · https://sabnzbd.devopshomelab.com/	Usenet NZB downloads
Forgejo	http://host:3002 · https://forgejo… / https://git…	Self-hosted Git (SSH :2222)
Quoting	https://quoting.devopshomelab.com/	Household quoting app (Caddy → QUOTING_UPSTREAM, default 192.168.8.135:8000)
RustDesk	21115–21119 TCP · 21116 UDP · https://rustdesk.devopshomelab.com/ (info)	Self-hosted remote desktop (hbbs + hbbr)
NetBird	https://netbird.devopshomelab.com/ · UDP 3478 (STUN) · UDP 51820 (WG)	Self-hosted mesh VPN; overlay 100.99.0.0/16 + fd99:8:123::/48; advertises LAN 192.168.8.0/24
Open WebUI	http://host:3000 · https://open-webui… / https://webui…	Chat UI, RAG, tools, code interpreter
Bonsai API	http://host:8081/v1 · https://bonsai.devopshomelab.com/	OpenAI-compatible llama-server
SearXNG	http://127.0.0.1:8888	Private web search (for RAG)
OpenHands	http://host:3001 · https://openhands.devopshomelab.com/	Coding agent (always deployed)
Cloudflare DDNS	(host network)	Keeps A records on devopshomelab.com
Stack admin	http://host:9090/	Flask control panel (compose, LLM, .env) — LAN only, systemd
Dozzle	http://host:9999/	Live Docker logs UI — LAN only (DOZZLE_USER / DOZZLE_PASS)

Keeping services documented

New services (compose or host) must update: docker-compose.yml (or unit), scripts/generate-specification.py → regenerate SPEC, landing/site/index.html, README/quickstart, .env.example, UFW ports. Agents: see AGENTS.md checklist and run ./scripts/check-stack-docs.py.

Public HTTPS (Caddy + Let’s Encrypt)

The landing container terminates TLS for public hostnames (grey-cloud DNS + WAN 80/443 → this host). Details: quickstart.md → HTTPS reverse proxy.

RustDesk (remote desktop)

Two containers always start with the stack:

Container	Role
rustdesk-hbbs	ID / rendezvous server (holds the public key clients need)
rustdesk-hbbr	Relay when peers cannot connect peer-to-peer

Public key for clients (paste into RustDesk → Network → Key):

# On the stack host (preferred path):
cat /opt/stack/config/rustdesk/id_ed25519.pub

Client field	Value
ID server	rustdesk.devopshomelab.com (or 192.168.8.123 on LAN)
Key	Full contents of id_ed25519.pub (no trailing spaces)
Relay	Advertised by hbbs as RUSTDESK_DOMAIN:21117 — usually automatic

Automated client config (churro) — generate install scripts + config string:

./scripts/RustDeskChurry.py              # uses .env RUSTDESK_DOMAIN + id_ed25519.pub
./scripts/RustDeskChurry.py --ensure-key # create key via hbbs if missing
# Output: config/rustdesk/clients/  (install-windows.ps1, install-linux.sh, churro.txt)

Windows (Administrator for install + password):

rustdesk.exe --silent-install
rustdesk.exe --config "<churro>"
rustdesk.exe --password "YourDesiredPassword123"

Linux: rustdesk --config '<churro>' then sudo rustdesk --password '…'

Router / firewall when used from the internet: TCP 21115–21119, UDP 21116.
The HTTPS page at https://rustdesk.devopshomelab.com/ is only a setup reminder — clients do not talk RustDesk over Caddy.

NetBird (mesh VPN + LAN)

Self-hosted control plane (netbird-server + netbird-dashboard) behind Caddy at https://netbird.devopshomelab.com/, plus a host-network routing peer (netbird) that advertises this house LAN.

Overlay IPv4	100.99.0.0/16 (not 10/8 or 192.168/16)
Overlay IPv6	fd99:8:123::/48 (chosen so it does not collide with Tailscale’s fd7a:115c:a1e0::/48)
Advertised LAN	LAN_SUBNET (192.168.8.0/24) so remotes use http://192.168.8.123:port
STUN	UDP 3478 on the WAN (cannot go through Caddy)

./scripts/render-netbird-config.sh
docker compose up -d netbird-server netbird-dashboard netbird landing
./scripts/netbird-bootstrap.sh   # overlay CIDRs, setup keys, LAN route

Dashboard login is NETBIRD_OWNER_* in .env (render-netbird-config.sh bcrypts the password for Dex). Phone/laptop clients use config/netbird/household.setup.key. Grey-cloud DNS for netbird.$PUBLIC_DOMAIN.

Public service names (sonarr.devopshomelab.com, radarr…, etc.) are Cloudflare DNS-only A records on the WAN IP, kept current by cloudflare-ddns. NetBird does not intercept DNS. Admin (:9090) and Dozzle (:9999) stay LAN-only.

Full project index (agents + humans): specification.html · specification.json
Regenerate both: ./scripts/generate-specification.py

---

Media stack

1. Prepare

cd arr-stack
cp .env.example .env          # then edit .env (PUID/PGID, TZ, LAN_SUBNET)

Drop your provider's WireGuard config into wireguard/wg0.conf and make the two edits described in wireguard/wg0.conf.example:

1. Remove IPv6 addresses from Address / DNS lines.
2. Endpoint must be an IP, not a hostname (dig +short your.endpoint.host).

Create the data tree (already scaffolded here; on a fresh host run):

mkdir -p config data/media/{movies,tv,music,books} data/torrents/{movies,tv,music,books}

The layout uses a single /data share — this lets the Arrs hardlink finished downloads into the media library instantly (no copy, no double disk usage, torrent keeps seeding):

data/
├── media/            <- Jellyfin libraries (+ books for LazyLibrarian)
│   ├── movies/
│   ├── tv/
│   ├── music/
│   └── books/
└── torrents/         <- qBittorrent downloads here
    ├── movies/
    ├── tv/
    ├── music/
    └── books/

2. Start media

docker compose up -d
docker compose ps            # gluetun must show (healthy)
docker logs -f gluetun       # wait for "You are running on the VPN" style line

qBittorrent waits for gluetun to be healthy before starting.

3. Verify the lockdown (do this once)

qBittorrent always uses gluetun's network namespace (network_mode: service:gluetun). It has no separate NIC: every external packet is forced through gluetun's WireGuard tunnel and firewall. That is the killswitch — the container must not reach the public internet over the host's normal ISP/LAN default route. Do not publish BitTorrent ports on the host WAN.

# IP seen from inside qBittorrent — must be your VPN's IP, NOT your home IP:
docker exec qbittorrent curl -s ifconfig.me && echo

# Your real IP for comparison:
curl -s ifconfig.me && echo

# Killswitch test — kill the tunnel, traffic must die with it:
docker exec gluetun sh -c "ip link del tun0" 2>/dev/null
docker exec qbittorrent curl -s --max-time 8 ifconfig.me || echo "NO CONNECTIVITY (good)"
docker restart gluetun       # bring the tunnel back

Belt-and-suspenders extra: in qBittorrent → Tools → Options → Advanced, set Network interface to tun0. Even in a hypothetical gluetun failure, qBittorrent itself would then refuse to use any other interface.

4. Wire it together

qBittorrent (http://host:8080 — default user admin, temporary password is printed in docker logs qbittorrent on first start; change it):

• Options → Downloads → Default Save Path: /data/torrents
• Create categories movies → /data/torrents/movies and tv → /data/torrents/tv

Prowlarr (http://host:9696):

• Add your indexers under Indexers.
• Settings → Apps: add Sonarr (http://sonarr:8989), Radarr (http://radarr:7878), each with its API key (found in the Arr under Settings → General). Indexers sync automatically from then on.

Sonarr & Radarr — Settings → Download Clients → add qBittorrent:

• Host: gluetun, Port: 8080, your qBittorrent credentials
• Category: tv (Sonarr) / movies (Radarr)
• Settings → Media Management: root folder /data/media/tv or /data/media/movies; enable hardlinks (on by default with this layout)

Bazarr: point it at Sonarr/Radarr (http://sonarr:8989 / http://radarr:7878 + API keys); paths /data/media/tv, /data/media/movies.

Lidarr (http://host:8686) — music, same patterns as Sonarr/Radarr:

• Settings → Download Clients → qBittorrent: Host gluetun, Port 8080
• Category: music → qBittorrent save path /data/torrents/music
• Settings → Media Management: root folder /data/media/music; hardlinks on
• In Prowlarr → Settings → Apps: add Lidarr (http://lidarr:8686 + API key)

LazyLibrarian (http://host:5299) — books:

• Dest dir: /data/media/books (ebooks; audiobooks can share or use a subfolder)
• Downloaders: SABnzbd (http://sabnzbd:8080, category books) and/or qBittorrent (http://gluetun:8080, user tim / pass asdfasdf, category books)
• Indexers: Newznab NZBgeek (https://api.nzbgeek.info + API key from secrets/.env)

Jellyfin (http://host:8096): add libraries → Movies: /data/media/movies, Shows: /data/media/tv, Music: /data/media/music (optional Books: /data/media/books). Media is mounted read-write (PUID/PGID) so library deletes work; prefer deleting via Radarr/Sonarr when you want the *arr DB updated too.

5. Optional: provider port forwarding

If your VPN provider gives you a forwarded port (AirVPN, ProtonVPN, …), inbound peers can reach you and swarm connectivity improves:

1. Uncomment FIREWALL_VPN_INPUT_PORTS in the compose file and set the port.
2. For providers with a NAT-PMP API also uncomment VPN_PORT_FORWARDING and VPN_PORT_FORWARDING_PROVIDER.
3. Set the same port as qBittorrent's listening port (Options → Connection) and disable UPnP/NAT-PMP inside qBittorrent.

Never publish qBittorrent's BitTorrent port on the host — inbound connections must arrive through the tunnel only.

---

Local AI stack

Why this design (and not plain Ollama)

Piece	Choice	Reason
Model	Ternary-Bonsai-27B (~7.2 GB Q2_0)	~95% of FP16 quality at laptop footprint; strong coding/math
Extra GGUF	Qwen3.8-27B Unsloth 2-bit (qwen38)	IQ2_XXS (~7.3GB, 64k) on mini/mid; Q2_K_XL (~9.8GB, 262k hybrid KV) on gaming
Runtime	PrismML llama.cpp (llama-server)	Custom Q2_0 g128 kernels for Bonsai; standard GGUFs (Qwen) share the same server
Hardware	SER5 PRO Ryzen 7 7735HS + Radeon 680M	No NVIDIA → default is CPU; optional Vulkan for the iGPU
Web UI	Open WebUI	Chat, document RAG, web search, tools, code interpreter
Search	SearXNG	Keyless private search for RAG
Coding	OpenHands (always on) + /workspace mount	Agent that edits files / runs commands in a sandbox

Memory ballpark (model alone, from Prism docs): ~8 GB at 4–10k context, ~14 GB at 100k with FP16 KV. With Arr + Jellyfin running, 32 GB system RAM is the practical minimum; 64 GB is comfortable for long context + media.

1. Download the model (~7.2 GB)

# Recommended (resume-friendly):
pip install -U "huggingface_hub[cli]"

./scripts/download-bonsai-model.sh
# files land in ./models/bonsai/

Optional:

DOWNLOAD_DRAFTER=1 ./scripts/download-bonsai-model.sh   # speculative decoding pack (CUDA-oriented)

2. Start AI services

# Build Prism llama-server image + start UI + search
docker compose up -d --build bonsai open-webui searxng

# Health
curl -s http://localhost:8081/health
curl -s http://localhost:8081/v1/models

Open http://host:3000 — create the admin account on first visit. Open WebUI is pre-pointed at http://bonsai:8080/v1 (see compose env).

3. RAG (documents & local files)

Upload / Knowledge collections (best for docs)

1. Open WebUI → Workspace → Knowledge → create a collection.
2. Upload PDFs, markdown, code, notes.
3. In a chat, attach the collection (or enable it on the model).

Embeddings download into the Open WebUI volume on first use (local, no cloud).

Whole folders via bind-mount

AI_WORKSPACE_DIR (default ./workspace) is mounted read-only at /workspace inside Open WebUI. Put projects/notes there, or point .env at your real code tree:

# .env
AI_WORKSPACE_DIR=/home/you/src

Then ask e.g. “Summarize the layout of /workspace/my-app” after adding those paths to a Knowledge collection, or use tools/OpenHands for active editing.

Web RAG

SearXNG is wired as the search engine (ENABLE_RAG_WEB_SEARCH=true). In chat settings enable web search when you want live results.

4. Coding helpers

Tool	How	Best for
Open WebUI + Knowledge	Upload repo docs / mount /workspace	Q&A over code, design docs
Open WebUI code interpreter	Enabled in compose	Python plots, quick analysis
OpenHands	Always up with the stack (:3001)	Multi-step “change the code / run tests” agents
Continue / Aider / Cursor (host apps)	Base URL http://host:8081/v1, any API key	IDE-native completion & edits

OpenHands (http://host:3001):

1. Settings → enable Advanced.
2. Custom model: openai/<id-from-/v1/models> (or keep the pre-set env).
3. Base URL: http://bonsai:8080/v1 (from inside Docker) or http://host.docker.internal:8081/v1.
4. API key: same as OPENAI_API_KEY in .env.
5. Workspace is ${AI_WORKSPACE_DIR} — agent sandboxes use Docker socket.

Note: Prism lists long-horizon multi-file agentic coding as a weaker area of this 27B release; it is still very usable for review, refactors, and smaller tasks. Prefer shorter goals and verify with tests.

5. Performance knobs (SER5 PRO)

# .env examples

# Leave headroom for Jellyfin/Arrs on a 16-thread 7735HS
BONSAI_THREADS=12

# Longer context for big repos / PDFs (needs RAM)
BONSAI_CTX=32768
BONSAI_KV4=1          # 4-bit KV cache if you approach OOM

# Optional: Vulkan offload on Radeon 680M
BONSAI_BACKEND=vulkan
BONSAI_NGL=99
# then uncomment devices/group_add under bonsai in docker-compose.yml
docker compose build --no-cache bonsai && docker compose up -d bonsai

Start with CPU (BONSAI_NGL=0). iGPU Vulkan can help but shares system RAM and is more fiddly in Docker; measure both with a short prompt if you care.

Recommended generation defaults (already set): temp=0.7, top_p=0.95, top_k=20. The 27B is a thinking model — answers may include a reasoning block; that is expected and improves quality.

6. Quick API smoke test

curl -s http://localhost:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-local-bonsai" \
  -d '{
    "model": "local",
    "messages": [{"role":"user","content":"Say hello in one short sentence."}],
    "temperature": 0.7,
    "max_tokens": 128
  }'

(model can be any string or the id returned by /v1/models — llama-server is usually lenient.)

7. What not to do

• Do not route bonsai/open-webui through gluetun — inference is local and the VPN killswitch would only add pain.
• Do not expect stock Ollama to load Ternary-Bonsai-27B-Q2_0.gguf without Prism’s kernels (use our bonsai service).
• Keep Open WebUI / OpenHands off the public internet; use LAN + auth (WEBUI_AUTH=true).

---

Layout after AI setup

arr-stack/
├── docker-compose.yml
├── .env / .env.example
├── secrets/.env             # Usenet + Cloudflare (gitignored)
├── ai/                      # Prism llama-server image
│   ├── Dockerfile
│   └── entrypoint.sh
├── landing/                 # Caddyfile + LAN map site + entrypoint
├── models/bonsai/           # GGUF weights (gitignored content)
├── workspace/               # files for RAG / OpenHands
├── searxng/settings.yml
├── scripts/                 # bootstrap, usenet, DDNS, generate-specification, …
├── config/                  # app configs (open-webui, sonarr, rustdesk, …)
│   └── rustdesk/
│       ├── id_ed25519       # server private key (do not share)
│       └── id_ed25519.pub   # ← give this string to every RustDesk client
├── data/                    # media + torrents
└── wireguard/wg0.conf

Notes

• All app-to-app traffic uses container DNS names (gluetun, sonarr, bonsai, …) on the internal arr-net bridge; only WebUIs (and RustDesk ports if you open them) are exposed to your LAN / WAN.
• Update media stack with docker compose pull && docker compose up -d. Gluetun is pinned to major version v3 — read its release notes before jumping majors.
• Rebuild AI runtime after Prism release bumps: edit PRISM_RELEASE in ai/Dockerfile, then docker compose build --no-cache bonsai.
• Optional media extras (commented in compose): Jellyseerr.
• OpenHands always deploys with the full stack (no compose profile required).
• After architecture changes: ./scripts/generate-specification.py and commit both specification.json and specification.html.