Skip to content

FastTAK Design Decisions

Significant architectural and design decisions, with reasoning. Newest first.

DD-064: The monitor refuses changes requested by another site

Date: 2026-11-11 Status: Decided

Decision: The monitor answers 403 to any POST, PUT, PATCH or DELETE whose Sec-Fetch-Site header is cross-site or same-site. Requests without the header pass. It also refuses bodies over 1 MiB, and records the address Caddy saw (the last X-Forwarded-For entry, honoured only with the proxy secret) instead of Caddy's own.

Why: Login is HTTP Basic, which the browser attaches to every request for the monitor's origin, including one made by a page on another site. Routes with a JSON body were already safe, because a cross-site page cannot send application/json without a preflight, but routes with no body (database vacuum, backup run, alert tests) needed nothing else. The body was read before any route check ran, with no limit. Audit rows all carried Caddy's container address, which made them useless for tracing a login.

Consequences: A browser too old to send Sec-Fetch-Site is not protected. same-site is refused too, so a page served from a sibling subdomain, such as a Node-RED dashboard, cannot drive the monitor from a browser; flows that call the API from the server are unaffected.

Alternatives considered: - CSRF tokens: rejected. They need session state and a change to every form and fetch in the dashboard, for the same protection in current browsers. - Checking Origin against SERVER_ADDRESS: rejected. It needs the public origin configured correctly in both deploy modes, and a wrong value locks the dashboard out.

DD-063: TAK Server reads the revocation list when it starts

Date: 2026-11-11, revised 2026-11-15 Status: Decided. The list is confirmed on TAK Server 5.8-RELEASE-65: a revoked cert connected until tak-server restarted, and was refused after. The database check for enrolled certs is taken from TAK Server's source (5.7) and has a test, which has not run yet.

Decision: init-config adds <crl _name="TAKServer CA" crlFile="certs/files/ca.crl"/> to the <tls> element inside <security> in CoreConfig.xml, and writes an empty ca.crl when there is none so the file exists before TAK Server starts. start.sh checks the element is present. When a revocation takes effect depends on who issued the cert:

Cert Check Takes effect
Enrolled through TAK Server (QR code, port 8446) TAK Server's database, at every connection (x509checkRevocation) At once
Generated by the monitor (downloaded .p12, service account) or by certs.sh The revocation list At the next tak-server start

An operator who needs a monitor-generated cert refused now restarts tak-server (docker compose restart tak-server), which disconnects every client for a minute or two. tests-integration/test_zz_revocation_enforced.py connects with a revoked cert of each kind.

Why: Revoking a cert in the monitor or with certs.sh adds it to ca.crl, and nothing told TAK Server to read that file. The only revocation setting was x509checkRevocation on <auth>, which covers certs TAK Server enrolled itself. A cert generated by the monitor for a lost tablet therefore kept connecting for ever after it was "revoked", while the dashboard showed it revoked and blocked its download. Every test stopped at "the serial is in the list". With the list wired in, such a cert is refused from the next start. TAK Server builds its TLS settings, the list included, once when the listener starts, and has no setting to reload them.

Consequences: Between a revocation and the next tak-server start, a monitor-generated cert still connects, and the dashboard already shows it revoked. This applies to a deactivated account's certs too: deactivation also removes the user's password and groups (DD-061), but the cert itself connects until the restart. DD-028's statement that a revoked cert is rejected at the next handshake holds only for enrolled certs. If the element cannot be added (no <tls> in <security>, or no ca.crl), init-config logs a warning and start.sh reports a failed check; the stack still starts, because refusing to would take a working deployment down. init-config rewrites the list on every start and the monitor on every revocation, both valid for ten years, because a list past its own expiry date might make TAK Server refuse every client; how TAK Server treats an expired list is not known. ./certs.sh revoke uses TAK Server's own script and its default validity, which the next start corrects.

Alternatives considered: - Restarting tak-server on every revocation: rejected as a default, because it disconnects every client to stop one. - Issuing monitor-generated certs through TAK Server's enrollment route, so that the database check covers every cert and there is one cert path instead of two: the better end state, and too large a change for a hardening release. Tracked as a follow-up issue. - Writing rows into TAK Server's certificate table directly: rejected, it depends on TAK Server's internal schema. - Removing the user's groups instead: that is what deactivation does (DD-061), but revoking one cert of an active user has no such fallback.

DD-062: Password guessing through TAK Server is limited per account

Date: 2026-11-11 Status: Decided

Decision: ldap-proxy counts failed binds on its LDAP listener per account name and locks the name out with the same limits as /auth/verify (LDAP_RATE_LIMIT_*: 10 failures in 5 minutes, 15 minutes locked). The service account TAK Server searches with is exempt. The monitor refuses passwords shorter than 12 characters. An enrollment token is accepted on the LDAP listener only; /auth/verify checks the password against LLDAP and ignores tokens.

Why: TAK Server's port 8446 is published in every deployment and takes passwords. Each attempt reaches ldap-proxy as a bind from TAK Server, which DD-037's limiter never saw because it covers only /auth/verify, and the monitor accepted one-character passwords. The client's address is not visible on the LDAP side, so the account name is the only key available. Tokens were also a 15-minute password for the monitor and Node-RED: a QR code shown for an admin opened both to whoever photographed it.

Consequences: Someone who knows a username can keep that user's password login locked by guessing. Certificate logins never bind as the user, so devices in the field stay connected, and the monitor and Node-RED stay reachable because their limit is per address. docker compose restart ldap-proxy clears a lockout. Existing short passwords keep working until changed. The exempt service account relies on its generated password.

Alternatives considered: - Limiting per source address: not possible here; every bind comes from TAK Server. - Putting 8446 behind Caddy only: rejected by DD-038, because enrollment needs TAK Server's own certificate on that port. - A password-strength score: rejected. Length is the part that matters against guessing and needs no dependency.

DD-061: Deactivation cuts credentials instead of setting a flag

Date: 2026-11-05 Status: Decided

Decision: Deactivating a user or service account deletes its enrollment tokens, replaces its password with a random one nobody keeps, removes it from every group, then sets is_active=false last. All three cuts are attempted even if one fails, and the account is marked inactive only when all three worked. PATCH is_active:false, DELETE and TTL expiry all do this. Reactivating restores neither the password nor the groups. webadmin and svc_fasttakapi cannot be deactivated. The response reports certs_revoked, and the dashboard says so when that is false.

Why: is_active was an attribute nothing enforced at login, so a deactivated user kept LDAP binds, the monitor, WebTAK and, with monitor_admin, the backup key. LLDAP has no disabled state. The tokens have to go separately because ldap-proxy accepts a token before it reaches LLDAP. Groups go too because TAK Server reads them for cert users, so a cert that outlives deactivation reaches nothing the next time it connects. is_active comes last because the TTL job only processes active users; written first, a failure would never be retried.

Consequences: An admin must set a new password and groups after reactivating someone, and the log records which groups were removed. Deactivation fails with a 502 if a credential cannot be cut, and certs are still revoked in that case; the call is safe to repeat. The TTL job also cuts any inactive account that still belongs to a group, which covers accounts deactivated before this change and anyone who gives groups to an inactive account (upgrade notes). A TTL cannot be set on webadmin, and the TTL job skips the accounts the stack depends on. A TAK session that is already open keeps the groups it connected with: TAK Server applies the removal when a client connects, not to an open session (seen on 5.8, 2026-11-16). Revoking an enrolled cert closes its session at once. A cert downloaded from the dashboard and a service account's cert keep an open session, with its old groups, until the client disconnects or tak-server restarts (DD-063); after someone with such a cert is deactivated, docker compose restart tak-server cuts them off. Video is separate: every viewer shares MEDIAMTX_VIEWER_PASSWORD (DD-056), so someone who has left can still watch streams until that password is rotated.

Alternatives considered: - Checking is_active in ldap-proxy on every bind: rejected. It adds a directory lookup to the login path, and a lookup failure either blocks all logins or lets the user in. It would keep the password for reactivation, but that convenience is not worth a second enforcement point. - Remembering groups for reactivation: rejected. Restoring old access, possibly monitor_admin, should be a deliberate act. - Letting webadmin be deactivated: rejected. Bootstrap re-creates it from .env on restart, so the deactivation would quietly undo itself.

DD-060: LLDAP gets its own JWT secret and, on fresh installs, its own key seed

Date: 2026-11-04 Status: Decided

Decision: LLDAP_JWT_SECRET is a required generated secret on every deployment. LLDAP_KEY_SEED is generated by setup.sh on fresh installs only. Where it is empty, Compose passes RanD0m STR1ng, the value the image's config template has always applied.

Why: With no jwt_secret, LLDAP signs tokens with the public default secretjwtsecret, and it reads a session's permissions from the token's claims. Anyone on the Docker network could therefore mint a lldap_admin token and own the directory, with no password. Rotating it costs only a re-login, so upgrades get it too.

The key seed is different. It derives the server key that password records depend on. LLDAP refuses to start when the key has changed, and forcing past that invalidates every password, so an upgrade must not generate one. Existing deployments stay on the public seed, whose exposure is smaller: it matters to someone who also holds a dump of the database.

Consequences: A new deployment and an old one now differ in one setting. Do not set LLDAP_KEY_SEED on a populated directory. Restoring a backup needs the .env from that backup. Moving an existing deployment to its own seed means setting LLDAP_FORCE_UPDATE_PRIVATE_KEY once and resetting every password; that is a planned migration, not an upgrade step. This relates to #103: LLDAP refuses to boot when a key_seed is set and a server_key file also exists. Passing the seed through the environment gives every lldap process in the container the same seed, including the healthcheck, which runs without the config file.

Alternatives considered: - Generating the seed on upgrade as well: rejected, because it would lock every user out or stop LLDAP. - A :? guard on the seed: rejected, because it would force the same choice on every existing deployment. - LLDAP_KEY_FILE: rejected. LLDAP recommends the seed, and a file needs a volume and a backup path.

DD-059: The monitor requires proof a request came through Caddy

Date: 2026-11-04 Status: Decided

Decision: Caddy adds X-FastTAK-Proxy-Secret to requests on the monitor route. AuthContextMiddleware honors Remote-User and Remote-Groups only when that header matches MONITOR_PROXY_SECRET; otherwise the request has no identity and every admin gate refuses it. The secret is generated by ensure-secrets.sh and given only to Caddy and the monitor. The generated Caddyfile reads it from Caddy's environment, so it is not written to disk.

Why: Issue #78, and the residual DD-047 and DD-050 left open. Any container on the Docker network could send Remote-Groups: monitor_admin and be an admin, including reading GET /api/backup/key and the backups, which hold .env and the CA key. Reachability is not authorization.

Consequences: A missing secret stops docker compose up. If the monitor somehow runs without one, it logs an error and every user gets 403: it fails closed. Tests that publish the monitor port directly must send the secret; the unit suites bypass the check through a fixture, and tests of the check itself use the real_proxy_check marker.

Alternatives considered: - A separate network for the monitor: rejected as the only control. The monitor has to share a network with Caddy, and Compose services that join it later would quietly regain access, whereas a credential fails closed. It remains worthwhile as a second layer. - mTLS between Caddy and the monitor: stronger, but it adds a CA and a TLS listener for a problem a shared secret already solves.

DD-058: Node-RED is limited to one LDAP group

Date: 2026-11-04 Status: Decided

Decision: /auth/verify takes an optional group parameter and returns 403 unless the user is a member. Caddy's Node-RED route passes NODERED_ADMIN_GROUP, default monitor_admin. The whole route is gated, including anything a flow serves over HTTP.

Why: Node-RED has no authorization of its own behind forward_auth, and its editor runs code in the container by design, so every LDAP user and every holder of an enrollment token had a shell on the Docker network. The monitor gates its own routes (DD-047) but Node-RED cannot, so the check has to happen in front of it.

Consequences: Users outside the group lose Node-RED on upgrade, including any dashboards or HTTP endpoints served by flows. A deployment that was reachable before this change should rebuild the nodered-data volume, because a planted flow, settings.js or npm package survives the upgrade. From inside the Docker network, nodered:1880 still answers with no login, so this closes the route from users, not from another compromised container.

Alternatives considered: - Node-RED adminAuth: rejected for now. It would add a second login on top of Basic auth, and it leaves flow HTTP endpoints open. - A new dedicated group: rejected as the default. The editor holds credentials an admin already has, and a new group would lock everyone out until someone joined it. NODERED_ADMIN_GROUP allows one where a deployment wants it. - Gating only / and the admin API: rejected, because Node-RED's paths are configurable and a missed path would fail open.

DD-057: Caddy strips identity and credential headers around forward_auth

Date: 2026-11-04 Status: Decided

Decision: Each forward_auth route removes Remote-User and Remote-Groups from the client's request before calling ldap-proxy, and removes Authorization before calling the backend. The Caddy image follows the major version (caddy:2-alpine). /auth/verify returns 503 when it cannot read a user's groups.

Why: copy_headers only overwrites a header that ldap-proxy returned with a value. Before Caddy 2.11.2, an empty Remote-Groups left the client's copy in place, so a user with no groups could send Remote-Groups: monitor_admin and reach the monitor as an admin. Removing the headers first holds whatever Caddy version runs, so the image does not need a pin to stay safe. Basic auth also sends the LDAP password on every request, and neither backend uses it, so leaving it in place gave any flow author the passwords of everyone who opened Node-RED.

Alternatives considered: - Pinning Caddy alone: rejected, because the guarantee would then rest on an upstream fix with no check in this repo. - Pinning Caddy by version and digest as well: rejected. A pinned image receives no Caddy security fix until someone edits the pin, and nothing in this repo opens that update. With caddy:2-alpine, docker compose pull caddy brings the fixes. The cost: hosts can run different 2.x versions, and a host keeps its cached image until it pulls. - Not returning Remote-Groups for a user with no groups: rejected, because a lookup failure and an empty membership still could not be told apart.

DD-056: MediaMTX has no anonymous access

Date: 2026-09-28 Status: Decided

Decision: MediaMTX has three fixed internal users — viewer (read, playback), publisher (publish), api (control API) — and no any entry. Their passwords are generated into .env and injected by Compose through MTX_AUTHINTERNALUSERS_<index>_PASS/_IPS, with ${VAR:?} so a missing password stops docker compose up. The entries in mediamtx.yml are loopback-only placeholders, so the file alone admits nobody. Publishing needs the publisher password and a source address in MEDIAMTX_PUBLISH_IPS, which defaults to 127.0.0.1/32 (off). RTSP uses Digest only. The users are per role, not per source: every source that sends a stream (an encoder, a camera, a TAK plugin such as UAS Tool) signs in as publisher and may use any path. Whatever announces a stream to TAK clients puts the viewer credential in the marker's <__video> url and ConnectionEntry address; the DroneSense flow and video-cot.sh do this today. The DroneSense flow, where MediaMTX pulls the video, is a stopgap until sources publish directly (#38).

Why: The previous config let anyone who reached 8554/1935 watch every stream or publish into a path operators watch — an integrity attack on the map. Operators worked around it by unpublishing the ports, which also shut out every permitted viewer and source outside the Docker network. The API needs a credential too: it can rewrite authInternalUsers at runtime, and the password keeps other containers on the Docker network from granting themselves access. Node-RED holds this credential in its environment, so anyone who can edit Node-RED flows (members of NODERED_ADMIN_GROUP, DD-058) effectively has MediaMTX admin.

Alternatives considered: - IP-only publish restriction — rejected: the source address MediaMTX sees depends on the platform (Docker Desktop NATs every client to its VM gateway; on Linux a localhost client arrives from the bridge gateway, inside the Docker subnet). - Rendering mediamtx.yml in init-config — unnecessary: MediaMTX 1.21 honors env overrides for user entries (1.15.5 did not; hence the 1.21 floor in check-env.sh), and the image has no shell. - An unguessable hashed placeholder password — rejected: MediaMTX refuses hashed credentials when Digest is enabled, even if overridden. - A boolean publish toggle — rejected: env overrides cannot delete a user entry, and an IP list also covers the VPN-subnet case. - Basic instead of Digest — MediaMTX's docs call Digest "compatibility only". True against an active man-in-the-middle, who gets the video anyway; but with generated 128-bit passwords Digest keeps a passive observer from capturing a reusable credential, at no cost to the TAK clients (WinTAK and ATAK both tested). RTSPS (#38) supersedes it. - Client-certificate auth — not possible: MediaMTX 1.21.1 has no client-CA setting, its HTTP auth hook receives no certificate, and TAK players show no sign of presenting client certificates.

DD-055: Status Reporters Never Fail Silent; Passes Are Quiet, Failures Explain Themselves

Decision: Every FastTAK surface that reports health or verifies the stack — start.sh's checks, the monitor's health probes, the TAK container's healthcheck, the alert engine — follows four rules:

  1. Never silent. A probe that cannot answer reports that it cannot, with the reason. None, an empty list, or a swallowed exception is not a result.
  2. Passes are quiet, failures explain themselves. Default output is the failures and one summary line. Every failure line has three parts: the label, what was expected or checked together with what came back, and the next command to run — e.g. ❌ init-config exited 0: expected "0", got "healthy". Next: docker compose logs init-config. --verbose prints passes and notes. An inspection command may print the inventory the operator asked for — --doctor's published-port table is the answer to the question, not a list of passes. Individual check pass lines still appear only under --verbose.
  3. Absent is not broken. A port this deployment deliberately does not publish, a cert not yet created, a service outside this deploy mode is a note, never a failure. "Not running" is never a note. "Could not read it" is not "absent": an item carrying an error is a warning with the reason, lowered to a note per service (error_status) only when the source is outside the deployment, as for the GitHub update check.
  4. Checks are re-runnable against a running stack without touching it.

Why: the failures that hurt were the quiet ones. A subdomain-mode start ended with ❌ Port 0 (Node-RED) on a healthy stack because Compose's "not published" answer was read as a port number (#120); the TAK container reported HEALTHY through a total outage because its probe only failed on no-response (#79); the cert probes returned None on any error, so an expired cert made the dashboard greener (#57); the alert engine never recorded a recovery and swallowed a recurring warning (#77). Each was an error path that returned "nothing" and a reporter that read "nothing" as "fine".

Alternatives considered: printing every check by default. Rejected — 29 lines on every healthy start trains operators to scroll past the block, which is how the one failing line was missed. The summary count carries the tally; --verbose is one flag away.

Consequences: Adding a check means writing its failure line with the three parts. A probe that catches an exception must return an error item or raise, never drop the entry. Epic #123 applies these across the stack.

DD-054: Restoring cot Pre-Creates Extensions as Superuser, Then Reassigns Ownership

Decision: Restoring the cot database — tests-integration/restore.sh, and the by-hand procedure in docs/backup-and-restore.md — pre-creates every extension the dump requires as the postgres superuser, over the Unix socket, before handing the dump to martiuser to restore. It then reassigns each extension's ownership (pg_extension.extowner) and the ownership of any tables in its extconfig (e.g. PostGIS's spatial_ref_sys) to martiuser, rather than leaving them owned by postgres.

Why creation needs a superuser: every TAK cot dump carries CREATE EXTENSION IF NOT EXISTS postgis, PostGIS is not a trusted extension, and the restore connects as martiuser, which is not a superuser. Restoring wholesale as martiuser fails outright with "permission denied to create extension postgis". Restoring wholesale as postgres avoids that, but the dumps are taken with pg_dump --no-owner (monitor/app/backup/runner.py) specifically so the restoring role ends up owning what it restores — doing the whole restore as postgres would leave TAK Server, which connects as martiuser, without write access to its own tables.

Why ownership has to be reassigned, not just the extension created: two statements later in the same dump act on objects the superuser just created — COMMENT ON EXTENSION postgis IS '...' requires owning the extension, and COPY public.spatial_ref_sys ... requires INSERT on a table that CREATE EXTENSION created as postgres. Both fail against a postgres-owned extension; confirmed against a live stack. PostgreSQL has no ALTER EXTENSION ... OWNER TO, so extowner is set directly — a superuser-only UPDATE of one OID column — and extconfig (the extension's own list of tables whose data pg_dump emits) gives the exact table set that needs the same treatment. The result is the state martiuser would have reached had PostGIS been trusted enough to create itself.

Why the extension set is discovered from the dump, not hardcoded: it varies by TAK release — the cot dump also carries pgcrypto, and postgis_topology/fuzzystrmatch appear on other releases — so a hardcoded postgis would put the next release back on the same failure path. Both call sites scan the dump's pre-data section for CREATE EXTENSION statements (stopping at the first COPY ... FROM stdin, since pg_dump never emits one before the other) and build the ownership-handoff SQL from what they find.

Alternatives considered:

  • Grant martiuser superuser for the restore, then revoke it. Rejected — broadens the blast radius of a compromised restore session for no benefit over a narrowly-scoped ownership handoff.
  • Mark postgis a trusted extension so martiuser can create it directly. Rejected — that is a cluster-wide ALTER EXTENSION ... TRUSTED change on an extension that manipulates filesystem-adjacent GEOS/PROJ libraries, a larger and less reviewable change than reassigning ownership after the fact.

Do not simplify this to a plain CREATE EXTENSION as martiuser. It works right up until the first dump that touches extconfig, then fails mid-restore with the data already dropped.

Kept in sync in two places: tests-integration/restore.sh carries this logic as upgrade_dump_extensions / upgrade_extension_sql, and docs/backup-and-restore.md spells the same SQL out longhand for an operator restoring by hand. The script is canonical; a change to it has to be mirrored in the doc.


DD-053: WAL Archiving Disabled

Decision: tak-database/start.sh appends archive_mode = off to both db-utils/postgresql.conf (what the vendor's takserver-setup-db.sh copies into PGDATA on the first boot of a fresh database) and the live $PGDATA/postgresql.conf (what the server reads on every boot after that) — see DD-020 for why both files need writing.

Why: TAK Server ships archive_mode = on with archive_command = 'cp "%p" "/var/lib/postgresql/archivedir/%f"', and nothing in any TAK bundle creates that directory. PostgreSQL will not recycle a WAL segment until its archive command succeeds, so every segment is retained and pg_wal grows until the filesystem fills.

This went unnoticed because the PGDATA persistence defect masked it — every container recreate wiped the accumulated WAL along with the database. Fixing persistence without also fixing this would have traded a data-loss bug for a disk-fill bug.

Alternatives considered:

  • Create archivedir so the cp succeeds. Rejected: it converts a fast disk-fill into a slow one, because nothing prunes the archives either.
  • Build point-in-time recovery on top of the archives. Out of scope — FastTAK's backups are logical (pg_dump per database) and no restore path consumes an archive. Worth revisiting as its own feature with its own retention design.

DD-052: tak-server Root Override on the Hardened Images

Decision: docker-compose.yml sets user: "0:0" on tak-server only, overriding the hardened image's own tak:0 (uid 1001). tak-database keeps the hardened image's own postgres:0 — it does not get the override.

Why tak-server needs it: TAK Server writes into the bind-mounted /opt/tak tree, and the monitor's cert_gen exec calls (monitor/app/api/service_accounts/cert_gen.py) pass no user= and so inherit the image user — twelve exec_run call sites, plus a put_archive writing PEMs into /opt/tak/certs/files. Adopting non-root properly needs a cert-tree permissioning scheme, the same blocker that defers the non-root monitor.

Why tak-database cannot take it, and does not need to: PostgreSQL categorically refuses to run as root (pg_ctl: cannot be run as root), and TAK 5.8's entrypoint (db-utils/configureInDocker.sh) calls pg_ctl directly rather than wrapping it in su - postgres the way 5.6 did — so user: "0:0" on this service would simply fail to start. It does not need the override either way: unlike tak-server, tak-database does not bind-mount /opt/tak over the image's copy — it uses the image's own — which removes the host-ownership conflict that motivated the override everywhere else. Running it as the image's own postgres:0 is both required and sufficient.

Not purely a regression: under rootless Podman, container uid 0 maps to the invoking user, so bind-mount writes to a user-owned ./tak work, while uid 1001 maps into the subuid range and owns nothing.

What is retained: the image hardening is independent of process user — UBI 10 with the DISA STIG remediations, FIPS-enabled OpenSSL, ChaCha20 disabled in the JDK TLS policy, DoD/WCF trust anchors, package minimisation, GPG-verified package sources, and the file-integrity healthchecks. What is given up, on tak-server only, is the non-root process user.

Consequence worth knowing: because tak-database uses the image's own /opt/tak, which ships CoreConfig.example.xml but no CoreConfig.xml, TAK's SchemaManager logs two WARN lines on every boot as it falls through to the example. tak-database/start.sh patches the DB password into whichever of the two exists, so the connection succeeds and the schema upgrade runs normally. The warnings are expected, not a fault.

Tracked in #97, sequenced behind #69 — moving cert-exec off the monitor removes most of the code that forces the override.


DD-051: TAK Server 5.8 Hardened Bundle Is the Supported Floor

Decision: setup.sh and scripts/check-env.sh both refuse a TAK Server release below 5.8 — setup.sh at extraction time (reading tak/version.txt), check-env.sh at every stack start, since .env can be hand-edited after setup.sh already ran. check-env.sh distinguishes a TAK_VERSION that is simply below the floor from one it cannot parse at all, so the two failures do not read as the same problem. FastTAK builds only the hardened Dockerfiles found in the release bundle's docker/ directory (Dockerfile.hardened-takserver, Dockerfile.hardened-takserver-db).

Why: tak.gov ships 5.8 as a hardened bundle. A 5.6 bundle would build cleanly but land PGDATA at /var/lib/postgresql/15/data rather than the volume FastTAK mounts, silently un-persisting the database on every container recreate — the defect this whole change fixes. Refusing outright is simpler and safer than tolerating a configuration that loses data quietly.

Alternatives considered:

  • Detect the bundle flavour and support both. Rejected as speculative: there is no standard (non-hardened) 5.8 bundle to support today. Support can be added back as filename detection plus a build arg if one appears.
  • Warn instead of refusing. Rejected — the failure mode is silent data loss, exactly the case a warning does not adequately cover.

Note: PGDATA location and the container user are properties of the hardened Dockerfile, not of TAK 5.8 itself. A future standard bundle could reintroduce the same problem, which is why tak-database/check-pgdata-persistent.sh stays in place even though 5.8's hardened image gets PGDATA right.

Building the images needs network access. The hardened images install packages from the Rocky, EPEL, Adoptium and PGDG repositories during the build — a change from earlier bundles, which built from postgres:15.1 with two extra packages. Worth noting for restricted-egress build environments.


DD-050: The Docker network is not a trust boundary

Decision: The ldap-proxy /tokens API requires a shared secret (TOKENS_API_SECRET, sent as a bearer token by the monitor), and the proxy rejects LDAP search from connections that have not completed a successful bind.

Why: Issue #54. /tokens mints an LDAP bind credential for any username, including admin accounts, and it was unauthenticated on the reasoning that only the monitor could reach it. That reasoning does not hold: every container on the default network can reach ldap-proxy:8080, and one of them — Node-RED — is a scripting environment reachable by every authenticated LDAP user. Anonymous search was the same shape: the proxy relayed queries upstream as the admin service account, so an unauthenticated caller could read the whole directory.

This supersedes the scope note in DD-036, which justified leaving /tokens/* unprotected because it was "internal-only … no Caddy route". Reachability is not authorization, and colocation on a Docker network is not a trust boundary. The same reasoning error appears in DD-047's residual (the monitor trusting Remote-Groups from any container), closed by DD-059 — this decision records the principle for both.

Consequences: TOKENS_API_SECRET becomes a required secret. setup.sh generates it on fresh install; existing deployments must add it, and the proxy fails closed (refusing to start) without it — which stops the whole stack, since tak-server waits on ldap-proxy being healthy. That upgrade path needs explicit handling rather than being left to fail loudly at the worst moment.

Residual: search authorization is binary — any connection that completed any successful bind may search, and the proxy still relays as the admin service account. That raises the bar from "anyone" to "anyone with a valid credential", not to "only what you are entitled to see". A user's own password, or a 15-minute enrollment token, is enough to enumerate the directory. Per-principal search scoping is out of scope here.

Testing note: the Go tests added with this change (connauth_test.go, server_authz_test.go, tokens_auth_test.go) encode the rules above — who may call /tokens, and who may search. They were initially unreachable by CI, since just test was shellcheck plus pytest with no go test step, which left the authorization rules verified only when someone ran them by hand. just test now runs them and CI installs Go from ldap-proxy/go.mod, so they gate every push. Locally they are skipped with a warning when Go is absent, because the image builds in-container and a contributor need not have a toolchain installed.


DD-049: The monitor reaches Docker through a path-allowlisting proxy

Decision: The monitor no longer mounts /var/run/docker.sock. A wollomatic/socket-proxy container holds the socket on an internal: true network with no egress, and the monitor reaches it over HTTP via DOCKER_HOST=tcp://docker-proxy:2375. The proxy allowlists by HTTP method and URL path, permitting exactly what the monitor calls: container list/inspect/ logs/stats, image inspect, exec create/start/inspect, and PUT .../archive. POST /containers/create matches nothing and is denied.

Why: Issue #53. Docker socket access is host root — anything that can reach that API can create a privileged container bind-mounting host /. The :ro flag protects the socket inode, not the API. The monitor is the most exposed component in the stack (a web-facing FastAPI app) and simultaneously holds LDAP admin credentials, the TAK admin mTLS cert, and database passwords, so a web bug there escalated to full host compromise.

Why wollomatic rather than tecnativa (which #53 suggested): tecnativa's proxy toggles whole API sections, so enabling containers permits POST /containers/create along with exec. The monitor genuinely needs exec (for cert issuance) and must not have create, and only per-path regex matching can express that distinction.

Configuration that is easy to get wrong: -allowfrom defaults to 127.0.0.1/32, which rejects the monitor and silently nulls its Docker client at API-version negotiation — every Docker-backed feature then fails at once while the container still reports healthy. It must name the client (-allowfrom=monitor, resolved by DNS per request). The allowlist must also cover the calls the SDK makes implicitly: exec_run() always calls exec_inspect(), container.image resolves through inspect_image, and image IDs arrive as sha256:… so the charset must admit a colon.

Residual — this does not close #53. The proxy matches paths and never inspects request bodies, and the container-ID pattern cannot be name-scoped because the SDK addresses containers by hex ID. A monitor compromise can still exec as root into any container (reaching ca-do-not-share.key in tak-server, the databases, the directory), pass "Privileged": true on that exec, and inspect any container's environment. What is retired is the clean one-shot host escape. #69 tracks moving cert-exec off the monitor, after which -allowPOST and -allowPUT can be dropped and the proxy made GET-only.

Non-root deferred: running the monitor as a non-root user was part of #53 and was removed from this change. tak-server generates certs as root at mode 0600 under a tree the monitor mounts read-only, so a non-root monitor gets EACCES on the mTLS client cert, on .p12 downloads, and on the backup tar. That needs a cert-tree permissioning scheme, tracked separately, and should not hold up the escape fix.


DD-048: Upgrade rules live in the scripts, not in a checklist

Decision: Anything an upgrade must always do is enforced by code, not documented as a step. start.sh and reconfig.sh pass --remove-orphans (as just up already did); init-identity bootstraps every gate group (ADMIN_GROUP and BACKUP_ADMIN_GROUP) and joins webadmin to each; scripts/check-env.sh refuses to start on a missing or unsafe value. docs/upgrading.md covers only what genuinely requires an operator decision — chiefly which accounts get admin access.

Why: Two failures on the same deployment showed that upgrade rules kept as prose don't get followed. docker compose up does not remove containers whose service was deleted from the compose file, so tak-portal — removed in DD-043 precisely because it had become unauthenticated — kept running for months, invisible to docker compose ps. Separately, DD-047 introduced ADMIN_GROUP while the bootstrap knew only BACKUP_ADMIN_GROUP; renaming the former produced a group nothing created and nobody joined, 403-ing every route on the monitor including for webadmin, with no route back in through the UI.

The dividing line: if the correct action is always the same, it belongs in a script. If it needs checking but no judgment, it belongs in the preflight. Only genuine decisions belong in prose.

Retroactive by construction: init-identity runs on every stack start and its ensure_group/add_to_group helpers are idempotent, so existing deployments repair themselves on the next restart rather than needing a migration.

Alternatives considered: A dedicated upgrade script — rejected, it would duplicate setup.sh and start.sh and become a third thing to keep in sync. Documenting the rules and trusting the checklist — rejected on the evidence above.


DD-047: The monitor is admin-only, end to end

Decision: Every route on the monitor requires membership in ADMIN_GROUP (default monitor_admin) — the JSON APIs (/api/users, /api/service-accounts, /api/ops, /api/health, /api/events, /api/tak) and every dashboard page and HTMX partial. /api/ping stays open as the liveness probe, and the backup surfaces keep their own BACKUP_ADMIN_GROUP gate so a backup operator need not be a full admin. Authorization is applied at include_router time for the JSON routers and per route on the dashboard router. FastAPI's built-in schema routes are disabled and /api/openapi.json and /api/docs are re-served behind the same gate, since the OpenAPI document is the full route inventory (#82).

Why: Caddy's forward_auth only authenticates — any valid LDAP bind passes, so every TAK user reached every monitor route. Gating writes alone was not enough: /ui/partials/user-list calls the identity client directly and returned the full user roster to any authenticated caller, and health telemetry (container inventory, cert expiry, config drift) plus /api/events (the audit trail) are operator data, not user data. Nothing on the monitor serves non-admin users — enrollment reaches TAK users as a QR code or tak:// link, generated by an admin — so a blanket gate costs no legitimate access.

Consequences: Callers that reach the monitor without Caddy in front must supply Remote-User/Remote-Groups and, since DD-059, the proxy secret. The test stack publishes the monitor port directly, so tests-integration/conftest.py sends all three on every request; production does not publish the port.

Limits of this gate (added after review). It authorized external callers but could not authenticate the header's origin, so any container on the Docker network could claim to be an admin. DD-059 closes that. Upgrading also has a membership prerequisite: accounts other than webadmin must join the gate group before the upgrade, and ADMIN_GROUP must be bootstrapped when renamed (see DD-048).

Alternatives considered: Gate only the mutating routes — rejected, it leaves the roster and audit trail readable by any TAK user. Gate the dashboard router as a whole — rejected, it would force ADMIN_GROUP onto the backup pages, so an operator who points BACKUP_ADMIN_GROUP at a separate group would find their backup admin locked out of the pages that group exists to grant.


DD-046: TAK federation is effectively disabled

Decision: FastTAK does not support TAK Server federation. It is doubly off: CoreConfig.xml does not set enableFederation="true", and docker-compose.yml does not publish the federation ports (9000 v1 / 9001 v2). Toggling federation on in the TAK admin UI does not make it work — the listener would start inside the container but remain unreachable from outside the Docker network.

Why: No current need, and federation is untested against this stack. Supporting it properly is more than a port mapping: it needs fed-truststore.jks management (exchanging CA certs with the federate), enrollment/rotation story, docs, and integration coverage. Until someone needs it, an explicit "not supported" beats a half-open door.

Revisit when: a deployment actually needs to federate. Tracked in #72.


DD-045: Traffic capture via an mitmproxy overlay presenting TAK's own certs

Decision: just up --capture layers docker-compose.capture.yml, adding an mitmdump reverse proxy (tak-mitm) in front of TAK Server on host ports 8443 (HTTPS/Marti) and 8089 (CoT stream). A one-shot init-capture container mints a mitm-proxy client cert via TAK's own makeCert.sh and materializes the cert bundles mitm needs. mitm presents TAK Server's own server cert downstream and the TAK-issued client cert upstream. Decrypted flows are written to ./captures/. It is a local dev tool for protocol documentation and test fixtures — no public-internet mode, no traffic rewriting.

Why: TAK Server's REST/CoT behaviors aren't canonically specified. Capturing a real client↔server exchange gives ground truth for API/client design and deterministic replay fixtures (mitmdump --server-replay).

Key choices:

  • Present TAK's own cert downstream rather than mitmproxy's CA, so any existing data package connects with no client-side trust change.
  • No data-package rewrite. The generated data package already embeds SERVER_ADDRESS:8089, and mitm binds those same host ports presenting TAK's cert, so clients hit the proxy transparently. SERVER_ADDRESS must be a hostname/IP the client can reach.
  • ports: !override drops tak-server's direct 8443/8089 host bindings when the overlay loads (8446 stays direct), freeing the ports for mitm. Same directive docker-compose.test.yml uses (Compose ≥2.24).
  • --ssl-insecure upstream. mitm dials the Docker service name tak-server, but the server cert's SAN is SERVER_ADDRESS; hostname verification would otherwise fail. Upstream is a container we control on the private network, so skipping verification is safe.
  • reverse:tls:// (not tcp://) for 8089. tcp:// dials TAK in plaintext, breaking the upstream mTLS handshake; tls:// does TLS both directions.

Limitation: mitm doesn't request the client's cert, so it authenticates upstream as the single mitm-proxy user. Captured flows are protocol-accurate but identity-flattened, and — because TAK routes situational awareness per user — multiple clients connected through the proxy can't see each other (DistributedSubscriptionManager: sendLatestReachableSA : User lookup failed for mitm-proxy). Capture is a single-client tool; multi-client operation must run without the overlay. This makes capture strictly a dev/diagnostic tool, not an operational path.


DD-044: DroneSense MediaMTX proxy paths are reconciled, not add-once

Decision: The DroneSense flow reconciles MediaMTX ds/* proxy paths against the live MediaMTX config every 5 s (add / replace-on-change / delete-orphan), on a timer decoupled from the per-mission poll fan-out.

Why: DroneSense relay endpoints are ephemeral and rotate per streaming session (new host/port/credentials/TLS cert). The original paths/add-once registration froze each path to its first endpoint; add rejects updates (path already exists), so rotated sensors silently stopped ingesting — recording and live playback failed intermittently. Reconciling against MediaMTX's actual state also self-heals after a MediaMTX restart and reaps orphaned paths.

Alternatives considered: Tracking registered paths only in Node-RED flow context (no GET) — rejected: desyncs on MediaMTX restart and leaves orphans. Driving reconcile off the Build CoT output — rejected: the per-mission fan-out means a single invocation sees only one mission and would delete other missions' paths.


DD-043: Remove TAK Portal

Date: 2026-05-28 Status: Decided

Decision: TAK Portal is removed from the stack. User, group, and certificate-enrollment management is handled by the FastTAK Monitor dashboard (Users page: create users, manage tak_ groups, per-user QR / tak:// enrollment with a TTL'd token). In direct mode the bare https://<SERVER_ADDRESS> redirects to the monitor (:8180); in subdomain mode the portal. subdomain is gone (no TAKPORTAL_SUBDOMAIN). The TAK_PORTAL_VERSION pin, the tak-portal service/container, and the tak/portal/ data directory are removed.

Why: TAK Portal depended on Authentik, which was already removed in favor of LLDAP + ldap-proxy (see DD-031). With Authentik gone the portal was incompatible and unauthenticated, and the monitor dashboard already provides equivalent user/group/cert management and QR enrollment. Removing the portal eliminates a redundant Node.js service and a dead dependency.


DD-042: Disable x509 EKU requirement for group cache

Status: Accepted

Decision: Set x509useGroupCacheRequiresExtKeyUsage="false" on the <auth> element in CoreConfig.xml so TAK Server's per-user channel filter (active group cache) works for every authenticated cert regardless of Extended Key Usage contents.

Why: TAK Server 5.6 has an undocumented gate in com.bbn.marti.groups.X509Authenticator.auth(). The cache-based group filter is consulted only when a cert's EKU contains the OID 1.2.840.113549.1.9.7 (PKCS#9 ChallengePassword) — or when this attribute is false. Default is true. The OID is injected onto enrolled certs by /Marti/api/tls/signClient/v2 only when the request includes a non-null ?version= query parameter; the CSR's requested extensions are not read for this. Hard certs from makeCert.sh and any auto-enrollment that omits ?version= produce certs without the OID, which silently disables channel toggles: PUT to /Marti/api/groups/active updates the cache, the broker re-auth runs (via SubscriptionApi.doSetActiveGroups → GroupManager.authenticateCoreUsers), but the file-groups path executes and overwrites Group.neighbors with the user's full static group set — undoing its own intended effect. Net result: cache writes look successful but the broker keeps forwarding filtered events.

Setting this attribute false makes the cache path the default for all x509-authenticated users, which is the desired behavior for any deployment that wants channel toggles to work.

Why this is safe: x509useGroupCacheDefaultActive="true" is already set on the same <auth> element. When a user has no cache rows yet (the case for any never-toggled identity, including service accounts like svc_adsb, svc_nr2, svc_wx), assignGroupsCheckCache defaults to "all groups active." Behavior change is opt-in via toggle; nothing breaks for users who have never called setActiveGroups.

Alternatives considered:

  • Per-client retrofit (each TAK client appends ?version=… on enrollment): fixes the symptom only for newly-enrolled certs, requires a code change in every TAK client implementation, and does nothing for hard-cert users. Adopted in parallel by the Wistful Weasel client (separate brief), but insufficient on its own.
  • Patch the TAK Server WAR to always inject the OID at sign time: requires rebuilding the upstream takserver.war for every TAK Server release we adopt. High maintenance burden for a config-equivalent outcome.

Known edge case: tak.server.cache.ActiveGroupCacheHelper.assignGroupsCheckCache contains a special case that force-sets active=true when a user's cached groups list has exactly two entries with the same name (a single channel as IN+OUT pair). Users with exactly one channel cannot toggle that channel off. Doesn't affect typical multi-channel deployments. Not patched here.

Reference: verified in deployed takserver-war-5.6-RELEASE-6.jar — com.bbn.marti.groups.X509Authenticator, tak.server.cache.ActiveGroupCacheHelper, com.bbn.marti.sync.api.SubscriptionApi, com.bbn.tak.tls.CertManager. Schema at tak/CoreConfig.xsd:279.


DD-041: Stale-alert cleanup in the NOAA Weather flow

Status: Accepted

Decision: The NOAA Weather Node-RED flow emits explicit t-x-d-d delete CoTs (with <__forcedelete/>) when an alert drops out of NOAA's active-alerts feed. UIDs we've previously sent are tracked in Node-RED flow context, in-memory only — we deliberately do not persist the set across Node-RED restarts.

Why explicit deletes: Observed behavior is that polygon drawings (u-d-f) stick on the ATAK map indefinitely past their stale time. Web research confirms this is by design across the TAK client family: ATAK's deleteStaleAfterMillis auto-purge applies to markers (a-*) only, not to drawings (u-d-*). WinTAK appears to behave the same. The CoT stale attribute is therefore not a reliable cleanup mechanism for shape-type events. The canonical removal pattern, documented in Toyon's LearnATAK plugin examples, is a t-x-d-d event that includes both <link uid="..." relation="p-p" type="..."/> and <__forcedelete/> in <detail>. We adopt that pattern.

Why no persistence: Persistent flow context would require enabling contextStorage in Node-RED's settings.js. That file currently lives only inside the fasttak_nodered-data named volume — the FastTAK repo does not own it. Adding persistence would mean either (a) bind-mounting a repo-managed settings.js (and now keeping it in sync with upstream Node-RED defaults), or (b) sed-patching the volume copy from start.sh. Neither is conceptually heavy, but both expand FastTAK's surface area for a narrow benefit:

The orphan window from in-memory tracking is bounded in practice. NWS Severe/Extreme alerts typically have validity windows of 30–90 minutes, and the orphan case requires three things to align: (1) Node-RED restarts, (2) a specific alert is canceled or expires during that downtime so it's missing from the next poll, and (3) the alert's canceled state was not already covered by a delete we sent before the restart. The damage is a small subset of alerts hanging on connected clients past their actual end time, until clients are manually cleared or restarted. That is a worse experience than a clean cleanup, but it is not a system-integrity problem and it self-bounds rather than accumulating without limit.

What we don't do, and why:

  • Per-feature stale from properties.expires: Would tighten the NOAA-emitted stale times to the actual expiry NOAA reports, which is more accurate than the flow's hard-coded 1-hour stale. Deferred — the client-side stale-on-drawings behavior makes this a backstop only, not a primary mechanism, and implementing it requires changing the shared geojson-to-cot subflow (new optional STALE_PROP env). Not worth the subflow change today; revisit if we find another flow that needs it.
  • Geographic / event-type filtering of the NOAA URL: Orthogonal to cleanup — addresses volume, not staleness. Operators can change the URL in the NOAA feed URL change node when scoping their feed.
  • _Switch from u-d-f polygons to a-n-X-i-m-_ point markers:* Orthogonal. Point markers do auto-purge on stale, so a future flow variant that emits points instead of (or alongside) shapes would reduce dependence on explicit deletes — but that is a different CoT product, not a fix to the polygon flow.

Operational caveat: The first poll after a Node-RED deploy or restart starts with an empty noaaSeenUids set. If alerts were canceled during the downtime and clients still have their polygons displayed, the flow will not emit deletes for them — clients clear those manually or by restarting. This is the documented orphan path and is consistent with the in-memory-only choice above.


DD-040: Persistent event store for audit + health activity

Status: Accepted

Decision: A fastak_events table on a new fastak database in app-db serves as a unified store for (a) audit events from mutating API calls and (b) health activity events previously held in-memory in the alert engine. Two FastAPI middlewares (auth-context, then audit) wrap every mutating route; one row is written per successful 2xx mutation. The alert engine's record_event is rewired from a 500-entry in-memory list to the same table. Reads via GET /api/events (filtered JSON) and GET /api/events.csv (export).

Schema includes a nullable agency_id UUID column from day 1, even though agency assignment is not implemented until issue #21. Adding the column on what will become the largest table in the system is free now; later it would be a real migration. NULL is meaningful — "this row was written before agencies existed" — and #21's agency filter will treat NULL as visible to non-superadmins so legacy events don't disappear.

Audit middleware order is significant. Starlette executes middleware in reverse of registration: the last-registered runs first. The auth-context middleware (reads Remote-User/Remote-Groups from Caddy's forwarded headers) MUST run before the audit middleware so the latter can read request.state.username. Therefore in main.py: register AuditMiddleware first, AuthContextMiddleware second.

Why same DB instance, separate database: The cot DB is owned by TAK Server — adding our schema there couples us to TAK Server's data lifecycle. The app-db Postgres already serves LLDAP and Node-RED; adding a third database alongside them is operationally cheap and keeps audit data under our control without standing up new infrastructure.

Caddyfile change: init-config/start.sh now adds copy_headers Remote-User Remote-Groups to the monitor route's forward_auth block in both direct and subdomain deploy modes. Previously the monitor saw no actor identity; every action would have attributed to "unknown". Node-RED's forward_auth block intentionally remains without copy_headers — Node-RED is not an audit consumer.

Best-effort writes: audit.record_event catches all exceptions and logs via log.exception(...) rather than re-raising. Rationale: failing the user-facing request because the audit DB hiccupped is worse than a missing row. Misconfiguration (ValueError from _build_dsn) surfaces loudly at startup via init_schema()'s logged failure; the broad runtime catch keeps every subsequent mutation safe from the same root cause.

Body sanitisation: The audit middleware redacts dictionary keys named password, token, secret, p12, or private_key (recursively, including nested dicts and lists) before persisting. Non-JSON request bodies are recorded as the literal string "<non-json or binary>".

Why mutations only, not reads: Audit volume on reads (every dashboard refresh, every poll) drowns the mutation events that actually matter for forensics. Reads can be added per-route later if a specific endpoint needs it.

since/until are typed as datetime, not str: FastAPI parses and validates ISO-8601 at the HTTP boundary, so malformed inputs return 422 instead of reaching Postgres as a 500. Same typing applied to the JSON endpoint and the CSV export.

Out of scope today (tracked as future work): read auditing, container log aggregation (Loki), retention policy (operators may manually DELETE FROM fastak_events WHERE timestamp < … until the policy lands), agency-scoped query filter (added once #21 lands), per-route action enrichment (today action is METHOD /path; a future route → action map is straightforward to add).

Follow-ups not blocking ship: every audit event opens a fresh psycopg connection on app-db. Realistic worst case is roughly 50 writes/minute from a stack of ~10 services flapping on 30s state cycles plus normal API mutations — comfortably under the max_connections=100 budget that also serves LLDAP and Node-RED. We deliberately ship without psycopg_pool.ConnectionPool and revisit only when pg_stat_activity or actual connection refused errors show pressure. init_schema() is idempotent but races on first-time bootstrap if the monitor ever runs multi-replica; replace with an advisory lock at that point.


DD-039: TAK Server proxy endpoints in FastTAK API

Status: Accepted

Decision: FastTAK exposes a thin HTTP proxy over selected TAK Server /Marti/api/* endpoints (groups, contacts, clients, missions). Consumers do not need the mTLS client certificate — the FastTAK monitor uses svc_fasttakapi.p12 internally and re-emits responses as plain HTTP behind Caddy's standard auth flow.

Why: TAK Server's Marti REST API requires client-cert authentication, which is impractical for browser-based dashboards, CLIs, and integrations. Centralising the cert in the FastTAK API removes the credential-distribution problem and lets the dashboard consume the same data with normal LLDAP-backed auth. Mutating endpoints (mission CRUD, plugin endpoints) are intentionally out of scope until the audit middleware in #13 lands.

LKP source: The connected-clients dashboard joins Marti contact/client lists with the most recent row per UID from TAK Server's cot_router table. We rely on TAK Server's persistence — FastTAK does not log CoT events itself.

Coupling: This deepens our existing read coupling to TAK Server's cot_router schema (already coupled for autovacuum tuning). Acceptable for a Docker-Compose-bundled deployment; a future non-bundled deployment story would need a different LKP source. The schema is TAK-internal and may change between TAK Server versions — bumping the TAK Server image requires re-verifying that uid, point_hae, servertime, cot_type, detail, and the event_pt PostGIS Point column still have the expected names and types. Lat/lon are extracted via ST_Y(event_pt) / ST_X(event_pt); TAK Server does not store them as plain columns. _parse_detail extracts <contact callsign="…" endpoint="…"> and <__group name="…" role="…"> from the detail XML column. The endpoint attribute is load-bearing: it gates the "real TAK user" classification.

Required index: TAK Server 5.x ships with uid_servertime_idx on (uid, servertime), which a B-tree index scans in either direction so the LKP query's ORDER BY uid, servertime DESC is served efficiently without a separate FastTAK-installed index. If a future TAK Server version drops or renames this index, operators MUST install an equivalent manually; the verification step in the plan flags absence.

Default time bound: /api/tak/contacts/recent?max_age=<seconds> defaults to 86400 (24h) when omitted. The UID list is sourced from cot_router (the durable source of truth), which retains entries for approximately 30 days. Omitting max_age gives a recent snapshot equivalent to the dashboard's Recently Seen card; larger windows enable historical lookups if needed.

Endpoint provenance: The connected-clients endpoint wraps /Marti/api/subscriptions/all (NOT /Marti/api/Subscription/GetAllRepeaters, which was removed in TAK Server 5.x). The FastTAK wrapper normalises TAK's clientUid field to uid so callers see a uniform identifier. The cot database is encoded as SQL_ASCII, so the LKP query layer decodes bytes → str for uid and cot_type; this is documented in monitor/app/api/tak/positions.py.

Real-TAK-user gate: /api/tak/contacts/recent filters rows to "real TAK clients" by default: the row must either be in the live /Marti/api/contacts/all roster (which TAK Server populates using the same gate server-side) or carry a <contact endpoint="…"> attribute in its cot_router.detail XML. This mirrors ATAK's own ContactListDetailHandler.java rule and excludes passive feeds (ADS-B, sensor tracks, drones without TAK plugins) that emit callsigns but no endpoint. Operators can globally disable the gate via LKP_REQUIRE_TAK_CLIENT=false, or opt back in to feeds per-request with ?include_feeds=true. The endpoint value for TAK-Server-proxied clients is typically *:-1:stcp; local-network clients use <ip>:<port>:tcp.


DD-038: Expose TAK Server 8446 Directly in Base Compose

Date: 2026-04-19 Status: Decided

Decision: tak-server binds port 8446 on the host in the base docker-compose.yml, regardless of DEPLOY_MODE. The Caddy port binding for 8446 is removed from docker-compose.direct.yml (no longer needed and would collide). The Caddy subdomain route takserver.${SERVER_ADDRESS} is preserved as a convenience for browser access to the admin UI with Caddy's Let's Encrypt cert, but enrollment flows hit SERVER_ADDRESS:8446 directly.

Why: Before this change, subdomain mode didn't expose 8446 on the host at all — TAK Server listened on it internally and Caddy optionally proxied via the takserver. subdomain. But the portal generates enrollment URLs using TAK_ENROLLMENT_PORT=8446 directly (https://SERVER_ADDRESS:8446/Marti/api/...). Clients scanning enrollment QRs tried to reach :8446 and got connection refused. Operators had to add a local compose override binding 8446 before enrollment would work.

In direct mode, Caddy previously handled 8446 with its internal-CA cert. That cert isn't in the TAK CA chain clients trust via the enrollment data package, so TLS validation could fail on strict clients. Routing 8446 directly to tak-server fixes both problems — clients consistently see the TAK CA cert regardless of mode.

Consequences:

  • Subdomain mode: operators no longer need a local override for enrollment to work. Existing overrides remain harmless but unnecessary.
  • Direct mode: 8446 now hits tak-server directly instead of Caddy. The cert presented switches from Caddy's internal CA to the TAK CA — matches what enrollment clients expect. No operator action required; Docker port binding moves automatically on docker compose up.
  • The docker-compose.test.yml override (port offset 18446:8446 on tak-server) was already consistent with this model — confirms the test harness and the base compose were misaligned before this change, not after.

Scope:

  • 8443 and 8089 remain bound on tak-server as before (HTTPS + CoT streaming, both mTLS).
  • 8446 is mTLS-protected for admin access (cert-based); unauthenticated enrollment (/Marti/api/tls/signClient/v2) requires Basic auth with a valid enrollment token. Public exposure is safe because both auth paths are cryptographically strong.

Alternatives considered:

  • Generate subdomain-style enrollment URLs (https://takserver.SERVER_ADDRESS) from the portal instead of :8446 URLs in subdomain mode — rejected as a larger change (touches monitor's Python URL-generation code) and inconsistent with how TAK ecosystem tooling and documentation refer to the enrollment port.
  • Document the local-override workaround permanently — rejected, puts the burden on every operator forever and foot-guns the deployment walkthrough.

Related: DD-029 (deploy modes), DD-033 (universal security defaults applied regardless of mode).


DD-037: Rate Limit Counts Only Failed Auth Attempts

Date: 2026-04-19 Status: Decided (refines DD-036)

Decision: The /auth/verify rate limiter counts only failed authentication attempts (401 responses). Successful auths do not consume the budget. The middleware's job is reduced to checking lockout state; the auth handler itself calls RateLimiter.RecordFailure(ip) on each 401 return.

Why: Caddy's forward_auth hits /auth/verify on every protected-route request, not just logins. Counting every attempt — the original DD-036 implementation — meant a legitimate user clicking around the portal could trivially exceed the budget and lock themselves out. The original defaults of 10 attempts per 5 minutes were correct for a brute-force threat model but catastrophically tight for the actual per-request-forward-auth traffic pattern.

Failures-only counting restores the intended semantics: brute-force attempts (which fail) are penalised; legitimate use (which succeeds) is not.

How it works:

  • Middleware → calls CheckLockout(ip) only. Does not record anything. Returns 429 if the IP is currently locked out.
  • HandleVerify → on every 401 return path (missing header, bad credentials, invalid bind), calls RecordFailure(ip).
  • RecordFailure → tracks per-IP failure timestamps in a sliding window; once the window-bounded failure count reaches maxAttempts, sets lockoutUntil.
  • CheckLockout → returns blocked if now < lockoutUntil, else allowed.

What does NOT count:

  • 200 responses (successful auth)
  • 400 responses (malformed Basic auth header — user error, not credential guessing)
  • 500 responses (internal errors on the ldap-proxy side)
  • Any request that reaches the handler via a non-/auth/verify route

Lockout is not extended by continued retries: once an attacker triggers lockout, their further attempts don't reset or extend the timer. The initial lockout duration is the penalty; they must stop or wait it out.

Alternatives considered:

  • Count all attempts but raise the threshold (e.g., 1000/min) — rejected, makes brute-force protection largely theatrical. The right fix is semantic, not numeric.
  • Session-level caching at Caddy so forward_auth doesn't hit ldap-proxy every request — rejected as scope creep; would need a second cookie/session layer on top of forward_auth.
  • Count based on response code via a wrapped ResponseWriter in middleware — rejected, tighter coupling and harder to reason about. Explicit RecordFailure call from the handler is clearer.

Related: DD-036 (the limiter this refines), DD-031 (ldap-proxy architecture).

DD-036: Rate Limit on LDAP /auth/verify

Date: 2026-04-18 Status: Decided

Decision: ldap-proxy rate-limits requests to /auth/verify: default 10 attempts per 5 minutes per source IP, 15-minute lockout on exceed. Keyed on X-Forwarded-For with fallback to RemoteAddr. Caddy's forward_auth blocks set no trusted_proxies, so Caddy replaces the header with the address it saw; with trusted_proxies private_ranges, a client on a LAN, a VPN or Docker Desktop could send a new address with every guess and never be locked out (#62). A deployment behind a load balancer sees the balancer's address for every client and needs its own trusted_proxies entry for that balancer. Returns HTTP 429 with Retry-After header on rejection. Rule applies unconditionally — no DEPLOY_MODE branching. Configurable via LDAP_RATE_LIMIT_WINDOW, LDAP_RATE_LIMIT_MAX_ATTEMPTS, LDAP_RATE_LIMIT_LOCKOUT env vars.

Why: Without a rate limit, an attacker with network reach to /auth/verify can brute-force passwords or enumerate usernames at line speed. Caddy's forward_auth makes this endpoint reachable from the public internet on every protected subdomain (portal, monitor, Node-RED). Defense in depth: LDAP auth must resist brute force regardless of whether the specific endpoint is "supposed to" be reachable.

Scope:

  • /auth/verify — rate-limited
  • /tokens/* — NOT rate-limited. The original reasoning was "internal-only (monitor service over the Docker network, no Caddy route)". DD-050 rejects that premise — "on the Docker network" is not a trust boundary, and the endpoint mints an LDAP bind credential for any username. It is now authenticated with a shared secret instead. Rate limiting remains unnecessary because an unauthenticated caller is rejected outright.
  • /healthz — NOT rate-limited; Docker health probes, not user traffic
  • LDAP bind port 3389 — NOT rate-limited at this layer; it's on the internal Docker network and TAK Server binds twice per enrollment by design (DD-031)

Implementation:

  • In-memory sliding-window limiter in Go, ~80 lines + tests
  • Process-local state (if ldap-proxy restarts, limiter history resets — acceptable tradeoff; token state is already ephemeral on tmpfs per DD-031)
  • Integration test (tests-integration/test_rate_limit.py) verifies 429 behavior end-to-end; docker-compose.test.yml overrides the env to short window/lockout (10s/5s) so tests recover quickly between runs

Alternatives considered:

  • Caddy-level rate limit plugin — rejected, requires a custom Caddy image, breaking the "stock caddy:2-alpine" simplicity.
  • fail2ban sidecar — rejected, log-parsing is fragile and adds another container.
  • Disk-backed limiter — rejected, unnecessary complexity; in-memory survives ldap-proxy's natural lifecycle fine.

Related: DD-031 (ldap-proxy architecture), DD-033 (universal security defaults).


DD-035: app-db Uses Official postgres:18-alpine Instead of postgis/postgis

Date: 2026-04-18 Status: Decided (amended 2026-08-18 — see below)

Decision: The app-db service (shared PostgreSQL for LLDAP + Node-RED) uses the official postgres:18-alpine image instead of postgis/postgis:15-3.4-alpine. PostGIS is no longer installed on app-db. Flows that need spatial queries should connect to tak-database, which has PostGIS natively.

Why: postgis/postgis ships amd64 only. The community multi-arch fork (imresamu/postgis) marks its arm64 support as experimental. Running FastTAK on ARM hosts (Apple Silicon development, AWS Graviton, Oracle Ampere A1, Hetzner Ampere) either fails outright or runs under emulation. The official postgres image is first-party, multi-arch-stable, and maintained by the PostgreSQL Docker team — no supply-chain or architecture-compatibility caveats.

The tradeoff is the loss of PostGIS on app-db. Stock FastTAK does not depend on it — the CREATE EXTENSION postgis call in app-db/start.sh was a convenience for user-authored Node-RED flows that might want spatial queries. tak-database already has PostGIS and already holds the spatial data that matters in a TAK context, so routing spatial-needing flows there is both cheaper and more honest about where the data lives.

Alternatives considered:

  • Keep postgis/postgis:15-3.4-alpine — rejected, blocks ARM deployments. Oracle A1 / Graviton / Apple Silicon all become non-starters.
  • Switch to imresamu/postgis:15-3.4-alpine (community multi-arch) — rejected, the maintainer's README explicitly marks arm64 as experimental. Not appropriate for a production-oriented stack.
  • Build our own PostGIS image — rejected as overkill. The feature (PostGIS on app-db) isn't used by stock FastTAK, so maintaining our own image for a user-convenience feature is disproportionate effort.

Related: DD-033 (same principle — security/reliability defaults stay universal across deploy modes and host architectures).

Amended 2026-08-18: app-db moved from postgres:15-alpine to postgres:18-alpine alongside TAK 5.8's own move to PostgreSQL 18 (tak-database moves with it — see DD-051). docker-compose.yml now pins PGDATA=/var/lib/postgresql/data explicitly on app-db: the postgres:18 image otherwise defaults PGDATA to a version-specific path (/var/lib/postgresql/18/docker) rather than the mount point, and the 18 entrypoint hard-fails on start rather than silently using the wrong directory. Pinning PGDATA to the mount keeps it exactly equal to what's mounted, which is what tak-database/check-pgdata-persistent.sh's exact-match guard requires — that guard does not accept a mount at PGDATA's parent. A future PostgreSQL major may move the default PGDATA path again; re-check this pin when it does.


DD-034: Memory Caps on Every Container

Date: 2026-04-18 Status: Decided

Decision: Every service in docker-compose.yml has deploy.resources.limits.memory set to a value that bounds its normal footprint with safety margin. Rule applies unconditionally — no DEPLOY_MODE branching.

Why: Bounding memory caps the blast radius of any container failure: an OOM loop, a runaway process, or a DoS attack that tries to exhaust RAM on the host. Without caps, a single compromised or misbehaving container can starve the rest of the stack. The cost is low (a missized limit just causes OOM-kill of the offending container, which Docker's restart policy then handles) and the benefit is constant.

Starting values:

Service Limit
tak-server 4G
tak-database 2G
app-db 1G
tak-portal 512M
nodered 512M
monitor 512M
mediamtx 512M
lldap 256M
caddy 256M
ldap-proxy 128M
init-config 128M
init-identity 256M
init-ldap-ready 64M

These were chosen based on observed idle/startup footprint plus safety margin, and validated via the integration test suite (81/81 tests pass under these caps). Operators with larger workloads (more concurrent clients, higher CoT volume) should override via docker-compose.override.yml rather than edit the base file.

Alternatives considered:

  • CPU limits — deliberately deferred. Memory limits give most of the safety benefit without introducing latency surprises. CPU throttling under load can manifest as mysterious timeouts; better to add them only after observing CPU-starvation issues.
  • Swarm-only deploy.* syntax — not relevant; Compose v2 honors deploy.resources.limits.memory in non-Swarm mode since 2020.
  • No limits (status quo) — rejected, leaves the stack exposed to single-container memory exhaustion.

Related: DD-033 (universal security defaults, same framing).


DD-033: Admin Password Generated Per-Install; Default Value Always Rejected

Date: 2026-04-18 Status: Decided

Decision: setup.sh auto-generates a random 24-char TAK_WEBADMIN_PASSWORD on fresh install. start.sh (via scripts/check-env.sh) refuses to start if TAK_WEBADMIN_PASSWORD is set to the documented default FastTAK-Admin-1!. The rule applies unconditionally — in every DEPLOY_MODE. Empty is permitted and preserves the existing "skip webadmin user creation" semantics.

Why: Security defaults should be unconditional. Keying the gate on DEPLOY_MODE=subdomain would leak the wrong assumption that direct mode is never publicly reachable — which is false: direct mode on a public IP is still public. Reachability can't be inferred from inside the stack, so the gate shouldn't try. The honest rule is simpler: the default password is public knowledge; no deployment should run it.

Scope — empty password is permitted: Preserves the existing escape hatch from .env.example ("Leave empty to skip webadmin user creation"). Operators who don't want a webadmin user (e.g., cert-only deployments) can set it empty and init-identity skips the LLDAP user creation.

Alternatives considered:

  • Warning-only — rejected, warnings get ignored on the exact deployments that need hardening most.
  • Gate by DEPLOY_MODE=subdomain only — rejected, conflates routing with security. A direct + public deployment would bypass the check.
  • Introduce a separate ENVIRONMENT=prod|dev flag — rejected, adds config surface for a use case (legitimate default-password use) that doesn't exist once setup.sh auto-generates.
  • Gate in init-identity instead of preflight — rejected, error would surface in Docker logs after the operator has been waiting. Preflight fails loud and fast.

Related: DD-029 keeps DEPLOY_MODE as a routing/cert-only knob; this decision preserves that boundary.


DD-032: User Type Attribute (fastak_user_type)

Date: 2026-04-13 Status: Decided

Decision: Every account has a fastak_user_type custom attribute in LLDAP, set at creation time. Three values: user (people), svc_data (data-mode service accounts), svc_admin (admin-mode service accounts). The API enforces group assignment rules based on type: user and svc_data require at least one group, svc_admin forbids groups.

Why: The Pydantic model validator on the service account creation endpoint enforced mode/group rules at creation time, but the PATCH endpoint and the general-purpose PUT /api/users/{id}/groups endpoint had no way to determine account type after creation. Without a persisted type, the rules could be circumvented by updating groups post-creation — which is how the bug was discovered (assigning groups to an admin-mode service account via the Monitor UI). Regular users also had no group requirement, allowing creation of users with no channel assignments (useless for TAK operations).

Alternatives considered:

  • fastak_svc_mode (admin/data) on service accounts only — doesn't cover regular users, requires combining prefix checks with mode checks.
  • Infer type from current state (no groups = admin) — circular; using the rule's expected outcome to enforce the rule.

Bootstrap: svc_nodered removed from default service accounts. Users create data-mode service accounts via the API when needed, which enforces the groups requirement at creation time. svc_fasttakapi is tagged svc_admin during bootstrap.


DD-031: Replace Authentik with LLDAP + ldap-proxy

Date: 2026-04-09 Status: Decided

Decision: Replace Authentik (6 containers, ~750MB, 5+ min startup) with LLDAP + a custom Go LDAP bind proxy (2 containers, ~60MB, ~15s startup).

Why: Authentik was the single largest source of operational issues — connection storms exhausting PostgreSQL (requiring PgBouncer), slow startup blocking the entire stack, heavy memory usage, and complex bootstrap logic (~850 lines). See issue #31 for the full history.

Architecture:

  • LLDAP — lightweight Rust LDAP server (stock Docker image), backed by PostgreSQL. Provides the user directory and a GraphQL management API.
  • ldap-proxy — custom Go binary sitting in front of LLDAP on port 3389. Intercepts LDAP bind requests to check enrollment tokens (SQLite on tmpfs) before falling through to LLDAP for real passwords. Also exposes /auth/verify for Caddy forward auth and a REST API for token management.
  • init-identity — one-shot Python container bootstrapping LLDAP via GraphQL (~250 lines replacing ~850).

Notable constraints discovered during implementation:

  • LLDAP requires email on user creation (v0.5+ security fix). Placeholder {username}@dummy.example.com used since TAK Server doesn't query email attributes.
  • LLDAP uses uid=/ou=people DN format (not cn=/ou=users like Authentik's LDAP outpost).
  • LLDAP uses the OPAQUE protocol for passwords — requires the lldap_set_password binary (copied from LLDAP image via multi-stage Docker build).
  • TAK Server's LdapAuthenticator binds as the user twice during enrollment (auth + group assignment), so enrollment tokens must be non-one-time. TTL (default 15 min) is the security boundary.
  • Enrollment token plaintext is stored alongside the hash to support idempotent get-or-create. The SQLite database lives on tmpfs (RAM only, dies with the container) to mitigate the exposure.
  • LLDAP custom attribute schemas (fastak_expires, fastak_certs_revoked, is_active) must be registered before use — handled by init-identity bootstrap.

DD-030: PgBouncer for Authentik connection pooling

Status: Superseded — Authentik removed in favor of LLDAP (see DD-031).

Decision: Add PgBouncer between Authentik's server and app-db in transaction-pool mode. The worker bypasses PgBouncer and connects directly to app-db.

Context: Authentik's Dramatiq worker creates hundreds of psycopg3 connection pools, exhausting PostgreSQL's max_connections. The Postgres tuning (idle_session_timeout, max_connections=300) mitigates but doesn't prevent the issue under load or with accumulated state.

Alternatives considered:

  1. Postgres tuning only (current state) — fragile, doesn't cap concurrent connections
  2. PgBouncer for all Authentik services — failed: worker migrations use advisory locks incompatible with transaction pooling
  3. Replace Authentik entirely — too disruptive for a connection management issue

Consequences:

  • One additional container (pgbouncer), ~10MB RAM
  • authentik-server connects to pgbouncer instead of app-db
  • authentik-worker environment overrides the shared anchor to connect directly to app-db
  • Server-side cursors disabled for Authentik server (required for transaction pooling)
  • Postgres tuning remains as defense-in-depth for the worker's direct connections

DD-029: Deploy Modes (direct / subdomain)

Date: 2026-04-04 Status: Decided

Decision: Replace the single FQDN-based deployment with two modes controlled by DEPLOY_MODE in .env.

Context: FastTAK required a domain name and public DNS, blocking field deployments, air-gapped networks, and Tailscale-style hostnames. The dev stack bypassed Caddy entirely.

Alternatives considered:

  1. Path-based routing (rejected — Authentik doesn't handle path prefixes well)
  2. Keep dev override separate (rejected — maintaining two routing models adds complexity)

Consequences:

  • docker-compose.dev.yml is deleted — direct mode replaces it
  • Caddyfile is generated by init-config, no longer a static file
  • Caddy is the single entry point for all HTTP/HTTPS in both modes
  • Self-signed TLS in direct mode (Caddy internal CA, separate from TAK CA)

DD-028: Allow Cert Download on CRL Check Failure

Date: 2026-04-01 Status: Decided

Decision: If the CRL revocation check fails (file unreadable, openssl error, timeout), the cert download is allowed rather than blocked.

Why: A revoked cert is useless — TAK Server rejects it on the next TLS handshake. Blocking downloads on transient CRL read failures would prevent operators from downloading valid certs when the CRL file is temporarily unavailable (e.g., during rotation or filesystem issues). The blast radius of allowing a revoked cert download is near zero since the cert can't be used.

Alternative considered: Fail closed (block download if CRL can't be checked). This is more conservative but creates a worse failure mode — a transient filesystem issue blocks all cert downloads, including valid ones.


DD-027: Single Cert Per Service Account

Date: 2026-03-30 Status: Decided

Decision: Service accounts get one certificate, named after the username (svc_{name}.p12). Multiple named certs per service account are not supported.

Why: YAGNI. The use case for multiple certs on a service account (e.g., a sensor network with 5 nodes under one identity) is real but hasn't materialized. Operators who need multiple nodes can create multiple service accounts (svc_sensor_node1, svc_sensor_node2), each with its own cert and independently manageable lifecycle. Users (people) support multiple named certs because a person genuinely uses multiple devices under one identity. Machines are simpler — one identity, one cert, one device.

Revisit when: An operator has a concrete need for multiple certs on a single service account and creating separate accounts is insufficient.


DD-026: Filter ROLE_ADMIN from Groups API

Date: 2026-03-30 Status: Decided

Decision: The GET /api/groups endpoint (and AuthentikClient.list_groups()) filters out tak_ROLE_ADMIN. It does not appear in group lists, dropdowns, or anywhere operators can assign it.

Why: ROLE_ADMIN is a TAK Server authorization role, not an operational channel. Admin access for service accounts is granted via certmod -A on the certificate, not via LDAP group membership. The tak_ROLE_ADMIN group in Authentik exists only for the webadmin user (who authenticates via LDAP password on port 8446 where group membership determines admin status). Exposing it in the groups API risks operators assigning it to users/service accounts, which creates a phantom channel visible in ATAK clients without actually granting admin API access.

Future: If webadmin can be migrated to cert-based auth with certmod -A, the tak_ROLE_ADMIN group can be eliminated entirely.


DD-025: Keep Default .p12 Password (atakatak)

Date: 2026-03-30 Status: Decided

Decision: All generated .p12 certificate bundles use the default password atakatak. This is not configurable per-cert.

Why: atakatak is a universal TAK ecosystem convention. Every TAK tool, tutorial, deployment guide, and client application assumes this password. The .p12 password protects the file at rest (prevents accidental import), not the connection — the real security is that the cert must be signed by the deployment's CA. Introducing unique passwords per cert would add operational burden (tracking and communicating passwords alongside certs) without meaningful security benefit, since the cert file itself is the credential.

Alternatives considered:

  • Per-cert auto-generated passwords shown once at download — adds friction for operators familiar with TAK conventions, marginal security gain
  • Configurable default password — breaks compatibility with existing TAK tooling that hardcodes atakatak

DD-024: Test Isolation via Port Offset Override

Date: 2026-03-29 Status: Decided

Decision: A docker-compose.test.yml override remaps all host-bound ports with a fixed +10000 offset so integration tests can run alongside a development stack. Uses Compose !reset directive (v2.24+) to replace base port arrays rather than append. The test override includes all ports (including dev convenience ports and 8443) so the test suite can exercise endpoints from the host.

Why: Integration tests previously collided with running stacks on every host-bound port. Dynamic port discovery was considered but rejected — fixed offset is predictable, easy to debug, and the only collision scenario (two simultaneous test stacks) is unlikely.


DD-023: Production-First Compose with Dev Override

Date: 2026-03-29 Status: Decided

Decision: The base docker-compose.yml is the production configuration — no direct-access convenience ports (TAK Portal 3000, Node-RED 1880, Monitor 8180), no host exposure of the TAK Server admin API (8443). A docker-compose.dev.yml override adds back convenience ports for local development. Justfile recipes (just up for production, just dev-up for development) hide the COMPOSE_FILE plumbing.

Why: The secure configuration should be the default. Developers opt in to convenience ports, not opt out of risk. A production deployment that forgets to apply an override is secure by default. Removing 8443 from host exposure closes the credential leak from TAK Server's /Marti/api/security/config and /Marti/api/authentication/config endpoints — the FastTAK API accesses 8443 internally over the Docker network, so no external consumer needs it.


DD-022: Health Monitoring Architecture Refactor

Date: 2026-03-28 Status: Decided (amended 2026-09-15 — see below)

Decision: Separate health data collection, evaluation, caching, and alerting into distinct layers. Health modules return raw data only (no status). An evaluator applies configurable thresholds from monitor/config/thresholds.yml (overridable via FASTAK_MON_* env vars) to produce status. The scheduler polls health modules on configured intervals, runs results through the evaluator, and writes to an in-memory cache. GET /api/health returns the full cache. Individual health endpoints (/api/health/database, etc.) remain live for debugging. Four status levels: ok, note, warning, critical.

Why: The previous architecture had health modules computing status with hardcoded thresholds, the scheduler re-querying the same data for alerting, and the dashboard querying a third time. Responsibilities were tangled, thresholds were unchangeable without code edits, and there was no way to distinguish informational changes (update available) from operational concerns (disk full). The refactor gives each layer a single clear responsibility and makes the entire monitoring policy configurable.

Alert design: The evaluator decides severity and whether a state warrants a notification (should_alert, from alert_min_level per service). The scheduler calls the alert engine on every poll with that verdict, not only when alerting. The engine tracks state on every call, records every transition (and recovered on an elevated → ok transition), and sends email/SMS only on a transition to a non-ok state when should_alert is true and the alert cooldown has elapsed. The first observation of a healthy service is not an event. Alert cooldown is configurable globally via alert_cooldown in thresholds.yml.

Amended with #77: the previous shape (call the engine only when alerting) left the engine's state pinned at the last elevated level, which suppressed recoveries and repeat warnings. The remaining escalation-vs-cooldown limit — a still-elevated service that worsens mid-cooldown does not re-alert until the cooldown clears — is tracked as #131.


DD-021: Live Data Size Uses Heap-Only Measurement

Date: 2026-03-28 Status: Decided

Decision: The database health endpoint reports "live data" using pg_total_relation_size(relid) summed across all user tables. This includes table data, indexes, and TOAST — everything that belongs to the user's data. The gap between pg_database_size and this sum is PostgreSQL system overhead (system catalogs, WAL, internal structures) — roughly 10-15 MB regardless of database size.

Why: When the gap between total size and live data is large, it indicates table bloat from dead tuples that VACUUM FULL would reclaim. On a 30 GB database with 15 GB live data, the user knows half is reclaimable bloat. On a 30 GB database with 29.8 GB live data, VACUUM FULL would be pointless. This gives users the information to make that call without FastTAK imposing a threshold.

Alternatives considered:

  • pg_relation_size(relid, 'main') (heap only) — excludes indexes and TOAST from "live data," making the gap include both reclaimable bloat and legitimate index space. Less accurate representation of what the user's data actually occupies.
  • pgstattuple extension — gives exact dead tuple byte counts but requires installing an extension that TAK Server doesn't ship.
  • Showing a computed "reclaimable" field — removed because the approximation includes system overhead, and labeling it "reclaimable" implies precision we don't have. Showing two raw values and letting users interpret the gap is more honest.

DD-020: Global Autovacuum Tuning via postgresql.conf

Date: 2026-03-27 Status: Decided (amended 2026-08-18 — see below)

Decision: PostgreSQL autovacuum is tuned globally, not via per-table ALTER TABLE SET statements. Three configurable .env variables with FastTAK defaults: PG_AUTOVACUUM_SCALE_FACTOR (default: 0.05), PG_AUTOVACUUM_COST_LIMIT (default: 1000), PG_MAINTENANCE_WORK_MEM (default: 256MB). These defaults are applied automatically — users who want stock PostgreSQL behavior must explicitly set them to 0.2, 200, 64MB. Autovacuum health status uses a minimum dead tuple threshold (AUTOVACUUM_MIN_DEAD_TUPLES, default 1000) — tables below this count are shown in the UI but don't affect the overall status. This prevents tiny config tables with a handful of dead rows from triggering false alerts on a fresh stack.

Why: Per-table tuning (ALTER TABLE cot_router SET (autovacuum_vacuum_scale_factor = 0.05)) is more precise but couples us to TAK Server's schema — table names could change across versions. The CoT tables are the dominant workload, so global settings effectively tune for them.

Amended 2026-08-18: the original mechanism — pg_ctl -o startup flags injected by tak-database/start.sh — no longer exists. TAK 5.8 dropped those flags entirely; the sed that used to splice FastTAK's flags in now matches nothing and exits 0, so the tuning would silently disappear with no error. tak-database/start.sh now appends FastTAK's block to db-utils/postgresql.conf (what the vendor's takserver-setup-db.sh copies into PGDATA on the first boot of a fresh database) and to the live $PGDATA/postgresql.conf (what the server reads on every boot after the first — the vendor's copy step prompts Type 'erase' on a database that already exists, which fails closed under this container's no-TTY entrypoint and never runs again after boot one). Because a config-file write is not a startup flag, "it took effect" is no longer implied by the process starting — start.sh now polls SHOW <setting> against the running server after the vendor entrypoint starts, and fails the container if the values it wanted are not what the server reports back.

Related: DD-053 — the same append-and-verify mechanism also disables WAL archiving, and the same first-boot-vs-live-config problem applies there too.


DD-019: Monitor Becomes the API Service

Date: 2026-03-27 Status: Decided (supersedes DD-005)

Decision: The monitor service expands from a read-only health dashboard to the FastTAK management API. It gains the Authentik API token for user management and communicates with TAK Server via mTLS on port 8443 (using the svc_fasttakapi cert) for cert operations. DD-005's read-only constraint is superseded.

Why: DD-012 established that the API is the single management layer. The monitor already has the FastAPI infrastructure, Docker client, scheduler, and config patterns. Splitting management into a separate service would duplicate this infrastructure for no benefit. The existing cert operations endpoints (/api/ops/certs/*) already broke DD-005's read-only constraint — this formalizes the expansion.

Portability: User management and cert operations use HTTP to Authentik and TAK Server, not docker exec. The Docker client remains optional — used only for container health monitoring, which degrades gracefully without it. This preserves DD-012's portability goal.

Security: The Authentik API token and TAK Server mTLS cert grant admin access to their respective services. These are scoped to the management API's purpose and protected by the reverse proxy authentication layer.


DD-018: User Cert Enrichment is Opt-In

Date: 2026-03-27 Status: Decided

Decision: GET /api/users returns Authentik user data only by default. TAK Server cert status is included only when the caller passes ?include=certs. Per-user detail (GET /api/users/{id}) always includes cert data.

Why: Cert status requires an mTLS HTTP call to TAK Server's certadmin API per user. For list endpoints this is an N+1 problem that scales poorly. Most list consumers (dashboards, dropdowns, search) don't need cert data. Making it opt-in keeps the default fast and lets consumers that need cert data request it explicitly.


DD-017: User TTL via Revocation Scheduler

Date: 2026-03-27 Status: Decided

Decision: Temporary users (e.g., external collaborators) are given an expiry timestamp stored as fastak_expires in Authentik's user attributes JSON. A FastTAK scheduled task (configurable interval via user_expiry_check_interval, default 60s) checks for expired users and enforces expiry by deactivating the Authentik account and revoking all TAK Server certs via the TAK Server certadmin REST API.

Why: TAK Server's enrollment cert validity (validityDays in CoreConfig.xml) is a global setting — no per-user control. Intercepting the enrollment flow to generate custom-TTL certs would require replicating TAK Server's undocumented data package format. Cert revocation is immediate (TAK Server rejects revoked certs on next connection), version-proof, and coordinates both Authentik deactivation and cert revocation as a single operation.

Alternatives considered:

  • Per-user cert validity during enrollment — not supported by TAK Server
  • Hijacking enrollment to generate certs with custom validity — fragile, requires reverse-engineering TAK Server's data package format
  • Authentik's built-in goauthentik.io/user/expires — deletes the user instead of deactivating, losing audit trail
  • Authentik Expression Policy — doesn't affect cert-based (mTLS) access, only password auth on port 8446

DD-016: Service Account Naming Convention

Date: 2026-03-25 Status: Decided

Decision: All service accounts (machine-to-machine credentials) use the prefix svc_ (e.g., svc_fasttakapi, svc_nodered). Human admin accounts use no prefix. The USERS_HIDDEN_PREFIXES in TAK Portal settings hides svc_ accounts from the user management UI.

Why: The original names (admin, nodered) are too generic and could collide with real usernames. A consistent prefix makes service accounts immediately identifiable, easy to filter in the UI, and easy to exclude from user-facing operations. svc_ was chosen over service_ (shorter), admin_ (confusing — implies human admin), and sa_ (less readable).

Applies to:

  • svc_fasttakapi — FastTAK API machine-to-machine cert (replaces admin)
  • svc_nodered — Node-RED service account (replaces nodered)
  • adm_ldapservice — kept as-is (Authentik LDAP convention, already hidden by adm_ prefix)

DD-015: Passwordless Users by Default

Date: 2026-03-25 Status: Decided

Decision: User creation does not set a password. Passwords for users requiring username/password authentication (e.g., for web-based clients like WebTAK or the admin UI) are set explicitly via POST /api/users/{id}/password. Device-only users never have an account password.

Why: The enrollment flow uses Authentik app passwords (15-minute TTL tokens), not the user's account password. A user with no password cannot authenticate via LDAP (verified by test — Authentik returns INVALID_CREDENTIALS). This is the most secure default: no credential exists to be compromised. Users who need browser access can have a password set explicitly.

Verified: Authentik's InbuiltBackend calls Django's check_password(), which returns False for users with no usable password. The TokenBackend (app passwords) works independently of the account password.


DD-014: Enrollment Token TTL is Configurable

Date: 2026-03-25 Status: Decided

Decision: The enrollment app password TTL defaults to 15 minutes but is configurable via environment variable (ENROLLMENT_TTL_MINUTES). Any value with different use cases should be a config option.

Why: 15 minutes is reasonable for in-person QR code scanning, but other use cases (emailing enrollment links, remote deployments, batch provisioning) may need longer windows. Making it configurable avoids hardcoding an assumption about deployment context.


DD-013: ROLE_ADMIN is Both a Permission Role and a Channel

Date: 2026-03-25 Status: Documented (not a decision — clarification of existing TAK Server behavior)

Context: TAK Server's LDAP config has adminGroup="ROLE_ADMIN". Membership in tak_ROLE_ADMIN grants admin API access. TAK Server does not distinguish between permission groups and communication channels — all groups are both. ROLE_ADMIN appears as a channel in TAK Aware alongside user-created channels like Field Team Alpha. This is TAK Server's design, not something FastTAK controls.


DD-012: API is the Management Layer, Dashboard is a Consumer

Date: 2026-03-25 Status: Decided

Decision: The FastTAK API is the single management interface for the stack. All management operations (user CRUD, plugin management, cert operations) go through the API. The dashboard and any future admin consoles are consumers of the API, not orchestrators.

Why: Decouples the management logic from any specific UI. Enables swapping dashboards, building CLI tools, or integrating with external systems without reimplementing orchestration. Makes the API portable — it can be used outside the Docker Compose stack by pointing it at any TAK Server + identity provider.

Implications:

  • All external service URLs are environment variables, never hardcoded Docker DNS names
  • Docker client is optional — API starts without it, container health endpoints degrade gracefully
  • See DD-019 for how the monitor service was formalized as the API service

DD-011: Docs Must Cover All Features

Date: 2026-03-24 Status: Decided

Decision: Documentation (mkdocs) must include user guides for all features. Users should be able to deploy and operate FastTAK entirely from the documentation.


DD-010: Release Path Filtering is Exclude-Only

Date: 2026-03-24 Status: Decided

Decision: Semantic release triggers on any feat:/fix: commit unless the commit only touches tests/, docs/, or .github/. New directories and services trigger releases by default.

Why: An include-list risks missing new services or folders. Exclude-only logic means everything triggers a release unless explicitly excluded, which is safer as the project grows.


DD-009: CI Runs Unit Tests Only

Date: 2026-03-24 Status: Decided (constraint-driven)

Decision: GitHub Actions CI runs unit tests (just test) and shellcheck only. Full integration tests (just test-integration) stay manual/local. A placeholder exists for a future integration job. Unit tests run on pre-commit; developers can bypass with --no-verify for WIP commits.

Why: TAK Server images are built from a proprietary tak.gov zip that isn't available in CI. No public TAK Server images exist.


DD-008: Tilde Pins at Patch Level for Python Dependencies

Date: 2026-03-24 Status: Decided

Decision: All Python dependencies use compatible release pins at patch level (~=X.Y.0), e.g., docker~=7.1.0, fastapi~=0.135.0.

Why: Pre-1.0 packages treat minor bumps as breaking changes. Patch-level is consistent across all deps — no need to reason about which packages follow semver strictly.


DD-007: Modern Ciphers for Cert Generation

Date: 2026-03-23 Status: Decided

Decision: Certificate generation uses ECDSA P-384 and AES-256-CBC instead of legacy RSA-2048/3DES.

Why: Modern cipher suites provide stronger security. Legacy software compatibility is not a design constraint.


DD-006: Node-RED TLS Uses PEM Files with servername Override

Date: 2026-03-23 Status: Decided

Decision: start.sh extracts PEM files from .p12 during init. The tls-config node uses cert/key/ca file paths (not inline PEM, not .p12 direct). verifyservercert = true with servername set to FQDN.

Why: Node-RED's certname/keyname are for editor-uploaded files. Inline PEM is treated as a file path. The servername override is needed because Node-RED connects via Docker hostname (tak-server) but the cert is issued for the external FQDN. Traffic stays on the Docker bridge — servername only affects cert validation.


DD-005: Monitor is Read-Only, Docker Socket is Read-Only

Date: 2026-03-23 Status: Superseded by DD-019

Decision: The monitor observes but does not control containers. Docker socket is mounted :ro. Service management (restart/stop/start), LDAP operations, and the Authentik API token are all excluded from the monitor.

Why: Reduces security surface. Containers should restart via Docker's own restart policies if they fail health checks. Management operations go through the API, not the monitoring layer.

Superseded: The monitor service was formalized as the FastTAK management API in DD-019. It now holds the Authentik API token and communicates with TAK Server via mTLS for cert and user operations.


DD-004: Two Init Containers with Different Lifecycles

Date: 2026-03-23 Status: Decided

Decision: Two separate one-shot init containers: init-core (needs DB healthy) and init-identity (needs both Authentik AND TAK Server healthy). Not combined into one.

Why: Different dependency chains. Combining them would force core startup to depend on Authentik being healthy, slowing the entire stack. Different languages (bash vs Python) and different concerns reinforce the split.


DD-003: CloudTAK Excluded from FastTAK Stack

Date: 2026-03-23 Status: Decided

Decision: CloudTAK is not part of the Docker Compose stack. All CloudTAK subdomain vars, Caddyfile routes, and .env.example entries are removed.

Why: FastTAK is a server-side infrastructure platform. It provides backend orchestration for TAK deployments, not client-side applications. Client platforms like CloudTAK have their own deployment requirements and are separate concerns.


DD-002: No Docker Compose Profiles — Single Unified Stack

Date: 2026-03-23 Status: Decided

Decision: The stack has no --profile flags. All services (TAK Server, Authentik, Caddy, MediaMTX, Node-RED, TAK Portal) are always present. TAK Server application files are served via a bind mount using the TAK_HOST_PATH variable (pointing to the user's tak/ directory on the host), not baked into Docker images.

Why: Profiles added complexity without benefit — Authentik/LDAP/TAK Portal were being used for all testing anyway. A single stack is simpler to document, test, and support. Bind mounts via TAK_HOST_PATH keep the user's TAK configuration on the host filesystem, making upgrades and customization straightforward without rebuilding images.


DD-001: TAK Server Images Are Built with Application Files Baked In

Date: 2026-03-23 Status: Decided

Decision: Users add COPY tak/ /opt/tak/ to the official TAK Dockerfiles before building. A setup.sh script automates extracting the TAK zip and building images. Named Docker volumes (not bind mounts) are used for persistence.

Why: The official tak.gov Dockerfiles are just base runtimes (Java, PostgreSQL+PostGIS) with no application files. Bind mounts were tried but rejected. Baking files in makes everything self-contained and compatible with named volumes. Volumes were consolidated from 6 to 4 by merging config + certs into tak-data.