CLI commands

Besides the server, the image ships maintenance commands under the litestar CLI. Run them inside the container with docker compose exec -u geometrikks app litestar <command> (or docker compose run --rm app litestar <command> when the stack is stopped). The image sets LITESTAR_APP so the bare command works there; outside the container, point the CLI at the app yourself: uv run litestar --app geometrikks.server.core:create_app <command>. Every command supports --help.

import-logs: backfill history

Live tailing only picks up lines written after the app starts. To backfill rotated or archived access logs (nginx, Traefik JSON, Caddy JSON or Nginx Proxy Manager’s own format, plain or gzip), use import-logs:

docker compose exec -u geometrikks app litestar import-logs /var/log/access/access.log.1.gz

It reuses the live ingestion pipeline (same parsing, GeoIP lookup and DB writes), uses the timestamps in each log line rather than wall-clock time, and refreshes the continuous aggregates for the imported range when done. The log format is auto-detected per file, as with live tailing; pass --format geometrikks-json, --format nginx, --format traefik-json, --format caddy-json or --format npm to pin it. You can pass several files in one invocation. Paths are container paths, and the import runs as the non-root geometrikks user (PUID:PGID, default 1000:1000), so host files must be readable by it (-u geometrikks keeps exec from running the import as root).

exec requires the app service to be running. If the stack is stopped, use run --rm instead:

docker compose run --rm app litestar import-logs /var/log/access/access.log.1.gz

[!IMPORTANT] Import archived (rotated) files only. Importing a file that is also being live-tailed double-counts its lines.

• Each imported file is fingerprinted by content checksum; importing the same content again (even under a different filename) is skipped. Pass --force to re-import. That updates the bookkeeping row but does not delete rows written by the earlier import.
• A file that matches no supported log format is rejected up front, before anything is written.
• --before takes a date or a datetime such as 2026-10-01T14:03:12+02:00 and leaves out lines stamped at or after it. The summary counts them. Use it when an archive overlaps a log GeoMetrikks already ingested, for example when a proxy keeps writing its own log next to the one GeoMetrikks tails. Without an offset the value is UTC. The cutoff is stored with the file’s checksum. Running the same file again with a later --before imports only the lines between the two cutoffs, and without --before the lines from the old cutoff on. An earlier --before is refused. --force ignores the stored cutoff and imports the whole file again.
• Without --format, the format is detected per file. If detection can only match the relaxed IP-and-timestamp pattern, the file imports as map events with no access-log rows. Pin the format (--format geometrikks-json or --format nginx) to require a full parse; lines that do not match then show up in the skipped count instead.
• The TimescaleDB retention policy drops rows older than the raw retention window (ANALYTICS_RAW_RETENTION_DAYS, default 180 days), so history beyond that window will not persist. Raise the retention setting before importing older archives if you want to keep them.

import-loki: backfill history from Loki

If GeoMetrikks reads from Loki, the history is in Loki, not in a file. import-loki reads a time range of every LOKI_QUERIES selector through the same pipeline as live reading:

docker compose exec -u geometrikks app litestar import-loki --since 2026-09-01

• --since is the start of the range, inclusive. --before is the end, exclusive. Both take a date or a datetime such as 2026-10-01T14:03:12+02:00. Without an offset the value is UTC.
• Without --before, the import stops where the data already in the database begins. On the first run that is the oldest access-log row for the selector’s server, which is where live reading started. After an earlier import-loki run, it is the start of that run, unless the server has access-log rows older than the ones that run wrote. The command prints the end it picked and the --since and --before pair for the same range.
• An interrupted run keeps its own end. Rerunning the same command with the same --since resumes it, even without --before.
• If --since is not earlier than the end, or nothing is stored yet to take an end from, the command skips that selector and says why.
• When the end is the oldest access-log row, it is the time written in the log line. Loki filters by the time the shipper stamped the entry, and the two differ by the shipper’s delay, so lines near that boundary can be read twice or missed. An earlier import’s start and an explicit --before are Loki times and exact. Pass --before when a tailed file shares the server name, because the oldest row may then come from the file.
• --query N imports only the Nth selector in LOKI_QUERIES, counting from 1. If several selectors share a server name, import them in one command without --query. The command picks every selector’s end before any of them imports, so one selector’s backfill can’t become another’s end.
• --batch-size sets how many records go into each database write. The default is 500.
• The command reads one hour at a time and records a selector’s progress after each complete hour. If Loki or a database write fails, or you stop the command, rerun it with the same --since and --before and without --force. It picks up at the start of the interrupted hour, so rows it had already written from that hour end up twice.
• A failure in one selector doesn’t stop the others. At the end the command lists the selectors that failed and exits with an error.
• The command refuses a range that overlaps an earlier import-loki range for the same selector, and names that range. It only knows about its own imports. Rows from live reading, from import-logs, or from another selector that matches the same streams don’t count, so pick --since and --before around them yourself.
• --force skips the overlap check. An unfinished run of the same range still resumes. A finished range is imported again, and the rows from the first import stay, so they end up twice.
• The command detects the format from the first lines it reads, or uses the one pinned in LOKI_FORMATS. If those lines match no format, or not the pinned one, it rejects the selector before writing anything.
• Loki can’t page within a single timestamp. With second-precision entry timestamps, the command reads at most 1000 lines from any one second and drops the rest, as live reading does.
• The retention policy drops imported rows older than ANALYTICS_RAW_RETENTION_DAYS, like any other rows. Loki’s own retention limits how far back there is anything to read.

backfill-server: fix up historical server names

Every ingested row records which server wrote it (LOGPARSER_SERVER_NAME; defaults to the machine hostname, which the compose file pins to geometrikks). Rows ingested by older versions have no server name, so the Server filter on Access Logs cannot see them. backfill-server stamps them retroactively:

docker compose exec -u geometrikks app litestar backfill-server myhost

The plain form fills only rows with no server name. It is idempotent, cannot overwrite stamped values, and runs immediately without a confirmation prompt.

If your database has accumulated many bogus server names, add --consolidate to rewrite all existing server names to the given name as well. The classic cause is running in Docker with LOGPARSER_SERVER_NAME unset before the compose file pinned a hostname: every container recreation got a new 12-hex container ID as its hostname, and so a new server name. Consolidate lists every server name it will rewrite, with row counts, and asks for confirmation first (--yes skips the prompt):

docker compose exec -u geometrikks app litestar backfill-server myhost --consolidate

Either form decompresses compressed history chunks first (a full-table update would trip TimescaleDB’s tuple decompression limit), so disk usage grows until the compression policy recompresses them. It then refreshes the affected continuous aggregates so the filter dropdowns update. It may run for minutes on a large database.

backfill-geo-servers: fill historical Geo Logs servers

Existing installations may show an empty Servers column for Geo Logs ranges over 24 hours. To fill in the missing server names from retained events, run:

docker compose exec -u geometrikks app litestar backfill-geo-servers

The command prints the pending ranges and asks for confirmation. It refreshes one day at a time and resumes with the next unfinished day if you rerun it. Use --yes to skip confirmation. The app can stay running, but the refresh uses database CPU and I/O and may take minutes on a large database.

The aggregate rows record which buckets are complete. If the command stops or a batch fails, rerun it to process the remaining buckets. A second simultaneous run is rejected. Hourly backfills respect hourly retention, and both views only refresh complete buckets within raw retention. Counts remain available while server history is incomplete. This command cannot recover server names after their raw events have expired.

The command reads geo_events and refreshes the aggregates. It does not rewrite server names or explicitly decompress every raw chunk. In-place column upgrades require TimescaleDB 2.28 or later. If adding the columns fails, the app keeps the existing aggregate and logs server_cagg_columns_unavailable. Fix the database error and restart before running the command.

backfill-asn: fill in ASN data for historical rows

Rows ingested before the ASN feature (or while the ASN database was missing) have no ASN data. backfill-asn resolves their IPs against the local GeoLite2 ASN database and stamps them retroactively. Choose the table explicitly. To backfill Access Logs:

docker compose exec -u geometrikks app litestar backfill-asn --table access-logs

Backfill Geo Logs without touching the Access Logs table:

docker compose exec -u geometrikks app litestar backfill-asn --table geo-events

Use --table all when both tables need a backfill.

It fills only rows with no ASN data (idempotent, never overwrites stamped values) and asks for confirmation after reporting how many rows and distinct IPs are affected (--yes skips the prompt). IPs the database cannot resolve stay empty. Like backfill-server, it decompresses the selected table’s compressed history first (disk usage grows until the compression policy recompresses it) and refreshes the corresponding continuous aggregates afterwards. It may run for minutes on a large database.

If the aggregate refresh fails, the command exits non-zero and names the stale aggregates: the rows are stamped, but the Top ASNs view will not show the backfilled range until they refresh. Rerunning is safe.

Today’s ASN database describes today’s network ownership; stamping years-old traffic with it is an approximation.

backfill-timings: clear placeholder response times

Archives in nginx’s built-in combined format have no $request_time. Older versions stored those rows with a response time of 0.0, which dragged every average and percentile toward zero. Rows ingested by this version store no timing at all for such lines; backfill-timings does the same for the rows that predate it:

docker compose exec -u geometrikks app litestar backfill-timings

It only touches rows the legacy nginx format wrote without a host, which is how a combined line looks after import (the custom format always logs $host), and whose response time is exactly 0. A genuine sub-millisecond timing on a row with a host is left alone. --server NAME and --before 2026-08-20 narrow the set; the command prints the row count and time span and asks for confirmation (--yes skips it). Like the other backfills it decompresses history chunks first and refreshes the affected continuous aggregates afterwards, so it may run for minutes on a large database.

Large imports and backfills

Bulk operations (import-logs over months of archives, either backfill command on a database with real history) write WAL much faster than live tailing does. With PostgreSQL’s default max_wal_size of 1GB the database checkpoints every few seconds and logs:

LOG:  checkpoints are occurring too frequently (21 seconds apart)
HINT: Consider increasing the configuration parameter "max_wal_size".

Nothing is at risk, but each checkpoint re-triggers full-page writes for the pages the run touches next, so the operation slows down the longer this goes on. Both settings reload without a restart, so you can raise them while a backfill is already running:

docker compose exec timescale_db psql -U geouser -d geometrikks \
  -c "ALTER SYSTEM SET max_wal_size = '4GB';" \
  -c "ALTER SYSTEM SET checkpoint_timeout = '15min';" \
  -c "SELECT pg_reload_conf();"

Expect up to max_wal_size of extra disk used for WAL during the run, on top of the temporarily decompressed chunks. To keep the settings permanently, add max_wal_size=4GB and checkpoint_timeout=15min to the database service’s command: block in the compose file, then run ALTER SYSTEM RESET max_wal_size and ALTER SYSTEM RESET checkpoint_timeout once. Command-line flags override ALTER SYSTEM values, and keeping both means the compose file no longer tells the whole truth.