Harden message log maintenance tooling

2026-06-26 05:44:38 +08:00 · 2026-05-27 09:58:56 +08:00 · 2026-05-27 09:58:56 +08:00 · 5240756f96
commit 5240756f96
parent 8b55a3d9ab
10 changed files with 369 additions and 34 deletions
--- a/8
+++ b/8
@ -34,7 +34,7 @@ MANDIR ?= $(PREFIX)/share/man
 SYSTEMD_UNIT_DIR ?= $(PREFIX)/lib/systemd/system
 CI_TEST_PORT ?= $(if $(PORT),$(PORT),2222)

-.PHONY: all clean install install-systemd uninstall uninstall-systemd debug release release-check release-check-strict asan valgrind check test test-advisory ci-test unit-test integration-test anonymous-access-test connection-limit-test security-test stress-test soak-test slow-client-test user-lifecycle-test info
+.PHONY: all clean install install-systemd uninstall uninstall-systemd debug release release-check release-check-strict asan valgrind check test test-advisory ci-test unit-test script-test integration-test anonymous-access-test connection-limit-test security-test stress-test soak-test slow-client-test user-lifecycle-test info

 all: $(TARGETS)

@ -108,7 +108,7 @@ check:
 	@command -v clang-tidy >/dev/null 2>&1 && clang-tidy src/*.c -- -Iinclude $(INCLUDES) || echo "clang-tidy not installed"

 # Test
-test: all unit-test integration-test
+test: all unit-test script-test integration-test

 test-advisory: all unit-test
 	@echo "Running integration tests..."
@ -120,6 +120,10 @@ unit-test:
 	@echo "Running unit tests..."
 	@$(MAKE) -C tests/unit run

+script-test:
+	@echo "Running script tests..."
+	@cd tests && ./test_logrotate.sh
+
 integration-test: all
 	@echo "Running integration tests..."
 	@cd tests && PORT=$${PORT:-2222} ./test_basic.sh
--- a/README.md
+++ b/README.md
@ -217,6 +217,19 @@ tntctl -p 2222 chat.example.com dump -n 100
 tntctl -l operator chat.example.com post "service notice"
 ```

+### Log Maintenance
+
+Persisted public history is stored as `messages.log` in the TNT state
+directory.  For manual maintenance, archive and compact it with:
+
+```sh
+scripts/logrotate.sh /var/lib/tnt/messages.log 100 10000
+```
+
+The script archives the full log, keeps the last `KEEP_LINES` records in the
+active file, compresses the archive when `gzip` is available, and can be
+previewed with `--dry-run`.
+
 ## Development

 ### Building
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@ -9,6 +9,8 @@
  including parser, sanitization, and partial-record recovery rules.
 - Added `dump [N]` / `dump -n N` to the SSH exec interface and `tntctl` for
  exporting valid persisted `messages.log` v1 records.
+- Added regression-tested manual log archive and compaction coverage for
+  `scripts/logrotate.sh`.
 - Added a public security policy, supported-version guidance, and GitHub issue
  templates for bug reports and feature requests.
 - Added `tntctl`, a thin local wrapper around the documented SSH exec
@ -58,6 +60,9 @@
 - Message-log replay and search now share one strict record parser and skip
  malformed, invalid UTF-8, extra-separator, oversized, or unterminated
  records instead of accepting partial replay data.
+- `scripts/logrotate.sh` now has validated arguments, stable exit statuses,
+  dry-run support, archive retention, gzip-aware archives, and a regression
+  test in the normal test suite.
 - The two-user lifecycle test now covers opening `:inbox` before a private
  message arrives, matching the way users often leave an inbox page open.
 - Private-message inbox access now uses its own mutex instead of sharing the
--- a/docs/DEPLOYMENT.md
+++ b/docs/DEPLOYMENT.md
@ -107,6 +107,24 @@ sudo rm /var/lib/tnt/motd.txt

 No restart required — TNT reads the file on each new connection.

+## Manual Log Maintenance
+
+TNT stores public chat history in `messages.log` under the state directory.
+Use the maintenance script from a source checkout when the service is stopped
+or during a quiet maintenance window:
+
+```bash
+sudo systemctl stop tnt
+sudo scripts/logrotate.sh /var/lib/tnt/messages.log 100 10000
+sudo systemctl start tnt
+```
+
+The arguments are `LOG_FILE MAX_SIZE_MB KEEP_LINES`.  The script archives the
+full log, compacts the active log to the last `KEEP_LINES` records, compresses
+the archive when `gzip` is available, and keeps the newest five archives by
+default.  Use `--dry-run` to preview actions, or `--keep-archives N` to change
+archive retention.
+
 ## Firewall

 ```bash
--- a/docs/MESSAGE_LOG.md
+++ b/docs/MESSAGE_LOG.md
@ -60,6 +60,21 @@ interface and `tntctl`.  The output format is exactly the v1 record format
 above.  Without `N`, `dump` exports all valid records; with `N`, it exports the
 last `N` valid records.

+## Maintenance
+
+`scripts/logrotate.sh` is the manual archive and compaction tool for
+`messages.log`:
+
+```sh
+scripts/logrotate.sh [--dry-run] [--keep-archives N] LOG_FILE MAX_SIZE_MB KEEP_LINES
+```
+
+When the log exceeds `MAX_SIZE_MB`, the script archives the full file, compacts
+the active file to the last `KEEP_LINES` records, compresses the archive when
+`gzip` is available, and removes older archives beyond the retention limit.
+Run it while TNT is stopped or during a quiet maintenance window if strict log
+consistency matters.
+
 ## Compatibility

 The v1 record format is stable for TNT 1.x.  Future incompatible storage
--- a/docs/QUICKREF.md
+++ b/docs/QUICKREF.md
@ -54,6 +54,12 @@ EXEC COMMANDS
  dump [N] / dump -n N   persisted messages.log v1 records
  post <message>         post as the SSH login name

+MAINTENANCE
+  scripts/logrotate.sh LOG_FILE MAX_SIZE_MB KEEP_LINES
+                           archive and compact messages.log
+  scripts/logrotate.sh --dry-run ...
+                           preview log maintenance actions
+
 STRUCTURE
  src/main.c          entry, signals
  src/cli_text.c      startup CLI text
--- a/docs/ROADMAP.md
+++ b/docs/ROADMAP.md
@ -58,7 +58,7 @@ Goal: make stored history durable, inspectable, and recoverable.
 - ✅ keep persisted timestamps in UTC throughout write and replay
 - ✅ validate persisted UTF-8 and record structure before replay/search
 - ✅ provide an inspection/export command for persisted records
- add log rotation and compaction tooling
+- ✅ add log rotation and compaction tooling
 - define broader recovery tooling for truncated or partially corrupted logs

 ## Stage 4: Interactive UX
--- a/scripts/logrotate.sh
+++ b/scripts/logrotate.sh
@ -1,44 +1,174 @@
-#!/bin/bash
-# TNT Log Rotation Script
-# Keeps chat history manageable and prevents disk space issues
+#!/bin/sh
+# Compact and archive a TNT messages.log file.
+#
+# This is an operator-run maintenance tool.  For strict consistency, stop TNT
+# or run it during a quiet maintenance window before compacting the active log.

-LOG_FILE="${1:-/var/lib/tnt/messages.log}"
-MAX_SIZE_MB="${2:-100}"
-KEEP_LINES="${3:-10000}"
+set -eu

-# Check if log file exists
-if [ ! -f "$LOG_FILE" ]; then
-    echo "Log file $LOG_FILE does not exist"
+DRY_RUN=0
+KEEP_ARCHIVES=5
+
+usage() {
+    cat <<'USAGE'
+Usage: scripts/logrotate.sh [--dry-run] [--keep-archives N] [LOG_FILE [MAX_SIZE_MB [KEEP_LINES]]]
+
+Defaults:
+  LOG_FILE       /var/lib/tnt/messages.log
+  MAX_SIZE_MB    100
+  KEEP_LINES     10000
+
+Exit status:
+  0   success, including missing log file
+  1   runtime error
+  64  invalid arguments
+USAGE
+}
+
+fail_usage() {
+    echo "logrotate: $*" >&2
+    usage >&2
+    exit 64
+}
+
+fail() {
+    echo "logrotate: $*" >&2
+    exit 1
+}
+
+is_uint() {
+    case "${1:-}" in
+        ''|*[!0-9]*)
+            return 1
+            ;;
+        *)
+            return 0
+            ;;
+    esac
+}
+
+is_positive_uint() {
+    is_uint "$1" && [ "$1" -gt 0 ]
+}
+
+while [ "$#" -gt 0 ]; do
+    case "$1" in
+        --dry-run)
+            DRY_RUN=1
+            shift
+            ;;
+        --keep-archives)
+            [ "$#" -ge 2 ] || fail_usage "missing value for --keep-archives"
+            is_uint "$2" || fail_usage "invalid archive count: $2"
+            KEEP_ARCHIVES=$2
+            shift 2
+            ;;
+        -h|--help)
+            usage
+            exit 0
+            ;;
+        --)
+            shift
+            break
+            ;;
+        -*)
+            fail_usage "unknown option: $1"
+            ;;
+        *)
+            break
+            ;;
+    esac
+done
+
+[ "$#" -le 3 ] || fail_usage "too many arguments"
+
+LOG_FILE=${1:-/var/lib/tnt/messages.log}
+MAX_SIZE_MB=${2:-100}
+KEEP_LINES=${3:-10000}
+
+case "$LOG_FILE" in
+    ''|-*)
+        fail_usage "invalid log path"
+        ;;
+esac
+is_uint "$MAX_SIZE_MB" || fail_usage "invalid max size: $MAX_SIZE_MB"
+is_positive_uint "$KEEP_LINES" || fail_usage "invalid keep lines: $KEEP_LINES"
+
+if [ ! -e "$LOG_FILE" ]; then
+    echo "logrotate: $LOG_FILE does not exist"
    exit 0
 fi
+[ -f "$LOG_FILE" ] || fail "$LOG_FILE is not a regular file"

-# Get file size in MB
-FILE_SIZE=$(du -m "$LOG_FILE" | cut -f1)
+MAX_BYTES=$((MAX_SIZE_MB * 1024 * 1024))
+FILE_SIZE=$(wc -c < "$LOG_FILE" | tr -d ' ')
+[ -n "$FILE_SIZE" ] || fail "could not read log size"

-# Rotate if file is too large
-if [ "$FILE_SIZE" -gt "$MAX_SIZE_MB" ]; then
-    echo "Log file size: ${FILE_SIZE}MB, rotating..."
+compact_log() {
+    timestamp=$(date -u +%Y%m%dT%H%M%SZ)
+    backup="${LOG_FILE}.${timestamp}"
+    suffix=1

-    # Create backup
-    BACKUP="${LOG_FILE}.$(date +%Y%m%d_%H%M%S)"
-    cp "$LOG_FILE" "$BACKUP"
+    while [ -e "$backup" ] || [ -e "${backup}.gz" ]; do
+        backup="${LOG_FILE}.${timestamp}.${suffix}"
+        suffix=$((suffix + 1))
+    done

-    # Keep only last N lines
-    tail -n "$KEEP_LINES" "$LOG_FILE" > "${LOG_FILE}.tmp"
-    mv "${LOG_FILE}.tmp" "$LOG_FILE"
-
-    # Compress old backup
-    gzip "$BACKUP"
-
-    echo "Log rotated. Backup: ${BACKUP}.gz"
-    echo "Kept last $KEEP_LINES lines"
-else
-    echo "Log file size: ${FILE_SIZE}MB (under ${MAX_SIZE_MB}MB limit)"
+    if [ "$DRY_RUN" -eq 1 ]; then
+        echo "logrotate: would archive $LOG_FILE to $backup"
+        echo "logrotate: would keep last $KEEP_LINES lines"
+        return 0
    fi

-# Clean up old compressed logs (keep last 5)
-LOG_DIR=$(dirname "$LOG_FILE")
-cd "$LOG_DIR" || exit
-ls -t messages.log.*.gz 2>/dev/null | tail -n +6 | xargs rm -f 2>/dev/null
+    tmp="${LOG_FILE}.tmp.$$"
+    rm -f "$tmp"
+    cp -p "$LOG_FILE" "$backup" || fail "failed to create archive"
+    if ! tail -n "$KEEP_LINES" "$LOG_FILE" > "$tmp"; then
+        rm -f "$tmp"
+        fail "failed to compact log"
+    fi
+    if ! cat "$tmp" > "$LOG_FILE"; then
+        rm -f "$tmp"
+        fail "failed to replace log"
+    fi
+    rm -f "$tmp"

-echo "Log rotation complete"
+    if command -v gzip >/dev/null 2>&1; then
+        gzip -f "$backup" || fail "failed to compress archive"
+        backup="${backup}.gz"
+    fi
+
+    echo "logrotate: archived $backup"
+    echo "logrotate: kept last $KEEP_LINES lines"
+}
+
+cleanup_archives() {
+    [ "$KEEP_ARCHIVES" -ge 0 ] || return 0
+
+    archives=$(
+        ls -1t "$LOG_FILE".*.gz "$LOG_FILE".[0-9]* 2>/dev/null || true
+    )
+    [ -n "$archives" ] || return 0
+
+    printf '%s\n' "$archives" |
+    awk '!seen[$0]++' |
+    awk -v keep="$KEEP_ARCHIVES" 'NR > keep' |
+    while IFS= read -r old; do
+        [ -n "$old" ] || continue
+        if [ "$DRY_RUN" -eq 1 ]; then
+            echo "logrotate: would remove $old"
+        else
+            rm -f "$old"
+        fi
+    done
+}
+
+if [ "$FILE_SIZE" -gt "$MAX_BYTES" ]; then
+    echo "logrotate: size ${FILE_SIZE} bytes exceeds ${MAX_BYTES} bytes"
+    compact_log
+else
+    echo "logrotate: size ${FILE_SIZE} bytes is within ${MAX_BYTES} bytes"
+fi
+
+cleanup_archives
+echo "logrotate: complete"
--- a/scripts/release_check.sh
+++ b/scripts/release_check.sh
@ -13,6 +13,7 @@ Default checks:
  - version metadata alignment
  - clean build
  - unit tests
+  - script tests
  - staged install layout with PREFIX=/usr and DESTDIR
  - installer shell syntax
  - Debian packaging metadata
@ -102,6 +103,9 @@ step "running unit tests"
 make -C tests/unit clean
 make -C tests/unit run

+step "running script tests"
+make script-test
+
 step "checking client I/O ownership boundaries"
 ! grep -R "client_send(target" src include >/dev/null ||
    fail "cross-client target writes must be queued through client_queue_bell"
--- a/tests/test_logrotate.sh
+++ b/tests/test_logrotate.sh
@ -0,0 +1,140 @@
+#!/bin/sh
+# Maintenance-script regression tests for scripts/logrotate.sh.
+
+set -u
+
+PASS=0
+FAIL=0
+SCRIPT="../scripts/logrotate.sh"
+STATE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/tnt-logrotate-test.XXXXXX")
+
+cleanup() {
+    rm -rf "$STATE_DIR"
+}
+trap cleanup EXIT
+
+pass() {
+    echo "✓ $1"
+    PASS=$((PASS + 1))
+}
+
+fail() {
+    echo "✗ $1"
+    FAIL=$((FAIL + 1))
+}
+
+archive_payload() {
+    archive=$1
+    case "$archive" in
+        *.gz) gzip -cd "$archive" ;;
+        *) cat "$archive" ;;
+    esac
+}
+
+echo "=== TNT Logrotate Tests ==="
+
+if [ ! -x "$SCRIPT" ]; then
+    echo "Error: script $SCRIPT not found or not executable."
+    exit 1
+fi
+
+MISSING_OUTPUT=$("$SCRIPT" "$STATE_DIR/missing.log" 100 10 2>&1)
+MISSING_STATUS=$?
+printf '%s\n' "$MISSING_OUTPUT" | grep -q 'does not exist'
+if [ "$MISSING_STATUS" -eq 0 ] && [ $? -eq 0 ]; then
+    pass "missing log is a successful no-op"
+else
+    fail "missing log handling"
+    printf '%s\n' "$MISSING_OUTPUT"
+fi
+
+LOG="$STATE_DIR/messages.log"
+cat > "$LOG" <<'EOF'
+2026-01-01T00:00:01Z|alice|one
+2026-01-01T00:00:02Z|bob|two
+2026-01-01T00:00:03Z|carol|three
+EOF
+
+if "$SCRIPT" "$LOG" 100 2 >/dev/null 2>&1 &&
+   grep -q 'alice|one' "$LOG" &&
+   [ "$(ls "$LOG".* 2>/dev/null | wc -l | tr -d ' ')" -eq 0 ]; then
+    pass "small log stays unmodified"
+else
+    fail "small log no-op"
+    cat "$LOG" 2>/dev/null
+fi
+
+ROTATE_OUTPUT=$("$SCRIPT" "$LOG" 0 2 2>&1)
+ROTATE_STATUS=$?
+ARCHIVE=$(ls "$LOG".*.gz "$LOG".[0-9]* 2>/dev/null | head -n 1)
+if [ "$ROTATE_STATUS" -eq 0 ] &&
+   printf '%s\n' "$ROTATE_OUTPUT" | grep -q 'kept last 2 lines' &&
+   ! grep -q 'alice|one' "$LOG" &&
+   grep -q 'bob|two' "$LOG" &&
+   grep -q 'carol|three' "$LOG" &&
+   [ -n "$ARCHIVE" ] &&
+   archive_payload "$ARCHIVE" | grep -q 'alice|one'; then
+    pass "oversize log is archived and compacted"
+else
+    fail "oversize rotation"
+    printf '%s\n' "$ROTATE_OUTPUT"
+    cat "$LOG" 2>/dev/null
+fi
+
+DRY_LOG="$STATE_DIR/dry.log"
+printf 'line1\nline2\nline3\n' > "$DRY_LOG"
+DRY_BEFORE=$(cat "$DRY_LOG")
+DRY_OUTPUT=$("$SCRIPT" --dry-run "$DRY_LOG" 0 1 2>&1)
+DRY_STATUS=$?
+if [ "$DRY_STATUS" -eq 0 ] &&
+   [ "$(cat "$DRY_LOG")" = "$DRY_BEFORE" ] &&
+   printf '%s\n' "$DRY_OUTPUT" | grep -q 'would archive'; then
+    pass "dry run does not modify the log"
+else
+    fail "dry run handling"
+    printf '%s\n' "$DRY_OUTPUT"
+fi
+
+INVALID_OUTPUT=$("$SCRIPT" "$LOG" nope 2 2>&1)
+INVALID_STATUS=$?
+if [ "$INVALID_STATUS" -eq 64 ] &&
+   printf '%s\n' "$INVALID_OUTPUT" | grep -q 'invalid max size'; then
+    pass "invalid arguments exit 64"
+else
+    fail "invalid argument status"
+    printf '%s\n' "$INVALID_OUTPUT"
+    echo "exit status: $INVALID_STATUS"
+fi
+
+DIR_OUTPUT=$("$SCRIPT" "$STATE_DIR" 0 1 2>&1)
+DIR_STATUS=$?
+if [ "$DIR_STATUS" -eq 1 ] &&
+   printf '%s\n' "$DIR_OUTPUT" | grep -q 'not a regular file'; then
+    pass "non-regular log path is rejected"
+else
+    fail "non-regular path handling"
+    printf '%s\n' "$DIR_OUTPUT"
+    echo "exit status: $DIR_STATUS"
+fi
+
+RET_LOG="$STATE_DIR/retention.log"
+printf 'a\nb\nc\n' > "$RET_LOG"
+printf old1 > "$RET_LOG.20000101T000000Z.gz"
+sleep 1
+printf old2 > "$RET_LOG.20010101T000000Z.gz"
+sleep 1
+printf old3 > "$RET_LOG.20020101T000000Z.gz"
+
+if "$SCRIPT" --keep-archives 2 "$RET_LOG" 100 2 >/dev/null 2>&1 &&
+   [ "$(ls "$RET_LOG".*.gz 2>/dev/null | wc -l | tr -d ' ')" -eq 2 ]; then
+    pass "archive retention removes older archives"
+else
+    fail "archive retention"
+    ls "$RET_LOG".* 2>/dev/null || true
+fi
+
+echo ""
+echo "PASSED: $PASS"
+echo "FAILED: $FAIL"
+[ "$FAIL" -eq 0 ] && echo "All tests passed" || echo "Some tests failed"
+exit "$FAIL"