docs: complete NaxOS architectural specs, migration guides, API reference, and CI workflows

docs/architecture/app-engine.md

@@ -0,0 +1,19 @@
+# App Engine & Workload Runtime Architecture
+
+NaxOS features a tri-tier runtime engine that selects the best execution strategy for each application:
+
+## 1. Native NixOS Systemd Services
+- **Best For**: High-throughput services requiring direct hardware device passthrough, minimal RAM overhead, and deep integration with OpenZFS.
+- **Example**: **Immich** Photo Hub.
+  - Native Nixpkgs package running as an isolated systemd service.
+  - Direct Intel QuickSync (`/dev/dri/renderD128`) access with `extraPackages = [ intel-media-driver intel-compute-runtime vpl-gpu-rt ]`.
+  - Native systemd CPU quota: `CPUQuota = "200%"` to avoid choking the storage daemon during intensive facial recognition or CLIP embedding indexing.
+  - Photos stored directly in OpenZFS dataset `/tank/media/photos`.
+
+## 2. Rootless Docker / Compose Engine
+- **Best For**: General third-party containerized workloads.
+- **Backend**: Configured with Docker's native `zfs` storage driver (`virtualisation.docker.storageDriver = "zfs"`). Container image layers become native ZFS subvolumes with instant snapshots and copy-on-write deduplication.
+
+## 3. Lightweight K3s
+- **Best For**: Cloud-native Helm charts and multi-service distributed workloads.
+- Can be activated on demand via `services.naxos.appEngine.k3s.enable = true`.

docs/architecture/declarative-backend.md

@@ -0,0 +1,37 @@
+# Declarative Backend & GitOps Synchronization Engine
+
+## Core Workflow
+
+NaxOS eliminates drift between what the Web Dashboard displays and what the underlying operating system executes.
+
+When an administrator performs an action in the UI (e.g. creating a share, setting a dataset quota, installing Immich):
+
+1. **REST API Request**: The frontend sends a strongly typed JSON payload to the NaxOS Management Daemon (`naxos-api`).
+2. **In-Memory & JSON State Update**: The daemon validates the payload against its schemas and updates `/etc/naxos/repo/naxos-config.json`.
+3. **NixOS Code Generation**: The daemon's compiler generates `/etc/naxos/repo/generated-naxos-config.nix`.
+4. **Git Transaction**:
+   - The daemon commits the modified files into the local Git repository at `/etc/naxos/repo`.
+   - If a remote repository is configured (e.g. on Gitea), it pushes the commit automatically to `origin/main`.
+5. **Dry-Run Validation**:
+   - Before any running service is altered, `naxos-rebuild-safe` performs a dry build:
+     ```bash
+     nixos-rebuild build --flake .#naxos
+     ```
+   - If evaluation or build fails (e.g. invalid option syntax or dependency conflict), the transaction is immediately aborted and the running system remains untouched.
+6. **Atomic Switch & Canary Verification**:
+   - The new generation is activated using `nixos-rebuild switch`.
+   - The canary runner tests daemon connectivity and storage accessibility.
+   - If canary verification fails, the system automatically runs:
+     ```bash
+     "$PREV_GEN/bin/switch-to-configuration" switch
+     ```
+   - The user is alerted in the dashboard with full build logs.
+
+---
+
+## 1-Click Rollback Mechanism
+
+Because every generation is tied to an explicit Git commit and NixOS system profile generation:
+- The user can browse the commit history in the Web UI.
+- Clicking **Rollback** checks out the selected commit and activates the previous generation in seconds.
+- No database dumps or manual backups are needed for appliance recovery.

docs/architecture/overview.md

@@ -0,0 +1,49 @@
+# NaxOS Architecture Overview
+
+## Design Philosophy
+
+**NaxOS** is an appliance-like Network Attached Storage (NAS) operating system combining the mathematical immutability of **NixOS**, the enterprise data integrity of **OpenZFS**, and the modern observability of **CNCF Perses**.
+
+Traditional NAS operating systems (like TrueNAS, Unraid, or Synology DSM) rely either on mutable root filesystems with hidden configuration databases, or proprietary management stacks that can be fragile to upgrade and difficult to replicate.
+
+NaxOS introduces an opinionated, cloud-native paradigm:
+1. **The Entire OS is Code**: Kernel parameters, network tuning, storage datasets, shares, users, and installed applications are compiled from a single declarative definition.
+2. **GitOps-by-Default**: Every modification made via the modern web dashboard or API is recorded as an atomic Git commit and synchronized to a remote Gitea repository.
+3. **True Disaster Recovery**: If an update or bad configuration breaks a service, instant rollbacks return the running system to the exact previous working generation with a single command.
+4. **Data Safety First**: Existing OpenZFS pools from legacy systems (such as `nixos-lukas` or TrueNAS) are imported read-safe without formatting, preserving critical application storage such as Immich photo libraries.
+5. **Modern Native Observability**: Replaces heavy standalone Grafana instances with embedded CNCF **Perses** dashboards that render natively inside the web UI.
+
+---
+
+## Architectural Subsystems
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                   NaxOS Modern Web UI                       │
+│        (React 18, Tailwind CSS, Embedded Perses)            │
+└──────────────────────────────┬──────────────────────────────┘
+                               │ REST / SSE / WebSockets
+┌──────────────────────────────▼──────────────────────────────┐
+│                  NaxOS Management Daemon                    │
+│    (Node.js / TypeScript, Fastify, ZFS Driver, GitOps)       │
+├──────────────────────────────┬──────────────────────────────┤
+│  • Storage Controller        │  • GitOps Synchronization    │
+│  • NixOS Module Generator    │  • App Engine Orchestrator   │
+│  • Journald Log Streamer     │  • Perses Metrics Coordinator│
+└──────────────┬───────────────┴───────────────┬──────────────┘
+               │                               │
+┌──────────────▼───────────────┐ ┌─────────────▼──────────────┐
+│     Declarative Engine       │ │      OpenZFS Storage       │
+│  /etc/naxos/repo (Git)       │ │  • zpool / zfs Primitives  │
+│  nixos-rebuild-safe          │ │  • Native Encryption/Comp  │
+│  System Generations          │ │  • Auto-Scrub & Snapshots  │
+└──────────────┬───────────────┘ └─────────────┬──────────────┘
+               │                               │
+┌──────────────▼───────────────────────────────▼──────────────┐
+│                    Linux / NixOS Core                       │
+│  • BBR Congestion Control & 10G/25G Kernel Buffers          │
+│  • Samba with Apple macOS Time Machine (vfs_fruit)          │
+│  • App Runtimes: Native Systemd, Rootless Docker, K3s       │
+│  • Intel QuickSync & OpenCL Hardware GPU Acceleration       │
+└─────────────────────────────────────────────────────────────┘
+```

docs/architecture/perses-analytics.md

@@ -0,0 +1,16 @@
+# CNCF Perses Native Analytics Architecture
+
+## Why Perses over Standalone Grafana?
+
+Traditional NAS appliances that provide telemetry either bundle a heavy standalone Grafana instance (which requires its own SQLite/PostgreSQL database, user accounts, and separate session cookies) or write rudimentary, un-configurable charts into their dashboard.
+
+NaxOS takes a pioneering approach by integrating **Perses** (a Cloud Native Computing Foundation project):
+
+1. **GitOps & Declarative Native**:
+   - Perses dashboards and datasources are written in clean JSON/CUE manifests.
+   - Dashboards are checked into `/var/lib/perses/projects/naxos/dashboards` as code.
+2. **Embedded Direct Visualization**:
+   - The NaxOS Web Dashboard communicates with the internal Perses metrics API.
+   - Eliminates Grafana's multi-megabyte bundle overhead, auth barriers, and database schema migrations.
+3. **Optimized for OpenZFS Metrics**:
+   - Out-of-the-box panels monitor OpenZFS ARC hits, misses, ARC target sizing, IOPS, and disk latencies scraped from Prometheus and Node Exporter.

docs/architecture/storage-engine.md

@@ -0,0 +1,36 @@
+# OpenZFS Storage Engine & Migration Architecture
+
+## OpenZFS Primitives in NaxOS
+
+NaxOS leverages upstream OpenZFS on Linux with specific architectural choices optimized for mixed workloads (streaming media, small-file databases, photo indexing, and Apple Time Machine backups).
+
+### 1. Default Pool Parameters
+When creating pools through NaxOS, the engine enforces optimal pool attributes:
+- `ashift=12`: 4096-byte sector alignment to eliminate write amplification on modern Advanced Format HDDs and NVMe SSDs.
+- `xattr=sa`: System-attribute storage for extended attributes, drastically improving Samba and POSIX ACL performance.
+- `acltype=posixacl`: POSIX Access Control Lists enabling granular multi-user permissions.
+- `compression=lz4`: Ultra-low overhead, transparent compression enabled globally on pools, with dataset-level overrides for `zstd` on cold backups and photo datasets.
+
+### 2. Record Size Strategy
+NaxOS tunes dataset record sizes based on application type:
+- **Media Datasets** (`tank/media`): `recordsize=1M` (maximizing sequential read throughput and reducing metadata overhead).
+- **General Storage** (`tank/backup`): `recordsize=128K` (default ZFS balance).
+- **Databases & Containers** (`tank/container`): `recordsize=16K` or `128K`.
+
+### 3. ZFS ARC Max Management
+To prevent OpenZFS from consuming all host RAM and starving machine learning workloads (such as Immich facial recognition models), NaxOS enforces a declarative ARC limit:
+```nix
+boot.kernelParams = [ "zfs.zfs_arc_max=4294967296" ]; # 4 GiB
+```
+
+---
+
+## Safe Foreign Pool Import Architecture
+
+A key tenet of NaxOS is **zero data loss**:
+
+When importing pools from existing setups (e.g. TrueNAS or `nixos-lukas`):
+1. The pool is initially inspected with `zpool import -N -f <pool>` (`-N` instructs ZFS not to mount any datasets).
+2. The dataset tree is scanned and properties are parsed without modifying data.
+3. Datasets are adopted declaratively into `services.naxos.storage.importExistingPools`.
+4. Original permissions and POSIX ACLs are preserved.