feat: multi-agent sessions, extraction filter, issue fixes

.gitignore

@@ -63,6 +63,7 @@ crawls/
 *.db
 *.sqlite
 *.sqlite3
+*.seoproject
 
 # Claude Code
 .claude

README.md

@@ -37,7 +37,7 @@ Every link type — images, CSS, JavaScript, media, canonicals, pagination, href
 
 Pull in real data from **Google Search Console, Analytics 4, PageSpeed Insights, and Chrome UX Report**, check spelling and grammar with **LanguageTool** — with a **bundled offline dictionary** for languages LanguageTool has no rules for, such as Turkish — and run your own **AI prompts** on each page (OpenAI, Claude, or Ollama). Open several projects side by side, each in its own window running its own crawl, and manage them from a Projects dialog with search, tags, and starter templates.
 
-Export to **Excel, CSV, JSON, XML**, an HTML report, sitemaps, **Google Sheets**, or **BigQuery**, save each project as **one compressed file** (or a **password-protected encrypted file**), and export any URL Details sub-table on its own. A built-in **MCP server** (96 tools) lets AI agents like Claude drive crawls live and read every result — every action a human can take in the desktop is callable from an agent. Everything runs **fully local** — no telemetry, no cloud, MIT-licensed.
+Export to **Excel, CSV, JSON, XML**, an HTML report, sitemaps, **Google Sheets**, or **BigQuery**, save each project as **one compressed file** (or a **password-protected encrypted file**), and export any URL Details sub-table on its own. A built-in **MCP server** (100+ tools) lets AI agents like Claude drive crawls live and read every result — every action a human can take in the desktop is callable from an agent, and **several agents can run in parallel**, each in its own isolated crawl session. Everything runs **fully local** — no telemetry, no cloud, MIT-licensed.
 
 <br />
 
@@ -209,7 +209,7 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 1. **Read-only data access** to the SQLite project — runs alongside the desktop app without contention (WAL allows concurrent readers).
 2. **Live crawl control** — when the desktop app is open, an agent can start / pause / resume / stop crawls and poll progress in real time. This goes through a localhost-only HTTP bridge (127.0.0.1, ephemeral random port, 32-byte Bearer token auth, discovery file written to `<userData>/mcp-bridge.json` on app launch).
 
-**96 tools** — every action a human can take in the desktop is callable from an agent. Representative groups:
+**100+ tools** — every action a human can take in the desktop is callable from an agent, plus multi-agent session management so several agents can work in parallel. Representative groups:
 
 | Group | Tools |
 | :--- | :--- |
@@ -224,9 +224,12 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 | ⚙ **Config & schedule** | `get_crawl_config`, `set_crawl_config`, `schedule_get`, `schedule_set` |
 | 📁 **Project management** | `list_projects`, `set_project`, `current_project` |
 | 🕷 **Crawl control** (desktop must be open) | `start_crawl`, `stop_crawl`, `pause_crawl`, `resume_crawl`, `clear_crawl`, `get_crawl_progress`, `get_desktop_project` |
+| 🧑‍🤝‍🧑 **Agent sessions** (parallel agents) | `session_create`, `session_close`, `session_save`, `session_list`, `session_status` |
 
 `start_crawl` accepts a `startUrl` plus optional whitelisted overrides (scope, maxDepth, maxUrls, maxConcurrency, maxRps, crawlDelayMs, requestTimeoutMs, respectRobotsTxt, followRedirects, crawlExternal, userAgent, include/excludePatterns) — anything you don't override keeps the desktop user's saved value. Crawls launched via MCP go through the **same code path** as the UI's Start button, so progress shows up in the desktop app live as the agent drives it. Call `clear_crawl` first when you want a fresh BFS after a completed crawl with the same seed URL (otherwise the crawler treats the re-start as a resume and exits because every URL is already in the DB).
 
+**Several agents at once.** Multiple MCP clients can drive one running desktop app safely. Crawl control is a *lease*: `start_crawl` returns a `crawlId` you own, a second start against a busy session fails with `crawl-in-progress` (or, with `onBusy: "queue"`, waits its turn — or `"takeover"` to replace it), and `stop`/`pause`/`resume` refuse to touch a crawl owned by another client unless you pass `force: true`. `clear_crawl` won't wipe the desktop window's project when it has unsaved work. For full isolation, call **`session_create`** to get your own headless session inside the same app — its own database, crawler and worker pool — so each agent crawls its own project without seeing the others; the MCP read tools automatically repoint to it. `session_create({ mode: "open", projectPath })` reopens a saved `.seoproject` to keep working on it (PageSpeed, spelling, exports, re-crawl), and `session_save` / `session_close({ save: true })` persist it — so the data no longer disappears when the session that produced it ends. Sessions are capped and idle ones are auto-closed (saving first if they have a document).
+
 **1. Build it once:**
 
 ```bash

apps/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/cli",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "type": "module",
   "bin": {

apps/desktop/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/desktop",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "description": "FreeCrawl SEO — desktop application",
   "homepage": "https://github.com/kemalai/FreeCrawl-SEO-Tool",

apps/desktop/src/main/agent-session-registry.ts

@@ -0,0 +1,307 @@
+/**
+ * Crawl-lease arbitration for the MCP bridge (Issue #12, Faz 1).
+ *
+ * Several MCP clients — typically parallel autonomous agents — drive crawls
+ * against the same running desktop app. Without arbitration a second
+ * `start_crawl` silently stops the first crawler and takes its place, so two
+ * callers' results interleave in one project. This registry makes the crawl
+ * slot a *lease*: at most one live crawl per session, with a FIFO queue and
+ * explicit ownership so a control call from another client can be refused.
+ *
+ * The module is deliberately Electron-free and side-effect-free (timers and
+ * id/clock sources are injectable) so it unit-tests without spinning up the
+ * app. The desktop `index.ts` owns the actual crawler; this only decides who
+ * is allowed to hold the slot and when a queued waiter is released.
+ *
+ * Faz 2 will grow session lifecycle (create/close/list) onto the same
+ * registry; the lease map is already keyed by an arbitrary `sessionId` so
+ * headless sessions slot in without a redesign.
+ */
+
+import { randomBytes } from 'node:crypto';
+import type { BridgeErrorCode, OnBusy } from '@freecrawl/shared-types';
+
+export interface CrawlLease {
+  crawlId: string;
+  ownerClientId: string | null;
+  startUrl: string;
+  startedAt: number;
+}
+
+/** Immediate outcome of an `acquireCrawl` — a granted lease or a typed error. */
+export type AcquireResult =
+  | { ok: true; crawlId: string; waitedMs: number; queued: boolean; tookOver: boolean }
+  | {
+      ok: false;
+      error: BridgeErrorCode;
+      message: string;
+      details?: Record<string, unknown>;
+    };
+
+export interface AcquireOptions {
+  clientId: string | null;
+  startUrl: string;
+  onBusy?: OnBusy;
+  queueTimeoutMs?: number;
+}
+
+interface QueueWaiter {
+  clientId: string | null;
+  startUrl: string;
+  enqueuedAt: number;
+  resolve: (r: AcquireResult) => void;
+  timer: unknown | null;
+}
+
+interface SessionSlot {
+  lease: CrawlLease | null;
+  queue: QueueWaiter[];
+}
+
+export interface RegistryOptions {
+  /** FIFO queue cap per session before `session-limit-reached`. */
+  maxQueueDepth?: number;
+  /** Injectable clock (defaults to Date.now) — keeps tests deterministic. */
+  now?: () => number;
+  /** Injectable crawl-id factory. */
+  genCrawlId?: () => string;
+  /** Injectable timer pair (defaults to global setTimeout/clearTimeout). */
+  setTimer?: (fn: () => void, ms: number) => unknown;
+  clearTimer?: (handle: unknown) => void;
+}
+
+const DEFAULT_MAX_QUEUE_DEPTH = 4;
+const DEFAULT_QUEUE_TIMEOUT_MS = 15 * 60_000;
+
+/**
+ * Pure: the uniform throttle scale (0.1–1) to apply to every live crawler so
+ * the summed concurrency fits a shared budget. Returns 1 when it already fits
+ * or nothing is running — so a single crawl (the normal single-user case) is
+ * never throttled by the multi-session budget. Lives here (not in the
+ * Electron-bound main module) so it unit-tests standalone.
+ */
+export function computeGlobalThrottleScale(
+  concurrencies: number[],
+  budget: number,
+): number {
+  const total = concurrencies.reduce((a, b) => a + b, 0);
+  if (total <= 0 || total <= budget) return 1;
+  return Math.max(0.1, budget / total);
+}
+
+function defaultCrawlId(): string {
+  return `cr_${randomBytes(4).toString('hex')}`;
+}
+
+export class SessionRegistry {
+  private readonly slots = new Map<string, SessionSlot>();
+  private readonly maxQueueDepth: number;
+  private readonly now: () => number;
+  private readonly genCrawlId: () => string;
+  private readonly setTimer: (fn: () => void, ms: number) => unknown;
+  private readonly clearTimer: (handle: unknown) => void;
+
+  constructor(opts: RegistryOptions = {}) {
+    this.maxQueueDepth = opts.maxQueueDepth ?? DEFAULT_MAX_QUEUE_DEPTH;
+    this.now = opts.now ?? (() => Date.now());
+    this.genCrawlId = opts.genCrawlId ?? defaultCrawlId;
+    this.setTimer =
+      opts.setTimer ?? ((fn, ms) => setTimeout(fn, ms));
+    this.clearTimer =
+      opts.clearTimer ?? ((h) => clearTimeout(h as ReturnType<typeof setTimeout>));
+  }
+
+  private slot(sessionId: string): SessionSlot {
+    let s = this.slots.get(sessionId);
+    if (!s) {
+      s = { lease: null, queue: [] };
+      this.slots.set(sessionId, s);
+    }
+    return s;
+  }
+
+  /** Current lease for a session, or null when the crawl slot is free. */
+  peekLease(sessionId: string): CrawlLease | null {
+    return this.slots.get(sessionId)?.lease ?? null;
+  }
+
+  /** True when `clientId` holds the live lease for `sessionId`. A null
+   *  client (headerless legacy call) is treated as the owner of a
+   *  null-owned lease so single-client setups keep working. */
+  isOwner(sessionId: string, clientId: string | null, crawlId?: string): boolean {
+    const lease = this.peekLease(sessionId);
+    if (!lease) return false;
+    if (crawlId !== undefined && lease.crawlId !== crawlId) return false;
+    if (lease.ownerClientId === null) return true;
+    return lease.ownerClientId === clientId;
+  }
+
+  /**
+   * Try to take the crawl slot for a session.
+   *
+   * - Free slot → immediate grant.
+   * - Busy + `reject` (default) → immediate `crawl-in-progress` error.
+   * - Busy + `takeover` → force-drop the current lease and grant. The caller
+   *   is responsible for stopping the superseded crawler; its later
+   *   release call is a no-op because the crawlId no longer matches.
+   * - Busy + `queue` → FIFO wait, resolved when the slot frees or rejected
+   *   with `queue-timeout` / `session-limit-reached`.
+   */
+  acquireCrawl(sessionId: string, opts: AcquireOptions): Promise<AcquireResult> {
+    const slot = this.slot(sessionId);
+    const onBusy: OnBusy = opts.onBusy ?? 'reject';
+
+    if (!slot.lease) {
+      return Promise.resolve(this.grant(slot, opts, 0, false, false));
+    }
+
+    if (onBusy === 'reject') {
+      const lease = slot.lease;
+      return Promise.resolve({
+        ok: false,
+        error: 'crawl-in-progress',
+        message:
+          `A crawl is already running in session '${sessionId}' ` +
+          `(crawlId=${lease.crawlId}, started ${Math.round(
+            (this.now() - lease.startedAt) / 1000,
+          )}s ago). Call start_crawl with onBusy:'queue' to wait for it, ` +
+          `onBusy:'takeover' to replace it, or session_create to get your own isolated session.`,
+        details: {
+          sessionId,
+          crawlId: lease.crawlId,
+          ownerClientId: lease.ownerClientId,
+          startUrl: lease.startUrl,
+          startedAt: lease.startedAt,
+        },
+      });
+    }
+
+    if (onBusy === 'takeover') {
+      // Drop the current lease outright; the superseded crawler is stopped by
+      // the caller and its release becomes a stale no-op. The queue is left
+      // intact — a takeover jumps the running crawl, not the waiting line.
+      slot.lease = null;
+      return Promise.resolve(this.grant(slot, opts, 0, false, true));
+    }
+
+    // onBusy === 'queue'
+    if (slot.queue.length >= this.maxQueueDepth) {
+      return Promise.resolve({
+        ok: false,
+        error: 'session-limit-reached',
+        message:
+          `The crawl queue for session '${sessionId}' is full ` +
+          `(${this.maxQueueDepth} waiting). Wait for it to drain or create your ` +
+          `own session with session_create.`,
+        details: { sessionId, maxQueueDepth: this.maxQueueDepth },
+      });
+    }
+
+    const timeoutMs =
+      opts.queueTimeoutMs && opts.queueTimeoutMs > 0
+        ? opts.queueTimeoutMs
+        : DEFAULT_QUEUE_TIMEOUT_MS;
+    return new Promise<AcquireResult>((resolve) => {
+      const waiter: QueueWaiter = {
+        clientId: opts.clientId,
+        startUrl: opts.startUrl,
+        enqueuedAt: this.now(),
+        resolve,
+        timer: null,
+      };
+      waiter.timer = this.setTimer(() => {
+        const idx = slot.queue.indexOf(waiter);
+        if (idx !== -1) slot.queue.splice(idx, 1);
+        resolve({
+          ok: false,
+          error: 'queue-timeout',
+          message:
+            `Waited ${Math.round(timeoutMs / 1000)}s for the crawl slot in ` +
+            `session '${sessionId}' without it freeing. Retry later or use ` +
+            `session_create for an isolated session.`,
+          details: { sessionId, queueTimeoutMs: timeoutMs },
+        });
+      }, timeoutMs);
+      slot.queue.push(waiter);
+    });
+  }
+
+  private grant(
+    slot: SessionSlot,
+    opts: AcquireOptions,
+    waitedMs: number,
+    queued: boolean,
+    tookOver: boolean,
+  ): AcquireResult {
+    const crawlId = this.genCrawlId();
+    slot.lease = {
+      crawlId,
+      ownerClientId: opts.clientId,
+      startUrl: opts.startUrl,
+      startedAt: this.now(),
+    };
+    return { ok: true, crawlId, waitedMs, queued, tookOver };
+  }
+
+  /**
+   * Release the lease identified by `crawlId` and hand the slot to the next
+   * queued waiter, if any. A `crawlId` that no longer matches the live lease
+   * (a superseded/stale crawler firing its terminal event) is ignored, so
+   * releases are safe to call from every crawl exit path.
+   */
+  releaseCrawl(sessionId: string, crawlId: string): void {
+    const slot = this.slots.get(sessionId);
+    if (!slot || !slot.lease || slot.lease.crawlId !== crawlId) return;
+    slot.lease = null;
+    this.promoteNext(slot);
+  }
+
+  /** Drop whatever lease a session currently holds (defensive cleanup for a
+   *  crawler that died without firing a terminal event) and promote the
+   *  queue. Returns the crawlId that was released, or null. */
+  forceRelease(sessionId: string): string | null {
+    const slot = this.slots.get(sessionId);
+    if (!slot || !slot.lease) return null;
+    const released = slot.lease.crawlId;
+    slot.lease = null;
+    this.promoteNext(slot);
+    return released;
+  }
+
+  private promoteNext(slot: SessionSlot): void {
+    const next = slot.queue.shift();
+    if (!next) return;
+    if (next.timer !== null) this.clearTimer(next.timer);
+    const waitedMs = this.now() - next.enqueuedAt;
+    const result = this.grant(
+      slot,
+      { clientId: next.clientId, startUrl: next.startUrl },
+      waitedMs,
+      true,
+      false,
+    );
+    next.resolve(result);
+  }
+
+  /** Clear a session entirely — lease dropped, every queued waiter rejected
+   *  with `session-expired`. Used when a session is closed/reaped. */
+  dropSession(sessionId: string): void {
+    const slot = this.slots.get(sessionId);
+    if (!slot) return;
+    for (const waiter of slot.queue) {
+      if (waiter.timer !== null) this.clearTimer(waiter.timer);
+      waiter.resolve({
+        ok: false,
+        error: 'session-expired',
+        message: `Session '${sessionId}' was closed while you waited for the crawl slot.`,
+        details: { sessionId },
+      });
+    }
+    this.slots.delete(sessionId);
+  }
+
+  /** Number of waiters queued behind a session's current crawl. */
+  queueDepth(sessionId: string): number {
+    return this.slots.get(sessionId)?.queue.length ?? 0;
+  }
+}

apps/desktop/src/main/mcp-bridge.ts

@@ -48,93 +48,147 @@ import { createServer, type Server, type IncomingMessage, type ServerResponse }
 import { randomBytes } from 'node:crypto';
 import { writeFileSync, unlinkSync, chmodSync, existsSync } from 'node:fs';
 import { join } from 'node:path';
-import type { CrawlConfig, CrawlProgress } from '@freecrawl/shared-types';
+import type {
+  BridgeCapabilities,
+  BridgeErrorBody,
+  BridgeErrorCode,
+  BridgeProgressResult,
+  BridgeSessionCloseInput,
+  BridgeSessionCloseResult,
+  BridgeSessionCreateInput,
+  BridgeSessionCreateResult,
+  BridgeSessionInfo,
+  BridgeSessionSaveInput,
+  BridgeSessionSaveResult,
+  McpStartCrawlInput,
+  McpStartCrawlResult,
+} from '@freecrawl/shared-types';
+import {
+  BRIDGE_CLIENT_HEADER,
+  BRIDGE_SESSION_HEADER,
+  PRIMARY_SESSION_ID,
+} from '@freecrawl/shared-types';
+
+/** Which client + session an incoming request is acting on. Parsed from the
+ *  `X-FreeCrawl-*` headers; a headerless (v1) request targets `primary` with
+ *  a null client so single-client setups keep working unchanged. */
+export interface BridgeRequestContext {
+  sessionId: string;
+  clientId: string | null;
+}
+
+/** Either a typed error the route should surface, or a success value `T`. */
+export type BridgeResult<T> = T | BridgeErrorBody;
+
+function isBridgeError(v: unknown): v is BridgeErrorBody {
+  return (
+    typeof v === 'object' &&
+    v !== null &&
+    'error' in v &&
+    'message' in v &&
+    typeof (v as { message: unknown }).message === 'string'
+  );
+}
+
+/** Map each error code to its HTTP status. */
+const ERROR_STATUS: Record<BridgeErrorCode, number> = {
+  'crawl-in-progress': 409,
+  'crawl-not-owned': 409,
+  'session-not-found': 404,
+  'session-expired': 410,
+  'session-limit-reached': 429,
+  'project-locked': 409,
+  'invalid-project-path': 400,
+  'unsupported-feature': 501,
+  'queue-timeout': 504,
+  'bad-request': 400,
+  'internal-error': 500,
+};
 
 /**
  * Functions the bridge needs from the main process. Wired by the
  * caller (`index.ts`) at startup — the bridge module stays
  * decoupled from the global desktop state.
+ *
+ * Protocol v2 (Issue #12): every crawl-control call carries a
+ * `BridgeRequestContext` so the main process can route it to the right
+ * session and enforce lease ownership. A dep may return a `BridgeErrorBody`
+ * instead of its success value; the route surfaces it with the mapped
+ * HTTP status.
  */
 export interface McpBridgeDeps {
   /** Discovery + token directory — usually `app.getPath('userData')`. */
   userDataDir: string;
+  /** Static capabilities snapshot for `GET /v1/capabilities`. */
+  getCapabilities: () => BridgeCapabilities;
   /**
-   * Kick off a crawl. The caller resolves any defaults/overrides and
-   * dispatches the existing crawler IPC pipeline. Returns synchronously
-   * once the crawl is launched (not when it finishes).
+   * Kick off a crawl. Resolves defaults/overrides, arbitrates the crawl
+   * lease (reject / queue / takeover), and dispatches the crawler pipeline.
+   * Resolves once the crawl is launched (not when it finishes), or with a
+   * typed error when the slot is busy / the project is locked.
    */
-  startCrawl: (input: McpStartCrawlInput) => Promise<{ ok: true; config: CrawlConfig } | { ok: false; error: string }>;
-  stopCrawl: () => void;
-  pauseCrawl: () => void;
-  resumeCrawl: () => void;
+  startCrawl: (
+    input: McpStartCrawlInput,
+    ctx: BridgeRequestContext,
+  ) => Promise<BridgeResult<McpStartCrawlResult>>;
+  stopCrawl: (ctx: BridgeRequestContext, opts: { force?: boolean }) => BridgeResult<{ ok: true }>;
+  pauseCrawl: (ctx: BridgeRequestContext, opts: { force?: boolean }) => BridgeResult<{ ok: true }>;
+  resumeCrawl: (ctx: BridgeRequestContext, opts: { force?: boolean }) => BridgeResult<{ ok: true }>;
   /**
    * Wipe the URLs table (and all dependent rows — links / images /
    * headers / cookies / sitemap_urls / crawl_queue). Same primitive
-   * the desktop's "Clear" button calls. Required because the crawler's
-   * resume-vs-fresh decision is "same start URL → keep existing rows"
-   * — without an explicit clear, an MCP-driven re-start after a
-   * completed crawl finds zero pending work and exits immediately.
+   * the desktop's "Clear" button calls. Refuses to wipe the desktop
+   * window's project when it has unsaved user work unless `force`.
    */
-  clearCrawl: () => Promise<void>;
+  clearCrawl: (
+    ctx: BridgeRequestContext,
+    opts: { force?: boolean },
+  ) => Promise<BridgeResult<{ ok: true }>>;
   /**
-   * Generic action dispatch table — keyed by short kebab-case action
-   * name, value is the closure that performs the action. The bridge's
-   * `POST /v1/action/<name>` route looks up by name and invokes with
-   * the request body. Keeps the bridge route surface flat (one route
-   * for N actions) while letting the main process explicitly allow-list
-   * which actions MCP can drive. Closures return `unknown` so they can
-   * be wired up to any util-function shape; the MCP server is the
-   * point that gives each action a typed contract.
+   * Generic action dispatch — `POST /v1/action/<name>` routes here. Split
+   * into a membership check and a session-scoped runner so an action's DB
+   * reads/writes land on the request's session (a headless agent exports
+   * from ITS project, not the desktop window's).
    */
-  actions: Record<string, (input: unknown) => Promise<unknown>>;
-  /**
-   * Snapshot of the latest progress event the Crawler emitted. Null
-   * when no crawl has ever run in this session. The bridge passes it
-   * verbatim to MCP — `running` / `paused` fields are authoritative.
-   */
-  getProgress: () => CrawlProgress | null;
-  /** Absolute path to the active `.seoproject`, or null if none. */
-  getProjectPath: () => string | null;
-  /** Row count for the active project — useful for confirming the
-   * MCP-side read-only view is targetting the same DB. */
-  getUrlCount: () => number;
+  hasAction: (name: string) => boolean;
+  runAction: (
+    name: string,
+    input: unknown,
+    ctx: BridgeRequestContext,
+  ) => Promise<unknown>;
+  /** Rich progress snapshot for a session: crawl counts + lease ownership. */
+  getCrawlProgress: (ctx: BridgeRequestContext) => BridgeProgressResult;
+  /** Project path + row count for a session. */
+  getProjectInfo: (
+    ctx: BridgeRequestContext,
+  ) => { projectPath: string | null; urlsCrawled: number };
+  /** Enumerate every live session (primary + windows + headless). */
+  listSessions?: () => BridgeSessionInfo[];
+  /** Create a headless agent session (scratch / open / attach). */
+  createSession?: (
+    input: BridgeSessionCreateInput,
+    ctx: BridgeRequestContext,
+  ) => Promise<BridgeResult<BridgeSessionCreateResult>>;
+  /** Close (optionally saving) the request's headless session. */
+  closeSession?: (
+    input: BridgeSessionCloseInput,
+    ctx: BridgeRequestContext,
+  ) => Promise<BridgeResult<BridgeSessionCloseResult>>;
+  /** Save the request's headless session to a `.seoproject`. */
+  saveSession?: (
+    input: BridgeSessionSaveInput,
+    ctx: BridgeRequestContext,
+  ) => Promise<BridgeResult<BridgeSessionSaveResult>>;
   /** Optional logger (info / warn / error). When omitted, the bridge
    *  swallows internal trace; only HTTP-visible errors propagate. */
   log?: (level: 'info' | 'warn' | 'error', msg: string) => void;
 }
 
-/**
- * Fields the MCP server may pass when starting a crawl. Everything is
- * optional except an effective startUrl must exist — either supplied
- * here or already saved in the last-used config. The caller layers
- * these on top of the last-used CrawlConfig and dispatches.
- */
-export interface McpStartCrawlInput {
-  startUrl?: string;
-  /** Whitelisted CrawlConfig field overrides — keeping the contract
-   *  narrow so MCP can't smuggle in fields we'd rather validate. */
-  configOverrides?: {
-    scope?: CrawlConfig['scope'];
-    maxDepth?: number;
-    maxUrls?: number;
-    maxConcurrency?: number;
-    maxRps?: number;
-    crawlDelayMs?: number;
-    requestTimeoutMs?: number;
-    respectRobotsTxt?: boolean;
-    followRedirects?: boolean;
-    crawlExternal?: boolean;
-    userAgent?: string;
-    includePatterns?: string[];
-    excludePatterns?: string[];
-  };
-}
-
 interface DiscoveryFile {
   port: number;
   token: string;
   pid: number;
-  version: 1;
+  version: 2;
 }
 
 const DISCOVERY_FILENAME = 'mcp-bridge.json';
@@ -228,7 +282,7 @@ export async function startMcpBridge(
   // reads this to find us. We `chmod 600` on POSIX so other local
   // users can't sniff the token — Windows ACLs already restrict
   // the userData directory to the current user.
-  const payload: DiscoveryFile = { port, token, pid: process.pid, version: 1 };
+  const payload: DiscoveryFile = { port, token, pid: process.pid, version: 2 };
   try {
     writeFileSync(discoveryPath, JSON.stringify(payload), { encoding: 'utf8' });
     if (process.platform !== 'win32') {
@@ -315,6 +369,25 @@ function send(res: ServerResponse, status: number, payload: unknown): void {
   res.end(JSON.stringify(payload));
 }
 
+/** Send a typed error body with the status mapped from its code. */
+function sendError(res: ServerResponse, body: BridgeErrorBody): void {
+  send(res, ERROR_STATUS[body.error] ?? 500, body);
+}
+
+/** Parse the routing headers. Absent session ⇒ `primary`; absent client ⇒
+ *  null (a legacy single-client caller), which the lease treats as owner. */
+function readContext(req: IncomingMessage): BridgeRequestContext {
+  const rawSession = req.headers[BRIDGE_SESSION_HEADER];
+  const rawClient = req.headers[BRIDGE_CLIENT_HEADER];
+  const sessionId =
+    typeof rawSession === 'string' && rawSession.trim() !== ''
+      ? rawSession.trim()
+      : PRIMARY_SESSION_ID;
+  const clientId =
+    typeof rawClient === 'string' && rawClient.trim() !== '' ? rawClient.trim() : null;
+  return { sessionId, clientId };
+}
+
 async function handleRequest(
   req: IncomingMessage,
   res: ServerResponse,
@@ -342,17 +415,66 @@ async function handleRequest(
   }
 
   // 3) Route — flat switch; the surface is small enough that we don't
-  //    need a router library.
+  //    need a router library. Every crawl-control route resolves the
+  //    calling client + session from headers first (v1 callers → primary).
+  const ctx = readContext(req);
+
+  if (method === 'GET' && path === '/v1/capabilities') {
+    send(res, 200, deps.getCapabilities());
+    return;
+  }
   if (method === 'GET' && path === '/v1/crawl/progress') {
-    const progress = deps.getProgress();
-    send(res, 200, { progress });
+    send(res, 200, deps.getCrawlProgress(ctx));
     return;
   }
   if (method === 'GET' && path === '/v1/project') {
-    send(res, 200, {
-      projectPath: deps.getProjectPath(),
-      urlsCrawled: deps.getUrlCount(),
-    });
+    send(res, 200, deps.getProjectInfo(ctx));
+    return;
+  }
+  if (method === 'GET' && path === '/v1/session/list') {
+    if (!deps.listSessions) {
+      sendError(res, {
+        error: 'unsupported-feature',
+        message: 'Session enumeration is not available on this desktop build.',
+      });
+      return;
+    }
+    send(res, 200, { sessions: deps.listSessions() });
+    return;
+  }
+  if (
+    method === 'POST' &&
+    (path === '/v1/session/create' ||
+      path === '/v1/session/close' ||
+      path === '/v1/session/save')
+  ) {
+    const dep =
+      path === '/v1/session/create'
+        ? deps.createSession
+        : path === '/v1/session/close'
+          ? deps.closeSession
+          : deps.saveSession;
+    if (!dep) {
+      sendError(res, {
+        error: 'unsupported-feature',
+        message: 'Headless sessions are not available on this desktop build.',
+      });
+      return;
+    }
+    let body: unknown;
+    try {
+      body = await readJsonBody(req);
+    } catch (err) {
+      sendError(res, {
+        error: 'bad-request',
+        message: err instanceof Error ? err.message : 'malformed request body',
+      });
+      return;
+    }
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    const result = await dep(body as any, ctx);
+    if (isBridgeError(result)) sendError(res, result);
+    else send(res, 200, result);
     return;
   }
   if (method === 'POST' && path === '/v1/crawl/start') {
@@ -360,70 +482,83 @@ async function handleRequest(
     try {
       body = await readJsonBody(req);
     } catch (err) {
-      send(res, 400, { error: err instanceof Error ? err.message : 'bad-body' });
-      return;
-    }
-    const input = body as McpStartCrawlInput;
-    const result = await deps.startCrawl(input);
-    if (!result.ok) {
-      send(res, 400, { error: result.error });
-      return;
-    }
-    send(res, 200, { ok: true, config: result.config });
-    return;
-  }
-  if (method === 'POST' && path === '/v1/crawl/stop') {
-    deps.stopCrawl();
-    send(res, 200, { ok: true });
-    return;
-  }
-  if (method === 'POST' && path === '/v1/crawl/pause') {
-    deps.pauseCrawl();
-    send(res, 200, { ok: true });
-    return;
-  }
-  if (method === 'POST' && path === '/v1/crawl/resume') {
-    deps.resumeCrawl();
-    send(res, 200, { ok: true });
-    return;
-  }
-  if (method === 'POST' && path === '/v1/crawl/clear') {
-    try {
-      await deps.clearCrawl();
-      send(res, 200, { ok: true });
-    } catch (err) {
-      send(res, 500, {
-        error: err instanceof Error ? err.message : 'clear-failed',
+      sendError(res, {
+        error: 'bad-request',
+        message: err instanceof Error ? err.message : 'malformed request body',
       });
+      return;
     }
+    const result = await deps.startCrawl(body as McpStartCrawlInput, ctx);
+    if (isBridgeError(result)) {
+      sendError(res, result);
+      return;
+    }
+    send(res, 200, result);
+    return;
+  }
+  if (
+    method === 'POST' &&
+    (path === '/v1/crawl/stop' ||
+      path === '/v1/crawl/pause' ||
+      path === '/v1/crawl/resume' ||
+      path === '/v1/crawl/clear')
+  ) {
+    // These carry an optional `{ force: true }` to override ownership /
+    // unsaved-work guards. Body is tiny and optional.
+    let body: unknown = {};
+    try {
+      body = await readJsonBody(req);
+    } catch (err) {
+      sendError(res, {
+        error: 'bad-request',
+        message: err instanceof Error ? err.message : 'malformed request body',
+      });
+      return;
+    }
+    const force = (body as { force?: unknown }).force === true;
+    let result: BridgeResult<{ ok: true }>;
+    if (path === '/v1/crawl/stop') result = deps.stopCrawl(ctx, { force });
+    else if (path === '/v1/crawl/pause') result = deps.pauseCrawl(ctx, { force });
+    else if (path === '/v1/crawl/resume') result = deps.resumeCrawl(ctx, { force });
+    else result = await deps.clearCrawl(ctx, { force });
+    if (isBridgeError(result)) sendError(res, result);
+    else send(res, 200, result);
     return;
   }
   if (method === 'POST' && path.startsWith('/v1/action/')) {
     const name = path.slice('/v1/action/'.length);
-    const action = deps.actions[name];
-    if (!action) {
-      send(res, 404, { error: `unknown-action: ${name}` });
+    if (!deps.hasAction(name)) {
+      sendError(res, {
+        error: 'bad-request',
+        message: `unknown action: ${name}`,
+        details: { action: name },
+      });
       return;
     }
     let body: unknown;
     try {
       body = await readJsonBody(req);
     } catch (err) {
-      send(res, 400, { error: err instanceof Error ? err.message : 'bad-body' });
+      sendError(res, {
+        error: 'bad-request',
+        message: err instanceof Error ? err.message : 'malformed request body',
+      });
       return;
     }
     try {
-      const result = await action(body);
+      const result = await deps.runAction(name, body, ctx);
       send(res, 200, result === undefined ? { ok: true } : result);
     } catch (err) {
-      send(res, 500, {
-        error: err instanceof Error ? err.message : 'action-failed',
+      sendError(res, {
+        error: 'internal-error',
+        message: err instanceof Error ? err.message : 'action failed',
+        details: { action: name },
       });
     }
     return;
   }
 
-  send(res, 404, { error: 'not-found' });
+  sendError(res, { error: 'bad-request', message: `no such route: ${method} ${path}` });
 }
 
 /**

apps/desktop/src/preload/index.ts

@@ -1,6 +1,7 @@
 import { contextBridge, ipcRenderer, type IpcRendererEvent } from 'electron';
 import {
   IPC,
+  type BridgeSessionInfo,
   type BrowserInstallState,
   type ConfirmClearResult,
   type CrawlConfig,
@@ -208,6 +209,12 @@ const api: FreeCrawlApi = {
     ipcRenderer.invoke(IPC.browserInstallStart),
   onBrowserInstallState: (cb: (state: BrowserInstallState) => void) =>
     subscribe<BrowserInstallState>(IPC.browserInstallState, cb),
+  agentsList: (): Promise<BridgeSessionInfo[]> =>
+    ipcRenderer.invoke(IPC.agentsList),
+  agentsClose: (sessionId: string): Promise<{ ok: boolean }> =>
+    ipcRenderer.invoke(IPC.agentsClose, sessionId),
+  onAgentsChanged: (cb: () => void) =>
+    subscribe<void>(IPC.agentsChanged, () => cb()),
   recentProjectsList: (): Promise<RecentProject[]> =>
     ipcRenderer.invoke(IPC.recentProjectsList),
   recentProjectSetArchived: (path: string, archived: boolean): Promise<void> =>

apps/desktop/src/renderer/src/App.tsx

@@ -250,6 +250,17 @@ export function App() {
     const offConfig = window.freecrawl.onProjectConfigChanged((cfg) => {
       if (cfg) applyProjectConfig(cfg);
     });
+    // The push above is missed on a cold start: double-clicking a
+    // `.seoproject` opens the project in the main process before this
+    // renderer has mounted and subscribed, so `projectConfigChanged` fires
+    // into the void and the URL bar / mode come up blank even though the
+    // data loaded. Pull the already-open project's config once on mount to
+    // close that race; the subscription still covers projects opened later
+    // while the app is running. Returns null on a fresh scratch session, so
+    // this is a no-op when there's nothing to restore.
+    void window.freecrawl.projectConfigGet().then((cfg) => {
+      if (cfg) applyProjectConfig(cfg);
+    });
     const off4 = window.freecrawl.onMenuEvent((event: MenuEvent) => {
       switch (event) {
         case 'edit-copy': {

apps/desktop/src/renderer/src/components/AdvancedFilterDialog.tsx

@@ -80,11 +80,15 @@ export function AdvancedFilterDialog({
   initial,
   onClose,
   onApply,
+  extractionFields = [],
 }: {
   open: boolean;
   initial: AdvancedFilter | null;
   onClose: () => void;
   onApply: (filter: AdvancedFilter | null) => void;
+  /** Names of the active custom-extraction rules, offered as filterable
+   *  fields ("Extract: <name>") in addition to the built-in columns. */
+  extractionFields?: string[];
 }) {
   const { t, i18n } = useTranslation();
   const lang = i18n.language;
@@ -212,6 +216,7 @@ export function AdvancedFilterDialog({
                     clause={clause}
                     showAndLabel={ci > 0}
                     lang={lang}
+                    extractionFields={extractionFields}
                     onChange={(patch) => updateClause(gi, ci, patch)}
                     onDelete={() => deleteClause(gi, ci)}
                   />
@@ -267,23 +272,34 @@ export function AdvancedFilterDialog({
   );
 }
 
+const EXTRACTION_PREFIX = 'extraction:';
+
 function ClauseRow({
   clause,
   showAndLabel,
   lang,
+  extractionFields,
   onChange,
   onDelete,
 }: {
   clause: FilterClause;
   showAndLabel: boolean;
   lang: string;
+  extractionFields: string[];
   onChange: (patch: Partial<FilterClause>) => void;
   onDelete: () => void;
 }) {
   const { t } = useTranslation();
-  const numeric = isNumericField(clause.field);
+  // Extraction fields are always text; built-in fields keep their own kind.
+  const numeric = clause.field !== 'extraction' && isNumericField(clause.field);
   const operators = numeric ? NUMERIC_OPS : TEXT_OPS;
   const needsValue = !NO_VALUE_OPS.includes(clause.operator);
+  // The <select> value for an extraction clause encodes the rule name so the
+  // dropdown can distinguish one extraction field from another.
+  const fieldValue =
+    clause.field === 'extraction'
+      ? `${EXTRACTION_PREFIX}${clause.extractionKey ?? ''}`
+      : clause.field;
 
   return (
     <div className="flex items-center gap-2">
@@ -297,15 +313,29 @@ function ClauseRow({
       </span>
       <select
         className="input w-52"
-        value={clause.field}
+        value={fieldValue}
         onChange={(e) => {
-          const field = e.target.value as FilterField;
+          const v = e.target.value;
+          if (v.startsWith(EXTRACTION_PREFIX)) {
+            // Extraction fields are text-only; keep the operator if it's a
+            // valid text operator, else fall back to the first text op.
+            const opStillValid = TEXT_OPS.some((o) => o.value === clause.operator);
+            onChange({
+              field: 'extraction',
+              extractionKey: v.slice(EXTRACTION_PREFIX.length),
+              operator: opStillValid ? clause.operator : TEXT_OPS[0]!.value,
+            });
+            return;
+          }
+          const field = v as FilterField;
           // Switching field may make the current operator invalid — reset
-          // to a safe default for the new field type.
+          // to a safe default for the new field type. Clearing extractionKey
+          // keeps the clause well-formed when leaving an extraction field.
           const nextOps = isNumericField(field) ? NUMERIC_OPS : TEXT_OPS;
           const opStillValid = nextOps.some((o) => o.value === clause.operator);
           onChange({
             field,
+            extractionKey: undefined,
             operator: opStillValid ? clause.operator : nextOps[0]!.value,
           });
         }}
@@ -315,6 +345,17 @@ function ClauseRow({
             {translateLabel(f.label, lang)}
           </option>
         ))}
+        {extractionFields.length > 0 && (
+          <optgroup
+            label={translateLabel('Custom Extraction', lang)}
+          >
+            {extractionFields.map((name) => (
+              <option key={`${EXTRACTION_PREFIX}${name}`} value={`${EXTRACTION_PREFIX}${name}`}>
+                {`Extract: ${name}`}
+              </option>
+            ))}
+          </optgroup>
+        )}
       </select>
       <select
         className="input w-48"

apps/desktop/src/renderer/src/components/AgentSessionsIndicator.tsx

@@ -0,0 +1,159 @@
+import { useEffect, useRef, useState } from 'react';
+import clsx from 'clsx';
+import { useTranslation } from 'react-i18next';
+import type { BridgeSessionInfo } from '@freecrawl/shared-types';
+import { useAgentSessions } from '../hooks/useAgentSessions.js';
+
+/** Compact "N ago" for the idle column. */
+function formatIdle(lastUsedAt: number): string {
+  const ms = Date.now() - lastUsedAt;
+  const s = Math.floor(ms / 1000);
+  if (s < 60) return `${s}s`;
+  const m = Math.floor(s / 60);
+  if (m < 60) return `${m}m`;
+  const h = Math.floor(m / 60);
+  return `${h}h`;
+}
+
+function basename(p: string | null): string | null {
+  if (!p) return null;
+  const parts = p.split(/[\\/]/);
+  return parts[parts.length - 1] || p;
+}
+
+function SessionRow({
+  s,
+  onClose,
+}: {
+  s: BridgeSessionInfo;
+  onClose: (id: string) => void;
+}) {
+  const { t } = useTranslation();
+  const doc = basename(s.documentPath);
+  const running = s.crawl?.running ?? false;
+  const paused = s.crawl?.paused ?? false;
+  return (
+    <div className="flex items-center gap-2 border-b border-surface-800/70 px-3 py-2 last:border-b-0">
+      <span
+        className={clsx(
+          'h-2 w-2 shrink-0 rounded-full',
+          running && !paused && 'animate-pulse bg-emerald-500',
+          running && paused && 'bg-amber-500',
+          !running && 'bg-surface-600',
+        )}
+        title={
+          running
+            ? paused
+              ? t('agents.statusPaused', { defaultValue: 'Paused' })
+              : t('agents.statusRunning', { defaultValue: 'Crawling' })
+            : t('agents.statusIdle', { defaultValue: 'Idle' })
+        }
+      />
+      <div className="min-w-0 flex-1">
+        <div className="flex items-center gap-2">
+          <span className="truncate text-[12px] font-medium text-surface-100">{s.label}</span>
+          <span className="shrink-0 font-mono text-[10px] text-surface-500">
+            {s.urlsCrawled.toLocaleString()} {t('agents.urls', { defaultValue: 'URLs' })}
+          </span>
+        </div>
+        <div className="truncate text-[10px] text-surface-500">
+          {doc ?? t('agents.scratchProject', { defaultValue: 'unsaved scratch project' })}
+          {' · '}
+          {t('agents.idleFor', { defaultValue: 'idle {{d}}', d: formatIdle(s.lastUsedAt) })}
+        </div>
+      </div>
+      <button
+        className="shrink-0 rounded border border-surface-700 px-2 py-0.5 text-[10px] text-surface-300 hover:bg-red-900/40 hover:text-red-200"
+        onClick={() => onClose(s.sessionId)}
+        title={t('agents.closeTitle', {
+          defaultValue: 'Close this agent session (stops its crawl and frees resources)',
+        })}
+      >
+        {t('agents.close', { defaultValue: 'Close' })}
+      </button>
+    </div>
+  );
+}
+
+/**
+ * Status-bar indicator for parallel MCP agent sessions (Issue #12). Hidden
+ * when no agent has created a session (the common single-user case). Shows a
+ * badge with the session count + a live dot for running crawls; clicking opens
+ * a popover listing each session with a Close button.
+ */
+export function AgentSessionsIndicator() {
+  const { t } = useTranslation();
+  const { sessions, runningCount, close } = useAgentSessions();
+  const [open, setOpen] = useState(false);
+  const ref = useRef<HTMLDivElement>(null);
+
+  // Close the popover on an outside click / Escape.
+  useEffect(() => {
+    if (!open) return;
+    const onDown = (e: MouseEvent) => {
+      if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
+    };
+    const onKey = (e: KeyboardEvent) => {
+      if (e.key === 'Escape') setOpen(false);
+    };
+    document.addEventListener('mousedown', onDown);
+    document.addEventListener('keydown', onKey);
+    return () => {
+      document.removeEventListener('mousedown', onDown);
+      document.removeEventListener('keydown', onKey);
+    };
+  }, [open]);
+
+  if (sessions.length === 0) return null;
+
+  return (
+    <div className="relative" ref={ref}>
+      <button
+        className={clsx(
+          'inline-flex items-center gap-1.5 rounded px-1.5 py-0.5',
+          open ? 'bg-surface-800' : 'hover:bg-surface-800/60',
+        )}
+        onClick={() => setOpen((v) => !v)}
+        title={t('agents.indicatorTitle', {
+          defaultValue: '{{n}} MCP agent session(s) active — click to manage',
+          n: sessions.length,
+        })}
+      >
+        <span
+          className={clsx(
+            'h-2 w-2 rounded-full',
+            runningCount > 0 ? 'animate-pulse bg-sky-400' : 'bg-sky-500/60',
+          )}
+        />
+        <span className="text-sky-300">
+          {t('agents.badge', { defaultValue: '{{n}} agents', n: sessions.length })}
+        </span>
+        {runningCount > 0 && (
+          <span className="font-mono text-sky-400/80">
+            {t('agents.runningBadge', { defaultValue: '{{n}} crawling', n: runningCount })}
+          </span>
+        )}
+      </button>
+
+      {open && (
+        <div className="absolute bottom-full right-0 z-50 mb-1.5 w-80 overflow-hidden rounded-md border border-surface-700 bg-surface-900 shadow-xl">
+          <div className="border-b border-surface-800 px-3 py-2">
+            <div className="text-[11px] font-semibold text-surface-200">
+              {t('agents.popoverTitle', { defaultValue: 'MCP Agent Sessions' })}
+            </div>
+            <div className="text-[10px] text-surface-500">
+              {t('agents.popoverSubtitle', {
+                defaultValue: 'Isolated crawl sessions driven by AI agents over MCP.',
+              })}
+            </div>
+          </div>
+          <div className="max-h-72 overflow-y-auto">
+            {sessions.map((s) => (
+              <SessionRow key={s.sessionId} s={s} onClose={close} />
+            ))}
+          </div>
+        </div>
+      )}
+    </div>
+  );
+}

apps/desktop/src/renderer/src/components/BottomDetailPanel.tsx

@@ -55,9 +55,30 @@ type SubTab =
   | 'analytics'
   | 'spelling';
 
+/**
+ * Canonical toolbar strip for every detail-panel sub-tab. Giving each tab the
+ * same header band — identical background, border and height, even when the
+ * tab has no controls — stops the panel from looking ragged (or losing the
+ * strip entirely) as you switch between tabs. `min-h` keeps an empty bar the
+ * same height as a populated one.
+ */
+const SUBTAB_BAR_CLASS =
+  'flex min-h-[29px] shrink-0 items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-500';
+
+function SubTabBar({
+  children,
+  className,
+}: {
+  children?: ReactNode;
+  className?: string;
+}) {
+  return <div className={clsx(SUBTAB_BAR_CLASS, className)}>{children}</div>;
+}
+
 /**
  * Right-aligned Export control shared by the detail-panel sub-tabs, with
- * optional status text (e.g. "Showing 500 of 1,703") on the left.
+ * optional status text (e.g. "Showing 500 of 1,703") on the left. Rendered
+ * inside the canonical {@link SubTabBar} so it reads as the same strip.
  */
 function SubTabExportBar({
   fileName,
@@ -73,7 +94,7 @@ function SubTabExportBar({
   children?: ReactNode;
 }) {
   return (
-    <div className="flex shrink-0 items-center gap-3 px-3 pt-2 text-[11px] text-surface-500">
+    <SubTabBar>
       {children}
       <GridExportButton
         className="ml-auto"
@@ -82,7 +103,7 @@ function SubTabExportBar({
         disabled={disabled}
         getData={getData}
       />
-    </div>
+    </SubTabBar>
   );
 }
 
@@ -651,24 +672,27 @@ function SpellingView({
   }
   if (data.matches.length === 0) {
     return (
-      <div className="p-3 text-[11px] text-emerald-400">
-        {/* Saying "no grammar issues" would be a claim the offline
-            dictionary never made — it only ever looked at spelling. */}
-        {data.engine === 'local'
-          ? t('spelling.cleanLocal', {
-              defaultValue:
-                'No spelling mistakes found. Grammar was not checked — LanguageTool has no rules for this language, so an offline dictionary was used instead.',
-            })
-          : t('spelling.clean', {
-              defaultValue: 'No spelling or grammar issues found.',
-            })}
+      <div className="flex h-full flex-col">
+        <SubTabBar />
+        <div className="p-3 text-[11px] text-emerald-400">
+          {/* Saying "no grammar issues" would be a claim the offline
+              dictionary never made — it only ever looked at spelling. */}
+          {data.engine === 'local'
+            ? t('spelling.cleanLocal', {
+                defaultValue:
+                  'No spelling mistakes found. Grammar was not checked — LanguageTool has no rules for this language, so an offline dictionary was used instead.',
+              })
+            : t('spelling.clean', {
+                defaultValue: 'No spelling or grammar issues found.',
+              })}
+        </div>
       </div>
     );
   }
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex items-center gap-3 border-b border-surface-800 px-3 py-1.5 text-[10px] text-surface-400">
+      <div className="flex items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[10px] text-surface-400">
         <span>
           {t('spelling.summary', {
             defaultValue: '{{n}} finding(s)',
@@ -1070,15 +1094,18 @@ function ScreenshotView({ urlId }: { urlId: number | null }) {
   }
   if (noneCaptured) {
     return (
-      <div className="p-4 text-[11px] text-surface-500">
-        {t('screenshot.empty', {
-          defaultValue: 'No screenshots stored for this URL.',
-        })}
-        <div className="mt-1 text-[10px] text-surface-600">
-          {t('screenshot.emptyHint', {
-            defaultValue:
-              'Screenshots are only captured when Rendering Mode = JavaScript Rendering and Screenshot Capture is enabled in Settings → Rendering. Re-crawl the URL with those options on.',
+      <div className="flex h-full flex-col">
+        <SubTabBar />
+        <div className="p-4 text-[11px] text-surface-500">
+          {t('screenshot.empty', {
+            defaultValue: 'No screenshots stored for this URL.',
           })}
+          <div className="mt-1 text-[10px] text-surface-600">
+            {t('screenshot.emptyHint', {
+              defaultValue:
+                'Screenshots are only captured when Rendering Mode = JavaScript Rendering and Screenshot Capture is enabled in Settings → Rendering. Re-crawl the URL with those options on.',
+            })}
+          </div>
         </div>
       </div>
     );
@@ -1086,7 +1113,7 @@ function ScreenshotView({ urlId }: { urlId: number | null }) {
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex items-center gap-1 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px]">
+      <div className="flex items-center gap-1 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px]">
         <span className="text-surface-500">
           {t('screenshot.variant', { defaultValue: 'Variant' })}:
         </span>
@@ -1293,7 +1320,7 @@ function ViewSourceView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-2 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px]">
+      <div className="flex flex-wrap items-center gap-2 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px]">
         <span className="text-surface-500">
           {(activeLength / 1024).toFixed(1)} KB
           {src.truncated && kind === 'raw' && (
@@ -2209,7 +2236,7 @@ function NameValueView({ row, exportName }: { row: CrawlUrlRow; exportName: stri
   ];
 
   return (
-    <div className="p-3">
+    <div className="flex h-full flex-col">
       <SubTabExportBar
         fileName={`${exportName}-url-details`}
         sheetName="URL Details"
@@ -2221,6 +2248,7 @@ function NameValueView({ row, exportName }: { row: CrawlUrlRow; exportName: stri
           ]),
         })}
       />
+      <div className="flex-1 overflow-auto p-3">
       <table className="w-full text-[11px]">
         <thead className="sticky top-0 bg-surface-900">
           <tr className="text-surface-400">
@@ -2246,6 +2274,7 @@ function NameValueView({ row, exportName }: { row: CrawlUrlRow; exportName: stri
           ))}
         </tbody>
       </table>
+      </div>
     </div>
   );
 }
@@ -2663,7 +2692,7 @@ function LinksView({
       className="relative flex h-full select-none flex-col"
       onMouseDown={() => markGridActive(GRID_ID)}
     >
-      <div className="flex shrink-0 items-center gap-3 px-3 pt-2 text-[11px] text-surface-500">
+      <div className="flex min-h-[29px] shrink-0 items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-500">
         <span>
           {t('links.showing', { defaultValue: 'Showing' })}{' '}
           <span className="font-mono text-surface-200">{shown.toLocaleString()}</span>{' '}
@@ -2984,7 +3013,11 @@ function SerpSnippet({ row }: { row: CrawlUrlRow }) {
   const charsSuffix = t('serp.chars', { defaultValue: 'chars' });
 
   return (
-    <div className="p-5">
+    <div className="flex h-full flex-col">
+      {/* Empty strip so the SERP Snippet tab carries the same header band as
+          every other sub-tab (this tab has no controls of its own). */}
+      <SubTabBar />
+      <div className="flex-1 overflow-auto p-5">
       <div className="max-w-[580px] rounded border border-surface-800 bg-surface-900 p-4">
         <div className="mb-1 truncate text-[12px] text-surface-400">{displayUrl(row.url)}</div>
         <div
@@ -3010,6 +3043,7 @@ function SerpSnippet({ row }: { row: CrawlUrlRow }) {
           value={String(row.metaDescriptionLength ?? 0) + ' ' + charsSuffix}
         />
       </div>
+      </div>
     </div>
   );
 }
@@ -3247,20 +3281,23 @@ function CookiesView({
 
   if (cookies.length === 0) {
     return (
-      <div className="p-4 text-[11px] text-surface-500">
-        {t('cookies.emptyPrefix', { defaultValue: 'This page did not set any cookies (no' })}{' '}
-        <span className="font-mono">Set-Cookie</span>{' '}
-        {t('cookies.emptySuffix', { defaultValue: 'response headers).' })}
-        <div className="mt-2 text-[10px] text-surface-600">
-          {t('cookies.emptyNote', { defaultValue: 'Note: only first-party cookies set by the page itself are listed here. Cookies set by third-party scripts (analytics, ads) are set in the browser at runtime and are not visible to a static crawler.' })}
+      <div className="flex h-full flex-col">
+        <SubTabBar />
+        <div className="p-4 text-[11px] text-surface-500">
+          {t('cookies.emptyPrefix', { defaultValue: 'This page did not set any cookies (no' })}{' '}
+          <span className="font-mono">Set-Cookie</span>{' '}
+          {t('cookies.emptySuffix', { defaultValue: 'response headers).' })}
+          <div className="mt-2 text-[10px] text-surface-600">
+            {t('cookies.emptyNote', { defaultValue: 'Note: only first-party cookies set by the page itself are listed here. Cookies set by third-party scripts (analytics, ads) are set in the browser at runtime and are not visible to a static crawler.' })}
+          </div>
         </div>
       </div>
     );
   }
 
   return (
-    <div className="p-3">
-      <div className="mb-2 flex flex-wrap gap-3 text-[11px] text-surface-400">
+    <div className="flex h-full flex-col">
+      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-400">
         <span>
           <span className="font-medium text-surface-200">{cookies.length}</span> {t('cookies.cookiesSet', { defaultValue: 'cookies set' })}
         </span>
@@ -3305,6 +3342,7 @@ function CookiesView({
           })}
         />
       </div>
+      <div className="flex-1 overflow-auto p-3">
       <table className="w-full text-[11px]">
         <thead className="sticky top-0 bg-surface-900">
           <tr className="text-surface-400">
@@ -3354,6 +3392,7 @@ function CookiesView({
           ))}
         </tbody>
       </table>
+      </div>
     </div>
   );
 }
@@ -3438,7 +3477,7 @@ function StructuredDataView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px] text-surface-400">
+      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-400">
         <span>
           <span className="font-medium text-surface-200">{row.schemaBlockCount}</span> JSON-LD{' '}
           {t('structured.blocks', { defaultValue: 'block(s)' })}
@@ -3606,7 +3645,7 @@ function ImagesView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px] text-surface-400">
+      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-400">
         <span>
           <span className="font-medium text-surface-200">{rows.length}</span> {t('imagesView.images', { defaultValue: 'images' })}
         </span>
@@ -3928,7 +3967,7 @@ function ResourcesView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-1.5 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px]">
+      <div className="flex flex-wrap items-center gap-1.5 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px]">
         {FILTERS.map((f) => (
           <button
             key={f.key}
@@ -4101,7 +4140,7 @@ function MultiImagesView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px] text-surface-400">
+      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-400">
         <span>
           <span className="font-medium text-surface-200">{total.toLocaleString()}</span>{' '}
           {t('imagesView.multiSummary', { defaultValue: 'images across {{n}} pages', n: pages.length })}
@@ -4332,7 +4371,7 @@ function MultiResourcesView({
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-1.5 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px]">
+      <div className="flex flex-wrap items-center gap-1.5 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px]">
         {FILTERS.map((f) => (
           <button
             key={f.key}
@@ -4455,20 +4494,23 @@ function ExtractedDataView({
 
   if (!hasExtraction && !hasSearch) {
     return (
-      <div className="p-4 text-[11px] text-surface-500">
-        {t('extraction.empty1', { defaultValue: 'No custom extraction rules or search terms have produced data for this page.' })}
-        <div className="mt-2 text-[10px] text-surface-600">
-          {t('extraction.empty2', { defaultValue: 'Configure rules in' })}{' '}
-          <span className="font-mono">Settings → Extraction</span>{' '}
-          {t('extraction.empty3', { defaultValue: 'or search terms in' })}{' '}
-          <span className="font-mono">Settings → Custom Search</span>, {t('extraction.empty4', { defaultValue: 'then re-crawl.' })}
+      <div className="flex h-full flex-col">
+        <SubTabBar />
+        <div className="p-4 text-[11px] text-surface-500">
+          {t('extraction.empty1', { defaultValue: 'No custom extraction rules or search terms have produced data for this page.' })}
+          <div className="mt-2 text-[10px] text-surface-600">
+            {t('extraction.empty2', { defaultValue: 'Configure rules in' })}{' '}
+            <span className="font-mono">Settings → Extraction</span>{' '}
+            {t('extraction.empty3', { defaultValue: 'or search terms in' })}{' '}
+            <span className="font-mono">Settings → Custom Search</span>, {t('extraction.empty4', { defaultValue: 'then re-crawl.' })}
+          </div>
         </div>
       </div>
     );
   }
 
   return (
-    <div className="flex h-full flex-col gap-3 overflow-auto p-3">
+    <div className="flex h-full flex-col">
       <SubTabExportBar
         fileName={`${exportName}-extracted-data`}
         sheetName="Extracted Data"
@@ -4504,6 +4546,7 @@ function ExtractedDataView({
           ] as (string | number | boolean | null)[][],
         })}
       />
+      <div className="flex-1 space-y-3 overflow-auto p-3">
       {hasExtraction && (
         <section>
           <div className="mb-2 text-[10px] uppercase tracking-wide text-surface-500">
@@ -4555,6 +4598,7 @@ function ExtractedDataView({
           </table>
         </section>
       )}
+      </div>
     </div>
   );
 }
@@ -4644,7 +4688,7 @@ function OutlineView({ row, exportName }: { row: CrawlUrlRow; exportName: string
 
   return (
     <div className="flex h-full flex-col">
-      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900/50 px-3 py-1.5 text-[11px] text-surface-400">
+      <div className="flex flex-wrap items-center gap-3 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[11px] text-surface-400">
         <span>
           <span className="font-medium text-surface-200">{outline.length}</span> {t('outline.headings', { defaultValue: 'headings' })}
         </span>

apps/desktop/src/renderer/src/components/SettingsDialog.tsx

@@ -4914,6 +4914,87 @@ function SpellingPanel() {
   );
 }
 
+/**
+ * Settings for parallel MCP agent sessions (Issue #12): how many headless
+ * sessions may run at once, and how long an idle one lives before it is
+ * auto-closed (saved first if it has a document). Persisted as prefs the main
+ * process reads directly (`maxAgentSessions`, `agentSessionIdleMinutes`).
+ */
+function AgentSessionsSettings() {
+  const { t } = useTranslation();
+  const [maxSessions, setMaxSessions] = useState<number>(() => {
+    const raw = window.freecrawl.prefsGet('maxAgentSessions');
+    return typeof raw === 'number' ? Math.max(1, Math.min(8, Math.floor(raw))) : 3;
+  });
+  const [idleMinutes, setIdleMinutes] = useState<number>(() => {
+    const raw = window.freecrawl.prefsGet('agentSessionIdleMinutes');
+    return typeof raw === 'number' ? Math.max(1, Math.min(1440, Math.floor(raw))) : 30;
+  });
+
+  function applyMax(n: number) {
+    const clamped = Math.max(1, Math.min(8, Math.floor(n)));
+    setMaxSessions(clamped);
+    if (clamped === 3) window.freecrawl.prefsDelete('maxAgentSessions');
+    else window.freecrawl.prefsSet('maxAgentSessions', clamped);
+  }
+  function applyIdle(n: number) {
+    const clamped = Math.max(1, Math.min(1440, Math.floor(n)));
+    setIdleMinutes(clamped);
+    if (clamped === 30) window.freecrawl.prefsDelete('agentSessionIdleMinutes');
+    else window.freecrawl.prefsSet('agentSessionIdleMinutes', clamped);
+  }
+
+  return (
+    <div className="mb-4 rounded border border-surface-800 bg-surface-950/40 p-3">
+      <div className="mb-2 text-[10px] font-semibold uppercase tracking-wider text-surface-400">
+        {t('settings.agents.title', { defaultValue: 'Agent Sessions (MCP)' })}
+      </div>
+      <p className="mb-3 text-[11px] text-surface-400">
+        {t('settings.agents.intro', {
+          defaultValue:
+            'AI agents can open their own isolated crawl sessions inside this app over MCP. These limits keep several parallel agents from oversubscribing the machine.',
+        })}
+      </p>
+      <div className="flex flex-col gap-3">
+        <label className="flex items-center justify-between gap-3">
+          <span className="text-[12px] text-surface-100">
+            {t('settings.agents.maxSessions', { defaultValue: 'Max concurrent agent sessions' })}
+          </span>
+          <input
+            type="number"
+            min={1}
+            max={8}
+            value={maxSessions}
+            onChange={(e) => applyMax(Number(e.target.value))}
+            className="w-20 rounded border border-surface-700 bg-surface-950 px-2 py-1 text-right text-[12px] text-surface-100 focus:border-blue-500 focus:outline-none"
+          />
+        </label>
+        <label className="flex items-center justify-between gap-3">
+          <span className="text-[12px] text-surface-100">
+            {t('settings.agents.idleMinutes', {
+              defaultValue: 'Auto-close idle agent sessions after (minutes)',
+            })}
+          </span>
+          <input
+            type="number"
+            min={1}
+            max={1440}
+            value={idleMinutes}
+            onChange={(e) => applyIdle(Number(e.target.value))}
+            className="w-20 rounded border border-surface-700 bg-surface-950 px-2 py-1 text-right text-[12px] text-surface-100 focus:border-blue-500 focus:outline-none"
+          />
+        </label>
+      </div>
+      <p className="mt-2 text-[10px] text-surface-500">
+        {t('settings.agents.note', {
+          defaultValue:
+            'An idle session with a saved project is saved before it is closed. Changes apply to sessions created from now on.',
+        })}
+      </p>
+    </div>
+  );
+}
+
 function StoragePanel() {
   const { t } = useTranslation();
   const [override, setOverride] = useState<string>(() => {
@@ -5025,6 +5106,8 @@ function StoragePanel() {
         )}
       </div>
 
+      <AgentSessionsSettings />
+
       <div className="mb-4 rounded border border-surface-800 bg-surface-950/40 p-3">
         <div className="mb-2 text-[10px] font-semibold uppercase tracking-wider text-surface-400">
           {t('settings.storage.defaultSaveFolder', { defaultValue: "Default Save Folder" })}

apps/desktop/src/renderer/src/components/StatsBar.tsx

@@ -4,6 +4,7 @@ import { useAppStore } from '../store.js';
 import { usePerfMeter } from '../hooks/usePerfMeter.js';
 import { useMemoryMonitor } from '../hooks/useMemoryMonitor.js';
 import { useBrowserInstall } from '../hooks/useBrowserInstall.js';
+import { AgentSessionsIndicator } from './AgentSessionsIndicator.js';
 
 function Stat({
   label,
@@ -227,6 +228,9 @@ export function StatsBar() {
       />
 
       <div className="ml-auto flex items-center gap-2">
+        {/* Parallel MCP agent sessions (Issue #12) — hidden unless an agent
+            has created one. */}
+        <AgentSessionsIndicator />
         {/* JS-rendering browser provisioning. Silent when everything is
             already in place — only surfaces while downloading or after a
             failure, where it doubles as the retry button. */}

apps/desktop/src/renderer/src/hooks/useAgentSessions.ts

@@ -0,0 +1,48 @@
+import { useCallback, useEffect, useState } from 'react';
+import type { BridgeSessionInfo } from '@freecrawl/shared-types';
+
+/**
+ * Live list of headless MCP agent sessions (Issue #12). Fetched on mount and
+ * re-fetched whenever the main process signals a change (session created /
+ * closed, crawl state transition). Empty for the common single-user case, so
+ * the status-bar indicator hides itself.
+ */
+export function useAgentSessions(): {
+  sessions: BridgeSessionInfo[];
+  runningCount: number;
+  close: (sessionId: string) => void;
+  refresh: () => void;
+} {
+  const [sessions, setSessions] = useState<BridgeSessionInfo[]>([]);
+
+  const refresh = useCallback(() => {
+    void window.freecrawl
+      .agentsList()
+      .then(setSessions)
+      .catch(() => {
+        /* main not ready — the change push will catch us up */
+      });
+  }, []);
+
+  useEffect(() => {
+    refresh();
+    const off = window.freecrawl.onAgentsChanged(refresh);
+    return off;
+  }, [refresh]);
+
+  const close = useCallback(
+    (sessionId: string) => {
+      void window.freecrawl
+        .agentsClose(sessionId)
+        .then(refresh)
+        .catch(() => {
+          /* the change push will reconcile */
+        });
+    },
+    [refresh],
+  );
+
+  const runningCount = sessions.filter((s) => s.crawl?.running).length;
+
+  return { sessions, runningCount, close, refresh };
+}

apps/desktop/src/renderer/src/hooks/useLazyUrlRows.ts

@@ -53,8 +53,10 @@ export interface LazyRowsState {
  *
  * Strategy: when sort/filter/category changes the cache is cleared and
  * the virtualizer re-fills through `ensureRange`. While a crawl is
- * running, a 1500ms timer re-queries only the currently visible chunks
- * and REPLACES them in place — no `.clear()`, no placeholder flicker.
+ * running, a progress-driven tick re-queries the currently visible span
+ * as ONE atomic snapshot and REPLACES those chunks in place — no
+ * `.clear()`, no placeholder flicker, and no cross-chunk tearing that
+ * would duplicate a row's key across a boundary.
  * Combined with `getItemKey: row.id` on the virtualizer, rows that still
  * exist keep their DOM nodes and only their cells re-render; rows that
  * have moved (because new higher-priority rows were inserted ahead of
@@ -90,11 +92,18 @@ export function useLazyUrlRows(opts: LazyRowsOpts): LazyRowsState {
     [opts.category, opts.search, opts.sortBy, opts.sortDir, opts.filter],
   );
 
-  const queryMeta = useCallback(
-    () =>
+  // Fetch an arbitrary contiguous span in ONE query. Used to refresh the
+  // whole visible range as a single consistent snapshot: fetching chunks
+  // one-by-one with awaits between them lets the crawler insert a row
+  // mid-sort, shifting a row across a 500-boundary so the same `row.id`
+  // lands in two adjacent chunks — which the virtualizer then renders with
+  // the same key, painting two rows on top of each other. A single
+  // LIMIT/OFFSET can't tear that way.
+  const queryRange = useCallback(
+    (offset: number, limit: number) =>
       window.freecrawl.urlsQuery({
-        limit: 0,
-        offset: 0,
+        limit,
+        offset,
         category: opts.category,
         search: opts.search || undefined,
         sortBy: opts.sortBy,
@@ -148,17 +157,19 @@ export function useLazyUrlRows(opts: LazyRowsOpts): LazyRowsState {
     let cancelled = false;
 
     const load = async (): Promise<void> => {
-      const metaPromise = queryMeta();
       const { first, last } = activeRange.current;
-      const chunkPromises: Promise<{ rows: CrawlUrlRow[]; total: number; idx: number }>[] = [];
-      for (let i = first; i <= last; i++) {
-        chunkPromises.push(queryChunk(i).then((r) => ({ ...r, idx: i })));
-      }
+      const spanCount = Math.max(1, last - first + 1);
       try {
-        const [{ total: t }, ...newChunks] = await Promise.all([metaPromise, ...chunkPromises]);
+        // One consistent snapshot of the whole visible span rather than a
+        // chunk-per-query fan-out — see queryRange for why tearing across
+        // chunk boundaries duplicates row keys.
+        const { rows, total: t } = await queryRange(
+          first * CHUNK_SIZE,
+          spanCount * CHUNK_SIZE,
+        );
         if (cancelled || token !== resetToken.current) return;
-        for (const ch of newChunks) {
-          storeChunk(ch.idx, ch.rows);
+        for (let c = 0; c < spanCount; c++) {
+          storeChunk(first + c, rows.slice(c * CHUNK_SIZE, (c + 1) * CHUNK_SIZE));
         }
         setTotal(t);
         setVersion((v) => v + 1);
@@ -205,23 +216,24 @@ export function useLazyUrlRows(opts: LazyRowsOpts): LazyRowsState {
       lastTickTs = Date.now();
       const token = resetToken.current;
       try {
+        const { first, last } = activeRange.current;
+        const spanCount = Math.max(1, last - first + 1);
         try {
-          const { total: t } = await queryMeta();
+          // Atomic snapshot of the visible span: rows + total in one query
+          // so a mid-crawl insert can't shift a row across a chunk boundary
+          // and duplicate its key (the row-overlap bug). See queryRange.
+          const { rows, total: t } = await queryRange(
+            first * CHUNK_SIZE,
+            spanCount * CHUNK_SIZE,
+          );
           if (cancelled || token !== resetToken.current) return;
           setTotal(t);
+          for (let c = 0; c < spanCount; c++) {
+            storeChunk(first + c, rows.slice(c * CHUNK_SIZE, (c + 1) * CHUNK_SIZE));
+          }
         } catch {
           /* ignore transient errors */
         }
-        const { first, last } = activeRange.current;
-        for (let i = first; i <= last; i++) {
-          try {
-            const { rows } = await queryChunk(i);
-            if (cancelled || token !== resetToken.current) return;
-            storeChunk(i, rows);
-          } catch {
-            /* ignore */
-          }
-        }
         if (cancelled || token !== resetToken.current) return;
         setVersion((v) => v + 1);
       } finally {

apps/desktop/src/renderer/src/i18n/locales/en.json

@@ -99,6 +99,21 @@
     "downloadingTitle": "Downloading the Chromium browser used for JavaScript rendering. Crawling in text mode is unaffected.",
     "failed": "Browser download failed — retry"
   },
+  "agents": {
+    "badge": "{{n}} agents",
+    "runningBadge": "{{n}} crawling",
+    "indicatorTitle": "{{n}} MCP agent session(s) active — click to manage",
+    "popoverTitle": "MCP Agent Sessions",
+    "popoverSubtitle": "Isolated crawl sessions driven by AI agents over MCP.",
+    "statusRunning": "Crawling",
+    "statusPaused": "Paused",
+    "statusIdle": "Idle",
+    "urls": "URLs",
+    "scratchProject": "unsaved scratch project",
+    "idleFor": "idle {{d}}",
+    "close": "Close",
+    "closeTitle": "Close this agent session (stops its crawl and frees resources)"
+  },
   "stats": {
     "discovered": "Discovered",
     "crawled": "Crawled",
@@ -1250,7 +1265,7 @@
     "days90": "Last 90 days",
     "fetch": "Fetch",
     "fetching": "Fetching from Search Console…",
-    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} pages · fetched {{ago}}",
+    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} pages with search data · fetched {{ago}}",
     "colUrl": "URL",
     "colStatus": "Status",
     "colVerdict": "Index",
@@ -1372,7 +1387,7 @@
     "days90": "Last 90 days",
     "fetch": "Fetch",
     "fetching": "Fetching from GA4…",
-    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} pages · fetched {{ago}}",
+    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} pages with search data · fetched {{ago}}",
     "colUrl": "URL",
     "colStatus": "Status",
     "colSessions": "Sessions",
@@ -1566,6 +1581,13 @@
       "resetTitle": "Reset to the OS Documents folder",
       "note": "Active crawl data lives in the app's user-data directory until you Save As — the path above only controls where Save As starts. Changing this won't move any existing project files."
     },
+    "agents": {
+      "title": "Agent Sessions (MCP)",
+      "intro": "AI agents can open their own isolated crawl sessions inside this app over MCP. These limits keep several parallel agents from oversubscribing the machine.",
+      "maxSessions": "Max concurrent agent sessions",
+      "idleMinutes": "Auto-close idle agent sessions after (minutes)",
+      "note": "An idle session with a saved project is saved before it is closed. Changes apply to sessions created from now on."
+    },
     "privacy": {
       "title": "Privacy",
       "intro": "FreeCrawl is local-first — crawl results, project files, and URLs you visit never leave your machine.",

apps/desktop/src/renderer/src/i18n/locales/tr.json

@@ -99,6 +99,21 @@
     "downloadingTitle": "JavaScript render için kullanılan Chromium tarayıcısı indiriliyor. Metin modunda tarama bundan etkilenmez.",
     "failed": "Tarayıcı indirme başarısız — tekrar dene"
   },
+  "agents": {
+    "badge": "{{n}} ajan",
+    "runningBadge": "{{n}} tarıyor",
+    "indicatorTitle": "{{n}} MCP ajan oturumu etkin — yönetmek için tıklayın",
+    "popoverTitle": "MCP Ajan Oturumları",
+    "popoverSubtitle": "Yapay zekâ ajanlarının MCP üzerinden sürdüğü izole crawl oturumları.",
+    "statusRunning": "Tarıyor",
+    "statusPaused": "Duraklatıldı",
+    "statusIdle": "Boşta",
+    "urls": "URL",
+    "scratchProject": "kaydedilmemiş taslak proje",
+    "idleFor": "{{d}} boşta",
+    "close": "Kapat",
+    "closeTitle": "Bu ajan oturumunu kapat (crawl'ını durdurur ve kaynakları serbest bırakır)"
+  },
   "stats": {
     "discovered": "Keşfedilen",
     "crawled": "Taranan",
@@ -1250,7 +1265,7 @@
     "days90": "Son 90 gün",
     "fetch": "Getir",
     "fetching": "Search Console'dan getiriliyor…",
-    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} sayfa · getirildi {{ago}}",
+    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} arama verili sayfa · getirildi {{ago}}",
     "colUrl": "URL",
     "colStatus": "Durum",
     "colVerdict": "İndeks",
@@ -1372,7 +1387,7 @@
     "days90": "Son 90 gün",
     "fetch": "Getir",
     "fetching": "GA4'ten getiriliyor…",
-    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} sayfa · getirildi {{ago}}",
+    "metaLine": "{{property}} · {{start}} → {{end}} · {{rows}} arama verili sayfa · getirildi {{ago}}",
     "colUrl": "URL",
     "colStatus": "Durum",
     "colSessions": "Oturum",
@@ -1566,6 +1581,13 @@
       "resetTitle": "OS Belgeler klasörüne sıfırla",
       "note": "Aktif crawl verisi siz Farklı Kaydet diyene kadar uygulamanın kullanıcı veri dizininde durur — yukarıdaki yol yalnızca Farklı Kaydet'in nereden başlayacağını belirler. Bunu değiştirmek mevcut proje dosyalarını taşımaz."
     },
+    "agents": {
+      "title": "Ajan Oturumları (MCP)",
+      "intro": "Yapay zekâ ajanları MCP üzerinden bu uygulama içinde kendi izole crawl oturumlarını açabilir. Bu limitler, paralel çalışan birden çok ajanın makineyi aşırı yüklemesini engeller.",
+      "maxSessions": "Eşzamanlı en fazla ajan oturumu",
+      "idleMinutes": "Boştaki ajan oturumlarını şu süre sonra otomatik kapat (dakika)",
+      "note": "Kaydedilmiş projesi olan boştaki bir oturum, kapatılmadan önce kaydedilir. Değişiklikler bundan sonra oluşturulan oturumlara uygulanır."
+    },
     "privacy": {
       "title": "Gizlilik",
       "intro": "FreeCrawl yerel önceliklidir — crawl sonuçları, proje dosyaları ve ziyaret ettiğiniz URL'ler makinenizden hiç çıkmaz.",

apps/desktop/src/renderer/src/tabs/SearchConsoleTab.tsx

@@ -573,7 +573,7 @@ export function SearchConsoleTab() {
           <span className="ml-auto">
             {t('gscTab.metaLine', {
               defaultValue:
-                '{{property}} · {{start}} → {{end}} · {{rows}} pages · fetched {{ago}}',
+                '{{property}} · {{start}} → {{end}} · {{rows}} pages with search data · fetched {{ago}}',
               property: meta.property,
               start: meta.startDate,
               end: meta.endDate,

apps/desktop/src/renderer/src/tabs/UrlsTab.tsx

@@ -150,6 +150,16 @@ export function UrlsTab() {
   const selectedUrlId = useAppStore((s) => s.selectedUrlId);
   const setSelectedUrlId = useAppStore((s) => s.setSelectedUrlId);
   const setSelectedUrlIds = useAppStore((s) => s.setSelectedUrlIds);
+  // Active custom-extraction rule names → offered as filterable fields in
+  // the Advanced Filter dialog. Reference is stable unless the config edits.
+  const customExtractionRules = useAppStore((s) => s.config.customExtractionRules);
+  const extractionFieldNames = useMemo(
+    () =>
+      (customExtractionRules ?? [])
+        .map((r) => r.name)
+        .filter((n) => n.trim().length > 0),
+    [customExtractionRules],
+  );
   const [search, setSearch] = useState('');
   const [sortBy, setSortBy] = useState<keyof CrawlUrlRow | undefined>(undefined);
   const [sortDir, setSortDir] = useState<'asc' | 'desc'>('asc');
@@ -1637,6 +1647,7 @@ export function UrlsTab() {
       <AdvancedFilterDialog
         open={filterDialogOpen}
         initial={filter}
+        extractionFields={extractionFieldNames}
         onClose={() => setFilterDialogOpen(false)}
         onApply={(f) => setFilter(f)}
       />
@@ -2294,6 +2305,16 @@ function cellText(row: CrawlUrlRow, spec: ColumnSpec, lang: string): string {
     return label === '' ? '' : translateLabel(label, lang);
   }
 
+  // Custom-extraction column holds a `{ruleName: value}` JSON map. Showing
+  // the raw `{"h2":"…"}` in the cell reads as noise — the preview and Detail
+  // panel already unwrap it, so match them here (single rule → just the
+  // value; multiple → `name: value` pairs). Full JSON stays in the Detail
+  // panel and in exports (which format from the DB, not this path).
+  if (spec.key === 'extractionResults') {
+    const rawEx = row[spec.key];
+    return typeof rawEx === 'string' && rawEx ? formatExtractionCell(rawEx) : '';
+  }
+
   const raw = row[spec.key];
   if (raw === null || raw === undefined) return '';
   // Booleans render as "Y" / blank rather than the JS-default
@@ -2302,6 +2323,35 @@ function cellText(row: CrawlUrlRow, spec: ColumnSpec, lang: string): string {
   return String(raw);
 }
 
+/**
+ * Flatten the `extraction_results` `{ruleName: value}` map into one compact
+ * line for the table cell. Single rule → the value alone; multiple rules →
+ * `name: value` pairs joined by " · ". Arrays (multi: all/concat) join with
+ * " | ". Malformed JSON falls back to the raw string so nothing is hidden.
+ */
+function formatExtractionCell(raw: string): string {
+  let obj: Record<string, unknown>;
+  try {
+    const parsed: unknown = JSON.parse(raw);
+    if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
+      return raw;
+    }
+    obj = parsed as Record<string, unknown>;
+  } catch {
+    return raw;
+  }
+  const fmt = (v: unknown): string => {
+    if (v === null || v === undefined) return '';
+    if (Array.isArray(v)) return v.map(String).join(' | ');
+    if (typeof v === 'object') return JSON.stringify(v);
+    return String(v);
+  };
+  const entries = Object.entries(obj);
+  if (entries.length === 0) return '';
+  if (entries.length === 1) return fmt(entries[0]![1]);
+  return entries.map(([name, v]) => `${name}: ${fmt(v)}`).join('  ·  ');
+}
+
 function indexabilityStatusLabel(v: CrawlUrlRow['indexability']): string {
   switch (v) {
     case 'indexable':

apps/mcp-server/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/mcp-server",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "type": "module",
   "bin": {

apps/mcp-server/src/desktop-bridge.ts

@@ -18,10 +18,41 @@
 
 import { readFileSync, existsSync } from 'node:fs';
 import { join } from 'node:path';
+import { randomUUID } from 'node:crypto';
+import type { BridgeCapabilities } from '@freecrawl/shared-types';
+import {
+  BRIDGE_CLIENT_HEADER,
+  BRIDGE_SESSION_HEADER,
+} from '@freecrawl/shared-types';
 import { userDataDir } from './project-resolver.js';
 
 const DISCOVERY_FILENAME = 'mcp-bridge.json';
 
+/**
+ * Stable per-process id for this MCP server. Every bridge request carries it
+ * so the desktop can tell which client owns a crawl lease — one autonomous
+ * agent's MCP process is one client. Generated once at import time and never
+ * changes for the life of the process.
+ */
+export const CLIENT_ID = randomUUID();
+
+/**
+ * The headless session this MCP process is currently driving, if any. Set by
+ * `session_create`, cleared by `session_close`. Every bridge request defaults
+ * its session header to this, so once an agent creates a session all of its
+ * crawl-control + action tools route there automatically — no per-call
+ * plumbing. A null value targets the desktop window (`primary`).
+ */
+let activeSessionId: string | null = null;
+
+export function setActiveSessionId(id: string | null): void {
+  activeSessionId = id;
+}
+
+export function getActiveSessionId(): string | null {
+  return activeSessionId;
+}
+
 interface DiscoveryFile {
   port: number;
   token: string;
@@ -75,6 +106,7 @@ export async function bridgeRequest<T>(
   method: 'GET' | 'POST',
   path: string,
   body?: Record<string, unknown>,
+  opts: { sessionId?: string | null } = {},
 ): Promise<T> {
   const disc = readDiscovery();
   if (!disc) {
@@ -83,12 +115,18 @@ export async function bridgeRequest<T>(
     );
   }
   const url = `http://127.0.0.1:${disc.port}${path}`;
+  // Explicit sessionId wins; otherwise fall back to the active session. Pass
+  // `sessionId: null` to force the desktop window (primary) regardless.
+  const effectiveSession =
+    opts.sessionId !== undefined ? opts.sessionId : activeSessionId;
   let res: Response;
   try {
     res = await fetch(url, {
       method,
       headers: {
         Authorization: `Bearer ${disc.token}`,
+        [BRIDGE_CLIENT_HEADER]: CLIENT_ID,
+        ...(effectiveSession ? { [BRIDGE_SESSION_HEADER]: effectiveSession } : {}),
         ...(body !== undefined ? { 'Content-Type': 'application/json' } : {}),
       },
       body: body !== undefined ? JSON.stringify(body) : undefined,
@@ -112,22 +150,58 @@ export async function bridgeRequest<T>(
   }
 
   if (!res.ok) {
-    const errMsg =
-      parsed && typeof parsed === 'object' && 'error' in parsed
-        ? String((parsed as { error: unknown }).error)
-        : `HTTP ${res.status}`;
+    const obj =
+      parsed && typeof parsed === 'object'
+        ? (parsed as { error?: unknown; message?: unknown })
+        : null;
+    const errCode = obj && 'error' in obj ? String(obj.error) : `HTTP ${res.status}`;
+    // Prefer the bridge's agent-facing `message` (protocol v2) — it says what
+    // to do next; fall back to the bare error code on a v1 desktop.
+    const guidance =
+      obj && typeof obj.message === 'string' && obj.message.length > 0
+        ? obj.message
+        : errCode;
     if (res.status === 401) {
       throw new Error(
-        `Desktop bridge auth failed (${errMsg}). The token may be stale — ` +
+        `Desktop bridge auth failed (${errCode}). The token may be stale — ` +
           'this happens when the desktop app was restarted between MCP calls. Retry.',
       );
     }
-    throw new Error(`Desktop bridge: ${errMsg}`);
+    throw new Error(`Desktop bridge [${errCode}]: ${guidance}`);
   }
 
   return parsed as T;
 }
 
+/**
+ * Cached `GET /v1/capabilities` fetch (60 s TTL). Returns null when the
+ * desktop is a v1 build (route 404s) or unreachable — callers degrade
+ * gracefully by treating every session feature as unavailable.
+ */
+let capsCache: { at: number; value: BridgeCapabilities | null } | null = null;
+const CAPS_TTL_MS = 60_000;
+
+export async function capabilities(): Promise<BridgeCapabilities | null> {
+  const now = Date.now();
+  if (capsCache && now - capsCache.at < CAPS_TTL_MS) return capsCache.value;
+  let value: BridgeCapabilities | null = null;
+  try {
+    value = await bridgeRequest<BridgeCapabilities>('GET', '/v1/capabilities');
+  } catch {
+    // v1 desktop (no such route) or bridge down — remember the miss so we
+    // don't hammer it, and let session tools report unsupported-feature.
+    value = null;
+  }
+  capsCache = { at: now, value };
+  return value;
+}
+
+/** Whether the running desktop advertises a given bridge feature flag. */
+export async function hasFeature(name: string): Promise<boolean> {
+  const caps = await capabilities();
+  return caps?.features.includes(name) ?? false;
+}
+
 /**
  * Convenience: report whether the bridge looks reachable without
  * making a network call. Used by tools that want to surface a

apps/mcp-server/src/index.ts

@@ -21,9 +21,22 @@
 
 import fs from 'node:fs';
 import { ProjectDb } from '@freecrawl/db';
+import { SESSIONS_MIN_APP_VERSION } from '@freecrawl/shared-types';
+import type {
+  BridgeSessionCreateResult,
+  BridgeSessionCloseResult,
+  BridgeSessionSaveResult,
+  BridgeSessionInfo,
+} from '@freecrawl/shared-types';
 import { JsonRpcServer } from './rpc.js';
 import { buildTools, type Tool } from './tools.js';
 import { defaultProjectPath, listProjectFiles } from './project-resolver.js';
+import {
+  bridgeRequest,
+  hasFeature,
+  setActiveSessionId,
+  getActiveSessionId,
+} from './desktop-bridge.js';
 
 const SERVER_NAME = 'freecrawl-seo';
 const SERVER_VERSION = '0.8.0';
@@ -67,6 +80,33 @@ function setProject(projectPath: string): Session {
   return session;
 }
 
+/** Drop the read-only DB handle (e.g. after a headless session's working DB
+ *  is torn down). The next DB tool reopens the default project lazily. */
+function closeReadSession(): void {
+  if (session) {
+    try {
+      session.db.close();
+    } catch {
+      // best-effort
+    }
+    session = null;
+  }
+}
+
+/**
+ * Fail a session tool early on a desktop that predates headless sessions,
+ * with guidance an agent can act on rather than a raw HTTP error.
+ */
+async function ensureSessionsSupported(): Promise<void> {
+  if (!(await hasFeature('sessions'))) {
+    throw new Error(
+      `This FreeCrawl desktop build does not support headless agent sessions ` +
+        `(needs v${SESSIONS_MIN_APP_VERSION}+). Update the desktop app, or drive the ` +
+        `desktop window's project directly with start_crawl (no session).`,
+    );
+  }
+}
+
 const tools: Tool[] = [
   ...buildTools(),
   {
@@ -112,12 +152,167 @@ const tools: Tool[] = [
   {
     name: 'current_project',
     description:
-      'Report which `.seoproject` the MCP server is currently reading from.',
+      'Report which `.seoproject` the MCP server is currently reading from, plus the active agent session id (if any). `mcpReadPath` is the read-only DB the query/report tools hit; `activeSessionId` is the headless session your crawl-control tools write to (null = the desktop window).',
     inputSchema: { type: 'object', properties: {} },
     requiresDb: false,
     // Report the open project without forcing one open — fall back to the
     // default path (which may not exist yet) purely as a hint.
-    handler: () => ({ projectPath: session ? session.projectPath : defaultProjectPath() }),
+    handler: () => ({
+      projectPath: session ? session.projectPath : defaultProjectPath(),
+      mcpReadPath: session ? session.projectPath : null,
+      activeSessionId: getActiveSessionId(),
+    }),
+  },
+
+  // ── Headless agent sessions (Issue #12) ────────────────────────────
+  {
+    name: 'session_create',
+    requiresDb: false,
+    description:
+      'Create your OWN isolated crawl session inside the running desktop app, so several agents can work in parallel without touching each other or the desktop window. `mode:"scratch"` (default) starts an empty project; `mode:"open"` unpacks an existing `.seoproject` to keep working on it (PageSpeed, spelling, exports, re-crawl); `mode:"attach"` reuses the session already bound to that path. Returns a `sessionId` — from here on your start_crawl / query / export tools automatically target this session, and the read-only view is repointed to its live database. Call session_close when done (optionally saving). Requires a desktop build with headless-session support.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        mode: {
+          type: 'string',
+          enum: ['scratch', 'open', 'attach'],
+          description: '"scratch" = new empty project (default); "open" = open an existing .seoproject; "attach" = reuse the session already bound to that path.',
+        },
+        projectPath: {
+          type: 'string',
+          description: 'Absolute path to a `.seoproject`. Required for mode "open"/"attach".',
+        },
+        label: {
+          type: 'string',
+          description: 'Optional human label shown in the app (defaults to agent-N).',
+        },
+        exclusive: {
+          type: 'boolean',
+          description: 'For mode "open": fail with project-locked if the file is already open elsewhere instead of sharing it.',
+        },
+      },
+    },
+    handler: async (args) => {
+      await ensureSessionsSupported();
+      const res = await bridgeRequest<BridgeSessionCreateResult>(
+        'POST',
+        '/v1/session/create',
+        {
+          mode: typeof args.mode === 'string' ? args.mode : undefined,
+          projectPath: typeof args.projectPath === 'string' ? args.projectPath : undefined,
+          label: typeof args.label === 'string' ? args.label : undefined,
+          exclusive: args.exclusive === true,
+        },
+        { sessionId: null },
+      );
+      setActiveSessionId(res.sessionId);
+      // Repoint the read-only view at this session's live working DB so the
+      // query/report tools read what this session crawls.
+      try {
+        setProject(res.dbPath);
+      } catch {
+        // The DB may not be flushed yet on a brand-new scratch session; the
+        // first read tool will retry openProject.
+      }
+      return res;
+    },
+  },
+  {
+    name: 'session_close',
+    requiresDb: false,
+    description:
+      'Close your active agent session and free its resources. Set `save:true` (optionally with `savePath`) to persist it to a `.seoproject` first so you can reopen it later with session_create mode:"open". After closing, your tools target the desktop window again until you create another session.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        save: { type: 'boolean', description: 'Persist the session to disk before closing.' },
+        savePath: { type: 'string', description: 'Where to save (defaults to the session\'s current document).' },
+      },
+    },
+    handler: async (args) => {
+      const id = getActiveSessionId();
+      if (!id) {
+        throw new Error('No active agent session to close — call session_create first.');
+      }
+      const res = await bridgeRequest<BridgeSessionCloseResult>(
+        'POST',
+        '/v1/session/close',
+        { save: args.save === true, savePath: typeof args.savePath === 'string' ? args.savePath : undefined },
+        { sessionId: id },
+      );
+      setActiveSessionId(null);
+      // The session's working DB is gone now — drop the read handle so the
+      // next DB tool reopens the default project instead of a dead file.
+      closeReadSession();
+      return res;
+    },
+  },
+  {
+    name: 'session_save',
+    requiresDb: false,
+    description:
+      'Save your active agent session to a `.seoproject` without closing it — the persistence half of "sessions you can come back to". Defaults to the session\'s current document; pass `projectPath` to write elsewhere.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        projectPath: { type: 'string', description: 'Destination .seoproject (defaults to the session\'s document).' },
+      },
+    },
+    handler: async (args) => {
+      await ensureSessionsSupported();
+      const id = getActiveSessionId();
+      if (!id) {
+        throw new Error('No active agent session to save — call session_create first.');
+      }
+      return bridgeRequest<BridgeSessionSaveResult>(
+        'POST',
+        '/v1/session/save',
+        { projectPath: typeof args.projectPath === 'string' ? args.projectPath : undefined },
+        { sessionId: id },
+      );
+    },
+  },
+  {
+    name: 'session_list',
+    requiresDb: false,
+    description:
+      'List every live session in the desktop app — the desktop window (`primary`) plus all headless agent sessions — with their project, row count, crawl state and owner. Use it to see what other agents are doing before you start.',
+    inputSchema: { type: 'object', properties: {} },
+    handler: async () => {
+      const res = await bridgeRequest<{ sessions: BridgeSessionInfo[] }>(
+        'GET',
+        '/v1/session/list',
+        undefined,
+        { sessionId: null },
+      );
+      return { ...res, activeSessionId: getActiveSessionId() };
+    },
+  },
+  {
+    name: 'session_status',
+    requiresDb: false,
+    description:
+      'Report your own active agent session: its id, document, live DB path, crawl state and idle time. Returns `activeSessionId: null` when you have not created a session (your tools target the desktop window).',
+    inputSchema: { type: 'object', properties: {} },
+    handler: async () => {
+      const id = getActiveSessionId();
+      if (!id) {
+        return { activeSessionId: null, mcpReadPath: session ? session.projectPath : null, session: null };
+      }
+      const res = await bridgeRequest<{ sessions: BridgeSessionInfo[] }>(
+        'GET',
+        '/v1/session/list',
+        undefined,
+        { sessionId: null },
+      );
+      const active = res.sessions.find((s) => s.sessionId === id) ?? null;
+      return {
+        activeSessionId: id,
+        mcpReadPath: session ? session.projectPath : null,
+        idleForMs: active ? Math.max(0, Date.now() - active.lastUsedAt) : null,
+        session: active,
+      };
+    },
   },
 ];
 

apps/mcp-server/src/tools.ts

@@ -263,7 +263,7 @@ export function buildTools(): Tool[] {
       requiresDb: false,
       name: 'start_crawl',
       description:
-        'Start a crawl in the running FreeCrawl desktop app. The desktop app must be open — this drives the SAME crawler the UI uses, so progress shows up in the app as it runs. `startUrl` is required unless the user has already run a crawl in this session (the last-used start URL is reused). Other config overrides are optional and layered on top of the desktop\'s saved settings. NOTE: when the previous crawl with the same start URL already completed, the crawler treats this as a resume and exits immediately because every URL is already in the DB. To force a fresh BFS from the seed, call `clear_crawl` first.',
+        'Start a crawl in the running FreeCrawl desktop app. The desktop app must be open — this drives the SAME crawler the UI uses, so progress shows up in the app as it runs. `startUrl` is required unless the user has already run a crawl in this session (the last-used start URL is reused). Other config overrides are optional and layered on top of the desktop\'s saved settings. Returns a `crawlId` you own — `stop_crawl`/`pause_crawl` refuse to touch a crawl owned by a different client unless forced. If a crawl is ALREADY running, this fails with `crawl-in-progress` by default; set `onBusy` to "queue" to wait your turn or "takeover" to replace it. NOTE: when the previous crawl with the same start URL already completed, the crawler treats a re-start as a resume and exits immediately; call `clear_crawl` first to force a fresh BFS from the seed.',
       inputSchema: {
         type: 'object',
         properties: {
@@ -271,6 +271,20 @@ export function buildTools(): Tool[] {
             type: 'string',
             description: 'Seed URL to crawl. Required unless the desktop already has a saved last-used crawl config.',
           },
+          projectPath: {
+            type: 'string',
+            description: 'Optional absolute path to a `.seoproject` this crawl should target. On the desktop-bound session it must match the project the desktop window already has open (otherwise `project-locked`); for an isolated per-agent project, open it with `session_create` (headless session) and crawl there.',
+          },
+          onBusy: {
+            type: 'string',
+            enum: ['reject', 'queue', 'takeover'],
+            description: 'What to do when a crawl is already running: "reject" (default) fails immediately with `crawl-in-progress`; "queue" waits FIFO until the slot frees (up to `queueTimeoutMs`); "takeover" stops the running crawl and replaces it.',
+          },
+          queueTimeoutMs: {
+            type: 'integer',
+            minimum: 1000,
+            description: 'For onBusy:"queue", the max time to wait for the crawl slot before giving up with `queue-timeout`. Default 15 minutes.',
+          },
           configOverrides: {
             type: 'object',
             description: 'Optional whitelisted CrawlConfig overrides. Anything not listed here keeps the desktop user\'s saved value.',
@@ -298,12 +312,23 @@ export function buildTools(): Tool[] {
       },
       handler: async (args) => {
         const startUrl = typeof args.startUrl === 'string' ? args.startUrl : undefined;
+        const projectPath =
+          typeof args.projectPath === 'string' ? args.projectPath : undefined;
+        const onBusy =
+          args.onBusy === 'queue' || args.onBusy === 'takeover' || args.onBusy === 'reject'
+            ? args.onBusy
+            : undefined;
+        const queueTimeoutMs =
+          typeof args.queueTimeoutMs === 'number' ? args.queueTimeoutMs : undefined;
         const configOverrides =
           args.configOverrides && typeof args.configOverrides === 'object'
             ? (args.configOverrides as Record<string, unknown>)
             : undefined;
-        return bridgeRequest<{ ok: true; config: unknown }>('POST', '/v1/crawl/start', {
+        return bridgeRequest('POST', '/v1/crawl/start', {
           startUrl,
+          projectPath,
+          onBusy,
+          queueTimeoutMs,
           configOverrides,
         });
       },
@@ -313,43 +338,79 @@ export function buildTools(): Tool[] {
       requiresDb: false,
       name: 'stop_crawl',
       description:
-        'Stop the active crawl in the desktop app immediately. Crawled URLs stay in the project DB; pending URLs are dropped. No-op when no crawl is running.',
-      inputSchema: { type: 'object', properties: {} },
-      handler: async () => bridgeRequest<{ ok: true }>('POST', '/v1/crawl/stop'),
+        'Stop the active crawl in the desktop app immediately. Crawled URLs stay in the project DB; pending URLs are dropped. No-op when no crawl is running. Refuses with `crawl-not-owned` if the running crawl belongs to a different client; pass `force: true` to stop it anyway.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          force: {
+            type: 'boolean',
+            description: 'Stop the crawl even if it is owned by another client.',
+          },
+        },
+      },
+      handler: async (args) =>
+        bridgeRequest<{ ok: true }>('POST', '/v1/crawl/stop', { force: args.force === true }),
     },
 
     {
       requiresDb: false,
       name: 'pause_crawl',
       description:
-        'Pause the active crawl in the desktop app. The pending queue is preserved — call resume_crawl to continue. No-op when no crawl is running.',
-      inputSchema: { type: 'object', properties: {} },
-      handler: async () => bridgeRequest<{ ok: true }>('POST', '/v1/crawl/pause'),
+        'Pause the active crawl in the desktop app. The pending queue is preserved — call resume_crawl to continue. No-op when no crawl is running. Refuses with `crawl-not-owned` for another client\'s crawl unless `force: true`.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          force: {
+            type: 'boolean',
+            description: 'Pause the crawl even if it is owned by another client.',
+          },
+        },
+      },
+      handler: async (args) =>
+        bridgeRequest<{ ok: true }>('POST', '/v1/crawl/pause', { force: args.force === true }),
     },
 
     {
       requiresDb: false,
       name: 'resume_crawl',
       description:
-        'Resume a paused crawl in the desktop app. No-op when the crawl is already running or no crawl is paused.',
-      inputSchema: { type: 'object', properties: {} },
-      handler: async () => bridgeRequest<{ ok: true }>('POST', '/v1/crawl/resume'),
+        'Resume a paused crawl in the desktop app. No-op when the crawl is already running or no crawl is paused. Refuses with `crawl-not-owned` for another client\'s crawl unless `force: true`.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          force: {
+            type: 'boolean',
+            description: 'Resume the crawl even if it is owned by another client.',
+          },
+        },
+      },
+      handler: async (args) =>
+        bridgeRequest<{ ok: true }>('POST', '/v1/crawl/resume', { force: args.force === true }),
     },
 
     {
       requiresDb: false,
       name: 'clear_crawl',
       description:
-        'Wipe every URL / link / image / header / cookie / sitemap entry from the active project — the same primitive the desktop "Clear" button calls. Stops any running crawl first. Use this before `start_crawl` when you want a fresh BFS from the seed and the previous crawl with that seed already completed (the crawler treats same-seed re-starts as a resume, so without clearing the new start would no-op). DESTRUCTIVE: there is no undo; the project file ends up empty until the new crawl populates it.',
-      inputSchema: { type: 'object', properties: {} },
-      handler: async () => bridgeRequest<{ ok: true }>('POST', '/v1/crawl/clear'),
+        'Wipe every URL / link / image / header / cookie / sitemap entry from the active project — the same primitive the desktop "Clear" button calls. Stops any running crawl first. Use this before `start_crawl` when you want a fresh BFS from the seed and the previous crawl with that seed already completed (the crawler treats same-seed re-starts as a resume, so without clearing the new start would no-op). Refuses with `project-locked` when the desktop window has unsaved user work, or `crawl-not-owned` for another client\'s crawl — pass `force: true` to override. DESTRUCTIVE: there is no undo; the project file ends up empty until the new crawl populates it.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          force: {
+            type: 'boolean',
+            description: 'Clear even if the desktop project has unsaved work or another client owns the crawl.',
+          },
+        },
+      },
+      handler: async (args) =>
+        bridgeRequest<{ ok: true }>('POST', '/v1/crawl/clear', { force: args.force === true }),
     },
 
     {
       requiresDb: false,
       name: 'get_crawl_progress',
       description:
-        'Snapshot of the current crawl progress in the desktop app: discovered / crawled / pending / failed counts, URLs per second, average response time, elapsed time, and running/paused flags. Returns `progress: null` when no crawl has run this session. Poll this every 1–2 s to watch a crawl live; once `running` flips to false the crawl is done.',
+        'Snapshot of the current crawl progress in the desktop app: discovered / crawled / pending / failed counts, URLs per second, average response time, elapsed time, and running/paused flags. Also returns `crawlId`, `ownerClientId`, and `ownedByCaller` so you can tell whether the running crawl is yours. Returns `progress: null` when no crawl has run this session. Poll this every 1–2 s to watch a crawl live; once `running` flips to false the crawl is done.',
       inputSchema: { type: 'object', properties: {} },
       handler: async () =>
         bridgeRequest<{ progress: unknown | null }>('GET', '/v1/crawl/progress'),

package.json

@@ -1,6 +1,6 @@
 {
   "name": "freecrawl-seo",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "description": "Open-source desktop SEO crawler — a free alternative to Screaming Frog",
   "license": "MIT",
@@ -31,7 +31,8 @@
     "lint": "eslint .",
     "format": "prettier --write \"**/*.{ts,tsx,js,json,md,css,html}\"",
     "format:check": "prettier --check \"**/*.{ts,tsx,js,json,md,css,html}\"",
-    "test": "npm run test --workspaces --if-present",
+    "test": "vitest run",
+    "test:watch": "vitest",
     "cli": "npm run start --workspace apps/cli --",
     "postinstall": "node scripts/install-playwright-browsers.mjs",
     "playwright:install": "node scripts/install-playwright-browsers.mjs --force"
@@ -42,6 +43,7 @@
     "eslint-plugin-react-hooks": "^7.1.1",
     "prettier": "^3.4.0",
     "typescript": "^5.7.0",
-    "typescript-eslint": "^8.20.0"
+    "typescript-eslint": "^8.20.0",
+    "vitest": "^2.1.9"
   }
 }

packages/core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/core",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "type": "module",
   "main": "./dist/index.js",

packages/core/src/crawler.ts

@@ -3675,8 +3675,18 @@ export class Crawler extends EventEmitter {
           }
         }
       }
-      for (const link of storableLinks) {
-        if (!link.isInternal) this.enqueueExternal(link.toUrl);
+      // List / Sitemap ("fixed-URL") modes crawl exactly the supplied set —
+      // nothing the pages point at. External links and internal subresources
+      // aren't part of that set, so probing/fetching them would balloon the
+      // crawl far past the sitemap (the "it keeps crawling page links / I see
+      // social links" report). Outlinks are still stored as link rows for the
+      // Outlinks tab; they're just not fetched as their own URL rows here.
+      const fixedUrlMode =
+        this.config.mode === 'list' || this.config.mode === 'sitemap';
+      if (!fixedUrlMode) {
+        for (const link of storableLinks) {
+          if (!link.isInternal) this.enqueueExternal(link.toUrl);
+        }
       }
       this.crawled++;
 
@@ -3686,7 +3696,10 @@ export class Crawler extends EventEmitter {
       // Done before the noindex/nofollow early-return so a page's assets are
       // crawled regardless of its own indexability. Resources are leaf nodes:
       // the non-HTML fetch path stores them without extracting further links.
-      this.enqueueResources(parsed, item.depth);
+      // Skipped in fixed-URL modes for the same reason as external probing.
+      if (!fixedUrlMode) {
+        this.enqueueResources(parsed, item.depth);
+      }
 
       if (parsed.hasNofollow || indexability === 'non-indexable:noindex') {
         return;
@@ -4307,6 +4320,16 @@ export class Crawler extends EventEmitter {
    * lifts.
    */
   private throttleScale = 1;
+  /**
+   * Per-source throttle scales, so independent pressure signals compose
+   * instead of overwriting each other. The effective scale is the MINIMUM
+   * across sources — the tightest constraint wins. The source key is the
+   * part of `reason` before the first `:` (e.g. `thermal:serious` →
+   * `thermal`, `multi-session` → `multi-session`), so a source updating its
+   * level (thermal fair→serious) replaces its own entry rather than piling
+   * up. A scale of ≥1 removes the source (no longer constraining).
+   */
+  private readonly throttleScales = new Map<string, number>();
 
   /** Configured ceiling with the external throttle scale applied. */
   private effectiveCeiling(): number {
@@ -4314,11 +4337,17 @@ export class Crawler extends EventEmitter {
     return Math.max(1, Math.round(base * this.throttleScale));
   }
 
-  /** Public — the desktop main process maps thermal state changes here. */
+  /** Public — the desktop main process maps thermal state + multi-session
+   *  concurrency budget here. Sources compose via minimum (see field doc). */
   setThrottleScale(scale: number, reason: string): void {
     const s = Math.min(1, Math.max(0.1, scale));
-    if (s === this.throttleScale) return;
-    this.throttleScale = s;
+    const source = reason.split(':')[0] || reason;
+    if (scale >= 1) this.throttleScales.delete(source);
+    else this.throttleScales.set(source, s);
+    let min = 1;
+    for (const v of this.throttleScales.values()) min = Math.min(min, v);
+    if (min === this.throttleScale) return;
+    this.throttleScale = min;
     const ceiling = this.effectiveCeiling();
     if (this.currentConcurrency === 0 || this.currentConcurrency > ceiling) {
       this.currentConcurrency = ceiling;

packages/db/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/db",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "type": "module",
   "main": "./dist/index.js",

packages/db/src/project-db.ts

@@ -775,6 +775,19 @@ const DUPLICATE_ALT_PREDICATE = `alt IS NOT NULL AND alt != '' AND alt IN (
  */
 const INTERNAL_HTML_SCOPE = "is_external = 0 AND content_kind = 'html'";
 
+/**
+ * Scope for on-page-element "missing" checks that only make sense on a real
+ * fetched page body (title / meta / h1 / lang / viewport / OG / Twitter /
+ * security headers / structured data). Redirects (3xx) and error responses
+ * (4xx/5xx) carry none of these by nature, so counting them as "missing"
+ * produces false positives on redirect rows — the fix for redirect false
+ * errors. Kept as `INTERNAL_HTML_SCOPE + status gate` so `runOverviewBatch`
+ * still recognises the shared prefix and hoists it (the extra status
+ * predicate rides along in the per-clause remainder).
+ */
+const INDEXABLE_HTML_SCOPE =
+  INTERNAL_HTML_SCOPE + ' AND status_code >= 200 AND status_code < 300';
+
 export class ProjectDb {
   private readonly db: DatabaseSync;
   /** Read-only connections can't write `sqlite_stat1`; see `optimize()`. */
@@ -4237,8 +4250,32 @@ export class ProjectDb {
     search?: string;
     filter?: GscPreset;
     accountId?: string;
+    /** Mirror Screaming Frog's URL-matching options when joining GSC data
+     *  onto crawled pages. Default slash-insensitive so a GSC `/page`
+     *  attaches to a crawled `/page/` (and vice versa) instead of showing
+     *  "—" — the same normalisation `gscOrphanUrls` already applies. */
+    matchSlash?: boolean;
+    matchCase?: boolean;
   }): { rows: GscRow[]; total: number } {
     const filter: GscPreset = params.filter ?? 'all';
+    const matchSlash = params.matchSlash ?? true;
+    const matchCase = params.matchCase ?? false;
+    // ON-condition builder between an indexed lookup column `probe` (the
+    // table joined INTO) and the driver column `drive`. Slash-tolerance
+    // keeps `probe` an equality target so its `url` index still serves the
+    // lookup (exact, add-slash, strip-slash all resolve to `probe = <value>`).
+    // Case-folding is opt-in and rare — it wraps both sides in LOWER and
+    // gives up the index, which is acceptable since `gsc_results` is small.
+    const urlMatch = (probe: string, drive: string): string => {
+      const eq = (p: string, d: string): string =>
+        matchCase ? `LOWER(${p}) = LOWER(${d})` : `${p} = ${d}`;
+      if (!matchSlash) return eq(probe, drive);
+      return (
+        `(${eq(probe, drive)}` +
+        ` OR ${eq(probe, `${drive} || '/'`)}` +
+        ` OR ${eq(probe, `RTRIM(${drive}, '/')`)})`
+      );
+    };
     // Every join below is account-scoped: with several linked accounts a
     // bare `g.url = u.url` would fan out one row per account.
     const accountId = params.accountId ?? '';
@@ -4311,8 +4348,8 @@ export class ProjectDb {
         oArgs.push(`%${params.search}%`);
       }
       const oJoin = `FROM gsc_results g
-         LEFT JOIN urls u ON u.url = g.url
-         LEFT JOIN gsc_inspection_results i ON i.url = g.url AND i.account_id = ?`;
+         LEFT JOIN urls u ON ${urlMatch('u.url', 'g.url')}
+         LEFT JOIN gsc_inspection_results i ON ${urlMatch('i.url', 'g.url')} AND i.account_id = ?`;
       const oWhereSql = `WHERE ${oWhere.join(' AND ')}`;
       const oTotal = this.db
         .prepare(`SELECT COUNT(*) AS c ${oJoin} ${oWhereSql}`)
@@ -4392,8 +4429,8 @@ export class ProjectDb {
     const whereSql = `WHERE ${where.join(' AND ')}`;
     const joinSql =
       `FROM urls u
-       LEFT JOIN gsc_results g ON g.url = u.url AND g.account_id = ?
-       LEFT JOIN gsc_inspection_results i ON i.url = u.url AND i.account_id = ?`;
+       LEFT JOIN gsc_results g ON ${urlMatch('g.url', 'u.url')} AND g.account_id = ?
+       LEFT JOIN gsc_inspection_results i ON ${urlMatch('i.url', 'u.url')} AND i.account_id = ?`;
 
     const totalRow = this.db
       .prepare(`SELECT COUNT(*) AS c ${joinSql} ${whereSql}`)
@@ -5372,6 +5409,10 @@ export class ProjectDb {
     // Common prefix for all issue checks — only crawled internal HTML pages
     // are eligible (is_external = 0, content_kind = 'html').
     const html = INTERNAL_HTML_SCOPE;
+    // Stricter prefix for on-page-element "missing" checks: also require a
+    // 2xx response so redirects / error pages (which never carry these
+    // elements) aren't flagged as "missing".
+    const indexable = INDEXABLE_HTML_SCOPE;
     const dup = (col: string): number =>
       this.scalarCount(
         `SELECT COALESCE(SUM(c), 0) AS c FROM (
@@ -5381,19 +5422,19 @@ export class ProjectDb {
          )`,
       );
     return {
-      titleMissing: countWhere(`${html} AND (title IS NULL OR title = '')`),
+      titleMissing: countWhere(`${indexable} AND (title IS NULL OR title = '')`),
       titleTooLong: countWhere(`${html} AND title_length > 60`),
       titleTooShort: countWhere(`${html} AND title_length > 0 AND title_length < 30`),
       titleDuplicate: dup('title'),
       metaMissing: countWhere(
-        `${html} AND (meta_description IS NULL OR meta_description = '')`,
+        `${indexable} AND (meta_description IS NULL OR meta_description = '')`,
       ),
       metaTooLong: countWhere(`${html} AND meta_description_length > 160`),
       metaTooShort: countWhere(
         `${html} AND meta_description_length > 0 AND meta_description_length < 120`,
       ),
       metaDuplicate: dup('meta_description'),
-      h1Missing: countWhere(`${html} AND (h1 IS NULL OR h1 = '')`),
+      h1Missing: countWhere(`${indexable} AND (h1 IS NULL OR h1 = '')`),
       h1Duplicate: dup('h1'),
       h1Multiple: countWhere(`${html} AND h1_count > 1`),
       headingSkippedLevel: countWhere(
@@ -5461,31 +5502,31 @@ export class ProjectDb {
         `${html} AND INSTR(SUBSTR(url, INSTR(url, '://') + 3), '//') > 0`,
       ),
       urlNonAscii: countWhere(`${html} AND LENGTH(CAST(url AS BLOB)) != LENGTH(url)`),
-      langMissing: countWhere(`${html} AND (lang IS NULL OR lang = '')`),
-      viewportMissing: countWhere(`${html} AND (viewport IS NULL OR viewport = '')`),
+      langMissing: countWhere(`${indexable} AND (lang IS NULL OR lang = '')`),
+      viewportMissing: countWhere(`${indexable} AND (viewport IS NULL OR viewport = '')`),
       ogMissing: countWhere(
-        `${html}
+        `${indexable}
          AND (og_title IS NULL OR og_title = '')
          AND (og_description IS NULL OR og_description = '')
          AND (og_image IS NULL OR og_image = '')`,
       ),
       twitterMissing: countWhere(
-        `${html}
+        `${indexable}
          AND (twitter_card IS NULL OR twitter_card = '')
          AND (twitter_image IS NULL OR twitter_image = '')`,
       ),
       hstsMissing: countWhere(
-        `${html} AND url LIKE 'https://%' AND (hsts IS NULL OR hsts = '')`,
+        `${indexable} AND url LIKE 'https://%' AND (hsts IS NULL OR hsts = '')`,
       ),
       xFrameOptionsMissing: countWhere(
-        `${html} AND (x_frame_options IS NULL OR x_frame_options = '')`,
+        `${indexable} AND (x_frame_options IS NULL OR x_frame_options = '')`,
       ),
       xContentTypeOptionsMissing: countWhere(
-        `${html} AND (x_content_type_options IS NULL OR x_content_type_options = '')`,
+        `${indexable} AND (x_content_type_options IS NULL OR x_content_type_options = '')`,
       ),
-      cspMissing: countWhere(`${html} AND (csp IS NULL OR csp = '')`),
+      cspMissing: countWhere(`${indexable} AND (csp IS NULL OR csp = '')`),
       structuredDataMissing: countWhere(
-        `${html} AND schema_block_count = 0 AND schema_invalid_count = 0
+        `${indexable} AND schema_block_count = 0 AND schema_invalid_count = 0
          AND microdata_count = 0 AND rdfa_count = 0`,
       ),
       structuredDataInvalid: countWhere(`${html} AND schema_invalid_count > 0`),
@@ -8784,10 +8825,28 @@ function buildAdvancedFilterSql(
 }
 
 function buildClauseSql(c: FilterClause, args: (string | number)[]): string | null {
-  // Whitelist-check the column name — it's interpolated into the SQL
-  // string, so allowing arbitrary values would be an injection vector.
-  if (!ALLOWED_FILTER_FIELDS.has(c.field)) return null;
-  const col = c.field;
+  // The column expression `col` is interpolated into the SQL string, so it
+  // must never carry unchecked input.
+  let col: string;
+  if (c.field === 'extraction') {
+    // Custom-extraction field → pull the named value out of the
+    // `extraction_results` JSON map. The key is embedded in a single-quoted
+    // SQL literal AND a JSON path, so: reject `"` / `\` (would break the
+    // path) and SQL-escape `'`. Unicode / spaces / normal punctuation pass.
+    const key = c.extractionKey ?? '';
+    if (key.length === 0 || key.length > 128 || key.includes('"') || key.includes('\\')) {
+      return null;
+    }
+    const safeKey = key.replace(/'/g, "''");
+    // NULLIF('' → NULL) so json_extract never sees a non-JSON empty string
+    // (which would raise "malformed JSON" and fail the whole query); NULL
+    // input yields NULL, matching a page that carries no extraction data.
+    col = `json_extract(NULLIF(extraction_results, ''), '$."${safeKey}"')`;
+  } else {
+    // Whitelist-check the column name for the same reason.
+    if (!ALLOWED_FILTER_FIELDS.has(c.field)) return null;
+    col = c.field;
+  }
   switch (c.operator) {
     case 'contains':
       args.push(`%${c.value}%`);
@@ -8890,7 +8949,11 @@ function categoryWhereClause(cat: UrlCategory): string | null {
       return `is_external = 0 AND content_kind = 'html'
               AND indexability = 'non-indexable:robots-blocked'`;
     case 'issues:title-missing':
-      return "is_external = 0 AND content_kind = 'html' AND (title IS NULL OR title = '')";
+      // Gate on 2xx: redirects / error pages have no body, so a NULL title
+      // there isn't a "missing title" — matches getIssuesCounts.titleMissing.
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND (title IS NULL OR title = '')`;
     case 'issues:title-too-long':
       return "is_external = 0 AND content_kind = 'html' AND title_length > 60";
     case 'issues:title-too-short':
@@ -8904,7 +8967,9 @@ function categoryWhereClause(cat: UrlCategory): string | null {
                 GROUP BY title HAVING COUNT(*) > 1
               )`;
     case 'issues:meta-missing':
-      return "is_external = 0 AND content_kind = 'html' AND (meta_description IS NULL OR meta_description = '')";
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND (meta_description IS NULL OR meta_description = '')`;
     case 'issues:meta-too-long':
       return "is_external = 0 AND content_kind = 'html' AND meta_description_length > 160";
     case 'issues:meta-too-short':
@@ -8918,7 +8983,9 @@ function categoryWhereClause(cat: UrlCategory): string | null {
                 GROUP BY meta_description HAVING COUNT(*) > 1
               )`;
     case 'issues:h1-missing':
-      return "is_external = 0 AND content_kind = 'html' AND (h1 IS NULL OR h1 = '')";
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND (h1 IS NULL OR h1 = '')`;
     case 'issues:h1-duplicate':
       return `is_external = 0 AND content_kind = 'html' AND h1 IS NOT NULL AND h1 != ''
               AND h1 IN (
@@ -9061,11 +9128,16 @@ function categoryWhereClause(cat: UrlCategory): string | null {
       // contains multi-byte UTF-8, i.e. any non-ASCII code point.
       return "is_external = 0 AND content_kind = 'html' AND LENGTH(CAST(url AS BLOB)) != LENGTH(url)";
     case 'issues:lang-missing':
-      return "is_external = 0 AND content_kind = 'html' AND (lang IS NULL OR lang = '')";
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND (lang IS NULL OR lang = '')`;
     case 'issues:viewport-missing':
-      return "is_external = 0 AND content_kind = 'html' AND (viewport IS NULL OR viewport = '')";
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND (viewport IS NULL OR viewport = '')`;
     case 'issues:og-missing':
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND (og_title IS NULL OR og_title = '')
               AND (og_description IS NULL OR og_description = '')
               AND (og_image IS NULL OR og_image = '')`;
@@ -9073,21 +9145,28 @@ function categoryWhereClause(cat: UrlCategory): string | null {
       // Twitter card is “missing” if there’s no twitter:card tag AND no
       // twitter:image — the minimum pair needed for a valid preview.
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND (twitter_card IS NULL OR twitter_card = '')
               AND (twitter_image IS NULL OR twitter_image = '')`;
     // HSTS on HTTP is meaningless — only flag HTTPS pages. X-Frame-Options
     // and X-Content-Type-Options matter on any HTML response regardless
-    // of scheme, so they're only scheme-gated on a per-page basis.
+    // of scheme, so they're only scheme-gated on a per-page basis. All three
+    // are gated on 2xx so redirect / error responses aren't flagged.
     case 'issues:hsts-missing':
-      return "is_external = 0 AND content_kind = 'html' AND url LIKE 'https://%' AND (hsts IS NULL OR hsts = '')";
+      return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
+              AND url LIKE 'https://%' AND (hsts IS NULL OR hsts = '')`;
     case 'issues:x-frame-options-missing':
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND (x_frame_options IS NULL OR x_frame_options = '')`;
     case 'issues:x-content-type-options-missing':
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND (x_content_type_options IS NULL OR x_content_type_options = '')`;
     case 'issues:csp-missing':
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND (csp IS NULL OR csp = '')`;
     case 'issues:structured-data-missing':
       // "Missing" = no valid JSON-LD block AND no malformed block either,
@@ -9095,6 +9174,7 @@ function categoryWhereClause(cat: UrlCategory): string | null {
       // accepts all three formats, so a page using only Microdata isn't
       // missing structured data even though JSON-LD is empty.
       return `is_external = 0 AND content_kind = 'html'
+              AND status_code >= 200 AND status_code < 300
               AND schema_block_count = 0 AND schema_invalid_count = 0
               AND microdata_count = 0 AND rdfa_count = 0`;
     case 'issues:structured-data-invalid':

packages/shared-types/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@freecrawl/shared-types",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "private": true,
   "type": "module",
   "main": "./dist/index.js",

packages/shared-types/src/bridge.ts

@@ -0,0 +1,227 @@
+/**
+ * Shared contract for the localhost MCP bridge that connects the desktop
+ * app (`apps/desktop/src/main/mcp-bridge.ts`, server side) to the FreeCrawl
+ * MCP server (`apps/mcp-server`, client side).
+ *
+ * Both ends import these types so the wire shape can never drift. Bridge
+ * protocol v2 (Issue #12) adds per-session routing so several MCP clients —
+ * typically parallel autonomous agents — can each drive their own crawl
+ * against their own project without racing the desktop window or each other.
+ *
+ * Backward compatibility: a request without a session header targets the
+ * `"primary"` session (the desktop window's project), which is exactly the
+ * v1 behaviour. The only visible difference for a single client is that a
+ * conflicting crawl now returns a typed error instead of silently taking
+ * over the running one.
+ */
+
+import type { CrawlConfig, CrawlProgress } from './crawl.js';
+
+/** Bump when the wire shape changes in a way old clients must notice. */
+export const BRIDGE_PROTOCOL_VERSION = 2 as const;
+
+/** HTTP header carrying the target session id (`"primary"` when absent). */
+export const BRIDGE_SESSION_HEADER = 'x-freecrawl-session';
+/** HTTP header carrying the MCP client's stable per-process id. */
+export const BRIDGE_CLIENT_HEADER = 'x-freecrawl-client';
+
+/** Well-known session id for the desktop window's own project. */
+export const PRIMARY_SESSION_ID = 'primary';
+
+/**
+ * Machine-readable error codes returned by the bridge. The MCP server maps
+ * these to guidance an agent can act on, so keep them stable.
+ */
+export type BridgeErrorCode =
+  | 'crawl-in-progress' // 409 — a crawl already owns this session
+  | 'crawl-not-owned' // 409 — control call from a client that isn't the owner
+  | 'session-not-found' // 404 — unknown session id
+  | 'session-expired' // 410 — session reaped for idleness
+  | 'session-limit-reached' // 429 — headless-session or queue cap hit
+  | 'project-locked' // 409 — target project is held by another session / has unsaved work
+  | 'invalid-project-path' // 400 — projectPath failed validation
+  | 'unsupported-feature' // 501 — feature not available on this desktop build
+  | 'queue-timeout' // 504 — queued crawl waited past its timeout
+  | 'bad-request' // 400 — malformed body / missing required field
+  | 'internal-error'; // 500 — unexpected failure
+
+/**
+ * Uniform error envelope for every non-2xx bridge response. `message` is
+ * written for the *agent* — it should say what to do next, not merely what
+ * went wrong.
+ */
+export interface BridgeErrorBody {
+  error: BridgeErrorCode;
+  message: string;
+  details?: Record<string, unknown>;
+}
+
+/** What a client can request when a session is already crawling. */
+export type OnBusy = 'reject' | 'queue' | 'takeover';
+
+/**
+ * Fields the MCP server may pass when starting a crawl. Everything is
+ * optional except an effective startUrl must exist — either supplied here or
+ * already saved in the session's last-used config. The desktop layers these
+ * on top of the last-used CrawlConfig and dispatches.
+ *
+ * Moved here from `mcp-bridge.ts` (v1) so both ends share one definition.
+ */
+export interface McpStartCrawlInput {
+  startUrl?: string;
+  /**
+   * Optional path to a `.seoproject` this crawl should target. On the
+   * primary (desktop-window) session it is validated against the open
+   * document (verify-only). For a real isolated project, open it in a
+   * headless session via `session_create` and crawl there instead.
+   */
+  projectPath?: string;
+  /** What to do when a crawl is already running in the target session. */
+  onBusy?: OnBusy;
+  /** Max wait for `onBusy: 'queue'` before giving up with `queue-timeout`. */
+  queueTimeoutMs?: number;
+  /** Whitelisted CrawlConfig field overrides — a narrow contract so MCP
+   *  can't smuggle in fields we'd rather validate. */
+  configOverrides?: {
+    scope?: CrawlConfig['scope'];
+    maxDepth?: number;
+    maxUrls?: number;
+    maxConcurrency?: number;
+    maxRps?: number;
+    crawlDelayMs?: number;
+    requestTimeoutMs?: number;
+    respectRobotsTxt?: boolean;
+    respectCrawlDelay?: boolean;
+    followRedirects?: boolean;
+    crawlExternal?: boolean;
+    userAgent?: string;
+    includePatterns?: string[];
+    excludePatterns?: string[];
+  };
+}
+
+/** Success payload for `POST /v1/crawl/start`. */
+export interface McpStartCrawlResult {
+  ok: true;
+  crawlId: string;
+  sessionId: string;
+  config: CrawlConfig;
+  /** True when the request was FIFO-queued behind a running crawl. */
+  queued: boolean;
+  /** Milliseconds spent waiting in the queue (0 when it started immediately). */
+  waitedMs: number;
+}
+
+/** A live crawl's ownership + run state, embedded in progress/list replies. */
+export interface BridgeCrawlInfo {
+  crawlId: string;
+  running: boolean;
+  paused: boolean;
+  ownerClientId: string | null;
+  startUrl: string;
+  startedAt: number;
+}
+
+/** Progress reply for `GET /v1/crawl/progress`. */
+export interface BridgeProgressResult {
+  progress: CrawlProgress | null;
+  crawlId: string | null;
+  sessionId: string;
+  ownerClientId: string | null;
+  /** True when the polling client owns the running crawl. */
+  ownedByCaller: boolean;
+}
+
+/** One session in `GET /v1/session/list`. */
+export interface BridgeSessionInfo {
+  sessionId: string;
+  label: string;
+  kind: 'primary' | 'window' | 'headless';
+  documentPath: string | null;
+  /** Live working-copy DB path — what the MCP read-only view should open. */
+  dbPath: string | null;
+  urlsCrawled: number;
+  crawl: BridgeCrawlInfo | null;
+  ownerClientId: string | null;
+  createdAt: number;
+  lastUsedAt: number;
+}
+
+/** How a new headless session should be initialised. */
+export type SessionCreateMode = 'scratch' | 'open' | 'attach';
+
+/** Body for `POST /v1/session/create`. */
+export interface BridgeSessionCreateInput {
+  /** `scratch` (default) = empty new project; `open` = unpack an existing
+   *  `.seoproject` into a fresh session; `attach` = reuse the session already
+   *  bound to that path, or open it if none. */
+  mode?: SessionCreateMode;
+  /** Required for `open`/`attach`: absolute path to a `.seoproject`. */
+  projectPath?: string;
+  /** Human label for logs / the Agents UI (defaults to `agent-N`). */
+  label?: string;
+  /** For `open`/`attach`: fail with `project-locked` if the path is already
+   *  held by another session instead of sharing it. */
+  exclusive?: boolean;
+}
+
+/** Reply for `POST /v1/session/create`. */
+export interface BridgeSessionCreateResult {
+  sessionId: string;
+  /** Live working-copy DB path — open this read-only from the MCP server. */
+  dbPath: string;
+  /** The `.seoproject` document backing the session, or null for scratch. */
+  documentPath: string | null;
+  /** False when `attach` returned an already-open session. */
+  created: boolean;
+  label: string;
+}
+
+/** Body for `POST /v1/session/close`. */
+export interface BridgeSessionCloseInput {
+  /** Persist the session to a `.seoproject` before tearing it down. */
+  save?: boolean;
+  /** Where to save when `save` is set; defaults to the session's document. */
+  savePath?: string;
+}
+
+/** Reply for `POST /v1/session/close`. */
+export interface BridgeSessionCloseResult {
+  ok: true;
+  savedTo?: string;
+}
+
+/** Body for `POST /v1/session/save`. */
+export interface BridgeSessionSaveInput {
+  /** Destination `.seoproject`; defaults to the session's current document. */
+  projectPath?: string;
+}
+
+/** Reply for `POST /v1/session/save`. */
+export interface BridgeSessionSaveResult {
+  filePath: string;
+  bytesWritten: number;
+}
+
+/** Reply for `GET /v1/capabilities`. Absent route ⇒ v1 desktop. */
+export interface BridgeCapabilities {
+  bridgeVersion: number;
+  appVersion: string;
+  features: string[];
+  limits: {
+    maxHeadlessSessions: number;
+    maxQueueDepth: number;
+    sessionIdleTtlMs: number;
+  };
+}
+
+/** Feature flags advertised in `BridgeCapabilities.features`. */
+export const BRIDGE_FEATURES = {
+  sessions: 'sessions',
+  crawlLease: 'crawl-lease',
+  crawlQueue: 'crawl-queue',
+} as const;
+
+/** Minimum desktop app version that supports headless sessions. Surfaced in
+ *  the MCP `unsupported-feature` guidance so an agent knows what to upgrade. */
+export const SESSIONS_MIN_APP_VERSION = '0.10.0';

packages/shared-types/src/crawl.ts

@@ -2024,7 +2024,11 @@ export type FilterField =
   | 'x_robots_tag'
   | 'redirect_target'
   | 'images_count'
-  | 'images_missing_alt';
+  | 'images_missing_alt'
+  // Sentinel for a custom-extraction field. The concrete rule is named by
+  // `FilterClause.extractionKey`; the query layer maps it to a
+  // `json_extract(extraction_results, …)` expression rather than a column.
+  | 'extraction';
 
 export type FilterOperator =
   | 'contains'
@@ -2044,6 +2048,9 @@ export interface FilterClause {
   field: FilterField;
   operator: FilterOperator;
   value: string;
+  /** Only when `field === 'extraction'`: the custom-extraction rule name
+   *  (the JSON key in `extraction_results`) this clause targets. */
+  extractionKey?: string;
 }
 
 /** Clauses inside a group are AND'd together. */

packages/shared-types/src/index.ts

@@ -2,6 +2,7 @@ export * from './crawl.js';
 export * from './issue-severity.js';
 export * from './ipc.js';
 export * from './project.js';
+export * from './bridge.js';
 export * from './integrations.js';
 export * from './pagespeed.js';
 export * from './crux.js';

packages/shared-types/src/ipc.ts

@@ -33,6 +33,7 @@ import type {
   SpellingStatus,
 } from './spelling.js';
 import type { RecentProject } from './project.js';
+import type { BridgeSessionInfo } from './bridge.js';
 import type {
   GoogleAuthState,
   GoogleAuthResult,
@@ -112,6 +113,13 @@ export const IPC = {
   browserInstallState: 'browser:install-state',
   browserInstallGet: 'browser:install-get',
   browserInstallStart: 'browser:install-start',
+  /** MCP agent sessions (Issue #12). `agentsList` returns the live headless
+   *  sessions; `agentsClose` tears one down; `agentsChanged` is pushed
+   *  main→renderer whenever a session is created / closed or its crawl state
+   *  changes, so the status-bar Agents indicator stays live. */
+  agentsList: 'agents:list',
+  agentsClose: 'agents:close',
+  agentsChanged: 'agents:changed',
   urlsQuery: 'urls:query',
   urlDetailGet: 'urls:detail',
   urlSourceGet: 'urls:source',
@@ -1766,6 +1774,14 @@ export interface FreeCrawlApi {
   browserInstallStart(): Promise<BrowserInstallState>;
   /** Fires on every provisioning state change (download progress included). */
   onBrowserInstallState(cb: (state: BrowserInstallState) => void): () => void;
+  /** Live headless MCP agent sessions (Issue #12). Empty when no agent has
+   *  created one — the desktop window itself is not included. */
+  agentsList(): Promise<BridgeSessionInfo[]>;
+  /** Close a headless agent session by id (stops its crawl, frees resources). */
+  agentsClose(sessionId: string): Promise<{ ok: boolean }>;
+  /** Fires when an agent session is created / closed or its crawl state
+   *  changes — the status-bar indicator re-fetches on this. */
+  onAgentsChanged(cb: () => void): () => void;
   /** Full recent-projects list including archived entries. */
   recentProjectsList(): Promise<RecentProject[]>;
   /** Archive / unarchive a recent project (hides it from Open Recent). */

tests/agent-session-registry.test.ts

@@ -0,0 +1,184 @@
+import { describe, it, expect } from 'vitest';
+import {
+  SessionRegistry,
+  computeGlobalThrottleScale,
+} from '../apps/desktop/src/main/agent-session-registry.js';
+
+/**
+ * Deterministic registry: injected clock + id factory + fake timers so queue
+ * timeout / promotion ordering is fully controllable.
+ */
+function makeRegistry() {
+  let id = 0;
+  const state = { clock: 1000 };
+  const timers: Array<{ fn: () => void; handle: symbol }> = [];
+  const reg = new SessionRegistry({
+    genCrawlId: () => `cr_${++id}`,
+    now: () => state.clock,
+    setTimer: (fn) => {
+      const handle = Symbol();
+      timers.push({ fn, handle });
+      return handle;
+    },
+    clearTimer: (h) => {
+      const i = timers.findIndex((t) => t.handle === h);
+      if (i !== -1) timers.splice(i, 1);
+    },
+    maxQueueDepth: 2,
+  });
+  return {
+    reg,
+    advance: (ms: number) => (state.clock += ms),
+    fireTimers: () => {
+      const pending = timers.splice(0, timers.length);
+      for (const t of pending) t.fn();
+    },
+  };
+}
+
+describe('SessionRegistry crawl lease', () => {
+  it('grants an idle slot immediately', async () => {
+    const { reg } = makeRegistry();
+    const g = await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    expect(g.ok).toBe(true);
+    if (g.ok) {
+      expect(g.queued).toBe(false);
+      expect(g.tookOver).toBe(false);
+    }
+    expect(reg.peekLease('primary')?.ownerClientId).toBe('A');
+  });
+
+  it('rejects a second start by default (crawl-in-progress)', async () => {
+    const { reg } = makeRegistry();
+    await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    const g = await reg.acquireCrawl('primary', { clientId: 'B', startUrl: 'y' });
+    expect(g.ok).toBe(false);
+    if (!g.ok) expect(g.error).toBe('crawl-in-progress');
+  });
+
+  it('queues and promotes FIFO when the lease releases', async () => {
+    const { reg, advance } = makeRegistry();
+    const a = await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    let bResolved = false;
+    const bPromise = reg
+      .acquireCrawl('primary', { clientId: 'B', startUrl: 'y', onBusy: 'queue' })
+      .then((v) => {
+        bResolved = true;
+        return v;
+      });
+    expect(bResolved).toBe(false);
+    expect(reg.queueDepth('primary')).toBe(1);
+    advance(5000);
+    if (a.ok) reg.releaseCrawl('primary', a.crawlId);
+    const b = await bPromise;
+    expect(b.ok).toBe(true);
+    if (b.ok) {
+      expect(b.queued).toBe(true);
+      expect(b.waitedMs).toBe(5000);
+    }
+    expect(reg.peekLease('primary')?.ownerClientId).toBe('B');
+  });
+
+  it('rejects with session-limit-reached when the queue is full', async () => {
+    const { reg } = makeRegistry();
+    await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    void reg.acquireCrawl('primary', { clientId: 'B', startUrl: 'y', onBusy: 'queue' });
+    void reg.acquireCrawl('primary', { clientId: 'C', startUrl: 'z', onBusy: 'queue' });
+    const g = await reg.acquireCrawl('primary', {
+      clientId: 'D',
+      startUrl: 'w',
+      onBusy: 'queue',
+    });
+    expect(g.ok).toBe(false);
+    if (!g.ok) expect(g.error).toBe('session-limit-reached');
+  });
+
+  it('takeover grants a new lease and the superseded release no-ops', async () => {
+    const { reg } = makeRegistry();
+    const a = await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    const g = await reg.acquireCrawl('primary', {
+      clientId: 'B',
+      startUrl: 'y',
+      onBusy: 'takeover',
+    });
+    expect(g.ok).toBe(true);
+    if (g.ok && a.ok) {
+      expect(g.tookOver).toBe(true);
+      expect(g.crawlId).not.toBe(a.crawlId);
+      // A's stale release must not free B's lease.
+      reg.releaseCrawl('primary', a.crawlId);
+      expect(reg.peekLease('primary')?.ownerClientId).toBe('B');
+    }
+  });
+
+  it('enforces ownership; a null-owner lease is owned by anyone (legacy)', async () => {
+    const { reg } = makeRegistry();
+    await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    expect(reg.isOwner('primary', 'A')).toBe(true);
+    expect(reg.isOwner('primary', 'B')).toBe(false);
+
+    const { reg: reg2 } = makeRegistry();
+    await reg2.acquireCrawl('primary', { clientId: null, startUrl: 'x' });
+    expect(reg2.isOwner('primary', 'anyone')).toBe(true);
+  });
+
+  it('times out a queued waiter and removes it', async () => {
+    const { reg, fireTimers } = makeRegistry();
+    const a = await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    const bPromise = reg.acquireCrawl('primary', {
+      clientId: 'B',
+      startUrl: 'y',
+      onBusy: 'queue',
+      queueTimeoutMs: 1000,
+    });
+    fireTimers();
+    const b = await bPromise;
+    expect(b.ok).toBe(false);
+    if (!b.ok) expect(b.error).toBe('queue-timeout');
+    expect(reg.queueDepth('primary')).toBe(0);
+    if (a.ok) reg.releaseCrawl('primary', a.crawlId);
+    expect(reg.peekLease('primary')).toBeNull();
+  });
+
+  it('dropSession expires queued waiters', async () => {
+    const { reg } = makeRegistry();
+    await reg.acquireCrawl('primary', { clientId: 'A', startUrl: 'x' });
+    const bPromise = reg.acquireCrawl('primary', {
+      clientId: 'B',
+      startUrl: 'y',
+      onBusy: 'queue',
+    });
+    reg.dropSession('primary');
+    const b = await bPromise;
+    expect(b.ok).toBe(false);
+    if (!b.ok) expect(b.error).toBe('session-expired');
+    expect(reg.peekLease('primary')).toBeNull();
+  });
+
+  it('keeps sessions independent — one lease does not block another', async () => {
+    const { reg } = makeRegistry();
+    const a = await reg.acquireCrawl('sess_1', { clientId: 'A', startUrl: 'x' });
+    const b = await reg.acquireCrawl('sess_2', { clientId: 'B', startUrl: 'y' });
+    expect(a.ok).toBe(true);
+    expect(b.ok).toBe(true);
+    expect(reg.peekLease('sess_1')?.ownerClientId).toBe('A');
+    expect(reg.peekLease('sess_2')?.ownerClientId).toBe('B');
+  });
+});
+
+describe('computeGlobalThrottleScale', () => {
+  it('returns 1 when the summed concurrency fits the budget', () => {
+    expect(computeGlobalThrottleScale([10, 10], 32)).toBe(1);
+    expect(computeGlobalThrottleScale([], 32)).toBe(1);
+    expect(computeGlobalThrottleScale([32], 32)).toBe(1);
+  });
+
+  it('scales down proportionally when over budget', () => {
+    expect(computeGlobalThrottleScale([40, 40], 40)).toBeCloseTo(0.5, 5);
+    expect(computeGlobalThrottleScale([30, 30, 30], 45)).toBeCloseTo(0.5, 5);
+  });
+
+  it('never drops below the 0.1 floor', () => {
+    expect(computeGlobalThrottleScale([200, 200], 1)).toBe(0.1);
+  });
+});

tests/bridge-routing.test.ts

@@ -0,0 +1,264 @@
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+  startMcpBridge,
+  stopMcpBridge,
+  type McpBridgeDeps,
+} from '../apps/desktop/src/main/mcp-bridge.js';
+
+/**
+ * Exercises the localhost bridge's HTTP layer end-to-end (auth, header-based
+ * session routing, typed error → status mapping, capabilities, session
+ * lifecycle dispatch) against fake deps — no Electron, no real crawler.
+ */
+
+let dir: string;
+let port: number;
+let token: string;
+const calls: Array<[string, ...unknown[]]> = [];
+const sessions = new Map<string, { label: string; doc: string | null }>();
+let seq = 0;
+
+const deps: McpBridgeDeps = {
+  userDataDir: '', // set in beforeAll
+  getCapabilities: () => ({
+    bridgeVersion: 2,
+    appVersion: '0.10.0',
+    features: ['crawl-lease', 'crawl-queue', 'sessions'],
+    limits: { maxHeadlessSessions: 3, maxQueueDepth: 4, sessionIdleTtlMs: 1_800_000 },
+  }),
+  startCrawl: async (input, ctx) => {
+    calls.push(['start', ctx.sessionId, ctx.clientId, input.onBusy ?? null]);
+    if (input.onBusy === 'reject')
+      return { error: 'crawl-in-progress', message: 'busy', details: { sessionId: ctx.sessionId } };
+    return {
+      ok: true,
+      crawlId: 'cr_x',
+      sessionId: ctx.sessionId,
+      config: { startUrl: input.startUrl ?? '' } as never,
+      queued: false,
+      waitedMs: 0,
+    };
+  },
+  stopCrawl: (ctx, opts) => {
+    calls.push(['stop', ctx.sessionId, opts.force]);
+    return { ok: true };
+  },
+  pauseCrawl: () => ({ ok: true }),
+  resumeCrawl: () => ({ ok: true }),
+  clearCrawl: async (ctx, opts) => {
+    if (ctx.sessionId === 'primary' && !opts.force)
+      return { error: 'project-locked', message: 'unsaved work', details: {} };
+    return { ok: true };
+  },
+  hasAction: (name) => name === 'export-csv',
+  runAction: async (name, _input, ctx) => {
+    calls.push(['action', name, ctx.sessionId]);
+    return { session: ctx.sessionId };
+  },
+  getCrawlProgress: (ctx) => ({
+    progress: null,
+    crawlId: null,
+    sessionId: ctx.sessionId,
+    ownerClientId: null,
+    ownedByCaller: false,
+  }),
+  getProjectInfo: (ctx) => ({ projectPath: `/p/${ctx.sessionId}`, urlsCrawled: 0 }),
+  listSessions: () => [
+    {
+      sessionId: 'primary',
+      label: 'Primary',
+      kind: 'primary',
+      documentPath: null,
+      dbPath: null,
+      urlsCrawled: 0,
+      crawl: null,
+      ownerClientId: null,
+      createdAt: 0,
+      lastUsedAt: 0,
+    },
+    ...[...sessions.entries()].map(([id, s]) => ({
+      sessionId: id,
+      label: s.label,
+      kind: 'headless' as const,
+      documentPath: s.doc,
+      dbPath: `/db/${id}`,
+      urlsCrawled: 0,
+      crawl: null,
+      ownerClientId: null,
+      createdAt: 0,
+      lastUsedAt: 0,
+    })),
+  ],
+  createSession: async (input) => {
+    const id = `sess_${++seq}`;
+    sessions.set(id, { label: input.label ?? `agent-${seq}`, doc: input.projectPath ?? null });
+    return {
+      sessionId: id,
+      dbPath: `/db/${id}`,
+      documentPath: input.projectPath ?? null,
+      created: true,
+      label: input.label ?? `agent-${seq}`,
+    };
+  },
+  closeSession: async (input, ctx) => {
+    if (!sessions.has(ctx.sessionId))
+      return { error: 'session-not-found', message: 'gone', details: {} };
+    sessions.delete(ctx.sessionId);
+    return input.save ? { ok: true, savedTo: input.savePath ?? '/s.seoproject' } : { ok: true };
+  },
+  saveSession: async (input, ctx) => {
+    if (!sessions.has(ctx.sessionId))
+      return { error: 'session-not-found', message: 'gone', details: {} };
+    return { filePath: input.projectPath ?? '/d.seoproject', bytesWritten: 1234 };
+  },
+  log: () => {},
+};
+
+beforeAll(async () => {
+  dir = mkdtempSync(join(tmpdir(), 'fc-bridge-'));
+  deps.userDataDir = dir;
+  const started = await startMcpBridge(deps);
+  if (!started) throw new Error('bridge failed to start');
+  port = started.port;
+  token = started.token;
+  const disc = JSON.parse(readFileSync(join(dir, 'mcp-bridge.json'), 'utf8'));
+  expect(disc.version).toBe(2);
+});
+
+afterAll(() => {
+  stopMcpBridge();
+  try {
+    rmSync(dir, { recursive: true, force: true });
+  } catch {
+    /* ignore */
+  }
+});
+
+const base = () => `http://127.0.0.1:${port}`;
+async function req(
+  method: string,
+  path: string,
+  body?: unknown,
+  headers: Record<string, string> = {},
+) {
+  const r = await fetch(base() + path, {
+    method,
+    headers: {
+      Authorization: `Bearer ${token}`,
+      'Content-Type': 'application/json',
+      ...headers,
+    },
+    body: body !== undefined ? JSON.stringify(body) : undefined,
+  });
+  let json: unknown = null;
+  try {
+    json = await r.json();
+  } catch {
+    /* no body */
+  }
+  return { status: r.status, body: json as Record<string, unknown> };
+}
+
+describe('MCP bridge routing', () => {
+  it('rejects a bad token with 401', async () => {
+    const r = await fetch(base() + '/v1/capabilities', {
+      headers: { Authorization: 'Bearer wrong' },
+    });
+    expect(r.status).toBe(401);
+  });
+
+  it('serves capabilities with the sessions feature', async () => {
+    const r = await req('GET', '/v1/capabilities');
+    expect(r.status).toBe(200);
+    expect(r.body.bridgeVersion).toBe(2);
+    expect((r.body.features as string[]).includes('sessions')).toBe(true);
+  });
+
+  it('routes a headerless request to the primary session with a null client', async () => {
+    const r = await req('POST', '/v1/crawl/start', { startUrl: 'x' });
+    expect(r.status).toBe(200);
+    expect(r.body.sessionId).toBe('primary');
+    expect(calls.at(-1)).toEqual(['start', 'primary', null, null]);
+  });
+
+  it('routes by session + client headers', async () => {
+    const r = await req(
+      'POST',
+      '/v1/crawl/start',
+      { startUrl: 'y' },
+      { 'x-freecrawl-session': 'sess_9', 'x-freecrawl-client': 'clientA' },
+    );
+    expect(r.status).toBe(200);
+    expect(r.body.sessionId).toBe('sess_9');
+    expect(calls.at(-1)).toEqual(['start', 'sess_9', 'clientA', null]);
+  });
+
+  it('maps crawl-in-progress to 409', async () => {
+    const r = await req('POST', '/v1/crawl/start', { onBusy: 'reject' });
+    expect(r.status).toBe(409);
+    expect(r.body.error).toBe('crawl-in-progress');
+  });
+
+  it('guards clear on primary with unsaved work (project-locked 409), overridable by force', async () => {
+    const locked = await req('POST', '/v1/crawl/clear', {});
+    expect(locked.status).toBe(409);
+    expect(locked.body.error).toBe('project-locked');
+    const forced = await req('POST', '/v1/crawl/clear', { force: true });
+    expect(forced.status).toBe(200);
+    expect(forced.body.ok).toBe(true);
+  });
+
+  it('propagates the force flag on stop', async () => {
+    await req('POST', '/v1/crawl/stop', { force: true }, { 'x-freecrawl-session': 'primary' });
+    expect(calls.at(-1)).toEqual(['stop', 'primary', true]);
+  });
+
+  it('routes actions to the request session; unknown actions 400', async () => {
+    const ok = await req('POST', '/v1/action/export-csv', {}, { 'x-freecrawl-session': 'sess_3' });
+    expect(ok.status).toBe(200);
+    expect(ok.body.session).toBe('sess_3');
+    const bad = await req('POST', '/v1/action/nope', {});
+    expect(bad.status).toBe(400);
+    expect(bad.body.error).toBe('bad-request');
+  });
+
+  it('runs the session create → list → save → close lifecycle', async () => {
+    const created = await req('POST', '/v1/session/create', { mode: 'scratch', label: 't' });
+    expect(created.status).toBe(200);
+    const id = created.body.sessionId as string;
+    expect(id.startsWith('sess_')).toBe(true);
+
+    const list = await req('GET', '/v1/session/list');
+    expect((list.body.sessions as Array<{ sessionId: string }>).some((s) => s.sessionId === id)).toBe(
+      true,
+    );
+
+    const saved = await req(
+      'POST',
+      '/v1/session/save',
+      { projectPath: '/x.seoproject' },
+      { 'x-freecrawl-session': id },
+    );
+    expect(saved.body.bytesWritten).toBe(1234);
+
+    const closed = await req(
+      'POST',
+      '/v1/session/close',
+      { save: true, savePath: '/y.seoproject' },
+      { 'x-freecrawl-session': id },
+    );
+    expect(closed.body.savedTo).toBe('/y.seoproject');
+
+    const closedAgain = await req('POST', '/v1/session/close', {}, { 'x-freecrawl-session': id });
+    expect(closedAgain.status).toBe(404);
+    expect(closedAgain.body.error).toBe('session-not-found');
+  });
+
+  it('returns 400 for an unknown route', async () => {
+    const r = await req('GET', '/v1/nope');
+    expect(r.status).toBe(400);
+  });
+});

tests/feedback-fixes.test.ts

@@ -0,0 +1,158 @@
+import { describe, it, expect } from 'vitest';
+import { ProjectDb } from '../packages/db/src/project-db.js';
+import type { AdvancedFilter, FilterClause } from '@freecrawl/shared-types';
+
+// These exercise the DB-layer fixes end-to-end against a real (in-memory)
+// migrated ProjectDb — the highest-risk SQL changes from the feedback pass:
+//   #13  redirect/error pages must NOT count as "missing" on-page elements
+//   #14  GSC data must attach across a trailing-slash URL variant
+//   #2   Advanced filter must query custom-extraction values (and stay safe)
+
+function makeDb(): ProjectDb {
+  // runMigrations runs inside the constructor for a writable connection, so
+  // an in-memory file is a fully-formed, empty project.
+  return new ProjectDb(':memory:');
+}
+
+const filter = (clauses: FilterClause[]): AdvancedFilter => ({
+  groups: [{ clauses }],
+});
+
+describe('#13 — no fake SEO errors on 3xx/4xx pages', () => {
+  it('excludes a 301 redirect from the "missing element" counters', () => {
+    const db = makeDb();
+    // A real 200 page with no title/meta/h1/lang/viewport — genuinely missing.
+    db.upsertUrl({ url: 'https://x.com/ok', depth: 0, statusCode: 200, contentKind: 'html' });
+    // A 301 redirect carries none of those by nature — must NOT be flagged.
+    db.upsertUrl({
+      url: 'https://x.com/redir',
+      depth: 0,
+      statusCode: 301,
+      contentKind: 'html',
+      indexability: 'non-indexable:redirect',
+    });
+
+    const issues = db.getOverviewCounts().issues;
+    expect(issues.titleMissing).toBe(1);
+    expect(issues.metaMissing).toBe(1);
+    expect(issues.h1Missing).toBe(1);
+    expect(issues.langMissing).toBe(1);
+    expect(issues.viewportMissing).toBe(1);
+
+    // The drill-down table must agree with the counter (only the 200 page).
+    const drill = db.queryUrls({ limit: 100, offset: 0, category: 'issues:title-missing' });
+    expect(drill.total).toBe(1);
+    expect(drill.rows[0]?.url).toBe('https://x.com/ok');
+    db.close();
+  });
+
+  it('still flags genuinely-missing elements on 200 pages', () => {
+    const db = makeDb();
+    db.upsertUrl({ url: 'https://x.com/a', depth: 0, statusCode: 200, contentKind: 'html' });
+    db.upsertUrl({
+      url: 'https://x.com/b',
+      depth: 0,
+      statusCode: 200,
+      contentKind: 'html',
+      title: 'Has title',
+      metaDescription: 'Has meta',
+      h1: 'Has H1',
+      lang: 'en',
+      viewport: 'width=device-width',
+    });
+    const issues = db.getOverviewCounts().issues;
+    expect(issues.titleMissing).toBe(1); // only /a
+    expect(issues.viewportMissing).toBe(1);
+    db.close();
+  });
+});
+
+describe('#14 — GSC join tolerates trailing-slash URL variants', () => {
+  it('attaches GSC data to a crawled /page/ from a GSC /page (slash-insensitive default)', () => {
+    const db = makeDb();
+    db.upsertUrl({ url: 'https://x.com/page/', depth: 0, statusCode: 200, contentKind: 'html' });
+    db.replaceGscResults(
+      [{ url: 'https://x.com/page', clicks: 5, impressions: 100, ctr: 0.05, position: 3 }],
+      '2026-01-01T00:00:00Z',
+    );
+
+    const tol = db.queryGsc({ limit: 100, offset: 0, filter: 'all', matchSlash: true });
+    const row = tol.rows.find((r) => r.url === 'https://x.com/page/');
+    expect(row?.gsc?.impressions).toBe(100);
+
+    // With slash-matching turned off the exact string differs → no attach.
+    const exact = db.queryGsc({ limit: 100, offset: 0, filter: 'all', matchSlash: false });
+    const rowExact = exact.rows.find((r) => r.url === 'https://x.com/page/');
+    expect(rowExact?.gsc).toBeNull();
+    db.close();
+  });
+});
+
+describe('#2 — Advanced filter on custom-extraction values', () => {
+  it('filters rows by an extracted field value', () => {
+    const db = makeDb();
+    db.upsertUrl({
+      url: 'https://x.com/a',
+      depth: 0,
+      statusCode: 200,
+      contentKind: 'html',
+      extractionResults: JSON.stringify({ h2: 'Recommended Products' }),
+    });
+    db.upsertUrl({
+      url: 'https://x.com/b',
+      depth: 0,
+      statusCode: 200,
+      contentKind: 'html',
+      extractionResults: JSON.stringify({ h2: 'Newsletter' }),
+    });
+    db.upsertUrl({ url: 'https://x.com/c', depth: 0, statusCode: 200, contentKind: 'html' });
+
+    const contains = db.queryUrls({
+      limit: 100,
+      offset: 0,
+      filter: filter([{ field: 'extraction', extractionKey: 'h2', operator: 'contains', value: 'Recommend' }]),
+    });
+    expect(contains.total).toBe(1);
+    expect(contains.rows[0]?.url).toBe('https://x.com/a');
+
+    const none = db.queryUrls({
+      limit: 100,
+      offset: 0,
+      filter: filter([{ field: 'extraction', extractionKey: 'h2', operator: 'contains', value: 'zzz' }]),
+    });
+    expect(none.total).toBe(0);
+
+    const empty = db.queryUrls({
+      limit: 100,
+      offset: 0,
+      filter: filter([{ field: 'extraction', extractionKey: 'h2', operator: 'is_empty', value: '' }]),
+    });
+    expect(empty.rows.map((r) => r.url)).toEqual(['https://x.com/c']);
+    db.close();
+  });
+
+  it('drops (does not execute) an unsafe extraction key instead of crashing', () => {
+    const db = makeDb();
+    db.upsertUrl({
+      url: 'https://x.com/a',
+      depth: 0,
+      statusCode: 200,
+      contentKind: 'html',
+      extractionResults: JSON.stringify({ h2: 'x' }),
+    });
+    db.upsertUrl({ url: 'https://x.com/b', depth: 0, statusCode: 200, contentKind: 'html' });
+
+    // A key with a double-quote would break the JSON path / could be an
+    // injection vector — the clause must be rejected, the query must not
+    // throw, and with no valid clause the filter is a no-op (all rows).
+    const res = db.queryUrls({
+      limit: 100,
+      offset: 0,
+      filter: filter([
+        { field: 'extraction', extractionKey: 'h2"} OR 1=1 --', operator: 'contains', value: 'x' },
+      ]),
+    });
+    expect(res.total).toBe(2);
+    db.close();
+  });
+});

vitest.config.ts

@@ -0,0 +1,41 @@
+import { defineConfig } from 'vitest/config';
+
+// Unit tests target the Electron-free, pure modules (crawl-lease registry,
+// the localhost bridge's HTTP routing). Anything that imports `electron` is
+// out of scope here — those paths are exercised by the app itself.
+export default defineConfig({
+  plugins: [
+    {
+      // `node:sqlite` is a newer Node builtin that vite's SSR resolver
+      // doesn't yet recognise: it strips the `node:` prefix, looks for a
+      // package called `sqlite`, and fails to load it. Marking it external
+      // isn't honoured by vitest's module runner here, so instead serve a
+      // tiny virtual shim that re-exports the real builtin via createRequire
+      // at run time (Node loads `node:sqlite` natively). Needed by the
+      // ProjectDb-backed tests; app/build paths are unaffected.
+      name: 'node-sqlite-shim',
+      enforce: 'pre',
+      resolveId(id) {
+        if (id === 'node:sqlite' || id === 'sqlite') return '\0node-sqlite-shim';
+        return null;
+      },
+      load(id) {
+        if (id === '\0node-sqlite-shim') {
+          return [
+            "import { createRequire } from 'node:module';",
+            'const req = createRequire(import.meta.url);',
+            "const sqlite = req('node:sqlite');",
+            'export const DatabaseSync = sqlite.DatabaseSync;',
+            'export const StatementSync = sqlite.StatementSync;',
+            'export default sqlite;',
+          ].join('\n');
+        }
+        return null;
+      },
+    },
+  ],
+  test: {
+    include: ['tests/**/*.test.ts'],
+    environment: 'node',
+  },
+});