feat: graph layouts + crawl path trace, XPath axes, browser login

2026-10-18 00:39:38 +02:00 · 2026-06-29 04:11:00 +03:00
1 parent c6ba4e4e41
commit 139ac0d5d7
@@ -31,9 +31,9 @@ A high-performance website crawler built for serious SEO audits. Targets <kbd>1M

 FreeCrawl is a desktop SEO crawler that scales to **1M+ URLs on a single machine** at 80–150 URL/s via undici with keep-alive, with optional **JavaScript rendering** through headless Chromium (Playwright) that captures the post-JS DOM, full-page / above-fold / mobile screenshots, LCP candidate elements, and a per-URL mobile usability audit. It runs **150+ on-page SEO checks across 30 top-level tabs** — exact and near-duplicate clustering (SimHash + LSH) with a dedicated Cluster view, full hreflang validation (reciprocity, self-reference, inconsistent lang), OpenGraph / Twitter Card / JSON-LD / Web App Manifest parsing, AMP smoke validation, structured-data validation across 17 schema types (duplicate `@id`, malformed `@type`, missing required + Google-recommended props), a configurable performance budget (response-time / page-size / LCP / CLS ceilings), WCAG accessibility checks (`<main>` landmark, skip-link, ARIA roles, heading order), security headers + SSL/TLS chain audit + active/passive mixed content split, readability scores (Flesch, Flesch–Kincaid, Gunning Fog), and a 10-rule custom extraction engine — **CSS selectors, XPath, regex, and JSONPath (for JSON API responses)** — with a **live Preview dialog and JSON import / export** for sharing rule sets.

-The **dense dark UI** renders virtualized 1M-row tables that live-stream rows during a crawl, with **List ↔ Tree ↔ Cluster view toggles**, **column pin (sticky-left) + drag-to-reorder + show/hide**, advanced AND/OR filters across 24 fields × 12 operators, per-tab quick-filter dropdowns, a 16-tab Details panel per URL (URL Details, Outline, Inlinks, Outlinks, Images, Resources, Extracted Data, SERP Snippet, HTTP Headers, Cookies, Structured Data, View Source, View Rendered, Screenshot, Duplicates, Analytics), an English + Turkish UI including every Settings panel, **in-app scheduled crawls** (hourly / daily / weekly / custom), a live memory monitor, robots.txt syntax validator, URL rewriting with live preview, project-vs-project compare diff, a Cytoscape link graph in its own native window, a **standalone Log File Analyzer window** (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, and crawl × log orphan detection — with 40+ bot detection, optional reverse-DNS verification, per-bot URL filtering, and one-click CSV / Excel export of every table), and a `.seoproject` file association for OS double-click.
+The **dense dark UI** renders virtualized 1M-row tables that live-stream rows during a crawl, with **List ↔ Tree ↔ Cluster view toggles**, **column pin (sticky-left) + drag-to-reorder + show/hide**, advanced AND/OR filters across 24 fields × 12 operators, per-tab quick-filter dropdowns, a 16-tab Details panel per URL (URL Details, Outline, Inlinks, Outlinks, Images, Resources, Extracted Data, SERP Snippet, HTTP Headers, Cookies, Structured Data, View Source, View Rendered, Screenshot, Duplicates, Analytics), an English + Turkish UI including every Settings panel, **in-app scheduled crawls** (hourly / daily / weekly / custom), a live memory monitor, robots.txt syntax validator, URL rewriting with live preview, project-vs-project compare diff, a Cytoscape link graph in its own native window (force-directed, BFS / radial / Sugiyama tree, and directory-tree layouts, a **By-LCP above-the-fold colour overlay**, and a click-to-trace **Crawl Path Report** that highlights the shortest discovery path from the homepage to any page), a **standalone Log File Analyzer window** (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, and crawl × log orphan detection — with 40+ bot detection, optional reverse-DNS verification, per-bot URL filtering, and one-click CSV / Excel export of every table), and a `.seoproject` file association for OS double-click.

-Integrations cover **Google Search Console** (clicks, impressions, CTR, position + URL Inspection coverage), **Google Analytics 4** (sessions, users, bounce, engagement), **PageSpeed Insights** Lighthouse audits, custom **AI prompts via OpenAI / Anthropic Claude / local Ollama** with `{url}`/`{title}`/`{description}`/`{h1}`/`{body}` variables, and **SEO authority providers** (Ahrefs / Majestic / Moz / Semrush) behind a single dropdown — each integration card now ships with an **in-app step-by-step Setup Guide modal** in English and Turkish that walks you through OAuth client creation, test-user setup, API enablement, and the most common errors. Exports go through a unified **Export Crawl Data dialog** (Excel `.xlsx` / CSV UTF-8 / JSON / XML with hierarchical category selection and nested folder output) plus a standalone HTML audit report, sitemap generator (standard / image / hreflang / sharded / gzipped), and direct streaming to **Google Sheets** and **BigQuery**. Project files can be saved as **password-protected encrypted snapshots** (`.seoproject.enc`, AES-256-GCM + PBKDF2). The **MCP server** exposes **89 tools** that let Claude Code or any MCP client drive crawls live, query every UI surface, run single or bulk exports, save / open encrypted project snapshots, fetch GSC/GA4 data on demand, and modify settings — every action a human user can take in the desktop is callable from an agent. Crawl-completion webhooks, OS notifications, 22 built-in reports (histograms, top/bottom URLs, link positions, top words, cross-source orphans), per-URL **Duplicates** view, and a custom-CSS theme override round out the suite. Everything runs **fully local** — no telemetry, no cloud, MIT-style license.
+Integrations cover **Google Search Console** (clicks, impressions, CTR, position + URL Inspection coverage), **Google Analytics 4** (sessions, users, bounce, engagement), **PageSpeed Insights** Lighthouse audits, custom **AI prompts via OpenAI / Anthropic Claude / local Ollama** with `{url}`/`{title}`/`{description}`/`{h1}`/`{body}` variables, and **SEO authority providers** (Ahrefs / Majestic / Moz / Semrush) behind a single dropdown — each integration card now ships with an **in-app step-by-step Setup Guide modal** in English and Turkish that walks you through OAuth client creation, test-user setup, API enablement, and the most common errors. Exports go through a unified **Export Crawl Data dialog** (Excel `.xlsx` / CSV UTF-8 / JSON / XML with hierarchical category selection and nested folder output) plus a standalone HTML audit report, sitemap generator (standard / image / hreflang / sharded / gzipped), and direct streaming to **Google Sheets** and **BigQuery**. Project files can be saved as **password-protected encrypted snapshots** (`.seoproject.enc`, AES-256-GCM + PBKDF2). The **MCP server** exposes **90 tools** that let Claude Code or any MCP client drive crawls live, query every UI surface, run single or bulk exports, save / open encrypted project snapshots, fetch GSC/GA4 data on demand, and modify settings — every action a human user can take in the desktop is callable from an agent. Crawl-completion webhooks, OS notifications, 22 built-in reports (histograms, top/bottom URLs, link positions, top words, cross-source orphans), per-URL **Duplicates** view, and a custom-CSS theme override round out the suite. Everything runs **fully local** — no telemetry, no cloud, MIT-style license.

 <br />

@@ -205,7 +205,7 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 1. **Read-only data access** to the SQLite project — runs alongside the desktop app without contention (WAL allows concurrent readers).
 2. **Live crawl control** — when the desktop app is open, an agent can start / pause / resume / stop crawls and poll progress in real time. This goes through a localhost-only HTTP bridge (127.0.0.1, ephemeral random port, 32-byte Bearer token auth, discovery file written to `<userData>/mcp-bridge.json` on app launch).

-**89 tools** — every action a human can take in the desktop is callable from an agent. Representative groups:
+**90 tools** — every action a human can take in the desktop is callable from an agent. Representative groups:

 | Group | Tools |
 | :--- | :--- |
@@ -214,7 +214,7 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 | 🧩 **Integration rows** | `query_gsc`, `query_ga4`, `query_pagespeed`, `query_ai`, `query_seo` |
 | 🔎 **Specialised queries** | `query_images`, `query_broken_links`, `list_duplicate_clusters` |
 | 📈 **Reports** | `report_status_code_histogram`, `report_indexability_distribution`, `report_content_kind_distribution`, `report_depth_histogram`, `report_response_time_histogram`, `report_inlinks_histogram`, `report_word_count_histogram`, `report_url_length_histogram`, `report_top_urls_by`, `report_top_anchor_texts`, `report_external_domain_health`, `report_image_weight_per_page`, `report_link_position_breakdown`, `report_pages_per_directory`, `report_word_count_per_directory`, `report_sitemap_orphans`, `report_analytics_coverage`, `report_server_header_breakdown`, `report_query_string_variants`, `report_sitemap_priority_mismatch`, `report_top_words` |
-| 📤 **Exports & mutations** | `export_csv`, `export_json`, `export_xml`, `export_html_report`, `export_tabular`, `export_bulk`, `export_broken_links`, `export_images`, `sitemap_generate`, `sitemap_validate`, `robots_test`, `robots_validate`, `compare_load`, `crawl_add_url`, `respider_urls`, `remove_urls`, `data_delete_by_domain`, `graph_snapshot`, `url_rewrite_preview`, `extraction_preview` |
+| 📤 **Exports & mutations** | `export_csv`, `export_json`, `export_xml`, `export_html_report`, `export_tabular`, `export_bulk`, `export_broken_links`, `export_images`, `sitemap_generate`, `sitemap_validate`, `robots_test`, `robots_validate`, `compare_load`, `crawl_add_url`, `respider_urls`, `remove_urls`, `data_delete_by_domain`, `graph_snapshot`, `get_crawl_path`, `url_rewrite_preview`, `extraction_preview` |
 | 🔐 **Project files** | `project_save_as`, `project_save_encrypted`, `project_open_encrypted` |
 | 🔌 **Integration fetch** | `google_auth_status`, `gsc_list_sites`, `gsc_fetch`, `ga4_list_properties`, `ga4_fetch`, `pagespeed_run`, `ai_run`, `seo_run`, `get_fetch_progress`, `cancel_fetch` |
 | ⚙ **Config & schedule** | `get_crawl_config`, `set_crawl_config`, `schedule_get`, `schedule_set` |
@@ -398,7 +398,7 @@ graph LR
 ## 📈 Status

 > [!NOTE]
-> **Active development.** All 29 analysis tabs (Internal, External, Response Codes, URL, Page Titles, Meta Description, H1, H2, Content, Images, Canonicals, Directives, Redirects, Pagination, Hreflang, AMP, Structured Data, Meta Refresh, Custom Extraction, Custom Search, Security, Duplicates, Links, Broken Links, SERP, **PageSpeed, Search Console, GA4, AI, SEO Authority**) plus standalone Visualization window, advanced search, **per-tab quick-filter dropdown** + **List/Tree view toggle**, 150+ issue categories, sitemap export variants, **Export Crawl Data dialog (XLSX / CSV-UTF-8 / JSON / XML with hierarchical tree picker + nested folder output)**, list mode, custom extraction, near-duplicate + exact-duplicate detection, hreflang validation, project compare, Cytoscape visualization, **Basic / Digest / Bearer + multi-step form-based login**, **HTTP / SOCKS5 proxy**, webhook, **MCP server with crawl control + live progress**, **Google PageSpeed / Search Console / URL Inspection / GA4 integrations**, **AI per-URL prompts (OpenAI / Anthropic / Ollama)**, **SEO Authority providers (Ahrefs / Majestic / Moz / Semrush)**, **Google Sheets + BigQuery direct export**, **encrypted project snapshots (AES-256-GCM)**, **cross-source orphan detection**, **JavaScript rendering with Playwright (post-JS DOM, screenshot capture, LCP candidate, Mobile Usability audit)**, **memory-limit auto-pause watchdog**, OS notifications, **robots.txt syntax validator**, URL rewriting + preview, **status-code diagnosis banner**, **live memory monitor**, **in-app scheduled crawl**, **multi-language UI (EN + TR) with full Settings coverage**, **standalone Log File Analyzer (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, crawl × log orphan detection, 40+ bot detection with reverse-DNS verification, per-bot URL filter, CSV / Excel export, and a headless `analyze-logs` CLI)**, **`.seoproject` file association**, in-app logs, and diagnostic popups are working. Cross-platform installers (Windows `.exe` + portable, macOS `.dmg`, Linux `.AppImage` / `.deb` / `.rpm`) — **release builds ship Playwright Chromium offline so JS rendering works on first launch**. Live-streaming UX with **first row in ~1 s**.
+> **Active development.** All 29 analysis tabs (Internal, External, Response Codes, URL, Page Titles, Meta Description, H1, H2, Content, Images, Canonicals, Directives, Redirects, Pagination, Hreflang, AMP, Structured Data, Meta Refresh, Custom Extraction, Custom Search, Security, Duplicates, Links, Broken Links, SERP, **PageSpeed, Search Console, GA4, AI, SEO Authority**) plus standalone Visualization window, advanced search, **per-tab quick-filter dropdown** + **List/Tree view toggle**, 150+ issue categories, sitemap export variants, **Export Crawl Data dialog (XLSX / CSV-UTF-8 / JSON / XML with hierarchical tree picker + nested folder output)**, list mode, custom extraction, near-duplicate + exact-duplicate detection, hreflang validation, project compare, **Cytoscape visualization (force-directed / BFS / radial / Sugiyama / directory-tree layouts + LCP above-the-fold overlay + click-to-trace Crawl Path Report)**, **Basic / Digest / Bearer + form-based login (HTTP multi-step + Playwright browser-driven SPA login)**, **HTTP / SOCKS5 proxy**, webhook, **MCP server with crawl control + live progress**, **Google PageSpeed / Search Console / URL Inspection / GA4 integrations**, **AI per-URL prompts (OpenAI / Anthropic / Ollama)**, **SEO Authority providers (Ahrefs / Majestic / Moz / Semrush)**, **Google Sheets + BigQuery direct export**, **encrypted project snapshots (AES-256-GCM)**, **cross-source orphan detection**, **JavaScript rendering with Playwright (post-JS DOM, screenshot capture, LCP candidate, Mobile Usability audit)**, **memory-limit auto-pause watchdog**, OS notifications, **robots.txt syntax validator**, URL rewriting + preview, **status-code diagnosis banner**, **live memory monitor**, **in-app scheduled crawl**, **multi-language UI (EN + TR) with full Settings coverage**, **standalone Log File Analyzer (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, crawl × log orphan detection, 40+ bot detection with reverse-DNS verification, per-bot URL filter, CSV / Excel export, and a headless `analyze-logs` CLI)**, **`.seoproject` file association**, in-app logs, and diagnostic popups are working. Cross-platform installers (Windows `.exe` + portable, macOS `.dmg`, Linux `.AppImage` / `.deb` / `.rpm`) — **release builds ship Playwright Chromium offline so JS rendering works on first launch**. Live-streaming UX with **first row in ~1 s**.
 >
 > **Upcoming (V2):** Plugin system, Light theme, Multi-window, Code-signing + auto-update.

@@ -37,6 +37,8 @@
    "@types/cytoscape": "^3.21.9",
    "clsx": "^2.1.1",
    "cytoscape": "^3.33.2",
+    "cytoscape-dagre": "^2.5.0",
+    "dagre": "^0.8.5",
    "i18next": "^26.2.0",
    "lucide-react": "^0.544.0",
    "react": "^19.2.0",
@@ -47,6 +49,8 @@
    "zustand": "^5.0.2"
  },
  "devDependencies": {
+    "@types/cytoscape-dagre": "^2.3.4",
+    "@types/dagre": "^0.7.54",
    "@types/react": "^19.2.0",
    "@types/react-dom": "^19.2.0",
    "@vitejs/plugin-react": "^5.0.0",
@@ -46,6 +46,8 @@ import {
  type CompareLoadResult,
  type GraphSnapshotInput,
  type GraphSnapshotResult,
+  type CrawlPathInput,
+  type CrawlPathResult,
  type AnchorTextRow,
  type RobotsTestInput,
  type SitemapValidateInput,
@@ -2304,6 +2306,7 @@ function registerIpc(): void {
          error: 'File does not contain a rules array.',
        };
      }
+      const validTypes = new Set(['css', 'regex', 'xpath', 'jsonpath']);
      const validOutputs = new Set([
        'text',
        'attribute',
@@ -2324,7 +2327,8 @@ function registerIpc(): void {
        if (
          typeof o['name'] !== 'string' ||
          typeof o['selector'] !== 'string' ||
-          (o['type'] !== 'css' && o['type'] !== 'regex') ||
+          typeof o['type'] !== 'string' ||
+          !validTypes.has(o['type']) ||
          typeof o['output'] !== 'string' ||
          !validOutputs.has(o['output'] as string) ||
          typeof o['multi'] !== 'string' ||
@@ -2341,7 +2345,7 @@ function registerIpc(): void {
        }
        const rule: CustomExtractionRule = {
          name: o['name'] as string,
-          type: o['type'] as 'css' | 'regex',
+          type: o['type'] as CustomExtractionRule['type'],
          selector: o['selector'] as string,
          output: o['output'] as CustomExtractionRule['output'],
          multi: o['multi'] as CustomExtractionRule['multi'],
@@ -4487,6 +4491,18 @@ function registerIpc(): void {
      return getDb().graphSnapshot(i.nodeLimit ?? 5000);
    },

+    'crawl-path': async (input) => {
+      const i = (input ?? {}) as { urlId?: number; url?: string };
+      let id = i.urlId;
+      if (id === undefined && typeof i.url === 'string') {
+        id = getDb().getUrlIdByUrl(i.url) ?? undefined;
+      }
+      if (id === undefined) {
+        return { path: [], reachedRoot: false };
+      }
+      return getDb().crawlPath(id);
+    },
+
    // ---- Faz 0.5 Increment 4 — Integration fetch triggers (simple ones). ----
    // GSC and GA4 fetches are single blocking API calls, no progress
    // streaming — safe to expose to MCP. PSI / AI / SEO have multi-URL
@@ -5961,6 +5977,13 @@ function registerIpc(): void {
    },
  );

+  ipcMain.handle(
+    IPC.crawlPath,
+    (_e, input: CrawlPathInput): CrawlPathResult => {
+      return getDb().crawlPath(input.urlId);
+    },
+  );
+
  ipcMain.handle(
    IPC.graphSnapshot,
    (_e, input: GraphSnapshotInput): GraphSnapshotResult => {
@@ -27,6 +27,8 @@ import {
  type CompareLoadResult,
  type GraphSnapshotInput,
  type GraphSnapshotResult,
+  type CrawlPathInput,
+  type CrawlPathResult,
  type AnchorTextRow,
  type FreeCrawlApi,
  type BrokenLinksQueryInput,
@@ -230,6 +232,8 @@ const api: FreeCrawlApi = {
    ipcRenderer.invoke(IPC.compareLoad, input),
  graphSnapshot: (input: GraphSnapshotInput): Promise<GraphSnapshotResult> =>
    ipcRenderer.invoke(IPC.graphSnapshot, input),
+  crawlPath: (input: CrawlPathInput): Promise<CrawlPathResult> =>
+    ipcRenderer.invoke(IPC.crawlPath, input),
  topAnchorTexts: (limit?: number): Promise<AnchorTextRow[]> =>
    ipcRenderer.invoke(IPC.topAnchorTexts, limit),
  sitemapGenerate: (input: SitemapGenerateInput): Promise<SitemapGenerateResult> =>
@@ -43,6 +43,7 @@ import {
 } from 'lucide-react';
 import clsx from 'clsx';
 import type {
+  BrowserLoginConfig,
  CrawlConfig,
  CrawlMode,
  CustomExtractionRule,
@@ -121,7 +122,9 @@ interface FormState {
  // auth + network
  auth: HttpAuth;
  formLoginEnabled: boolean;
+  formLoginMode: 'http' | 'browser';
  formLoginSteps: FormLoginStep[];
+  formLoginBrowser: BrowserLoginConfig;
  proxyUrl: string;
  excludeExtensionsText: string;
  maxRedirects: string;
@@ -433,11 +436,22 @@ function configToForm(c: CrawlConfig): FormState {
    webhookUrl: c.webhookUrl ?? '',
    auth: { ...(c.auth ?? { type: 'none' }) },
    formLoginEnabled: c.formLogin?.enabled ?? false,
+    formLoginMode: c.formLogin?.mode ?? 'http',
    formLoginSteps: (c.formLogin?.steps ?? []).map((s) => ({
      ...s,
      fields: (s.fields ?? []).map((f) => ({ ...f })),
      captures: (s.captures ?? []).map((cp) => ({ ...cp })),
    })),
+    formLoginBrowser: {
+      loginUrl: c.formLogin?.browser?.loginUrl ?? '',
+      usernameSelector: c.formLogin?.browser?.usernameSelector ?? '',
+      usernameValue: c.formLogin?.browser?.usernameValue ?? '',
+      passwordSelector: c.formLogin?.browser?.passwordSelector ?? '',
+      passwordValue: c.formLogin?.browser?.passwordValue ?? '',
+      submitSelector: c.formLogin?.browser?.submitSelector ?? '',
+      successSelector: c.formLogin?.browser?.successSelector ?? '',
+      waitMs: c.formLogin?.browser?.waitMs ?? 0,
+    },
    proxyUrl: c.proxyUrl ?? '',
    excludeExtensionsText: (c.excludeExtensions ?? []).join(', '),
    maxRedirects: String(c.maxRedirects ?? 10),
@@ -649,6 +663,7 @@ export function SettingsDialog({ open, onClose }: Props) {
      auth: form.auth,
      formLogin: {
        enabled: form.formLoginEnabled,
+        mode: form.formLoginMode,
        steps: form.formLoginSteps
          .map((s) => ({
            url: s.url.trim(),
@@ -657,6 +672,16 @@ export function SettingsDialog({ open, onClose }: Props) {
            captures: (s.captures ?? []).filter((c) => c.name.trim() && c.selector.trim()),
          }))
          .filter((s) => s.url),
+        browser: {
+          loginUrl: form.formLoginBrowser.loginUrl.trim(),
+          usernameSelector: form.formLoginBrowser.usernameSelector.trim(),
+          usernameValue: form.formLoginBrowser.usernameValue,
+          passwordSelector: form.formLoginBrowser.passwordSelector.trim(),
+          passwordValue: form.formLoginBrowser.passwordValue,
+          submitSelector: form.formLoginBrowser.submitSelector.trim(),
+          successSelector: (form.formLoginBrowser.successSelector ?? '').trim(),
+          waitMs: Math.max(0, num(String(form.formLoginBrowser.waitMs ?? 0), 0)),
+        },
      },
      proxyUrl: form.proxyUrl.trim(),
      excludeExtensions: form.excludeExtensionsText
@@ -2251,6 +2276,9 @@ function FormLoginEditor({ form, update }: PanelProps) {
  };
  const addStep = () =>
    setSteps([...steps, { url: '', method: 'POST', fields: [], captures: [] }]);
+  const browser = form.formLoginBrowser;
+  const updateBrowser = (patch: Partial<BrowserLoginConfig>) =>
+    update('formLoginBrowser', { ...browser, ...patch });

  return (
    <div className="mb-4 rounded border border-surface-800 bg-surface-950/40 p-3">
@@ -2272,6 +2300,80 @@ function FormLoginEditor({ form, update }: PanelProps) {
        })}
      </p>

+      <label className="mb-3 flex items-center gap-2 text-[11px] text-surface-300">
+        {t('settingsPanels.formLogin.mode', { defaultValue: 'Login method' })}
+        <select
+          className="rounded border border-surface-700 bg-surface-950 px-2 py-1 text-[12px] text-surface-100 focus:border-blue-500 focus:outline-none"
+          value={form.formLoginMode}
+          onChange={(e) => update('formLoginMode', e.target.value as 'http' | 'browser')}
+        >
+          <option value="http">
+            {t('settingsPanels.formLogin.modeHttp', { defaultValue: 'HTTP (form POST + CSRF)' })}
+          </option>
+          <option value="browser">
+            {t('settingsPanels.formLogin.modeBrowser', { defaultValue: 'Browser (SPA / JS login)' })}
+          </option>
+        </select>
+      </label>
+
+      {form.formLoginMode === 'browser' && (
+        <div className="mb-2 rounded border border-surface-800 bg-surface-900/40 p-2">
+          <p className="mb-2 text-[10px] text-surface-500">
+            {t('settingsPanels.formLogin.browserIntro', {
+              defaultValue:
+                'Drives a real Chromium through the login form once (for JS-heavy / SPA logins), then replays the captured session cookies on the undici crawl. Requires the bundled Playwright browser.',
+            })}
+          </p>
+          <Text
+            label={t('settingsPanels.formLogin.loginUrl', { defaultValue: 'Login page URL' })}
+            value={browser.loginUrl}
+            onChange={(v) => updateBrowser({ loginUrl: v })}
+            placeholder="https://app.example.com/login"
+          />
+          <div className="grid grid-cols-2 gap-2">
+            <Text
+              label={t('settingsPanels.formLogin.userSelector', { defaultValue: 'Username selector' })}
+              value={browser.usernameSelector}
+              onChange={(v) => updateBrowser({ usernameSelector: v })}
+              placeholder="#email"
+            />
+            <Text
+              label={t('settingsPanels.formLogin.userValue', { defaultValue: 'Username' })}
+              value={browser.usernameValue}
+              onChange={(v) => updateBrowser({ usernameValue: v })}
+            />
+            <Text
+              label={t('settingsPanels.formLogin.passSelector', { defaultValue: 'Password selector' })}
+              value={browser.passwordSelector}
+              onChange={(v) => updateBrowser({ passwordSelector: v })}
+              placeholder="#password"
+            />
+            <Text
+              label={t('settingsPanels.formLogin.passValue', { defaultValue: 'Password' })}
+              value={browser.passwordValue}
+              onChange={(v) => updateBrowser({ passwordValue: v })}
+              type="password"
+            />
+            <Text
+              label={t('settingsPanels.formLogin.submitSelector', { defaultValue: 'Submit selector' })}
+              value={browser.submitSelector}
+              onChange={(v) => updateBrowser({ submitSelector: v })}
+              placeholder="button[type=submit]"
+            />
+            <Text
+              label={t('settingsPanels.formLogin.successSelector', {
+                defaultValue: 'Logged-in selector (optional)',
+              })}
+              value={browser.successSelector ?? ''}
+              onChange={(v) => updateBrowser({ successSelector: v })}
+              placeholder="a[href*='logout']"
+            />
+          </div>
+        </div>
+      )}
+
+      {form.formLoginMode === 'http' && (
+        <>
      {steps.length === 0 && (
        <p className="mb-2 text-[11px] italic text-surface-500">
          {t('settingsPanels.formLogin.empty', { defaultValue: 'No steps — add one to define the login flow.' })}
@@ -2418,6 +2520,8 @@ function FormLoginEditor({ form, update }: PanelProps) {
      >
        <Plus className="h-3 w-3" /> {t('settingsPanels.formLogin.addStep', { defaultValue: 'Add login step' })}
      </button>
+        </>
+      )}
    </div>
  );
 }
@@ -3048,18 +3152,23 @@ function Text({
  onChange,
  info,
  example,
+  placeholder,
+  type = 'text',
 }: {
  label: string;
  value: string;
  onChange: (v: string) => void;
+  placeholder?: string;
+  type?: string;
 } & FieldInfo) {
  return (
    <label className="mb-3 flex flex-col gap-1">
      <FieldLabel label={label} info={info} example={example} />
      <input
-        type="text"
+        type={type}
        className="rounded border border-surface-700 bg-surface-950 px-2 py-1 text-[12px] text-surface-100 focus:border-blue-500 focus:outline-none"
        value={value}
+        placeholder={placeholder}
        onChange={(e) => onChange(e.target.value)}
        spellCheck={false}
      />
@@ -396,6 +396,12 @@ const TR_LABELS: Record<string, string> = {
  'Compound spring embedder': 'Bileşik yay yerleştirici',
  'Tree (BFS)': 'Ağaç (BFS)',
  'Roots-to-leaves layered': 'Kök-yaprak katmanlı',
+  'Force-Directed Tree': 'Force-Directed Ağaç',
+  'Sugiyama layered DAG (dagre)': 'Sugiyama katmanlı DAG (dagre)',
+  'Radial Tree': 'Radyal Ağaç',
+  'Roots at centre, leaves outward': 'Kök merkezde, yapraklar dışa',
+  'Directory Tree': 'Dizin Ağacı',
+  'Grouped by URL path segments': 'URL yol segmentlerine göre gruplu',
  Circle: 'Çember',
  'Equal radial spacing': 'Eşit radyal boşluk',
  Concentric: 'Konsantrik',
@@ -518,7 +518,18 @@
      "addField": "Add field",
      "captures": "Capture from response (name ← CSS selector)",
      "addCapture": "Add capture",
-      "addStep": "Add login step"
+      "addStep": "Add login step",
+      "mode": "Login method",
+      "modeHttp": "HTTP (form POST + CSRF)",
+      "modeBrowser": "Browser (SPA / JS login)",
+      "browserIntro": "Drives a real Chromium through the login form once (for JS-heavy / SPA logins), then replays the captured session cookies on the undici crawl. Requires the bundled Playwright browser.",
+      "loginUrl": "Login page URL",
+      "userSelector": "Username selector",
+      "userValue": "Username",
+      "passSelector": "Password selector",
+      "passValue": "Password",
+      "submitSelector": "Submit selector",
+      "successSelector": "Logged-in selector (optional)"
    },
    "network": {
      "intro": "Network-level controls: proxy override, file-extension exclusion, redirect hop cap.",
@@ -879,6 +890,13 @@
    "byStatus": "By Status",
    "byDepth": "By Depth",
    "byIndexability": "By Indexability",
+    "byLcp": "By LCP (above-fold)",
+    "crawlPath": "Crawl Path",
+    "crawlPathTitle": "Crawl Path mode — click a page node to trace its shortest path from the crawl root",
+    "crawlPathHint": "Select a page node to trace its shortest path from the crawl root.",
+    "crawlPathNone": "No path data for this node.",
+    "crawlPathDepth": "Crawl depth",
+    "crawlPathOrphan": "Did not reach the crawl root — possible orphan (sitemap-only) page.",
    "hoverOnly": "Hover Only",
    "top20": "Top 20",
    "all": "All",
@@ -517,7 +517,18 @@
      "addField": "Alan ekle",
      "captures": "Yanıttan yakala (ad ← CSS seçici)",
      "addCapture": "Yakalama ekle",
-      "addStep": "Giriş adımı ekle"
+      "addStep": "Giriş adımı ekle",
+      "mode": "Giriş yöntemi",
+      "modeHttp": "HTTP (form POST + CSRF)",
+      "modeBrowser": "Tarayıcı (SPA / JS girişi)",
+      "browserIntro": "Giriş formunu bir kez gerçek Chromium ile sürer (JS ağırlıklı / SPA girişleri için), ardından yakalanan oturum çerezlerini undici taramasında yeniden kullanır. Pakete dahil Playwright tarayıcısı gerekir.",
+      "loginUrl": "Giriş sayfası URL'si",
+      "userSelector": "Kullanıcı adı seçici",
+      "userValue": "Kullanıcı adı",
+      "passSelector": "Parola seçici",
+      "passValue": "Parola",
+      "submitSelector": "Gönder seçici",
+      "successSelector": "Giriş yapıldı seçici (opsiyonel)"
    },
    "network": {
      "intro": "Ağ seviyesi kontroller: proxy override, dosya-uzantısı hariç tutma, yönlendirme hop limiti.",
@@ -878,6 +889,13 @@
    "byStatus": "Duruma Göre",
    "byDepth": "Derinliğe Göre",
    "byIndexability": "İndekslenebilirliğe Göre",
+    "byLcp": "LCP'ye Göre (fold üstü)",
+    "crawlPath": "Tarama Yolu",
+    "crawlPathTitle": "Tarama Yolu modu — bir sayfa düğümüne tıklayarak tarama kökünden en kısa yolu izleyin",
+    "crawlPathHint": "Tarama kökünden en kısa yolu izlemek için bir sayfa düğümü seçin.",
+    "crawlPathNone": "Bu düğüm için yol verisi yok.",
+    "crawlPathDepth": "Tarama derinliği",
+    "crawlPathOrphan": "Tarama köküne ulaşılamadı — olası yetim (yalnızca sitemap) sayfa.",
    "hoverOnly": "Yalnızca Hover",
    "top20": "Top 20",
    "all": "Tümü",
@@ -1,15 +1,26 @@
 import { useEffect, useRef, useState } from 'react';
-import { RefreshCw, Sparkles, Settings2, RotateCcw, Download } from 'lucide-react';
+import { RefreshCw, Sparkles, Settings2, RotateCcw, Download, Route } from 'lucide-react';
 import cytoscape, { type Core } from 'cytoscape';
+import dagre from 'cytoscape-dagre';
 import { useTranslation } from 'react-i18next';
 import type {
  AnchorTextRow,
+  CrawlPathResult,
  GraphSnapshotResult,
  Indexability,
 } from '@freecrawl/shared-types';
 import { useAppStore } from '../store.js';
 import { translateLabel } from '../i18n/labels.js';

+// Register the dagre (Sugiyama layered DAG) layout extension once at module
+// load. Wrapped because Vite HMR can re-evaluate this module and cytoscape
+// throws on a duplicate registration.
+try {
+  cytoscape.use(dagre);
+} catch {
+  /* already registered (HMR re-import) */
+}
+
 /**
 * User-tunable layout knobs. Persisted in prefs under the
 * `vis-tuning` key so the settings survive across sessions.
@@ -160,16 +171,102 @@ function exportStandaloneHtml(
  downloadBlob(new Blob([html], { type: 'text/html' }), filename);
 }

-type LayoutKind = 'cose' | 'breadthfirst' | 'circle' | 'concentric';
+type LayoutKind =
+  | 'cose'
+  | 'breadthfirst'
+  | 'dagre'
+  | 'radial'
+  | 'directory'
+  | 'circle'
+  | 'concentric';

 const LAYOUTS: { key: LayoutKind; label: string; hint: string }[] = [
  { key: 'cose', label: 'Force-Directed', hint: 'Compound spring embedder' },
  { key: 'breadthfirst', label: 'Tree (BFS)', hint: 'Roots-to-leaves layered' },
+  { key: 'dagre', label: 'Force-Directed Tree', hint: 'Sugiyama layered DAG (dagre)' },
+  { key: 'radial', label: 'Radial Tree', hint: 'Roots at centre, leaves outward' },
+  { key: 'directory', label: 'Directory Tree', hint: 'Grouped by URL path segments' },
  { key: 'circle', label: 'Circle', hint: 'Equal radial spacing' },
  { key: 'concentric', label: 'Concentric', hint: 'By inlinks (centre = most-linked)' },
 ];

-type ColorMode = 'status' | 'depth' | 'indexability';
+type ColorMode = 'status' | 'depth' | 'indexability' | 'lcp';
+
+/** Colour a node by its Largest-Contentful-Paint candidate: grey when no
+ *  render data, amber when the LCP is an image (a prime optimisation
+ *  target), green when it is text / other markup. */
+function lcpColor(n: GraphSnapshotResult['nodes'][number]): string {
+  if (!n.lcpTag) return '#737373';
+  const tag = n.lcpTag.toLowerCase();
+  const isImage = !!n.lcpResourceUrl || tag === 'img' || tag === 'image' || tag === 'svg';
+  return isImage ? '#ea580c' : '#16a34a';
+}
+
+/**
+ * Build a directory-hierarchy element set from the link-graph nodes: a
+ * synthetic folder node per URL path segment, with each page hanging under
+ * its containing directory. Pure renderer-side transform — no extra query.
+ */
+function buildDirectoryElements(
+  nodes: GraphSnapshotResult['nodes'],
+  colorFn: (n: GraphSnapshotResult['nodes'][number]) => string,
+  sizeScale: number,
+): { data: Record<string, unknown> }[] {
+  const elements: { data: Record<string, unknown> }[] = [];
+  const dirSeen = new Set<string>();
+  const edgeSeen = new Set<string>();
+  const ensureDir = (id: string, label: string) => {
+    if (dirSeen.has(id)) return;
+    dirSeen.add(id);
+    elements.push({ data: { id, label, kind: 'dir', color: '#475569', size: 12 } });
+  };
+  const ensureEdge = (source: string, target: string) => {
+    const key = `${source}>${target}`;
+    if (edgeSeen.has(key)) return;
+    edgeSeen.add(key);
+    elements.push({ data: { id: `de:${key}`, source, target } });
+  };
+  for (const n of nodes) {
+    let host = '';
+    let segs: string[] = [];
+    try {
+      const u = new URL(n.url);
+      host = u.host;
+      segs = u.pathname.split('/').filter(Boolean);
+    } catch {
+      continue;
+    }
+    const rootId = `dir:${host}/`;
+    ensureDir(rootId, host);
+    // All but the final segment are the page's containing folders.
+    const dirSegs = segs.slice(0, Math.max(0, segs.length - 1));
+    let parentId = rootId;
+    let acc = '';
+    for (const seg of dirSegs) {
+      acc += `/${seg}`;
+      const id = `dir:${host}${acc}`;
+      ensureDir(id, `${seg}/`);
+      ensureEdge(parentId, id);
+      parentId = id;
+    }
+    const leafLabel = segs.length > 0 ? segs[segs.length - 1]! : host;
+    elements.push({
+      data: {
+        id: String(n.id),
+        label: leafLabel,
+        fullUrl: n.url,
+        statusCode: n.statusCode ?? '',
+        inlinks: n.inlinks,
+        color: colorFn(n),
+        size: nodeSize(n.inlinks, sizeScale),
+        kind: 'page',
+        isTop: 0,
+      },
+    });
+    ensureEdge(parentId, String(n.id));
+  }
+  return elements;
+}

 function statusColor(code: number | null): string {
  if (code === null) return '#737373';
@@ -213,6 +310,18 @@ export function VisualizationTab() {
  const [colorMode, setColorMode] = useState<ColorMode>('status');
  const [nodeLimit, setNodeLimit] = useState(150);
  const [labelMode, setLabelMode] = useState<LabelMode>('hover');
+  // Crawl-path trace: when enabled, selecting a node fetches & highlights
+  // the shortest discovery path from the crawl root to that page.
+  const [pathMode, setPathMode] = useState(false);
+  const [crawlPath, setCrawlPath] = useState<CrawlPathResult | null>(null);
+  const pathModeRef = useRef(false);
+  useEffect(() => {
+    pathModeRef.current = pathMode;
+    if (!pathMode) {
+      setCrawlPath(null);
+      cyRef.current?.elements().removeClass('onpath pathedge');
+    }
+  }, [pathMode]);
  const [loading, setLoading] = useState(false);
  const containerRef = useRef<HTMLDivElement | null>(null);
  const cyRef = useRef<Core | null>(null);
@@ -286,6 +395,7 @@ export function VisualizationTab() {
    const colorFn = (n: GraphSnapshotResult['nodes'][number]) => {
      if (colorMode === 'depth') return depthColor(n.depth);
      if (colorMode === 'indexability') return indexColor(n.indexability);
+      if (colorMode === 'lcp') return lcpColor(n);
      return statusColor(n.statusCode);
    };

@@ -295,27 +405,32 @@ export function VisualizationTab() {
      .slice(0, TOP_LABEL_COUNT);
    const topIds = new Set(topByInlinks.map((n) => String(n.id)));

-    const elements = [
-      ...graph.nodes.map((n) => ({
-        data: {
-          id: String(n.id),
-          label: shortenUrl(n.url),
-          fullUrl: n.url,
-          statusCode: n.statusCode ?? '',
-          inlinks: n.inlinks,
-          color: colorFn(n),
-          size: nodeSize(n.inlinks, debouncedTuning.nodeSizeScale),
-          isTop: topIds.has(String(n.id)) ? 1 : 0,
-        },
-      })),
-      ...graph.edges.map((e) => ({
-        data: {
-          id: `e${e.source}-${e.target}`,
-          source: String(e.source),
-          target: String(e.target),
-        },
-      })),
-    ];
+    // Directory mode swaps the link graph for a path-segment hierarchy; all
+    // other layouts render the actual internal link graph.
+    const elements =
+      layout === 'directory'
+        ? buildDirectoryElements(graph.nodes, colorFn, debouncedTuning.nodeSizeScale)
+        : [
+            ...graph.nodes.map((n) => ({
+              data: {
+                id: String(n.id),
+                label: shortenUrl(n.url),
+                fullUrl: n.url,
+                statusCode: n.statusCode ?? '',
+                inlinks: n.inlinks,
+                color: colorFn(n),
+                size: nodeSize(n.inlinks, debouncedTuning.nodeSizeScale),
+                isTop: topIds.has(String(n.id)) ? 1 : 0,
+              },
+            })),
+            ...graph.edges.map((e) => ({
+              data: {
+                id: `e${e.source}-${e.target}`,
+                source: String(e.source),
+                target: String(e.target),
+              },
+            })),
+          ];

    if (cyRef.current) {
      cyRef.current.destroy();
@@ -329,12 +444,27 @@ export function VisualizationTab() {
          ? 'node[isTop = 1]'
          : 'node.focus';

+    // `radial` and `directory` are UI choices that map onto real cytoscape
+    // layout algorithms (breadthfirst-circle / dagre respectively).
+    const layoutName =
+      layout === 'radial' ? 'breadthfirst' : layout === 'directory' ? 'dagre' : layout;
    const layoutCfg: Record<string, unknown> = {
-      name: layout,
+      name: layoutName,
      animate: false,
      padding: 30,
    };
-    if (layout === 'cose') {
+    if (layout === 'dagre' || layout === 'directory') {
+      // Sugiyama layered DAG — top-to-bottom ranks, crossing-minimised.
+      layoutCfg.rankDir = 'TB';
+      layoutCfg.nodeSep = layout === 'directory' ? 24 : 36;
+      layoutCfg.rankSep = layout === 'directory' ? 55 : 70;
+      layoutCfg.edgeSep = 12;
+      layoutCfg.ranker = 'network-simplex';
+    } else if (layout === 'radial') {
+      layoutCfg.circle = true;
+      layoutCfg.directed = true;
+      layoutCfg.spacingFactor = 1.5;
+    } else if (layout === 'cose') {
      layoutCfg.nodeRepulsion = () => 1_000_000 * debouncedTuning.repulsionScale;
      layoutCfg.idealEdgeLength = () => 400 * debouncedTuning.edgeLengthScale;
      layoutCfg.edgeElasticity = () => 20;
@@ -413,6 +543,32 @@ export function VisualizationTab() {
            'z-index': 1000,
          },
        },
+        {
+          selector: "node[kind = 'dir']",
+          style: {
+            shape: 'round-rectangle',
+            'background-color': '#334155',
+            'border-width': 1,
+            'border-color': '#475569',
+            label: 'data(label)',
+            color: '#94a3b8',
+            'font-size': 9,
+            'text-valign': 'center',
+            'text-halign': 'center',
+            'text-outline-width': 0,
+            'text-background-opacity': 0,
+            width: 'data(size)',
+            height: 'data(size)',
+          },
+        },
+        {
+          selector: 'node.onpath',
+          style: {
+            'border-width': 3,
+            'border-color': '#22d3ee',
+            'z-index': 1001,
+          },
+        },
        {
          selector: 'node.faded',
          style: {
@@ -448,6 +604,16 @@ export function VisualizationTab() {
            opacity: 0.08,
          },
        },
+        {
+          selector: 'edge.pathedge',
+          style: {
+            'line-color': '#22d3ee',
+            'target-arrow-color': '#22d3ee',
+            width: 2,
+            opacity: 1,
+            'z-index': 1000,
+          },
+        },
        // eslint-disable-next-line @typescript-eslint/no-explicit-any
      ] as any),
      // eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -470,6 +636,23 @@ export function VisualizationTab() {
      });
    };

+    const highlightPath = (res: CrawlPathResult) => {
+      cy.batch(() => {
+        cy.elements().removeClass('onpath pathedge');
+        const ids = res.path.map((p) => String(p.id));
+        const idSet = new Set(ids);
+        cy.nodes().forEach((nd) => {
+          if (idSet.has(nd.id())) nd.addClass('onpath');
+        });
+        // Highlight link-graph edges that connect consecutive path hops
+        // (present only when both endpoints survived the node cap).
+        for (let i = 0; i < ids.length - 1; i++) {
+          const e1 = cy.$id(`e${ids[i]}-${ids[i + 1]}`);
+          if (e1.nonempty()) e1.addClass('pathedge');
+        }
+      });
+    };
+
    cy.on('mouseover', 'node', (e) => {
      const node = e.target;
      setHover(String(node.data('fullUrl')));
@@ -513,16 +696,26 @@ export function VisualizationTab() {
        keep.removeClass('faded');
        keep.edges().addClass('focus');
      });
+      const kind = String(node.data('kind') ?? 'page');
+      const idNum = Number(node.data('id'));
+      if (pathModeRef.current && kind !== 'dir' && Number.isFinite(idNum)) {
+        void window.freecrawl.crawlPath({ urlId: idNum }).then((res) => {
+          setCrawlPath(res);
+          highlightPath(res);
+        });
+      }
    });
    cy.on('tap', (e) => {
      if (e.target !== cy) return;
      if (selectedUrlRef.current) {
        setSelectedUrl(null);
        setLabelOverlay(null);
+        setCrawlPath(null);
        cy.batch(() => {
          cy.elements().removeClass('selected');
          cy.elements().removeClass('faded');
          cy.elements().removeClass('focus');
+          cy.elements().removeClass('onpath pathedge');
        });
      }
    });
@@ -584,6 +777,7 @@ export function VisualizationTab() {
              <option value="status">{t('viz.byStatus', { defaultValue: 'By Status' })}</option>
              <option value="depth">{t('viz.byDepth', { defaultValue: 'By Depth' })}</option>
              <option value="indexability">{t('viz.byIndexability', { defaultValue: 'By Indexability' })}</option>
+              <option value="lcp">{t('viz.byLcp', { defaultValue: 'By LCP (above-fold)' })}</option>
            </select>
          </label>
          <label className="flex items-center gap-1 text-surface-400">
@@ -622,6 +816,20 @@ export function VisualizationTab() {
          >
            {t('viz.fit', { defaultValue: 'Fit' })}
          </button>
+          <button
+            className={`flex items-center gap-1 rounded border px-2 py-1 text-[11px] ${
+              pathMode
+                ? 'border-cyan-500 bg-surface-800 text-cyan-200'
+                : 'border-surface-700 text-surface-200 hover:border-blue-500 hover:bg-surface-800'
+            }`}
+            onClick={() => setPathMode((v) => !v)}
+            title={t('viz.crawlPathTitle', {
+              defaultValue: 'Crawl Path mode — click a page node to trace its shortest path from the crawl root',
+            })}
+          >
+            <Route className="h-3 w-3" />
+            {t('viz.crawlPath', { defaultValue: 'Crawl Path' })}
+          </button>
          <button
            className="flex items-center gap-1 rounded border border-surface-700 px-2 py-1 text-[11px] text-surface-200 hover:border-blue-500 hover:bg-surface-800"
            onClick={() => loadGraph()}
@@ -759,6 +967,61 @@ export function VisualizationTab() {
        </div>

        <aside className="flex w-72 flex-col border-l border-surface-800 bg-surface-950/40">
+          {pathMode && (
+            <div className="border-b border-surface-800">
+              <div className="px-3 py-2 text-[11px] font-semibold uppercase tracking-wider text-surface-400">
+                <div className="flex items-center gap-1">
+                  <Route className="h-3 w-3" /> {t('viz.crawlPath', { defaultValue: 'Crawl Path' })}
+                </div>
+              </div>
+              <div className="max-h-64 overflow-auto px-3 pb-2">
+                {!crawlPath && (
+                  <div className="px-1 py-2 text-[11px] italic text-surface-500">
+                    {t('viz.crawlPathHint', {
+                      defaultValue: 'Select a page node to trace its shortest path from the crawl root.',
+                    })}
+                  </div>
+                )}
+                {crawlPath && crawlPath.path.length === 0 && (
+                  <div className="px-1 py-2 text-[11px] italic text-surface-500">
+                    {t('viz.crawlPathNone', { defaultValue: 'No path data for this node.' })}
+                  </div>
+                )}
+                {crawlPath && crawlPath.path.length > 0 && (
+                  <ol className="flex flex-col gap-1">
+                    {crawlPath.path.map((p, i) => (
+                      <li
+                        key={p.id}
+                        className="flex items-center gap-1.5"
+                        style={{ paddingLeft: `${Math.min(i, 8) * 8}px` }}
+                      >
+                        <span
+                          className="shrink-0 rounded bg-surface-800 px-1 text-[9px] font-mono text-surface-400"
+                          title={t('viz.crawlPathDepth', { defaultValue: 'Crawl depth' })}
+                        >
+                          {p.depth}
+                        </span>
+                        <span
+                          className="truncate font-mono text-[10px] text-surface-200"
+                          title={p.url}
+                        >
+                          {shortenUrl(p.url)}
+                        </span>
+                      </li>
+                    ))}
+                  </ol>
+                )}
+                {crawlPath && crawlPath.path.length > 0 && !crawlPath.reachedRoot && (
+                  <div className="mt-1.5 px-1 text-[10px] leading-snug text-amber-400/80">
+                    {t('viz.crawlPathOrphan', {
+                      defaultValue:
+                        'Did not reach the crawl root — possible orphan (sitemap-only) page.',
+                    })}
+                  </div>
+                )}
+              </div>
+            </div>
+          )}
          <div className="border-b border-surface-800 px-3 py-2 text-[11px] font-semibold uppercase tracking-wider text-surface-400">
            <div className="flex items-center gap-1">
              <Sparkles className="h-3 w-3" /> {t('viz.topAnchorTexts', { defaultValue: 'Top Anchor Texts' })}
@@ -1528,6 +1528,22 @@ export function buildTools(): Tool[] {
        bridgeRequest<unknown>('POST', '/v1/action/graph-snapshot', args),
    },

+    {
+      requiresDb: false,
+      name: 'get_crawl_path',
+      description:
+        "Crawl Path Report for one page — the shortest discovery path from the crawl root (homepage, depth 0) to the target, following internal links. Answers 'how did the crawler reach this URL'. Pass `urlId` or `url`. `reachedRoot` is false for orphan pages reachable only via sitemap.",
+      inputSchema: {
+        type: 'object',
+        properties: {
+          urlId: { type: 'integer', minimum: 1 },
+          url: { type: 'string' },
+        },
+      },
+      handler: async (args) =>
+        bridgeRequest<unknown>('POST', '/v1/action/crawl-path', args),
+    },
+
    // ---- Faz 0.5 Increment 4 — Integration fetch (GSC + GA4 — single
    // blocking API calls, no progress streaming). PSI / AI / SEO have
    // multi-URL batches with progress + cancellation and are exposed via
@@ -56,6 +56,8 @@
        "@types/cytoscape": "^3.21.9",
        "clsx": "^2.1.1",
        "cytoscape": "^3.33.2",
+        "cytoscape-dagre": "^2.5.0",
+        "dagre": "^0.8.5",
        "i18next": "^26.2.0",
        "lucide-react": "^0.544.0",
        "react": "^19.2.0",
@@ -66,6 +68,8 @@
        "zustand": "^5.0.2"
      },
      "devDependencies": {
+        "@types/cytoscape-dagre": "^2.3.4",
+        "@types/dagre": "^0.7.54",
        "@types/react": "^19.2.0",
        "@types/react-dom": "^19.2.0",
        "@vitejs/plugin-react": "^5.0.0",
@@ -2277,6 +2281,23 @@
      "integrity": "sha512-JyrG4tllI6jvuISPjHK9j2Xv/LTbnLekLke5otGStjFluIyA9JjgnvgZrSBsp8cEDpiTjwgZUZwpPv8TSBcoLw==",
      "license": "MIT"
    },
+    "node_modules/@types/cytoscape-dagre": {
+      "version": "2.3.4",
+      "resolved": "https://registry.npmjs.org/@types/cytoscape-dagre/-/cytoscape-dagre-2.3.4.tgz",
+      "integrity": "sha512-uOGXuPfPLFoKZaegjHl9oj4tqONNJuhUl180FiJgRZ35rVijBs6J4UP1Ah6mA6S46h+7pv4ICqpgfdC3EADZlw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "cytoscape": "^3.31"
+      }
+    },
+    "node_modules/@types/dagre": {
+      "version": "0.7.54",
+      "resolved": "https://registry.npmjs.org/@types/dagre/-/dagre-0.7.54.tgz",
+      "integrity": "sha512-QjcRY+adGbYvBFS7cwv5txhVIwX1XXIUswWl+kSQTbI6NjgZydrZkEKX/etzVd7i+bCsCb40Z/xlBY5eoFuvWQ==",
+      "dev": true,
+      "license": "MIT"
+    },
    "node_modules/@types/debug": {
      "version": "4.1.13",
      "resolved": "https://registry.npmjs.org/@types/debug/-/debug-4.1.13.tgz",
@@ -3890,10 +3911,33 @@
      "resolved": "https://registry.npmjs.org/cytoscape/-/cytoscape-3.33.2.tgz",
      "integrity": "sha512-sj4HXd3DokGhzZAdjDejGvTPLqlt84vNFN8m7bGsOzDY5DyVcxIb2ejIXat2Iy7HxWhdT/N1oKyheJ5YdpsGuw==",
      "license": "MIT",
+      "peer": true,
      "engines": {
        "node": ">=0.10"
      }
    },
+    "node_modules/cytoscape-dagre": {
+      "version": "2.5.0",
+      "resolved": "https://registry.npmjs.org/cytoscape-dagre/-/cytoscape-dagre-2.5.0.tgz",
+      "integrity": "sha512-VG2Knemmshop4kh5fpLO27rYcyUaaDkRw+6PiX4bstpB+QFt0p2oauMrsjVbUamGWQ6YNavh7x2em2uZlzV44g==",
+      "license": "MIT",
+      "dependencies": {
+        "dagre": "^0.8.5"
+      },
+      "peerDependencies": {
+        "cytoscape": "^3.2.22"
+      }
+    },
+    "node_modules/dagre": {
+      "version": "0.8.5",
+      "resolved": "https://registry.npmjs.org/dagre/-/dagre-0.8.5.tgz",
+      "integrity": "sha512-/aTqmnRta7x7MCCpExk7HQL2O4owCT2h8NT//9I1OQ9vt29Pa0BzSAkR5lwFUcQ7491yVi/3CXU9jQ5o0Mn2Sw==",
+      "license": "MIT",
+      "dependencies": {
+        "graphlib": "^2.1.8",
+        "lodash": "^4.17.15"
+      }
+    },
    "node_modules/debug": {
      "version": "4.4.3",
      "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
@@ -5329,6 +5373,15 @@
      "dev": true,
      "license": "ISC"
    },
+    "node_modules/graphlib": {
+      "version": "2.1.8",
+      "resolved": "https://registry.npmjs.org/graphlib/-/graphlib-2.1.8.tgz",
+      "integrity": "sha512-jcLLfkpoVGmH7/InMC/1hIvOPSUh38oJtGhvrOFGzioE1DZ+0YW16RgmOJhHiuWTvGiJQ9Z1Ik43JvkRPRvE+A==",
+      "license": "MIT",
+      "dependencies": {
+        "lodash": "^4.17.15"
+      }
+    },
    "node_modules/has-flag": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
@@ -5939,7 +5992,6 @@
      "version": "4.18.1",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.18.1.tgz",
      "integrity": "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==",
-      "dev": true,
      "license": "MIT"
    },
    "node_modules/lodash.merge": {
@@ -0,0 +1,150 @@
+/**
+ * Browser-driven login for JS-heavy SPA login forms.
+ *
+ * The plain undici + cheerio form-login flow (`crawler.runFormLogin`) can't
+ * authenticate against login pages whose fields are client-rendered, whose
+ * CSRF token is built in JavaScript, or that submit via XHR. This module
+ * drives a real Chromium (via Playwright — already bundled for Faz 1 JS
+ * rendering) through the login: navigate → fill credential selectors →
+ * submit → wait for the logged-in state → capture session cookies.
+ *
+ * The captured cookies are returned to the crawler, which bridges them into
+ * the same `SessionCookieJar` the undici crawl path replays — so the heavy
+ * browser is used once, for login only, and the crawl itself stays fast.
+ *
+ * A one-shot browser is launched and closed here rather than reusing the
+ * render `BrowserPool`: login must work even when the crawl isn't in JS
+ * mode, and the pool actively clears cookies between renders (which would
+ * wipe the session we just established).
+ */
+
+import { existsSync } from 'node:fs';
+import { chromium, type LaunchOptions } from 'playwright';
+import { PlaywrightBrowserMissingError } from './browser-pool.js';
+
+export interface BrowserLoginInput {
+  loginUrl: string;
+  usernameSelector: string;
+  usernameValue: string;
+  passwordSelector: string;
+  passwordValue: string;
+  submitSelector: string;
+  successSelector?: string;
+  waitMs?: number;
+}
+
+export interface BrowserLoginOptions {
+  /** Show the browser window (debug). Default false. */
+  headless: boolean;
+  /** Chromium channel ('chrome' / 'msedge'); empty = bundled Playwright. */
+  channel?: string;
+  /** Custom Chromium executable path override. */
+  executablePath?: string;
+  userAgent?: string;
+  acceptLanguage?: string;
+  viewport?: { width: number; height: number };
+  /** Per-step timeout (navigation, selector waits), ms. Default 30000. */
+  timeoutMs?: number;
+}
+
+/** A single captured cookie — shape compatible with Playwright's
+ *  `context.cookies()` and the {@link import('./cookie-jar.js')} bridge. */
+export interface BrowserCookie {
+  name: string;
+  value: string;
+  domain: string;
+  path?: string;
+  secure?: boolean;
+  httpOnly?: boolean;
+  /** Seconds since epoch; -1 for a session cookie. */
+  expires?: number;
+}
+
+export interface BrowserLoginResult {
+  cookies: BrowserCookie[];
+  /** The URL the page settled on after a successful login. */
+  finalUrl: string;
+}
+
+/**
+ * Run the scripted login in a one-shot Chromium and return the resulting
+ * session cookies. Throws {@link PlaywrightBrowserMissingError} when the
+ * browser binary isn't installed; other failures surface as the underlying
+ * Playwright error (the caller logs it and proceeds unauthenticated).
+ */
+export async function runBrowserLogin(
+  cfg: BrowserLoginInput,
+  opts: BrowserLoginOptions,
+): Promise<BrowserLoginResult> {
+  const loginUrl = cfg.loginUrl.trim();
+  if (!loginUrl) throw new Error('browser login: loginUrl is empty');
+
+  // Pre-flight the bundled-browser case so a missing binary surfaces a
+  // single clean install prompt instead of Playwright's per-call banner
+  // (mirrors BrowserPool.doStart).
+  if (!opts.channel && !opts.executablePath) {
+    let resolved = '';
+    try {
+      resolved = chromium.executablePath();
+    } catch {
+      /* unresolved → treat as missing below */
+    }
+    if (!resolved || !existsSync(resolved)) {
+      throw new PlaywrightBrowserMissingError(resolved, opts.channel);
+    }
+  }
+
+  const launchOpts: LaunchOptions = { headless: opts.headless };
+  if (opts.channel) launchOpts.channel = opts.channel;
+  if (opts.executablePath) launchOpts.executablePath = opts.executablePath;
+
+  const timeout = Math.max(5000, opts.timeoutMs ?? 30_000);
+  const browser = await chromium.launch(launchOpts);
+  try {
+    const context = await browser.newContext({
+      viewport: opts.viewport ?? { width: 1366, height: 900 },
+      userAgent: opts.userAgent,
+      locale: opts.acceptLanguage,
+      extraHTTPHeaders: opts.acceptLanguage
+        ? { 'Accept-Language': opts.acceptLanguage }
+        : undefined,
+      ignoreHTTPSErrors: true,
+    });
+    const page = await context.newPage();
+    page.setDefaultTimeout(timeout);
+
+    await page.goto(loginUrl, { waitUntil: 'domcontentloaded', timeout });
+    await page.fill(cfg.usernameSelector, cfg.usernameValue);
+    await page.fill(cfg.passwordSelector, cfg.passwordValue);
+    await page.click(cfg.submitSelector);
+
+    // Settle: let the post-submit navigation / XHR auth complete. networkidle
+    // covers SPA XHR logins that don't trigger a full navigation; a missed
+    // idle window isn't fatal (we still capture whatever cookies exist).
+    await page.waitForLoadState('networkidle', { timeout }).catch(() => {});
+    const successSel = cfg.successSelector?.trim();
+    if (successSel) {
+      await page.waitForSelector(successSel, { timeout }).catch(() => {});
+    }
+    if (cfg.waitMs && cfg.waitMs > 0) {
+      await page.waitForTimeout(Math.min(cfg.waitMs, 60_000));
+    }
+
+    const cookies = await context.cookies();
+    const finalUrl = page.url();
+    return {
+      cookies: cookies.map((c) => ({
+        name: c.name,
+        value: c.value,
+        domain: c.domain,
+        path: c.path,
+        secure: c.secure,
+        httpOnly: c.httpOnly,
+        expires: c.expires,
+      })),
+      finalUrl,
+    };
+  } finally {
+    await browser.close().catch(() => {});
+  }
+}
@@ -31,6 +31,7 @@ import {
  collectNetworkDiagnostics,
  defaultRequestHeaders,
  detectHttpProtocol,
+  type DigestChallenge,
  formatFetchError,
  initHttpClient,
  parseDigestChallenge,
@@ -39,6 +40,7 @@ import { setActiveDnsHook } from './dns-resolver.js';
 import { discoverSitemapUrls, fetchSitemaps } from './sitemap.js';
 import { runJsonExtractionRules } from './extraction.js';
 import { SessionCookieJar } from './cookie-jar.js';
+import { runBrowserLogin } from './browser-login.js';

 export interface CrawlerEvents {
  progress: (p: CrawlProgress) => void;
@@ -3028,6 +3030,43 @@ export class Crawler extends EventEmitter {
    return this.config.userAgent;
  }

+  /**
+   * Per-origin Digest challenge cache. The first request to a protection
+   * space still costs one 401 to learn the realm/nonce; every subsequent
+   * request to the same origin reuses the cached challenge and sends the
+   * `Authorization: Digest …` header proactively (no 401 round-trip),
+   * incrementing the nonce-count (`nc`) each time per RFC 2617/7616.
+   */
+  private readonly digestChallenges = new Map<
+    string,
+    { challenge: DigestChallenge; nc: number }
+  >();
+
+  /** Compute a proactive Digest Authorization header for `fetchUrl` from a
+   *  previously-cached same-origin challenge, bumping the nonce count.
+   *  Returns null when digest auth isn't configured or the origin's realm
+   *  hasn't been learned yet (that first request still pays one 401). */
+  private digestAuthHeader(fetchUrl: string, method = 'GET'): string | null {
+    if (this.config.auth.type !== 'digest' || !(this.config.auth.username ?? '')) return null;
+    let origin: string;
+    try {
+      origin = new URL(fetchUrl).origin;
+    } catch {
+      return null;
+    }
+    const entry = this.digestChallenges.get(origin);
+    if (!entry) return null;
+    entry.nc += 1;
+    const u = new URL(fetchUrl);
+    return buildDigestAuthHeader(entry.challenge, {
+      method,
+      uri: u.pathname + u.search,
+      username: this.config.auth.username ?? '',
+      password: this.config.auth.password ?? '',
+      nc: entry.nc.toString(16).padStart(8, '0'),
+    });
+  }
+
  /**
   * Build request headers for a crawl fetch: defaults + auth + any
   * established session cookies. A user-supplied `Cookie` header (via
@@ -3075,7 +3114,12 @@ export class Crawler extends EventEmitter {
   */
  private async runFormLogin(): Promise<void> {
    const cfg = this.config.formLogin;
-    if (!cfg || !cfg.enabled || cfg.steps.length === 0) return;
+    if (!cfg || !cfg.enabled) return;
+    if (cfg.mode === 'browser') {
+      await this.runBrowserLogin();
+      return;
+    }
+    if (cfg.steps.length === 0) return;

    const jar = new SessionCookieJar();
    const vars: Record<string, string> = {};
@@ -3185,6 +3229,69 @@ export class Crawler extends EventEmitter {
    }
  }

+  /**
+   * Browser-driven login (`formLogin.mode === 'browser'`). Drives a one-shot
+   * Playwright Chromium through the SPA login form, then bridges the
+   * resulting session cookies into the same jar the undici crawl replays.
+   * Best-effort — a failure logs a warning and the crawl proceeds
+   * unauthenticated.
+   */
+  private async runBrowserLogin(): Promise<void> {
+    const b = this.config.formLogin.browser;
+    if (!b || !b.loginUrl.trim()) {
+      this.emit(
+        'warn',
+        'Browser login enabled but no login URL configured — running unauthenticated.',
+      );
+      return;
+    }
+    this.setOp('crawl:login');
+    const jr = this.config.jsRender;
+    try {
+      const result = await runBrowserLogin(
+        {
+          loginUrl: b.loginUrl,
+          usernameSelector: b.usernameSelector,
+          usernameValue: b.usernameValue,
+          passwordSelector: b.passwordSelector,
+          passwordValue: b.passwordValue,
+          submitSelector: b.submitSelector,
+          successSelector: b.successSelector,
+          waitMs: b.waitMs,
+        },
+        {
+          headless: jr?.headless ?? true,
+          channel: jr?.browserChannel || undefined,
+          userAgent: this.config.userAgent,
+          acceptLanguage: this.config.acceptLanguage,
+          viewport: jr
+            ? { width: jr.viewportWidth, height: jr.viewportHeight }
+            : undefined,
+          timeoutMs: Math.max(5000, this.config.requestTimeoutMs),
+        },
+      );
+      const jar = new SessionCookieJar();
+      jar.setFromBrowserCookies(result.cookies);
+      if (jar.size > 0) {
+        this.sessionJar = jar;
+        this.emit(
+          'info',
+          `Browser login established ${jar.size} session cookie(s) (settled at ${result.finalUrl}).`,
+        );
+      } else {
+        this.emit(
+          'warn',
+          'Browser login produced no cookies — the crawl will run unauthenticated.',
+        );
+      }
+    } catch (err) {
+      this.emit(
+        'warn',
+        `Browser login failed: ${formatFetchError(err)} — continuing unauthenticated.`,
+      );
+    }
+  }
+
  private async fetchWithRetry(
    url: string,
    signal: AbortSignal,
@@ -3210,15 +3317,22 @@ export class Crawler extends EventEmitter {
          this.config.renderingMode === 'ajax'
            ? toEscapedFragmentUrl(url)
            : url;
+        // Proactive Digest — if we already learned this origin's realm on
+        // an earlier URL, send the Authorization header up-front so the
+        // server doesn't need to issue a 401 first.
+        const proactiveDigest = this.digestAuthHeader(fetchUrl);
        let res = await undiciFetch(fetchUrl, {
          method: 'GET',
-          headers: this.requestHeaders(url),
+          headers: proactiveDigest
+            ? this.requestHeaders(url, { authorization: proactiveDigest })
+            : this.requestHeaders(url),
          redirect: 'manual',
          signal,
        });
-        // HTTP Digest — the first 401 carries the `WWW-Authenticate: Digest`
-        // challenge; recompute the Authorization header from it and retry
-        // once. Only when digest auth is configured with a username.
+        // HTTP Digest — a 401 carries the `WWW-Authenticate: Digest`
+        // challenge (or a fresh nonce on `stale=true`); cache it per-origin,
+        // recompute the Authorization header, and retry once. Only when
+        // digest auth is configured with a username.
        if (
          res.status === 401 &&
          this.config.auth.type === 'digest' &&
@@ -3231,19 +3345,24 @@ export class Crawler extends EventEmitter {
            } catch {
              /* ignore */
            }
-            const u = new URL(fetchUrl);
-            const authHeader = buildDigestAuthHeader(challenge, {
-              method: 'GET',
-              uri: u.pathname + u.search,
-              username: this.config.auth.username ?? '',
-              password: this.config.auth.password ?? '',
-            });
-            res = await undiciFetch(fetchUrl, {
-              method: 'GET',
-              headers: this.requestHeaders(url, { authorization: authHeader }),
-              redirect: 'manual',
-              signal,
-            });
+            let origin: string;
+            try {
+              origin = new URL(fetchUrl).origin;
+            } catch {
+              origin = fetchUrl;
+            }
+            // Cache the (possibly rotated) challenge and reset its nonce
+            // counter; digestAuthHeader bumps it to 1 for this retry.
+            this.digestChallenges.set(origin, { challenge, nc: 0 });
+            const authHeader = this.digestAuthHeader(fetchUrl);
+            if (authHeader) {
+              res = await undiciFetch(fetchUrl, {
+                method: 'GET',
+                headers: this.requestHeaders(url, { authorization: authHeader }),
+                redirect: 'manual',
+                signal,
+              });
+            }
          }
        }
        // Keep the login session alive — capture any rotated session cookies.
@@ -13,6 +13,13 @@ export {
  type LcpCandidate,
  type MobileUsabilityResult,
 } from './render-fetcher.js';
+export {
+  runBrowserLogin,
+  type BrowserLoginInput,
+  type BrowserLoginOptions,
+  type BrowserLoginResult,
+  type BrowserCookie,
+} from './browser-login.js';
 export {
  normalizeUrl,
  isSameHost,
@@ -13,6 +13,11 @@
 *   /step  //step                absolute child / descendant axis
 *   //div   //*                  element name test / wildcard
 *   a/b//c                       chained child + descendant steps
+ *   ..   parent::x               parent axis (abbreviated + explicit)
+ *   ancestor::x  ancestor-or-self::x   upward traversal cheerio CSS can't do
+ *   following-sibling::x  preceding-sibling::x   sibling axes
+ *   self::x   .                  self axis
+ *   attribute::x                 == @x
 *   .../@attr   .../@*           attribute terminal (reads attribute values)
 *   .../text()                   direct text-node terminal
 *   [n]  [last()]  [last()-1]    positional predicates (per-parent, like XPath)
@@ -64,8 +69,37 @@ interface PredEnv {

 type PredFn = (env: PredEnv) => XVal;

+/** Axis a step navigates. `child` / `descendant` are the abbreviated `/`
+ *  and `//` defaults; the rest are explicit `axis::` forms. */
+type Axis =
+  | 'child'
+  | 'descendant'
+  | 'descendant-or-self'
+  | 'parent'
+  | 'ancestor'
+  | 'ancestor-or-self'
+  | 'self'
+  | 'following-sibling'
+  | 'preceding-sibling';
+
+/** The axis implied purely by the step separator (`/` vs `//`). An explicit
+ *  `axis::` prefix or `..` / `.` abbreviation overrides it. */
+type SepAxis = 'child' | 'descendant';
+
+const KNOWN_AXES = new Set<string>([
+  'child',
+  'descendant',
+  'descendant-or-self',
+  'parent',
+  'ancestor',
+  'ancestor-or-self',
+  'self',
+  'following-sibling',
+  'preceding-sibling',
+]);
+
 interface PathStep {
-  axis: 'child' | 'descendant';
+  axis: Axis;
  nodeTest: NodeTest;
  predicates: PredFn[];
 }
@@ -112,7 +146,7 @@ function evalElementStep($: CheerioAPI, ctxNodes: AnyNode[], step: PathStep): El
  const seen = new Set<AnyNode>();
  const cand: Element[] = [];
  for (const ctx of ctxNodes) {
-    const pool = step.axis === 'descendant' ? descendantElements(ctx) : elementChildren(ctx);
+    const pool = axisPool(ctx, step.axis);
    for (const el of pool) {
      if (matchNodeTest(el, step.nodeTest) && !seen.has(el)) {
        seen.add(el);
@@ -123,6 +157,38 @@ function evalElementStep($: CheerioAPI, ctxNodes: AnyNode[], step: PathStep): El
  return applyPredicates($, cand, step.predicates);
 }

+/** Candidate elements reachable from `ctx` along `axis`, in the order XPath
+ *  predicates expect (reverse axes return nearest-first). Note: positional
+ *  predicates on reverse / sibling axes are evaluated per-parent in
+ *  {@link applyPredicates}, which is exact for the common single-context
+ *  case and a documented approximation for multi-context node-sets. */
+function axisPool(ctx: AnyNode, axis: Axis): Element[] {
+  switch (axis) {
+    case 'child':
+      return elementChildren(ctx);
+    case 'descendant':
+      return descendantElements(ctx);
+    case 'descendant-or-self':
+      return isTag(ctx) ? [ctx, ...descendantElements(ctx)] : descendantElements(ctx);
+    case 'parent': {
+      const p = parentElement(ctx);
+      return p ? [p] : [];
+    }
+    case 'ancestor':
+      return ancestorElements(ctx);
+    case 'ancestor-or-self':
+      return isTag(ctx) ? [ctx, ...ancestorElements(ctx)] : ancestorElements(ctx);
+    case 'self':
+      return isTag(ctx) ? [ctx] : [];
+    case 'following-sibling':
+      return followingSiblings(ctx);
+    case 'preceding-sibling':
+      return precedingSiblings(ctx);
+    default:
+      return [];
+  }
+}
+
 function applyPredicates($: CheerioAPI, cand: Element[], predicates: PredFn[]): Element[] {
  let list = cand;
  for (const pred of predicates) {
@@ -191,6 +257,46 @@ function elementChildren(node: AnyNode): Element[] {
  return kids.filter(isTag);
 }

+function parentOf(node: AnyNode): AnyNode | null {
+  return (node as { parent?: AnyNode | null }).parent ?? null;
+}
+
+/** Immediate parent element (skips the document root / non-element parents). */
+function parentElement(node: AnyNode): Element | null {
+  const p = parentOf(node);
+  return p && isTag(p) ? p : null;
+}
+
+/** Ancestor elements, nearest-first (XPath reverse document order). */
+function ancestorElements(node: AnyNode): Element[] {
+  const out: Element[] = [];
+  let p = parentOf(node);
+  while (p) {
+    if (isTag(p)) out.push(p);
+    p = parentOf(p);
+  }
+  return out;
+}
+
+/** Sibling elements after `node` among its parent's element children, in
+ *  document order. */
+function followingSiblings(node: AnyNode): Element[] {
+  const parent = parentOf(node);
+  if (!parent) return [];
+  const sibs = elementChildren(parent);
+  const idx = sibs.indexOf(node as Element);
+  return idx < 0 ? [] : sibs.slice(idx + 1);
+}
+
+/** Sibling elements before `node`, nearest-first (XPath reverse order). */
+function precedingSiblings(node: AnyNode): Element[] {
+  const parent = parentOf(node);
+  if (!parent) return [];
+  const sibs = elementChildren(parent);
+  const idx = sibs.indexOf(node as Element);
+  return idx <= 0 ? [] : sibs.slice(0, idx).reverse();
+}
+
 /** All descendant elements of `node` in document order (pre-order DFS),
 *  excluding `node` itself. */
 function descendantElements(node: AnyNode): Element[] {
@@ -279,7 +385,7 @@ function readStepRaw(expr: string, start: number): { raw: string; next: number }
  return { raw: expr.slice(start, i).trim(), next: i };
 }

-function parseStep(raw: string, axis: 'child' | 'descendant'): PathStep {
+function parseStep(raw: string, axis: SepAxis): PathStep {
  // Node test ends at the first top-level '['.
  let testEnd = raw.length;
  let depth = 0;
@@ -300,7 +406,7 @@ function parseStep(raw: string, axis: 'child' | 'descendant'): PathStep {
    }
  }
  const testStr = raw.slice(0, testEnd).trim();
-  const nodeTest = parseNodeTest(testStr);
+  const { axis: resolvedAxis, nodeTest } = parseAxisAndTest(testStr, axis);
  const predicates: PredFn[] = [];
  let i = testEnd;
  while (i < raw.length) {
@@ -314,7 +420,31 @@ function parseStep(raw: string, axis: 'child' | 'descendant'): PathStep {
      throw new XPathError(`unexpected '${raw[i]}' after node test in '${raw}'`);
    }
  }
-  return { axis, nodeTest, predicates };
+  return { axis: resolvedAxis, nodeTest, predicates };
+}
+
+/** Resolve an explicit `axis::test`, the `..` / `.` abbreviations, or a bare
+ *  node test evaluated against the separator-implied default axis. */
+function parseAxisAndTest(
+  testStr: string,
+  defaultAxis: SepAxis,
+): { axis: Axis; nodeTest: NodeTest } {
+  if (testStr === '..') return { axis: 'parent', nodeTest: { kind: 'node' } };
+  if (testStr === '.') return { axis: 'self', nodeTest: { kind: 'node' } };
+  const m = /^([A-Za-z][\w-]*)::(.*)$/.exec(testStr);
+  if (m) {
+    const axisName = m[1]!;
+    const rest = m[2]!.trim();
+    if (axisName === 'attribute') {
+      // attribute::x is the long form of @x — an attribute terminal.
+      return { axis: defaultAxis, nodeTest: parseNodeTest('@' + rest) };
+    }
+    if (!KNOWN_AXES.has(axisName)) {
+      throw new XPathError(`unsupported axis: ${axisName}::`);
+    }
+    return { axis: axisName as Axis, nodeTest: parseNodeTest(rest) };
+  }
+  return { axis: defaultAxis, nodeTest: parseNodeTest(testStr) };
 }

 function parseNodeTest(s: string): NodeTest {
@@ -5445,12 +5445,23 @@ export class ProjectDb {
   * Cost: two indexed SELECTs + a JOIN. ~200 ms at 100K URLs / 5K cap.
   */
  graphSnapshot(nodeLimit = 1000): {
-    nodes: { id: number; url: string; statusCode: number | null; depth: number; inlinks: number; indexability: Indexability }[];
+    nodes: {
+      id: number;
+      url: string;
+      statusCode: number | null;
+      depth: number;
+      inlinks: number;
+      indexability: Indexability;
+      lcpTag: string | null;
+      lcpResourceUrl: string | null;
+      lcpCoverage: number | null;
+    }[];
    edges: { source: number; target: number }[];
  } {
    const nodes = this.db
      .prepare(
-        `SELECT id, url, status_code, depth, inlinks, indexability
+        `SELECT id, url, status_code, depth, inlinks, indexability,
+                lcp_tag, lcp_resource_url, lcp_coverage
           FROM urls
          WHERE is_external = 0 AND content_kind = 'html'
          ORDER BY inlinks DESC, id ASC
@@ -5463,6 +5474,9 @@ export class ProjectDb {
      depth: number;
      inlinks: number;
      indexability: Indexability;
+      lcp_tag: string | null;
+      lcp_resource_url: string | null;
+      lcp_coverage: number | null;
    }[];

    if (nodes.length === 0) return { nodes: [], edges: [] };
@@ -5498,11 +5512,75 @@ export class ProjectDb {
        depth: n.depth,
        inlinks: n.inlinks,
        indexability: n.indexability,
+        lcpTag: n.lcp_tag,
+        lcpResourceUrl: n.lcp_resource_url,
+        lcpCoverage: n.lcp_coverage,
      })),
      edges,
    };
  }

+  /**
+   * Reconstruct the shortest discovery path from the crawl root (depth 0)
+   * to `targetUrlId` — the SF-style "Crawl Path Report". Walks backwards
+   * from the target along internal inlinks, preferring at each hop a parent
+   * exactly one depth shallower (a BFS-tree parent, which guarantees the
+   * shortest path). Falls back to the shallowest available parent when the
+   * link graph is irregular (sitemap-discovered pages, redirect chains).
+   *
+   * Returns the path ordered root → target. `reachedRoot` is false when the
+   * walk stalled before depth 0 (e.g. an orphan reachable only via sitemap).
+   */
+  crawlPath(
+    targetUrlId: number,
+    maxHops = 60,
+  ): {
+    path: { id: number; url: string; depth: number; statusCode: number | null }[];
+    reachedRoot: boolean;
+  } {
+    type PathRow = { id: number; url: string; depth: number; status_code: number | null };
+    const target = this.db
+      .prepare('SELECT id, url, depth, status_code FROM urls WHERE id = ?')
+      .get(targetUrlId) as PathRow | undefined;
+    if (!target) return { path: [], reachedRoot: false };
+
+    const parentStmt = this.db.prepare(
+      `SELECT f.id AS id, f.url AS url, f.depth AS depth, f.status_code AS status_code
+         FROM links l
+         JOIN urls f ON l.from_url_id = f.id
+        WHERE l.to_url = ? AND l.is_internal = 1 AND f.id != ?
+        ORDER BY f.depth ASC, f.inlinks DESC
+        LIMIT 200`,
+    );
+
+    const chain: PathRow[] = [target];
+    const seen = new Set<number>([target.id]);
+    let cur = target;
+    let reachedRoot = cur.depth <= 0;
+    while (!reachedRoot && chain.length < maxHops) {
+      const parents = parentStmt.all(cur.url, cur.id) as PathRow[];
+      const next =
+        parents.find((p) => p.depth === cur.depth - 1 && !seen.has(p.id)) ??
+        parents.find((p) => p.depth < cur.depth && !seen.has(p.id)) ??
+        parents.find((p) => !seen.has(p.id));
+      if (!next) break;
+      chain.push(next);
+      seen.add(next.id);
+      cur = next;
+      if (cur.depth <= 0) reachedRoot = true;
+    }
+    chain.reverse();
+    return {
+      path: chain.map((r) => ({
+        id: r.id,
+        url: r.url,
+        depth: r.depth,
+        statusCode: r.status_code,
+      })),
+      reachedRoot,
+    };
+  }
+
  // ===================================================================
  // V2 Faz 2 — Log File Analyzer. Aggregate-on-ingest storage + the
  // crawl × log joins. The desktop main process flattens the core
@@ -1173,12 +1173,49 @@ export interface FormLoginStep {
  captures: { name: string; selector: string; attribute?: string }[];
 }

+/**
+ * Browser-driven login for JS-heavy SPA login forms that the plain
+ * undici+cheerio `steps` flow can't handle (client-rendered fields,
+ * JS-built CSRF tokens, XHR-based auth). A one-shot Playwright browser
+ * navigates the login page, fills the credential selectors, submits, waits
+ * for the logged-in state, and its session cookies are bridged into the
+ * same jar that the undici crawl path replays. Requires Playwright
+ * (Faz 1 — already bundled).
+ */
+export interface BrowserLoginConfig {
+  /** Login page URL to navigate to. */
+  loginUrl: string;
+  /** CSS selector for the username/email input. */
+  usernameSelector: string;
+  /** Username/email to type. */
+  usernameValue: string;
+  /** CSS selector for the password input. */
+  passwordSelector: string;
+  /** Password to type. */
+  passwordValue: string;
+  /** CSS selector for the submit button (clicked after filling). */
+  submitSelector: string;
+  /** Optional selector that appears only once logged in — waited for
+   *  before cookies are captured (e.g. an avatar / logout link). */
+  successSelector?: string;
+  /** Extra settle time after submit, ms (for slow SPA redirects). */
+  waitMs?: number;
+}
+
 export interface FormLoginConfig {
  /** Master switch — when off the login sequence never runs and no
   *  session cookies are injected. */
  enabled: boolean;
-  /** Ordered login steps. Empty = nothing to do even when enabled. */
+  /**
+   * Login strategy:
+   *  - `http`    — undici + cheerio `steps` flow (default; fast, no browser).
+   *  - `browser` — Playwright-driven `browser` flow for SPA login forms.
+   */
+  mode?: 'http' | 'browser';
+  /** Ordered login steps for `http` mode. Empty = nothing to do. */
  steps: FormLoginStep[];
+  /** Configuration for `browser` mode. */
+  browser?: BrowserLoginConfig;
 }

 /**
@@ -116,6 +116,7 @@ export const IPC = {
  exportBulk: 'export:bulk',
  compareLoad: 'compare:load',
  graphSnapshot: 'graph:snapshot',
+  crawlPath: 'graph:crawl-path',
  topAnchorTexts: 'graph:anchor-texts',
  sitemapGenerate: 'sitemap:generate',
  menuEvent: 'menu:event',
@@ -644,6 +645,12 @@ export interface GraphNode {
  depth: number;
  inlinks: number;
  indexability: Indexability;
+  /** Largest-Contentful-Paint candidate captured during JS render (null
+   *  when JS rendering wasn't used or no candidate was found). Powers the
+   *  "LCP / above-the-fold" visualization overlay. */
+  lcpTag: string | null;
+  lcpResourceUrl: string | null;
+  lcpCoverage: number | null;
 }

 export interface GraphEdge {
@@ -660,6 +667,24 @@ export interface GraphSnapshotResult {
  edges: GraphEdge[];
 }

+export interface CrawlPathInput {
+  urlId: number;
+}
+
+export interface CrawlPathNode {
+  id: number;
+  url: string;
+  depth: number;
+  statusCode: number | null;
+}
+
+export interface CrawlPathResult {
+  /** Ordered crawl root → target. Empty when the target id is unknown. */
+  path: CrawlPathNode[];
+  /** False when the walk stalled before the depth-0 root (orphan page). */
+  reachedRoot: boolean;
+}
+
 export interface AnchorTextRow {
  anchor: string;
  count: number;
@@ -1405,6 +1430,7 @@ export interface FreeCrawlApi {
  exportBulk(): Promise<BulkExportResult>;
  compareLoad(input: CompareLoadInput): Promise<CompareLoadResult>;
  graphSnapshot(input: GraphSnapshotInput): Promise<GraphSnapshotResult>;
+  crawlPath(input: CrawlPathInput): Promise<CrawlPathResult>;
  topAnchorTexts(limit?: number): Promise<AnchorTextRow[]>;
  sitemapGenerate(input: SitemapGenerateInput): Promise<SitemapGenerateResult>;
  appVersion(): Promise<string>;