feat: log file analyzer + bulk/project MCP export tools

README.md

@@ -31,9 +31,9 @@ A high-performance website crawler built for serious SEO audits. Targets <kbd>1M
 
 FreeCrawl is a desktop SEO crawler that scales to **1M+ URLs on a single machine** at 80–150 URL/s via undici with keep-alive, with optional **JavaScript rendering** through headless Chromium (Playwright) that captures the post-JS DOM, full-page / above-fold / mobile screenshots, LCP candidate elements, and a per-URL mobile usability audit. It runs **150+ on-page SEO checks across 30 top-level tabs** — exact and near-duplicate clustering (SimHash + LSH) with a dedicated Cluster view, full hreflang validation (reciprocity, self-reference, inconsistent lang), OpenGraph / Twitter Card / JSON-LD / Web App Manifest parsing, AMP smoke validation, structured-data validation across 17 schema types (duplicate `@id`, malformed `@type`, missing required + Google-recommended props), a configurable performance budget (response-time / page-size / LCP / CLS ceilings), WCAG accessibility checks (`<main>` landmark, skip-link, ARIA roles, heading order), security headers + SSL/TLS chain audit + active/passive mixed content split, readability scores (Flesch, Flesch–Kincaid, Gunning Fog), and a 10-rule custom extraction engine — **CSS selectors, XPath, regex, and JSONPath (for JSON API responses)** — with a **live Preview dialog and JSON import / export** for sharing rule sets.
 
-The **dense dark UI** renders virtualized 1M-row tables that live-stream rows during a crawl, with **List ↔ Tree ↔ Cluster view toggles**, **column pin (sticky-left) + drag-to-reorder + show/hide**, advanced AND/OR filters across 24 fields × 12 operators, per-tab quick-filter dropdowns, a 16-tab Details panel per URL (URL Details, Outline, Inlinks, Outlinks, Images, Resources, Extracted Data, SERP Snippet, HTTP Headers, Cookies, Structured Data, View Source, View Rendered, Screenshot, Duplicates, Analytics), an English + Turkish UI including every Settings panel, **in-app scheduled crawls** (hourly / daily / weekly / custom), a live memory monitor, robots.txt syntax validator, URL rewriting with live preview, project-vs-project compare diff, a Cytoscape link graph in its own native window, and a `.seoproject` file association for OS double-click.
+The **dense dark UI** renders virtualized 1M-row tables that live-stream rows during a crawl, with **List ↔ Tree ↔ Cluster view toggles**, **column pin (sticky-left) + drag-to-reorder + show/hide**, advanced AND/OR filters across 24 fields × 12 operators, per-tab quick-filter dropdowns, a 16-tab Details panel per URL (URL Details, Outline, Inlinks, Outlinks, Images, Resources, Extracted Data, SERP Snippet, HTTP Headers, Cookies, Structured Data, View Source, View Rendered, Screenshot, Duplicates, Analytics), an English + Turkish UI including every Settings panel, **in-app scheduled crawls** (hourly / daily / weekly / custom), a live memory monitor, robots.txt syntax validator, URL rewriting with live preview, project-vs-project compare diff, a Cytoscape link graph in its own native window, a **standalone Log File Analyzer window** (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, and crawl × log orphan detection — with 40+ bot detection, optional reverse-DNS verification, per-bot URL filtering, and one-click CSV / Excel export of every table), and a `.seoproject` file association for OS double-click.
 
-Integrations cover **Google Search Console** (clicks, impressions, CTR, position + URL Inspection coverage), **Google Analytics 4** (sessions, users, bounce, engagement), **PageSpeed Insights** Lighthouse audits, custom **AI prompts via OpenAI / Anthropic Claude / local Ollama** with `{url}`/`{title}`/`{description}`/`{h1}`/`{body}` variables, and **SEO authority providers** (Ahrefs / Majestic / Moz / Semrush) behind a single dropdown — each integration card now ships with an **in-app step-by-step Setup Guide modal** in English and Turkish that walks you through OAuth client creation, test-user setup, API enablement, and the most common errors. Exports go through a unified **Export Crawl Data dialog** (Excel `.xlsx` / CSV UTF-8 / JSON / XML with hierarchical category selection and nested folder output) plus a standalone HTML audit report, sitemap generator (standard / image / hreflang / sharded / gzipped), and direct streaming to **Google Sheets** and **BigQuery**. Project files can be saved as **password-protected encrypted snapshots** (`.seoproject.enc`, AES-256-GCM + PBKDF2). The **MCP server** exposes **76 tools** that let Claude Code or any MCP client drive crawls live, query every UI surface, trigger exports, fetch GSC/GA4 data on demand, and modify settings — every action a human user can take in the desktop is callable from an agent. Crawl-completion webhooks, OS notifications, 22 built-in reports (histograms, top/bottom URLs, link positions, top words, cross-source orphans), per-URL **Duplicates** view, and a custom-CSS theme override round out the suite. Everything runs **fully local** — no telemetry, no cloud, MIT-style license.
+Integrations cover **Google Search Console** (clicks, impressions, CTR, position + URL Inspection coverage), **Google Analytics 4** (sessions, users, bounce, engagement), **PageSpeed Insights** Lighthouse audits, custom **AI prompts via OpenAI / Anthropic Claude / local Ollama** with `{url}`/`{title}`/`{description}`/`{h1}`/`{body}` variables, and **SEO authority providers** (Ahrefs / Majestic / Moz / Semrush) behind a single dropdown — each integration card now ships with an **in-app step-by-step Setup Guide modal** in English and Turkish that walks you through OAuth client creation, test-user setup, API enablement, and the most common errors. Exports go through a unified **Export Crawl Data dialog** (Excel `.xlsx` / CSV UTF-8 / JSON / XML with hierarchical category selection and nested folder output) plus a standalone HTML audit report, sitemap generator (standard / image / hreflang / sharded / gzipped), and direct streaming to **Google Sheets** and **BigQuery**. Project files can be saved as **password-protected encrypted snapshots** (`.seoproject.enc`, AES-256-GCM + PBKDF2). The **MCP server** exposes **89 tools** that let Claude Code or any MCP client drive crawls live, query every UI surface, run single or bulk exports, save / open encrypted project snapshots, fetch GSC/GA4 data on demand, and modify settings — every action a human user can take in the desktop is callable from an agent. Crawl-completion webhooks, OS notifications, 22 built-in reports (histograms, top/bottom URLs, link positions, top words, cross-source orphans), per-URL **Duplicates** view, and a custom-CSS theme override round out the suite. Everything runs **fully local** — no telemetry, no cloud, MIT-style license.
 
 <br />
 
@@ -169,8 +169,14 @@ npm run dev
 npm run build:cli
 node apps/cli/dist/index.js https://example.com --depth 2 --max 500 --out out.csv
 node apps/cli/dist/index.js --list urls.txt --out out.json     # list mode + JSON
+
+# Log file analysis — parse an access log into the project for crawl × log reporting
+node apps/cli/dist/index.js analyze-logs samples/apache-access.txt --project crawl.seoproject
+node apps/cli/dist/index.js analyze-logs access.log --format iis-w3c --verify-bots --json
 ```
 
+> **Sample logs** ship in [`samples/`](samples/) (`apache-access.txt`, `iis-access.txt`) so you can try the Log File Analyzer immediately — in the desktop app, open **Log Analyzer → Open Log Analyzer Window…** and import one.
+
 **CI / CD recipes** — ready-to-use [GitHub Actions](docs/ci/github-actions-example.yml) and [GitLab CI](docs/ci/gitlab-ci-example.yml) examples that crawl your site on a schedule, fail the build when broken-URL count exceeds a threshold, and upload the crawl as an artifact.
 
 </details>
@@ -199,7 +205,7 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 1. **Read-only data access** to the SQLite project — runs alongside the desktop app without contention (WAL allows concurrent readers).
 2. **Live crawl control** — when the desktop app is open, an agent can start / pause / resume / stop crawls and poll progress in real time. This goes through a localhost-only HTTP bridge (127.0.0.1, ephemeral random port, 32-byte Bearer token auth, discovery file written to `<userData>/mcp-bridge.json` on app launch).
 
-**47 tools**:
+**89 tools** — every action a human can take in the desktop is callable from an agent. Representative groups:
 
 | Group | Tools |
 | :--- | :--- |
@@ -207,7 +213,11 @@ FreeCrawl ships an **MCP (Model Context Protocol) server** that exposes the acti
 | 🔬 **Per-URL detail sub-tabs** | `get_url_source` (raw + rendered + screenshot paths), `get_url_inlinks`, `get_url_outlinks`, `get_url_images`, `get_url_headers`, `get_url_duplicates`, `get_url_analytics`, `get_url_cert` |
 | 🧩 **Integration rows** | `query_gsc`, `query_ga4`, `query_pagespeed`, `query_ai`, `query_seo` |
 | 🔎 **Specialised queries** | `query_images`, `query_broken_links`, `list_duplicate_clusters` |
-| 📈 **Reports** | `report_status_code_histogram`, `report_indexability_distribution`, `report_content_kind_distribution`, `report_depth_histogram`, `report_response_time_histogram`, `report_inlinks_histogram`, `report_word_count_histogram`, `report_url_length_histogram`, `report_top_urls_by`, `report_top_anchor_texts`, `report_external_domain_health`, `report_image_weight_per_page`, `report_link_position_breakdown`, `report_pages_per_directory`, `report_word_count_per_directory`, `report_sitemap_orphans`, `report_analytics_coverage`, `report_server_header_breakdown` |
+| 📈 **Reports** | `report_status_code_histogram`, `report_indexability_distribution`, `report_content_kind_distribution`, `report_depth_histogram`, `report_response_time_histogram`, `report_inlinks_histogram`, `report_word_count_histogram`, `report_url_length_histogram`, `report_top_urls_by`, `report_top_anchor_texts`, `report_external_domain_health`, `report_image_weight_per_page`, `report_link_position_breakdown`, `report_pages_per_directory`, `report_word_count_per_directory`, `report_sitemap_orphans`, `report_analytics_coverage`, `report_server_header_breakdown`, `report_query_string_variants`, `report_sitemap_priority_mismatch`, `report_top_words` |
+| 📤 **Exports & mutations** | `export_csv`, `export_json`, `export_xml`, `export_html_report`, `export_tabular`, `export_bulk`, `export_broken_links`, `export_images`, `sitemap_generate`, `sitemap_validate`, `robots_test`, `robots_validate`, `compare_load`, `crawl_add_url`, `respider_urls`, `remove_urls`, `data_delete_by_domain`, `graph_snapshot`, `url_rewrite_preview`, `extraction_preview` |
+| 🔐 **Project files** | `project_save_as`, `project_save_encrypted`, `project_open_encrypted` |
+| 🔌 **Integration fetch** | `google_auth_status`, `gsc_list_sites`, `gsc_fetch`, `ga4_list_properties`, `ga4_fetch`, `pagespeed_run`, `ai_run`, `seo_run`, `get_fetch_progress`, `cancel_fetch` |
+| ⚙ **Config & schedule** | `get_crawl_config`, `set_crawl_config`, `schedule_get`, `schedule_set` |
 | 📁 **Project management** | `list_projects`, `set_project`, `current_project` |
 | 🕷 **Crawl control** (desktop must be open) | `start_crawl`, `stop_crawl`, `pause_crawl`, `resume_crawl`, `clear_crawl`, `get_crawl_progress`, `get_desktop_project` |
 
@@ -388,9 +398,9 @@ graph LR
 ## 📈 Status
 
 > [!NOTE]
-> **Active development.** All 29 analysis tabs (Internal, External, Response Codes, URL, Page Titles, Meta Description, H1, H2, Content, Images, Canonicals, Directives, Redirects, Pagination, Hreflang, AMP, Structured Data, Meta Refresh, Custom Extraction, Custom Search, Security, Duplicates, Links, Broken Links, SERP, **PageSpeed, Search Console, GA4, AI, SEO Authority**) plus standalone Visualization window, advanced search, **per-tab quick-filter dropdown** + **List/Tree view toggle**, 150+ issue categories, sitemap export variants, **Export Crawl Data dialog (XLSX / CSV-UTF-8 / JSON / XML with hierarchical tree picker + nested folder output)**, list mode, custom extraction, near-duplicate + exact-duplicate detection, hreflang validation, project compare, Cytoscape visualization, **Basic / Digest / Bearer + multi-step form-based login**, **HTTP / SOCKS5 proxy**, webhook, **MCP server with crawl control + live progress**, **Google PageSpeed / Search Console / URL Inspection / GA4 integrations**, **AI per-URL prompts (OpenAI / Anthropic / Ollama)**, **SEO Authority providers (Ahrefs / Majestic / Moz / Semrush)**, **Google Sheets + BigQuery direct export**, **encrypted project snapshots (AES-256-GCM)**, **cross-source orphan detection**, **JavaScript rendering with Playwright (post-JS DOM, screenshot capture, LCP candidate, Mobile Usability audit)**, **memory-limit auto-pause watchdog**, OS notifications, **robots.txt syntax validator**, URL rewriting + preview, **status-code diagnosis banner**, **live memory monitor**, **in-app scheduled crawl**, **multi-language UI (EN + TR) with full Settings coverage**, **`.seoproject` file association**, in-app logs, and diagnostic popups are working. Cross-platform installers (Windows `.exe` + portable, macOS `.dmg`, Linux `.AppImage` / `.deb` / `.rpm`) — **release builds ship Playwright Chromium offline so JS rendering works on first launch**. Live-streaming UX with **first row in ~1 s**.
+> **Active development.** All 29 analysis tabs (Internal, External, Response Codes, URL, Page Titles, Meta Description, H1, H2, Content, Images, Canonicals, Directives, Redirects, Pagination, Hreflang, AMP, Structured Data, Meta Refresh, Custom Extraction, Custom Search, Security, Duplicates, Links, Broken Links, SERP, **PageSpeed, Search Console, GA4, AI, SEO Authority**) plus standalone Visualization window, advanced search, **per-tab quick-filter dropdown** + **List/Tree view toggle**, 150+ issue categories, sitemap export variants, **Export Crawl Data dialog (XLSX / CSV-UTF-8 / JSON / XML with hierarchical tree picker + nested folder output)**, list mode, custom extraction, near-duplicate + exact-duplicate detection, hreflang validation, project compare, Cytoscape visualization, **Basic / Digest / Bearer + multi-step form-based login**, **HTTP / SOCKS5 proxy**, webhook, **MCP server with crawl control + live progress**, **Google PageSpeed / Search Console / URL Inspection / GA4 integrations**, **AI per-URL prompts (OpenAI / Anthropic / Ollama)**, **SEO Authority providers (Ahrefs / Majestic / Moz / Semrush)**, **Google Sheets + BigQuery direct export**, **encrypted project snapshots (AES-256-GCM)**, **cross-source orphan detection**, **JavaScript rendering with Playwright (post-JS DOM, screenshot capture, LCP candidate, Mobile Usability audit)**, **memory-limit auto-pause watchdog**, OS notifications, **robots.txt syntax validator**, URL rewriting + preview, **status-code diagnosis banner**, **live memory monitor**, **in-app scheduled crawl**, **multi-language UI (EN + TR) with full Settings coverage**, **standalone Log File Analyzer (Apache / Nginx / IIS / custom access logs → bot hits per URL, crawl budget, response-code distribution, daily trend, crawl × log orphan detection, 40+ bot detection with reverse-DNS verification, per-bot URL filter, CSV / Excel export, and a headless `analyze-logs` CLI)**, **`.seoproject` file association**, in-app logs, and diagnostic popups are working. Cross-platform installers (Windows `.exe` + portable, macOS `.dmg`, Linux `.AppImage` / `.deb` / `.rpm`) — **release builds ship Playwright Chromium offline so JS rendering works on first launch**. Live-streaming UX with **first row in ~1 s**.
 >
-> **Upcoming (V2):** Log file analyzer, Plugin system, Light theme, Multi-window, Code-signing + auto-update.
+> **Upcoming (V2):** Plugin system, Light theme, Multi-window, Code-signing + auto-update.
 
 <br />
 

apps/cli/src/index.ts

@@ -1,8 +1,8 @@
 #!/usr/bin/env node
 import { parseArgs } from 'node:util';
-import { resolve } from 'node:path';
+import { basename, resolve } from 'node:path';
 import { readFileSync } from 'node:fs';
-import { DEFAULT_CRAWL_CONFIG } from '@freecrawl/shared-types';
+import { DEFAULT_CRAWL_CONFIG, type LogIngestInput } from '@freecrawl/shared-types';
 import {
   Crawler,
   compareCrawls,
@@ -11,6 +11,8 @@ import {
   testUrlAgainstRobots,
   fetchSitemaps,
   validateSitemap,
+  analyzeLogFile,
+  type LogAnalysisResult,
 } from '@freecrawl/core';
 import { ProjectDb } from '@freecrawl/db';
 
@@ -24,6 +26,9 @@ Usage:
   freecrawl audit-robots <url> [--user-agent UA]  Test if a URL is allowed by robots.txt
   freecrawl compare <before.seoproject> <after.seoproject>
                                                   Cross-project diff (added / removed / status / title / meta / h1 / canonical / indexability / response_time)
+  freecrawl analyze-logs <access.log> [--project file]
+                                                  Parse a server access log (Apache / Nginx / IIS / custom), detect bots,
+                                                  and merge the aggregates into a project for crawl × log analysis
 
 Options:
   --depth <n>         Max crawl depth (default: ${DEFAULT_CRAWL_CONFIG.maxDepth})
@@ -46,6 +51,12 @@ Options:
                         any other → CSV (subset of common columns)
   --json              Print a machine-readable JSON summary to stdout (instead of human progress).
                         Useful for CI/CD pipelines.
+
+analyze-logs options:
+  --project <file>    Project to merge log aggregates into (default: ./crawl.seoproject)
+  --format <fmt>      auto | apache-combined | apache-common | nginx | iis-w3c | custom (default: auto)
+  --regex <pattern>   Named-group regex for --format custom (groups: ip ts method path status bytes ua referer)
+  --verify-bots       Reverse-DNS verify a sample of bot IPs (slower; catches spoofed user-agents)
   -h, --help          Show this help
 `);
 }
@@ -69,6 +80,10 @@ async function main(): Promise<void> {
       db: { type: 'string' },
       out: { type: 'string' },
       json: { type: 'boolean' },
+      project: { type: 'string' },
+      format: { type: 'string' },
+      regex: { type: 'string' },
+      'verify-bots': { type: 'boolean' },
       help: { type: 'boolean', short: 'h' },
     },
   });
@@ -111,6 +126,20 @@ async function main(): Promise<void> {
     );
     process.exit(exitCode);
   }
+  if (positionals[0] === 'analyze-logs') {
+    if (!positionals[1]) {
+      console.error('Usage: freecrawl analyze-logs <access.log> [--project file] [--format f] [--regex p] [--verify-bots]');
+      process.exit(2);
+    }
+    const exitCode = await runAnalyzeLogs(resolve(positionals[1]), {
+      projectPath: resolve(values.project ?? values.db ?? 'crawl.seoproject'),
+      format: (values.format as LogIngestInput['file']['format'] | 'auto' | undefined) ?? 'auto',
+      regex: values.regex,
+      verifyBots: Boolean(values['verify-bots']),
+      json: Boolean(values.json),
+    });
+    process.exit(exitCode);
+  }
 
   // List mode is selected by `--list`; otherwise we need a positional URL.
   const listFile = values.list;
@@ -409,6 +438,128 @@ function runCompare(beforePath: string, afterPath: string): number {
   }
 }
 
+/**
+ * V2 Faz 2 — flatten the core analyzer's Map aggregates into the plain
+ * arrays `ProjectDb.ingestLogAnalysis` expects (mirrors the desktop's
+ * `ingestLogResult`). Kept in the CLI so headless log ingestion needs no
+ * Electron / desktop process.
+ */
+function flattenLogResult(filePath: string, result: LogAnalysisResult): LogIngestInput {
+  return {
+    file: {
+      filePath,
+      fileName: basename(filePath),
+      format: result.format,
+      totalLines: result.totalLines,
+      parsedLines: result.parsedLines,
+      skippedLines: result.skippedLines,
+      minTs: result.minTs,
+      maxTs: result.maxTs,
+    },
+    urlStats: Array.from(result.urlStats.values()).map((u) => ({
+      path: u.path,
+      totalHits: u.totalHits,
+      botHits: u.botHits,
+      googlebotHits: u.googlebotHits,
+      bingbotHits: u.bingbotHits,
+      yandexbotHits: u.yandexbotHits,
+      otherBotHits: u.otherBotHits,
+      lastStatus: u.lastStatus,
+      firstTs: u.firstTs,
+      lastTs: u.lastTs,
+    })),
+    daily: Array.from(result.daily.entries()).map(([key, hits]) => {
+      const sp = key.indexOf(' ');
+      return { day: key.slice(0, sp), bucket: key.slice(sp + 1), hits };
+    }),
+    status: Array.from(result.status.entries()).map(([status, v]) => ({
+      status,
+      count: v.count,
+      botCount: v.botCount,
+    })),
+    bots: Array.from(result.bots.entries()).map(([bot, agg]) => ({
+      bot,
+      family: agg.family,
+      hits: agg.hits,
+      totalIps: agg.ips.size,
+      verifiedIps: agg.verifiedIps,
+      verifiable: agg.verifiable,
+    })),
+    urlBots: Array.from(result.urlStats.values()).flatMap((u) =>
+      Array.from(u.botCounts.entries()).map(([bot, hits]) => ({ path: u.path, bot, hits })),
+    ),
+  };
+}
+
+/**
+ * Parse an access log, detect bots, and merge the aggregates into a
+ * project DB. Prints a summary (or JSON). Exit 0 on success, 2 on error.
+ */
+async function runAnalyzeLogs(
+  logPath: string,
+  opts: {
+    projectPath: string;
+    format: 'auto' | 'apache-combined' | 'apache-common' | 'nginx' | 'iis-w3c' | 'custom';
+    regex: string | undefined;
+    verifyBots: boolean;
+    json: boolean;
+  },
+): Promise<number> {
+  let db: ProjectDb;
+  try {
+    db = new ProjectDb(opts.projectPath);
+  } catch (err) {
+    console.error(`Cannot open project ${opts.projectPath}: ${(err as Error).message}`);
+    return 2;
+  }
+  try {
+    const result = await analyzeLogFile(logPath, {
+      format: opts.format,
+      customRegex: opts.regex,
+      verifyBots: opts.verifyBots,
+    });
+    const summary = db.ingestLogAnalysis(flattenLogResult(logPath, result));
+    const overview = db.getLogOverview();
+    const bots = db.getLogBots().slice(0, 10);
+    const topUrls = db
+      .queryLogUrlStats({ limit: 10, offset: 0, sortBy: 'botHits', filter: 'bots' })
+      .rows;
+
+    if (opts.json) {
+      process.stdout.write(
+        JSON.stringify({ ok: true, imported: summary, overview, topBots: bots, topBotUrls: topUrls }, null, 2) + '\n',
+      );
+    } else {
+      console.log(`Log: ${summary.fileName} (${summary.format})`);
+      console.log(`  Lines:  ${summary.parsedLines}/${summary.totalLines} parsed, ${summary.skippedLines} skipped`);
+      console.log(`  Hits:   ${overview.totalHits} total · ${overview.botHits} bot · ${overview.humanHits} human`);
+      if (overview.minTs) {
+        const d = (ms: number): string => new Date(ms).toISOString().slice(0, 16).replace('T', ' ');
+        console.log(`  Range:  ${d(overview.minTs)} → ${overview.maxTs ? d(overview.maxTs) : '?'}`);
+      }
+      if (bots.length > 0) {
+        console.log('  Top bots:');
+        for (const b of bots) {
+          const v = b.verifiable ? ` (${b.verifiedIps}/${b.totalIps} IPs verified)` : '';
+          console.log(`    ${b.bot.padEnd(28)} ${String(b.hits).padStart(8)}${v}`);
+        }
+      }
+      if (topUrls.length > 0) {
+        console.log('  Top bot-hit URLs:');
+        for (const u of topUrls) {
+          console.log(`    ${String(u.botHits).padStart(8)}  ${u.path}${u.inCrawl ? '' : '  [orphan]'}`);
+        }
+      }
+    }
+    return 0;
+  } catch (err) {
+    console.error(`analyze-logs failed: ${(err as Error).message}`);
+    return 2;
+  } finally {
+    db.close();
+  }
+}
+
 main().catch((err: unknown) => {
   console.error(err instanceof Error ? err.stack : err);
   process.exit(2);

apps/desktop/src/main/index.ts

@@ -40,7 +40,6 @@ import {
   type CrashRecoveryStatus,
   type ExportHtmlReportInput,
   type ExportHtmlReportResult,
-  type BulkExportFile,
   type BulkExportResult,
   type UrlCategory,
   type CompareLoadInput,
@@ -132,6 +131,21 @@ import {
   type ExportBigqueryResult,
   type GscInspectionResult,
   type UrlAnalyticsDetail,
+  type LogAnalyzeInput,
+  type LogAnalyzeResult,
+  type LogOverview,
+  type LogUrlStatsInput,
+  type LogUrlStatsResult,
+  type LogBotRow,
+  type LogStatusRow,
+  type LogTrendRow,
+  type LogCrawlBudgetRow,
+  type LogDiscoveryRow,
+  type LogSeedDiscoveryResult,
+  type LogIngestInput,
+  type LogImportSummary,
+  type LogExportInput,
+  type LogExportResult,
 } from '@freecrawl/shared-types';
 import {
   BrowserPool,
@@ -156,6 +170,12 @@ import {
   normalizeUrl,
   compileUrlRegexRewrites,
   previewExtractionRules,
+  runBulkExport,
+  analyzeLogFile,
+  buildLogExportXlsx,
+  buildLogExportCsv,
+  type LogAnalysisResult,
+  type LogExportTables,
 } from '@freecrawl/core';
 import { fetch as undiciFetch } from 'undici';
 import { ProjectDb } from '@freecrawl/db';
@@ -202,6 +222,8 @@ let mainWindow: BrowserWindow | null = null;
 let logsWindow: BrowserWindow | null = null;
 /** Standalone Visualization popup — single-instance, native window. */
 let visualizationWindow: BrowserWindow | null = null;
+/** V2 Faz 2 — standalone Log File Analyzer popup, single-instance. */
+let logAnalyzerWindow: BrowserWindow | null = null;
 let db: ProjectDb | null = null;
 let activeCrawler: Crawler | null = null;
 /** V2 Faz 1 — Playwright browser pool, lazily started when a JS-mode
@@ -363,6 +385,11 @@ function fireDataChanged(): void {
   if (visualizationWindow && !visualizationWindow.isDestroyed()) {
     visualizationWindow.webContents.send(IPC.dataChanged);
   }
+  // V2 Faz 2 — keep the Log Analyzer window's crawl × log joins fresh
+  // when the crawl mutates (orphan/budget reports depend on the URL set).
+  if (logAnalyzerWindow && !logAnalyzerWindow.isDestroyed()) {
+    logAnalyzerWindow.webContents.send(IPC.dataChanged);
+  }
 }
 
 /**
@@ -929,6 +956,7 @@ function rebuildMenu(): void {
       onResetDiagnosticDialogs: () => resetDiagnosticDialogs(),
       onOpenLogsFolder: () => openLogsFolder(),
       onCheckForUpdates: () => void checkForUpdates(),
+      onOpenLogAnalyzer: () => openLogAnalyzerWindow(),
     }),
   );
 }
@@ -1706,6 +1734,160 @@ function openVisualizationWindow(): void {
   logger.log('info', 'main', 'Visualization window opened');
 }
 
+/**
+ * V2 Faz 2 — Standalone Log File Analyzer window. Same renderer bundle
+ * with `?loganalyzer=1` so React mounts `LogAnalyzerView` only. Keeps the
+ * log-analysis surface off the main tab strip; single-instance.
+ */
+function openLogAnalyzerWindow(): void {
+  if (logAnalyzerWindow && !logAnalyzerWindow.isDestroyed()) {
+    logAnalyzerWindow.show();
+    logAnalyzerWindow.focus();
+    return;
+  }
+  const win = new BrowserWindow({
+    width: 1280,
+    height: 820,
+    minWidth: 720,
+    minHeight: 480,
+    show: false,
+    backgroundColor: '#0a0a0a',
+    title: 'FreeCrawl — Log Analyzer',
+    webPreferences: {
+      preload: join(__dirname, '../preload/index.js'),
+      sandbox: false,
+      contextIsolation: true,
+      nodeIntegration: false,
+      backgroundThrottling: false,
+    },
+  });
+  win.setMenu(null);
+  win.on('ready-to-show', () => win.show());
+  win.on('page-title-updated', (e) => e.preventDefault());
+  win.on('closed', () => {
+    if (logAnalyzerWindow === win) logAnalyzerWindow = null;
+  });
+  win.webContents.on('before-input-event', (e, input) => {
+    if (input.type !== 'keyDown') return;
+    const key = input.key.toLowerCase();
+    const mod = input.control || input.meta;
+    const isF12 = key === 'f12';
+    const isCtrlShiftI = mod && input.shift && key === 'i';
+    const isCtrlAltI = mod && input.alt && key === 'i';
+    const isCtrlShiftJ = mod && input.shift && key === 'j';
+    const isCtrlShiftC = mod && input.shift && key === 'c';
+    if (isF12 || isCtrlShiftI || isCtrlAltI || isCtrlShiftJ || isCtrlShiftC) {
+      e.preventDefault();
+    }
+  });
+  win.webContents.on('devtools-opened', () => {
+    win.webContents.closeDevTools();
+  });
+  if (process.env['ELECTRON_RENDERER_URL']) {
+    void win.loadURL(process.env['ELECTRON_RENDERER_URL'] + '?loganalyzer=1');
+  } else {
+    void win.loadFile(join(__dirname, '../renderer/index.html'), {
+      search: 'loganalyzer=1',
+    });
+  }
+  logAnalyzerWindow = win;
+  logger.log('info', 'main', 'Log Analyzer window opened');
+}
+
+/**
+ * V2 Faz 2 — flatten the core analyzer's Map aggregates into the plain
+ * arrays the DB ingest expects, then persist. Returns the import summary.
+ */
+function ingestLogResult(filePath: string, result: LogAnalysisResult): LogImportSummary {
+  const fileName = basename(filePath);
+  const payload: LogIngestInput = {
+    file: {
+      filePath,
+      fileName,
+      format: result.format,
+      totalLines: result.totalLines,
+      parsedLines: result.parsedLines,
+      skippedLines: result.skippedLines,
+      minTs: result.minTs,
+      maxTs: result.maxTs,
+    },
+    urlStats: Array.from(result.urlStats.values()).map((u) => ({
+      path: u.path,
+      totalHits: u.totalHits,
+      botHits: u.botHits,
+      googlebotHits: u.googlebotHits,
+      bingbotHits: u.bingbotHits,
+      yandexbotHits: u.yandexbotHits,
+      otherBotHits: u.otherBotHits,
+      lastStatus: u.lastStatus,
+      firstTs: u.firstTs,
+      lastTs: u.lastTs,
+    })),
+    daily: Array.from(result.daily.entries()).map(([key, hits]) => {
+      const sp = key.indexOf(' ');
+      return { day: key.slice(0, sp), bucket: key.slice(sp + 1), hits };
+    }),
+    status: Array.from(result.status.entries()).map(([status, v]) => ({
+      status,
+      count: v.count,
+      botCount: v.botCount,
+    })),
+    bots: Array.from(result.bots.entries()).map(([bot, agg]) => ({
+      bot,
+      family: agg.family,
+      hits: agg.hits,
+      totalIps: agg.ips.size,
+      verifiedIps: agg.verifiedIps,
+      verifiable: agg.verifiable,
+    })),
+    urlBots: Array.from(result.urlStats.values()).flatMap((u) =>
+      Array.from(u.botCounts.entries()).map(([bot, hits]) => ({ path: u.path, bot, hits })),
+    ),
+  };
+  return getDb().ingestLogAnalysis(payload);
+}
+
+/**
+ * V2 Faz 2 (item 12) — inject log-discovered URLs (paths the crawl never
+ * reached) into the active crawl as fresh seeds. Reconstructs absolute
+ * URLs from the project's base origin; no-op when no crawl is running.
+ */
+function seedDiscoveryUrls(limit: number): LogSeedDiscoveryResult {
+  const db = getDb();
+  let origin: string | null = null;
+  const start = db.getMeta('lastStartUrl');
+  if (start) {
+    try {
+      origin = new URL(start).origin;
+    } catch {
+      origin = null;
+    }
+  }
+  if (!origin) {
+    const sample = db.queryUrls({ category: 'internal:all', limit: 1, offset: 0 }).rows[0];
+    if (sample?.url) {
+      try {
+        origin = new URL(sample.url).origin;
+      } catch {
+        origin = null;
+      }
+    }
+  }
+  if (!origin) {
+    return { enqueued: 0, hasActiveCrawl: activeCrawler?.isRunning === true, reason: 'no-base-origin' };
+  }
+  if (!activeCrawler || !activeCrawler.isRunning) {
+    return { enqueued: 0, hasActiveCrawl: false, reason: 'no-active-crawl' };
+  }
+  const rows = db.getLogDiscovery(limit);
+  let enqueued = 0;
+  for (const r of rows) {
+    if (activeCrawler.enqueueManual(origin + r.path)) enqueued++;
+  }
+  if (enqueued > 0) fireDataChanged();
+  return { enqueued, hasActiveCrawl: true };
+}
+
 function registerIpc(): void {
   ipcMain.handle(IPC.appVersion, () => app.getVersion());
 
@@ -1756,6 +1938,132 @@ function registerIpc(): void {
   });
   ipcMain.handle(IPC.logsOpenWindow, () => openLogsWindow());
   ipcMain.handle(IPC.visualizationOpenWindow, () => openVisualizationWindow());
+
+  // ── V2 Faz 2 — Log File Analyzer IPC surface ───────────────────────
+  ipcMain.handle(IPC.logAnalyzerOpenWindow, () => openLogAnalyzerWindow());
+
+  ipcMain.handle(
+    IPC.logAnalyze,
+    async (e, input: LogAnalyzeInput): Promise<LogAnalyzeResult> => {
+      let filePath = input?.filePath;
+      if (!filePath) {
+        const parent =
+          BrowserWindow.fromWebContents(e.sender) ?? logAnalyzerWindow ?? mainWindow;
+        if (!parent) {
+          return { ok: false, error: 'No window available for the file picker.', imported: null, overview: null };
+        }
+        const res = await dialog.showOpenDialog(parent, {
+          title: 'Open Access Log',
+          properties: ['openFile'],
+          filters: [
+            { name: 'Log Files', extensions: ['log', 'txt', 'gz', 'out', 'access'] },
+            { name: 'All Files', extensions: ['*'] },
+          ],
+        });
+        if (res.canceled || res.filePaths.length === 0) {
+          return { ok: false, error: undefined, imported: null, overview: getDb().getLogOverview() };
+        }
+        filePath = res.filePaths[0]!;
+      }
+      try {
+        const result = await analyzeLogFile(filePath, {
+          format: input?.format ?? 'auto',
+          customRegex: input?.customRegex,
+          verifyBots: input?.verifyBots === true,
+        });
+        const imported = ingestLogResult(filePath, result);
+        if (result.urlCapHit) {
+          logger.log('warn', 'main', `Log Analyzer: distinct-URL cap hit while ingesting ${basename(filePath)}; some long-tail URLs were dropped.`);
+        }
+        logger.log(
+          'info',
+          'main',
+          `Log Analyzer ingested ${basename(filePath)} (${result.format}): ${result.parsedLines}/${result.totalLines} lines, ${result.bots.size} bot type(s).`,
+        );
+        fireDataChanged();
+        return { ok: true, imported, overview: getDb().getLogOverview() };
+      } catch (err) {
+        const error = err instanceof Error ? err.message : String(err);
+        logger.log('warn', 'main', `Log Analyzer failed on ${filePath}: ${error}`);
+        return { ok: false, error, imported: null, overview: getDb().getLogOverview() };
+      }
+    },
+  );
+
+  ipcMain.handle(IPC.logOverview, (): LogOverview => getDb().getLogOverview());
+  ipcMain.handle(
+    IPC.logUrlStats,
+    (_e, input: LogUrlStatsInput): LogUrlStatsResult => getDb().queryLogUrlStats(input),
+  );
+  ipcMain.handle(IPC.logBots, (): LogBotRow[] => getDb().getLogBots());
+  ipcMain.handle(IPC.logStatus, (): LogStatusRow[] => getDb().getLogStatusDistribution());
+  ipcMain.handle(IPC.logTrend, (): LogTrendRow[] => getDb().getLogTrend());
+  ipcMain.handle(
+    IPC.logCrawlBudget,
+    (_e, limit?: number): LogCrawlBudgetRow[] => getDb().getLogCrawlBudget(limit ?? 200),
+  );
+  ipcMain.handle(
+    IPC.logOrphans,
+    (_e, input: LogUrlStatsInput): LogUrlStatsResult =>
+      getDb().queryLogUrlStats({ ...input, filter: 'orphans' }),
+  );
+  ipcMain.handle(
+    IPC.logDiscovery,
+    (_e, limit?: number): LogDiscoveryRow[] => getDb().getLogDiscovery(limit ?? 500),
+  );
+  ipcMain.handle(
+    IPC.logSeedDiscovery,
+    (_e, limit?: number): LogSeedDiscoveryResult => seedDiscoveryUrls(limit ?? 500),
+  );
+  ipcMain.handle(
+    IPC.logExport,
+    async (e, input: LogExportInput): Promise<LogExportResult> => {
+      const format = input?.format === 'csv' ? 'csv' : 'xlsx';
+      let filePath = input?.filePath;
+      if (!filePath) {
+        const parent =
+          BrowserWindow.fromWebContents(e.sender) ?? logAnalyzerWindow ?? mainWindow;
+        if (!parent) return { filePath: '', bytesWritten: 0 };
+        const res = await dialog.showSaveDialog(parent, {
+          title: 'Export Log Analysis',
+          defaultPath: `log-analysis.${format}`,
+          filters: [
+            format === 'csv'
+              ? { name: 'CSV', extensions: ['csv'] }
+              : { name: 'Excel Workbook', extensions: ['xlsx'] },
+          ],
+        });
+        if (res.canceled || !res.filePath) return { filePath: '', bytesWritten: 0 };
+        filePath = res.filePath;
+      }
+      const db = getDb();
+      const tables: LogExportTables = {
+        overview: db.getLogOverview(),
+        urlStats: db.queryLogUrlStats({ limit: 100_000, offset: 0, sortBy: 'totalHits', filter: 'all' }).rows,
+        orphans: db.queryLogUrlStats({ limit: 100_000, offset: 0, filter: 'orphans' }).rows,
+        bots: db.getLogBots(),
+        status: db.getLogStatusDistribution(),
+        trend: db.getLogTrend(),
+        crawlBudget: db.getLogCrawlBudget(5000),
+        discovery: db.getLogDiscovery(50_000),
+      };
+      const { writeFileSync } = await import('node:fs');
+      if (format === 'csv') {
+        const csv = buildLogExportCsv(tables);
+        writeFileSync(filePath, csv, 'utf8');
+        return { filePath, bytesWritten: Buffer.byteLength(csv, 'utf8') };
+      }
+      const buf = buildLogExportXlsx(tables);
+      writeFileSync(filePath, buf);
+      return { filePath, bytesWritten: buf.length };
+    },
+  );
+  ipcMain.handle(IPC.logClear, (): void => {
+    getDb().clearLogData();
+    fireDataChanged();
+    logger.log('info', 'main', 'Log Analyzer data cleared');
+  });
+
   ipcMain.handle(
     IPC.screenshotRead,
     async (_e, absolutePath: string): Promise<string | null> => {
@@ -4518,6 +4826,76 @@ function registerIpc(): void {
       }
       return { ok: true, cancelled };
     },
+
+    // ---- Faz 0.5 closeout — bulk export + project-file actions. These
+    // were the last `[⏸]` items: dialog-driven in the desktop, exposed
+    // here with explicit paths (an agent can't drive a save/open dialog). ----
+
+    'export-bulk': async (input) => {
+      const dir = (input as { outputDir?: string }).outputDir;
+      if (!dir) throw new Error('outputDir is required (MCP cannot open dialogs).');
+      return runBulkExport(getDb(), dir);
+    },
+
+    'project-save-as': async (input) => {
+      // Snapshot the live DB to a new .seoproject via VACUUM INTO (atomic,
+      // WAL-safe). Unlike the desktop handler this does NOT switch the
+      // active project — an agent snapshotting shouldn't yank the UI.
+      const target = (input as { filePath?: string }).filePath;
+      if (!target) throw new Error('filePath is required (MCP cannot open dialogs).');
+      const database = getDb();
+      try {
+        database.walCheckpoint();
+      } catch {
+        /* best-effort */
+      }
+      // eslint-disable-next-line @typescript-eslint/no-explicit-any
+      const rawDb = (database as any).db as { exec: (sql: string) => void };
+      rawDb.exec(`VACUUM INTO '${target.replace(/'/g, "''")}'`);
+      const { statSync } = await import('node:fs');
+      return { filePath: target, bytesWritten: statSync(target).size };
+    },
+
+    'project-save-encrypted': async (input) => {
+      const i = input as { filePath?: string; password?: string };
+      if (!i.filePath) throw new Error('filePath is required.');
+      if (!i.password) throw new Error('password is required.');
+      const tmpPath = join(app.getPath('temp'), `freecrawl-enc-${Date.now()}.seoproject`);
+      try {
+        const database = getDb();
+        try {
+          database.walCheckpoint();
+        } catch {
+          /* best-effort */
+        }
+        // eslint-disable-next-line @typescript-eslint/no-explicit-any
+        const rawDb = (database as any).db as { exec: (sql: string) => void };
+        rawDb.exec(`VACUUM INTO '${tmpPath.replace(/'/g, "''")}'`);
+        const { encryptFile } = await import('./project-encryption.js');
+        const result = encryptFile(tmpPath, i.filePath, i.password);
+        return { filePath: i.filePath, bytesWritten: result.bytesWritten };
+      } finally {
+        try {
+          const { unlinkSync } = await import('node:fs');
+          unlinkSync(tmpPath);
+        } catch {
+          /* best-effort tmp cleanup */
+        }
+      }
+    },
+
+    'project-open-encrypted': async (input) => {
+      // Decrypt a .seoproject.enc to an explicit destination, then make it
+      // the desktop's active project (this DOES switch the UI).
+      const i = input as { filePath?: string; password?: string; destPath?: string };
+      if (!i.filePath) throw new Error('filePath is required (source .seoproject.enc).');
+      if (!i.password) throw new Error('password is required.');
+      if (!i.destPath) throw new Error('destPath is required (output .seoproject path).');
+      const { decryptFile } = await import('./project-encryption.js');
+      decryptFile(i.filePath, i.destPath, i.password);
+      openProjectAtPath(i.destPath);
+      return { filePath: i.destPath };
+    },
   };
 
   crawlController = {
@@ -5411,279 +5789,9 @@ function registerIpc(): void {
       return { outputDir: '', files: [], errors: [] };
     }
     const outputDir = dirRes.filePaths[0]!;
-    const tasks: { label: string; file: string; category: UrlCategory }[] = [
+    // Task list + per-category writing live in @freecrawl/core's runBulkExport
-      { label: 'All URLs', file: 'all-urls.csv', category: 'all' },
+    // so the desktop handler and the MCP `export_bulk` action stay in sync.
-      { label: 'Internal HTML', file: 'internal-html.csv', category: 'internal:html' },
+    const { files, errors } = await runBulkExport(getDb(), outputDir);
-      { label: 'Internal All', file: 'internal-all.csv', category: 'internal:all' },
-      { label: 'External All', file: 'external-all.csv', category: 'external:all' },
-      { label: '2xx Success', file: 'status-2xx.csv', category: 'status:2xx' },
-      { label: '3xx Redirects', file: 'status-3xx.csv', category: 'status:3xx' },
-      { label: '4xx Client Errors', file: 'status-4xx.csv', category: 'status:4xx' },
-      { label: '5xx Server Errors', file: 'status-5xx.csv', category: 'status:5xx' },
-      { label: 'Indexable', file: 'indexable.csv', category: 'indexability:indexable' },
-      {
-        label: 'Non-Indexable',
-        file: 'non-indexable.csv',
-        category: 'indexability:non-indexable',
-      },
-      {
-        label: 'Title Issues — Missing',
-        file: 'issues-title-missing.csv',
-        category: 'issues:title-missing',
-      },
-      {
-        label: 'Title Issues — Duplicate',
-        file: 'issues-title-duplicate.csv',
-        category: 'issues:title-duplicate',
-      },
-      {
-        label: 'Meta Description Issues — Missing',
-        file: 'issues-meta-missing.csv',
-        category: 'issues:meta-missing',
-      },
-      {
-        label: 'H1 Issues — Missing',
-        file: 'issues-h1-missing.csv',
-        category: 'issues:h1-missing',
-      },
-      {
-        label: 'Canonical Issues — Missing',
-        file: 'issues-canonical-missing.csv',
-        category: 'issues:canonical-missing',
-      },
-      {
-        label: 'Pagination Broken',
-        file: 'issues-pagination-broken.csv',
-        category: 'issues:pagination-broken',
-      },
-      {
-        label: 'Mixed Content',
-        file: 'issues-mixed-content.csv',
-        category: 'issues:mixed-content',
-      },
-      {
-        label: 'Insecure Form Action',
-        file: 'issues-insecure-form-action.csv',
-        category: 'issues:insecure-form-action',
-      },
-      {
-        label: 'Hreflang — Reciprocity Missing',
-        file: 'hreflang-reciprocity-missing.csv',
-        category: 'issues:hreflang-reciprocity-missing',
-      },
-      {
-        label: 'Sitemap — Crawled, Not Listed',
-        file: 'sitemap-crawled-not-in-sitemap.csv',
-        category: 'issues:crawled-not-in-sitemap',
-      },
-      {
-        label: 'Image Missing Alt',
-        file: 'issues-image-missing-alt.csv',
-        category: 'issues:image-missing-alt',
-      },
-      {
-        label: 'Near-Duplicate Content',
-        file: 'issues-near-duplicate.csv',
-        category: 'issues:near-duplicate',
-      },
-      // V1 Faz 4 — Redirects family.
-      { label: 'All Redirects', file: 'all-redirects.csv', category: 'tab:redirects' },
-      {
-        label: 'Redirect Chains (Long)',
-        file: 'redirect-chains-long.csv',
-        category: 'issues:redirect-chain-long',
-      },
-      { label: 'Redirect Loops', file: 'redirect-loops.csv', category: 'issues:redirect-loop' },
-      {
-        label: 'Self-Redirects',
-        file: 'redirect-self.csv',
-        category: 'issues:redirect-self',
-      },
-      // V1 Faz 4 — Canonical family.
-      { label: 'All Canonicals', file: 'all-canonicals.csv', category: 'tab:canonicals' },
-      {
-        label: 'Canonical — Non Self-Referencing',
-        file: 'canonical-non-self.csv',
-        category: 'issues:canonical-non-self',
-      },
-      {
-        label: 'Canonical — To Non-200',
-        file: 'canonical-to-non-200.csv',
-        category: 'issues:canonical-to-non-200',
-      },
-      {
-        label: 'Canonical — To Redirect',
-        file: 'canonical-to-redirect.csv',
-        category: 'issues:canonical-to-redirect',
-      },
-      {
-        label: 'Canonical — To Noindex',
-        file: 'canonical-to-noindex.csv',
-        category: 'issues:canonical-to-noindex',
-      },
-      {
-        label: 'Canonical — Multi-Hop Chain',
-        file: 'canonical-chain-multi-hop.csv',
-        category: 'issues:canonical-chain-multi-hop',
-      },
-      {
-        label: 'Multiple Canonicals',
-        file: 'canonical-multiple.csv',
-        category: 'issues:multiple-canonicals',
-      },
-      // V1 Faz 4 — Pagination family.
-      { label: 'All Pagination', file: 'all-pagination.csv', category: 'tab:pagination' },
-      {
-        label: 'Pagination — Sequence Break',
-        file: 'pagination-sequence-break.csv',
-        category: 'issues:pagination-sequence-break',
-      },
-      {
-        label: 'Pagination — Canonical Conflict',
-        file: 'pagination-canonical-conflict.csv',
-        category: 'issues:pagination-canonical-conflict',
-      },
-      // V1 Faz 4 — Hreflang family (7 variants).
-      { label: 'Hreflang — All', file: 'hreflang-all.csv', category: 'tab:hreflang' },
-      {
-        label: 'Hreflang — x-default Missing',
-        file: 'hreflang-x-default-missing.csv',
-        category: 'issues:hreflang-x-default-missing',
-      },
-      {
-        label: 'Hreflang — Invalid Language Code',
-        file: 'hreflang-invalid-code.csv',
-        category: 'issues:hreflang-invalid-code',
-      },
-      {
-        label: 'Hreflang — Self-Ref Missing',
-        file: 'hreflang-self-ref-missing.csv',
-        category: 'issues:hreflang-self-ref-missing',
-      },
-      {
-        label: 'Hreflang — Target Issues',
-        file: 'hreflang-target-issues.csv',
-        category: 'issues:hreflang-target-issues',
-      },
-      {
-        label: 'Hreflang — Inconsistent Language',
-        file: 'hreflang-inconsistent-lang.csv',
-        category: 'issues:hreflang-inconsistent-lang',
-      },
-      // V1 Faz 4 — Duplicates (cluster + exact).
-      { label: 'Duplicate Pages', file: 'duplicate-pages.csv', category: 'tab:duplicates' },
-      {
-        label: 'Duplicate Content — Exact',
-        file: 'duplicate-content-exact.csv',
-        category: 'issues:duplicate-content-exact',
-      },
-      // V1 Faz 4 — Custom data exports.
-      {
-        label: 'Custom Extraction Results',
-        file: 'custom-extraction.csv',
-        category: 'tab:custom-extraction',
-      },
-      { label: 'Custom Search Hits', file: 'custom-search.csv', category: 'tab:custom-search' },
-      // V1 Faz 4 — Structured data validation errors.
-      {
-        label: 'Structured Data — Missing',
-        file: 'structured-data-missing.csv',
-        category: 'issues:structured-data-missing',
-      },
-      {
-        label: 'Structured Data — Invalid',
-        file: 'structured-data-invalid.csv',
-        category: 'issues:structured-data-invalid',
-      },
-      {
-        label: 'Structured Data — Duplicate @id',
-        file: 'structured-data-duplicate-id.csv',
-        category: 'issues:schema-duplicate-id',
-      },
-      {
-        label: 'Structured Data — Missing Required Property',
-        file: 'structured-data-missing-required.csv',
-        category: 'issues:schema-missing-required',
-      },
-      // V1 Faz 4 — AMP.
-      { label: 'AMP Pages', file: 'amp-pages.csv', category: 'tab:amp' },
-      // V1 Faz 4 — Image variants.
-      {
-        label: 'Image — Missing Alt',
-        file: 'image-missing-alt.csv',
-        category: 'issues:image-missing-alt',
-      },
-      {
-        label: 'Image — Empty Alt',
-        file: 'image-empty-alt.csv',
-        category: 'issues:image-empty-alt',
-      },
-      {
-        label: 'Image — Too Large',
-        file: 'image-too-large.csv',
-        category: 'issues:image-too-large',
-      },
-      {
-        label: 'Image — Broken Src',
-        file: 'image-broken-src.csv',
-        category: 'issues:image-broken-src',
-      },
-      // V1 Faz 4 — Security.
-      { label: 'Security Audit', file: 'security-audit.csv', category: 'tab:security' },
-      {
-        label: 'Mixed Content — Active',
-        file: 'mixed-content-active.csv',
-        category: 'issues:mixed-content-active',
-      },
-      {
-        label: 'Mixed Content — Passive',
-        file: 'mixed-content-passive.csv',
-        category: 'issues:mixed-content-passive',
-      },
-      {
-        label: 'HSTS Missing',
-        file: 'security-hsts-missing.csv',
-        category: 'issues:hsts-missing',
-      },
-      {
-        label: 'CSP Missing',
-        file: 'security-csp-missing.csv',
-        category: 'issues:csp-missing',
-      },
-      // V1 Faz 4 — SERP summary.
-      { label: 'SERP Summary', file: 'serp-summary.csv', category: 'tab:serp' },
-      // V1 Faz 4 — Outlinks Zero (dead-end pages: no outbound links).
-      {
-        label: 'Outlinks Zero',
-        file: 'outlinks-zero.csv',
-        category: 'issues:outlinks-zero',
-      },
-    ];
-    const files: BulkExportFile[] = [];
-    const errors: { label: string; error: string }[] = [];
-    const database = getDb();
-    for (const task of tasks) {
-      const filePath = join(outputDir, task.file);
-      try {
-        const { rowsWritten } = await exportUrlsToCsv(database, filePath, {
-          category: task.category,
-        });
-        // Skip 0-row files — they bloat the bulk dump and the user almost
-        // certainly doesn't want empty CSVs cluttering the directory.
-        if (rowsWritten === 0) {
-          try {
-            const { unlinkSync } = await import('node:fs');
-            unlinkSync(filePath);
-          } catch {
-            /* ignore */
-          }
-          continue;
-        }
-        files.push({ filePath, label: task.label, category: task.category, rowsWritten });
-      } catch (err) {
-        errors.push({ label: task.label, error: (err as Error).message });
-      }
-    }
     if (mainWindow) {
       const totalRows = files.reduce((s, f) => s + f.rowsWritten, 0);
       await dialog.showMessageBox(mainWindow, {
@@ -6152,6 +6260,14 @@ function performShutdown(): void {
     }
     visualizationWindow = null;
   }
+  if (logAnalyzerWindow && !logAnalyzerWindow.isDestroyed()) {
+    try {
+      logAnalyzerWindow.close();
+    } catch {
+      /* ignore */
+    }
+    logAnalyzerWindow = null;
+  }
   activeCrawler?.stop();
   // V2 Faz 1 — Tear down Playwright before worker pools so a hung
   // Chromium process can't keep the app alive after the renderer

apps/desktop/src/main/menu-i18n.ts

@@ -46,6 +46,9 @@ export interface MenuLabels {
   openVisualizationWindow: string;
   reports: string;
   reportsItem: string;
+  logAnalyzer: string;
+  openLogAnalyzerWindow: string;
+  openLogAnalyzerWindowTooltip: string;
   help: string;
   documentation: string;
   showLogs: string;
@@ -98,6 +101,10 @@ const MENU_EN: MenuLabels = {
   openVisualizationWindow: 'Open Visualization Window…',
   reports: 'Reports',
   reportsItem: 'Reports…',
+  logAnalyzer: 'Log Analyzer',
+  openLogAnalyzerWindow: 'Open Log Analyzer Window…',
+  openLogAnalyzerWindowTooltip:
+    'Analyze server access logs (Apache / Nginx / IIS) — bot hits per URL, crawl budget, and crawl × log orphan detection in a standalone window.',
   help: 'Help',
   documentation: 'Documentation',
   showLogs: 'Show Logs…',
@@ -155,6 +162,10 @@ const MENU_TR: MenuLabels = {
   openVisualizationWindow: 'Görselleştirme Penceresini Aç…',
   reports: 'Raporlar',
   reportsItem: 'Raporlar…',
+  logAnalyzer: 'Log Analizi',
+  openLogAnalyzerWindow: 'Log Analiz Penceresini Aç…',
+  openLogAnalyzerWindowTooltip:
+    'Sunucu erişim loglarını analiz et (Apache / Nginx / IIS) — URL başına bot isabeti, crawl bütçesi ve crawl × log yetim tespiti ayrı bir pencerede.',
   help: 'Yardım',
   documentation: 'Dokümantasyon',
   showLogs: 'Logları Göster…',

apps/desktop/src/main/menu.ts

@@ -28,6 +28,8 @@ export interface AppMenuHandlers {
    * update available with "Open release page" button / network error).
    */
   onCheckForUpdates: () => void;
+  /** V2 Faz 2 — open the standalone Log File Analyzer window. */
+  onOpenLogAnalyzer: () => void;
   /** UI language for menu labels. Falls back to `en` when missing. */
   lang?: MenuLang;
 }
@@ -182,6 +184,16 @@ export function buildAppMenu(handlers: AppMenuHandlers): Menu {
         },
       ],
     },
+    {
+      label: L.logAnalyzer,
+      submenu: [
+        {
+          label: L.openLogAnalyzerWindow,
+          toolTip: L.openLogAnalyzerWindowTooltip,
+          click: () => handlers.onOpenLogAnalyzer(),
+        },
+      ],
+    },
     { role: 'windowMenu' as const },
     {
       label: L.help,

apps/desktop/src/preload/index.ts

@@ -128,6 +128,19 @@ import {
   type ExportBigqueryInput,
   type ExportBigqueryResult,
   type UrlAnalyticsDetail,
+  type LogAnalyzeInput,
+  type LogAnalyzeResult,
+  type LogOverview,
+  type LogUrlStatsInput,
+  type LogUrlStatsResult,
+  type LogBotRow,
+  type LogStatusRow,
+  type LogTrendRow,
+  type LogCrawlBudgetRow,
+  type LogDiscoveryRow,
+  type LogSeedDiscoveryResult,
+  type LogExportInput,
+  type LogExportResult,
 } from '@freecrawl/shared-types';
 
 function subscribe<T>(
@@ -382,6 +395,27 @@ const api: FreeCrawlApi = {
     ipcRenderer.invoke(IPC.exportBigquery, input),
   urlAnalyticsGet: (url: string): Promise<UrlAnalyticsDetail | null> =>
     ipcRenderer.invoke(IPC.urlAnalyticsGet, url),
+  logAnalyzerOpenWindow: (): Promise<void> =>
+    ipcRenderer.invoke(IPC.logAnalyzerOpenWindow),
+  logAnalyze: (input: LogAnalyzeInput): Promise<LogAnalyzeResult> =>
+    ipcRenderer.invoke(IPC.logAnalyze, input),
+  logOverview: (): Promise<LogOverview> => ipcRenderer.invoke(IPC.logOverview),
+  logUrlStats: (input: LogUrlStatsInput): Promise<LogUrlStatsResult> =>
+    ipcRenderer.invoke(IPC.logUrlStats, input),
+  logBots: (): Promise<LogBotRow[]> => ipcRenderer.invoke(IPC.logBots),
+  logStatus: (): Promise<LogStatusRow[]> => ipcRenderer.invoke(IPC.logStatus),
+  logTrend: (): Promise<LogTrendRow[]> => ipcRenderer.invoke(IPC.logTrend),
+  logCrawlBudget: (limit?: number): Promise<LogCrawlBudgetRow[]> =>
+    ipcRenderer.invoke(IPC.logCrawlBudget, limit),
+  logOrphans: (input: LogUrlStatsInput): Promise<LogUrlStatsResult> =>
+    ipcRenderer.invoke(IPC.logOrphans, input),
+  logDiscovery: (limit?: number): Promise<LogDiscoveryRow[]> =>
+    ipcRenderer.invoke(IPC.logDiscovery, limit),
+  logSeedDiscovery: (limit?: number): Promise<LogSeedDiscoveryResult> =>
+    ipcRenderer.invoke(IPC.logSeedDiscovery, limit),
+  logExport: (input: LogExportInput): Promise<LogExportResult> =>
+    ipcRenderer.invoke(IPC.logExport, input),
+  logClear: (): Promise<void> => ipcRenderer.invoke(IPC.logClear),
   onLogEntry: (cb) => subscribe<LogEntry>(IPC.logsEntry, cb),
   onLogsBatch: (cb) => subscribe<LogEntry[]>(IPC.logsBatch, cb),
   onLogsBusy: (cb) => subscribe<boolean>(IPC.logsBusy, cb),

apps/desktop/src/renderer/src/LogAnalyzerView.tsx

@@ -0,0 +1,755 @@
+import { useCallback, useEffect, useMemo, useState } from 'react';
+import { useTranslation } from 'react-i18next';
+import type {
+  LogAnalyzeResult,
+  LogBotRow,
+  LogCrawlBudgetRow,
+  LogDiscoveryRow,
+  LogFormatChoice,
+  LogOverview,
+  LogStatusRow,
+  LogTrendRow,
+  LogUrlStatRow,
+  LogUrlStatsInput,
+} from '@freecrawl/shared-types';
+
+/**
+ * V2 Faz 2 — standalone Log File Analyzer window (renderer `?loganalyzer=1`).
+ * Ingests server access logs and surfaces every Faz 2 feature: bot hits per
+ * URL, crawl-budget, response-code distribution, daily trend, crawl × log
+ * orphan detection, and seeding log-discovered URLs into the active crawl.
+ */
+
+type Tab = 'urls' | 'bots' | 'status' | 'trend' | 'budget' | 'orphans' | 'discovery';
+
+const FAMILY_COLORS: Record<string, string> = {
+  googlebot: 'text-blue-300 bg-blue-500/15',
+  bingbot: 'text-teal-300 bg-teal-500/15',
+  yandexbot: 'text-red-300 bg-red-500/15',
+  search: 'text-green-300 bg-green-500/15',
+  ai: 'text-purple-300 bg-purple-500/15',
+  social: 'text-pink-300 bg-pink-500/15',
+  'seo-tool': 'text-amber-300 bg-amber-500/15',
+  other: 'text-surface-300 bg-surface-700/40',
+  human: 'text-surface-400 bg-surface-700/30',
+};
+
+function fmtNum(n: number | null | undefined): string {
+  return typeof n === 'number' ? n.toLocaleString() : '—';
+}
+
+function fmtTs(ts: number | null): string {
+  if (ts === null) return '—';
+  try {
+    return new Date(ts).toISOString().replace('T', ' ').slice(0, 16);
+  } catch {
+    return '—';
+  }
+}
+
+export function LogAnalyzerView() {
+  const { t } = useTranslation();
+  const [overview, setOverview] = useState<LogOverview | null>(null);
+  const [tab, setTab] = useState<Tab>('urls');
+  const [format, setFormat] = useState<LogFormatChoice>('auto');
+  const [customRegex, setCustomRegex] = useState('');
+  const [verifyBots, setVerifyBots] = useState(false);
+  const [busy, setBusy] = useState(false);
+  const [toast, setToast] = useState<string | null>(null);
+  const [version, setVersion] = useState(0);
+
+  const refreshOverview = useCallback(() => {
+    void window.freecrawl.logOverview().then(setOverview);
+  }, []);
+
+  useEffect(() => {
+    refreshOverview();
+    const off = window.freecrawl.onDataChanged(() => {
+      refreshOverview();
+      setVersion((v) => v + 1);
+    });
+    return () => off();
+  }, [refreshOverview]);
+
+  const showToast = useCallback((msg: string) => {
+    setToast(msg);
+    setTimeout(() => setToast(null), 4000);
+  }, []);
+
+  const onImport = useCallback(async () => {
+    setBusy(true);
+    try {
+      const res: LogAnalyzeResult = await window.freecrawl.logAnalyze({
+        format,
+        customRegex: format === 'custom' ? customRegex : undefined,
+        verifyBots,
+      });
+      if (res.overview) setOverview(res.overview);
+      if (res.ok && res.imported) {
+        const i = res.imported;
+        showToast(
+          t('logAnalyzer.imported', {
+            defaultValue: '{{file}} — parsed {{parsed}}/{{total}} lines ({{format}})',
+            file: i.fileName,
+            parsed: i.parsedLines.toLocaleString(),
+            total: i.totalLines.toLocaleString(),
+            format: i.format,
+          }),
+        );
+        setVersion((v) => v + 1);
+      } else if (res.error) {
+        showToast(t('logAnalyzer.importError', { defaultValue: 'Import failed: {{err}}', err: res.error }));
+      }
+    } finally {
+      setBusy(false);
+    }
+  }, [format, customRegex, verifyBots, showToast, t]);
+
+  const onClear = useCallback(async () => {
+    if (!window.confirm(t('logAnalyzer.clearConfirm', { defaultValue: 'Clear all ingested log data from this project?' }))) {
+      return;
+    }
+    await window.freecrawl.logClear();
+    refreshOverview();
+    setVersion((v) => v + 1);
+  }, [refreshOverview, t]);
+
+  const onExport = useCallback(
+    async (format: 'csv' | 'xlsx') => {
+      const r = await window.freecrawl.logExport({ format });
+      if (r.filePath) {
+        showToast(
+          t('logAnalyzer.exported', {
+            defaultValue: 'Exported {{size}} KB → {{path}}',
+            size: Math.max(1, Math.round(r.bytesWritten / 1024)),
+            path: r.filePath,
+          }),
+        );
+      }
+    },
+    [showToast, t],
+  );
+
+  const hasData = overview?.hasData ?? false;
+
+  return (
+    <div className="flex h-screen w-screen flex-col bg-surface-950 text-surface-100">
+      {/* Title bar */}
+      <div className="flex shrink-0 items-center gap-2 border-b border-surface-800 bg-surface-900 px-3 py-1.5 text-[12px]">
+        <span className="font-semibold tracking-wide text-surface-200">
+          {t('logAnalyzer.title', { defaultValue: 'Log File Analyzer' })}
+        </span>
+        <span className="text-surface-500">·</span>
+        <span className="text-surface-500">
+          {t('logAnalyzer.windowHint', {
+            defaultValue: 'Apache / Nginx / IIS access logs — bot activity, crawl budget, crawl × log join',
+          })}
+        </span>
+      </div>
+
+      {/* Toolbar */}
+      <div className="flex shrink-0 flex-wrap items-center gap-2 border-b border-surface-800 bg-surface-900/60 px-3 py-2 text-[12px]">
+        <label className="text-surface-400">{t('logAnalyzer.format', { defaultValue: 'Format' })}</label>
+        <select
+          value={format}
+          onChange={(e) => setFormat(e.target.value as LogFormatChoice)}
+          className="rounded border border-surface-700 bg-surface-800 px-2 py-1 text-surface-100"
+        >
+          <option value="auto">{t('logAnalyzer.fmtAuto', { defaultValue: 'Auto-detect' })}</option>
+          <option value="apache-combined">{t('logAnalyzer.fmtApacheCombined', { defaultValue: 'Apache / Nginx Combined' })}</option>
+          <option value="apache-common">{t('logAnalyzer.fmtApacheCommon', { defaultValue: 'Apache Common (CLF)' })}</option>
+          <option value="nginx">{t('logAnalyzer.fmtNginx', { defaultValue: 'Nginx' })}</option>
+          <option value="iis-w3c">{t('logAnalyzer.fmtIis', { defaultValue: 'IIS W3C Extended' })}</option>
+          <option value="custom">{t('logAnalyzer.fmtCustom', { defaultValue: 'Custom (regex)' })}</option>
+        </select>
+        {format === 'custom' && (
+          <input
+            value={customRegex}
+            onChange={(e) => setCustomRegex(e.target.value)}
+            placeholder={t('logAnalyzer.customRegexPlaceholder', {
+              defaultValue: 'Named-group regex: (?<ip>…) (?<ts>…) (?<method>…) (?<path>…) (?<status>…) (?<ua>…)',
+            })}
+            className="min-w-[320px] flex-1 rounded border border-surface-700 bg-surface-800 px-2 py-1 font-mono text-[11px] text-surface-100"
+          />
+        )}
+        <label className="ml-1 flex items-center gap-1 text-surface-300" title={t('logAnalyzer.verifyBotsHint', { defaultValue: 'Reverse-DNS verify a sample of bot IPs (slower; catches spoofed user-agents)' })}>
+          <input type="checkbox" checked={verifyBots} onChange={(e) => setVerifyBots(e.target.checked)} />
+          {t('logAnalyzer.verifyBots', { defaultValue: 'Verify bots (rDNS)' })}
+        </label>
+        <div className="ml-auto flex items-center gap-2">
+          <button
+            onClick={() => void onImport()}
+            disabled={busy}
+            className="rounded bg-blue-600 px-3 py-1 font-medium text-white hover:bg-blue-500 disabled:opacity-50"
+          >
+            {busy
+              ? t('logAnalyzer.analyzing', { defaultValue: 'Analyzing…' })
+              : t('logAnalyzer.import', { defaultValue: 'Import Log File…' })}
+          </button>
+          {hasData && <ExportMenu onExport={onExport} />}
+          {hasData && (
+            <button
+              onClick={() => void onClear()}
+              className="rounded border border-surface-700 px-2 py-1 text-surface-300 hover:bg-surface-800"
+            >
+              {t('logAnalyzer.clear', { defaultValue: 'Clear' })}
+            </button>
+          )}
+        </div>
+      </div>
+
+      {!hasData ? (
+        <EmptyState />
+      ) : (
+        <>
+          <SummaryCards overview={overview!} />
+          <TabStrip tab={tab} setTab={setTab} />
+          <div className="min-h-0 flex-1 overflow-auto">
+            {tab === 'urls' && <UrlStatsTab version={version} mode="all" />}
+            {tab === 'orphans' && <UrlStatsTab version={version} mode="orphans" />}
+            {tab === 'bots' && <BotsTab version={version} />}
+            {tab === 'status' && <StatusTab version={version} />}
+            {tab === 'trend' && <TrendTab version={version} />}
+            {tab === 'budget' && <BudgetTab version={version} />}
+            {tab === 'discovery' && <DiscoveryTab version={version} onToast={showToast} />}
+          </div>
+        </>
+      )}
+
+      {toast && (
+        <div className="pointer-events-none absolute bottom-4 left-1/2 -translate-x-1/2 rounded bg-surface-800 px-4 py-2 text-[12px] text-surface-100 shadow-lg ring-1 ring-surface-700">
+          {toast}
+        </div>
+      )}
+    </div>
+  );
+}
+
+function ExportMenu({ onExport }: { onExport: (format: 'csv' | 'xlsx') => void }) {
+  const { t } = useTranslation();
+  const [open, setOpen] = useState(false);
+  const choose = (format: 'csv' | 'xlsx'): void => {
+    setOpen(false);
+    onExport(format);
+  };
+  return (
+    <div className="relative">
+      <button
+        onClick={() => setOpen((o) => !o)}
+        className="flex items-center gap-1 rounded border border-surface-700 px-2 py-1 text-surface-200 hover:bg-surface-800"
+      >
+        {t('logAnalyzer.export', { defaultValue: 'Export' })}
+        <span className="text-[9px] text-surface-400">▼</span>
+      </button>
+      {open && (
+        <>
+          {/* click-away backdrop */}
+          <div className="fixed inset-0 z-40" onClick={() => setOpen(false)} />
+          <div className="absolute right-0 z-50 mt-1 w-44 overflow-hidden rounded border border-surface-700 bg-surface-800 shadow-lg">
+            <button
+              onClick={() => choose('csv')}
+              className="block w-full px-3 py-1.5 text-left text-surface-200 hover:bg-surface-700"
+            >
+              {t('logAnalyzer.exportCsv', { defaultValue: 'CSV (all tables)' })}
+            </button>
+            <button
+              onClick={() => choose('xlsx')}
+              className="block w-full px-3 py-1.5 text-left text-surface-200 hover:bg-surface-700"
+            >
+              {t('logAnalyzer.exportXlsx', { defaultValue: 'Excel (.xlsx, one sheet each)' })}
+            </button>
+          </div>
+        </>
+      )}
+    </div>
+  );
+}
+
+function EmptyState() {
+  const { t } = useTranslation();
+  return (
+    <div className="flex flex-1 flex-col items-center justify-center gap-2 px-6 text-center text-surface-400">
+      <div className="text-[14px] font-medium text-surface-200">
+        {t('logAnalyzer.noData', { defaultValue: 'No log data yet' })}
+      </div>
+      <p className="max-w-md text-[12px] leading-relaxed">
+        {t('logAnalyzer.noDataHint', {
+          defaultValue:
+            'Import a server access log (Apache, Nginx, or IIS W3C) to see which URLs search-engine bots crawl, how response codes are distributed in the log, and which logged URLs your crawl never reached.',
+        })}
+      </p>
+    </div>
+  );
+}
+
+function SummaryCards({ overview }: { overview: LogOverview }) {
+  const { t } = useTranslation();
+  const cards: { label: string; value: string }[] = [
+    { label: t('logAnalyzer.cardTotalHits', { defaultValue: 'Total Hits' }), value: fmtNum(overview.totalHits) },
+    { label: t('logAnalyzer.cardBotHits', { defaultValue: 'Bot Hits' }), value: fmtNum(overview.botHits) },
+    { label: t('logAnalyzer.cardHumanHits', { defaultValue: 'Human Hits' }), value: fmtNum(overview.humanHits) },
+    { label: t('logAnalyzer.cardVerified', { defaultValue: 'Verified Bot Hits' }), value: fmtNum(overview.verifiedBotHits) },
+    { label: t('logAnalyzer.cardUrls', { defaultValue: 'Distinct URLs' }), value: fmtNum(overview.distinctUrls) },
+    {
+      label: t('logAnalyzer.cardRange', { defaultValue: 'Date Range' }),
+      value: overview.minTs ? `${fmtTs(overview.minTs)} → ${fmtTs(overview.maxTs)}` : '—',
+    },
+  ];
+  return (
+    <div className="shrink-0 border-b border-surface-800 bg-surface-900/40 px-3 py-2">
+      <div className="flex flex-wrap gap-2">
+        {cards.map((c) => (
+          <div key={c.label} className="min-w-[120px] flex-1 rounded border border-surface-800 bg-surface-900 px-3 py-1.5">
+            <div className="text-[10px] uppercase tracking-wide text-surface-500">{c.label}</div>
+            <div className="font-mono text-[13px] text-surface-100">{c.value}</div>
+          </div>
+        ))}
+      </div>
+      {overview.files.length > 0 && (
+        <div className="mt-1.5 text-[10px] text-surface-500">
+          {t('logAnalyzer.filesIngested', { defaultValue: '{{n}} file(s) ingested', n: overview.files.length })}:{' '}
+          {overview.files.map((f) => f.fileName).join(', ')}
+        </div>
+      )}
+    </div>
+  );
+}
+
+function TabStrip({ tab, setTab }: { tab: Tab; setTab: (t: Tab) => void }) {
+  const { t } = useTranslation();
+  const tabs: { id: Tab; label: string }[] = [
+    { id: 'urls', label: t('logAnalyzer.tabUrls', { defaultValue: 'URL Hits' }) },
+    { id: 'bots', label: t('logAnalyzer.tabBots', { defaultValue: 'Bots' }) },
+    { id: 'status', label: t('logAnalyzer.tabStatus', { defaultValue: 'Status Codes' }) },
+    { id: 'trend', label: t('logAnalyzer.tabTrend', { defaultValue: 'Trend' }) },
+    { id: 'budget', label: t('logAnalyzer.tabBudget', { defaultValue: 'Crawl Budget' }) },
+    { id: 'orphans', label: t('logAnalyzer.tabOrphans', { defaultValue: 'Orphans' }) },
+    { id: 'discovery', label: t('logAnalyzer.tabDiscovery', { defaultValue: 'Discovery' }) },
+  ];
+  return (
+    <div className="flex shrink-0 gap-1 border-b border-surface-800 bg-surface-900/60 px-2 text-[12px]">
+      {tabs.map((tb) => (
+        <button
+          key={tb.id}
+          onClick={() => setTab(tb.id)}
+          className={`border-b-2 px-3 py-1.5 ${
+            tab === tb.id
+              ? 'border-blue-500 text-surface-100'
+              : 'border-transparent text-surface-400 hover:text-surface-200'
+          }`}
+        >
+          {tb.label}
+        </button>
+      ))}
+    </div>
+  );
+}
+
+function Th({ children, className = '' }: { children: React.ReactNode; className?: string }) {
+  return (
+    <th className={`sticky top-0 z-10 bg-surface-900 px-2 py-1 text-left font-medium text-surface-400 ${className}`}>
+      {children}
+    </th>
+  );
+}
+
+function FamilyBadge({ family }: { family: string }) {
+  return (
+    <span className={`rounded px-1.5 py-0.5 text-[10px] ${FAMILY_COLORS[family] ?? FAMILY_COLORS.other}`}>
+      {family}
+    </span>
+  );
+}
+
+const PAGE = 100;
+
+function UrlStatsTab({ version, mode }: { version: number; mode: 'all' | 'orphans' }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogUrlStatRow[]>([]);
+  const [total, setTotal] = useState(0);
+  const [offset, setOffset] = useState(0);
+  const [search, setSearch] = useState('');
+  const [filter, setFilter] = useState<NonNullable<LogUrlStatsInput['filter']>>(mode === 'orphans' ? 'orphans' : 'all');
+  const [bot, setBot] = useState('');
+  const [botList, setBotList] = useState<LogBotRow[]>([]);
+  const [sortBy, setSortBy] = useState<NonNullable<LogUrlStatsInput['sortBy']>>('totalHits');
+
+  useEffect(() => {
+    if (mode === 'all') void window.freecrawl.logBots().then(setBotList);
+  }, [version, mode]);
+
+  useEffect(() => {
+    setOffset(0);
+  }, [search, filter, bot, sortBy, version]);
+
+  useEffect(() => {
+    const input: LogUrlStatsInput = { limit: PAGE, offset, search, sortBy, filter, bot: bot || undefined };
+    const call = mode === 'orphans' ? window.freecrawl.logOrphans(input) : window.freecrawl.logUrlStats(input);
+    void call.then((r) => {
+      setRows(r.rows);
+      setTotal(r.total);
+    });
+  }, [offset, search, filter, bot, sortBy, version, mode]);
+
+  // The single filter <select> carries either a membership token
+  // (all/bots/orphans/crawled) or a `bot:<name>` per-bot selection.
+  const selValue = bot ? `bot:${bot}` : filter;
+  const onFilterChange = (v: string): void => {
+    if (v.startsWith('bot:')) {
+      setBot(v.slice(4));
+      setFilter('all');
+    } else {
+      setFilter(v as NonNullable<LogUrlStatsInput['filter']>);
+      setBot('');
+    }
+  };
+
+  return (
+    <div className="flex h-full flex-col">
+      <div className="flex shrink-0 items-center gap-2 border-b border-surface-800 px-3 py-1.5 text-[11px]">
+        <input
+          value={search}
+          onChange={(e) => setSearch(e.target.value)}
+          placeholder={t('logAnalyzer.searchPath', { defaultValue: 'Filter by path…' })}
+          className="w-56 rounded border border-surface-700 bg-surface-800 px-2 py-1 text-surface-100"
+        />
+        {mode === 'all' && (
+          <select
+            value={selValue}
+            onChange={(e) => onFilterChange(e.target.value)}
+            className="max-w-[220px] rounded border border-surface-700 bg-surface-800 px-2 py-1 text-surface-100"
+          >
+            <option value="all">{t('logAnalyzer.filterAll', { defaultValue: 'All URLs' })}</option>
+            <option value="bots">{t('logAnalyzer.filterBots', { defaultValue: 'Bot-touched' })}</option>
+            <option value="orphans">{t('logAnalyzer.filterOrphans', { defaultValue: 'Orphans (not crawled)' })}</option>
+            <option value="crawled">{t('logAnalyzer.filterCrawled', { defaultValue: 'Crawled' })}</option>
+            {botList.length > 0 && (
+              <optgroup label={t('logAnalyzer.filterByBot', { defaultValue: 'By bot' })}>
+                {botList.map((b) => (
+                  <option key={b.bot} value={`bot:${b.bot}`}>
+                    {b.bot} ({b.hits.toLocaleString()})
+                  </option>
+                ))}
+              </optgroup>
+            )}
+          </select>
+        )}
+        <select
+          value={sortBy}
+          onChange={(e) => setSortBy(e.target.value as NonNullable<LogUrlStatsInput['sortBy']>)}
+          className="rounded border border-surface-700 bg-surface-800 px-2 py-1 text-surface-100"
+        >
+          <option value="totalHits">{t('logAnalyzer.sortTotal', { defaultValue: 'Sort: Total hits' })}</option>
+          <option value="botHits">{t('logAnalyzer.sortBots', { defaultValue: 'Sort: Bot hits' })}</option>
+          <option value="googlebotHits">{t('logAnalyzer.sortGoogle', { defaultValue: 'Sort: Googlebot hits' })}</option>
+          <option value="lastHitAt">{t('logAnalyzer.sortRecent', { defaultValue: 'Sort: Most recent' })}</option>
+        </select>
+        <Pager offset={offset} total={total} onPrev={() => setOffset(Math.max(0, offset - PAGE))} onNext={() => setOffset(offset + PAGE)} />
+      </div>
+      <div className="min-h-0 flex-1 overflow-auto">
+        <table className="w-full border-collapse text-[11px]">
+          <thead>
+            <tr>
+              <Th>{t('logAnalyzer.colPath', { defaultValue: 'Path' })}</Th>
+              <Th className="text-right">{t('logAnalyzer.colTotal', { defaultValue: 'Total' })}</Th>
+              <Th className="text-right">{t('logAnalyzer.colBots', { defaultValue: 'Bots' })}</Th>
+              <Th className="text-right">Googlebot</Th>
+              <Th className="text-right">Bingbot</Th>
+              <Th className="text-right">Yandex</Th>
+              <Th className="text-right">{t('logAnalyzer.colStatus', { defaultValue: 'Status' })}</Th>
+              <Th>{t('logAnalyzer.colLast', { defaultValue: 'Last Hit' })}</Th>
+              <Th>{t('logAnalyzer.colCrawl', { defaultValue: 'In Crawl' })}</Th>
+            </tr>
+          </thead>
+          <tbody>
+            {rows.map((r) => (
+              <tr key={r.path} className="border-b border-surface-800/60 hover:bg-surface-900/40">
+                <td className="max-w-[420px] truncate px-2 py-1 font-mono text-surface-200" title={r.path}>{r.path}</td>
+                <td className="px-2 py-1 text-right font-mono">{fmtNum(r.totalHits)}</td>
+                <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(r.botHits)}</td>
+                <td className="px-2 py-1 text-right font-mono text-blue-300">{fmtNum(r.googlebotHits)}</td>
+                <td className="px-2 py-1 text-right font-mono text-teal-300">{fmtNum(r.bingbotHits)}</td>
+                <td className="px-2 py-1 text-right font-mono text-red-300">{fmtNum(r.yandexbotHits)}</td>
+                <td className="px-2 py-1 text-right font-mono">{r.lastStatus ?? '—'}</td>
+                <td className="px-2 py-1 text-surface-400">{fmtTs(r.lastHitAt)}</td>
+                <td className="px-2 py-1">
+                  {r.inCrawl ? (
+                    <span className="text-green-400">✓</span>
+                  ) : (
+                    <span className="text-amber-400">{t('logAnalyzer.orphan', { defaultValue: 'orphan' })}</span>
+                  )}
+                </td>
+              </tr>
+            ))}
+            {rows.length === 0 && (
+              <tr>
+                <td colSpan={9} className="px-3 py-6 text-center text-surface-500">
+                  {t('logAnalyzer.noRows', { defaultValue: 'No matching URLs.' })}
+                </td>
+              </tr>
+            )}
+          </tbody>
+        </table>
+      </div>
+    </div>
+  );
+}
+
+function Pager({ offset, total, onPrev, onNext }: { offset: number; total: number; onPrev: () => void; onNext: () => void }) {
+  const { t } = useTranslation();
+  const from = total === 0 ? 0 : offset + 1;
+  const to = Math.min(offset + PAGE, total);
+  return (
+    <div className="ml-auto flex items-center gap-2 text-surface-400">
+      <span>
+        {t('logAnalyzer.pageRange', { defaultValue: '{{from}}–{{to}} of {{total}}', from, to, total: total.toLocaleString() })}
+      </span>
+      <button onClick={onPrev} disabled={offset === 0} className="rounded border border-surface-700 px-2 py-0.5 disabled:opacity-40">‹</button>
+      <button onClick={onNext} disabled={to >= total} className="rounded border border-surface-700 px-2 py-0.5 disabled:opacity-40">›</button>
+    </div>
+  );
+}
+
+function BotsTab({ version }: { version: number }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogBotRow[]>([]);
+  useEffect(() => {
+    void window.freecrawl.logBots().then(setRows);
+  }, [version]);
+  return (
+    <table className="w-full border-collapse text-[11px]">
+      <thead>
+        <tr>
+          <Th>{t('logAnalyzer.colBot', { defaultValue: 'Bot' })}</Th>
+          <Th>{t('logAnalyzer.colFamily', { defaultValue: 'Family' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colHits', { defaultValue: 'Hits' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colIps', { defaultValue: 'IPs' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colVerified', { defaultValue: 'Verified IPs' })}</Th>
+        </tr>
+      </thead>
+      <tbody>
+        {rows.map((b) => (
+          <tr key={b.bot} className="border-b border-surface-800/60 hover:bg-surface-900/40">
+            <td className="px-2 py-1 text-surface-100">{b.bot}</td>
+            <td className="px-2 py-1"><FamilyBadge family={b.family} /></td>
+            <td className="px-2 py-1 text-right font-mono">{fmtNum(b.hits)}</td>
+            <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(b.totalIps)}</td>
+            <td className="px-2 py-1 text-right font-mono">
+              {b.verifiable ? (
+                <span className={b.verifiedIps > 0 ? 'text-green-400' : 'text-surface-500'}>
+                  {b.verifiedIps}/{b.totalIps}
+                </span>
+              ) : (
+                <span className="text-surface-600">{t('logAnalyzer.naVerify', { defaultValue: 'n/a' })}</span>
+              )}
+            </td>
+          </tr>
+        ))}
+        {rows.length === 0 && <EmptyRow cols={5} />}
+      </tbody>
+    </table>
+  );
+}
+
+function StatusTab({ version }: { version: number }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogStatusRow[]>([]);
+  useEffect(() => {
+    void window.freecrawl.logStatus().then(setRows);
+  }, [version]);
+  const max = useMemo(() => Math.max(1, ...rows.map((r) => r.count)), [rows]);
+  return (
+    <table className="w-full border-collapse text-[11px]">
+      <thead>
+        <tr>
+          <Th>{t('logAnalyzer.colStatus', { defaultValue: 'Status' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colCount', { defaultValue: 'Count' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colBotCount', { defaultValue: 'Bot Count' })}</Th>
+          <Th className="w-1/2">{t('logAnalyzer.colShare', { defaultValue: 'Share' })}</Th>
+        </tr>
+      </thead>
+      <tbody>
+        {rows.map((r) => (
+          <tr key={r.status} className="border-b border-surface-800/60">
+            <td className="px-2 py-1 font-mono">{r.status}</td>
+            <td className="px-2 py-1 text-right font-mono">{fmtNum(r.count)}</td>
+            <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(r.botCount)}</td>
+            <td className="px-2 py-1">
+              <div className="h-2 rounded bg-blue-500/60" style={{ width: `${(r.count / max) * 100}%` }} />
+            </td>
+          </tr>
+        ))}
+        {rows.length === 0 && <EmptyRow cols={4} />}
+      </tbody>
+    </table>
+  );
+}
+
+function TrendTab({ version }: { version: number }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogTrendRow[]>([]);
+  useEffect(() => {
+    void window.freecrawl.logTrend().then(setRows);
+  }, [version]);
+  const days = useMemo(() => {
+    const map = new Map<string, { total: number; googlebot: number; bingbot: number; yandexbot: number; otherBot: number; human: number }>();
+    for (const r of rows) {
+      const d = map.get(r.day) ?? { total: 0, googlebot: 0, bingbot: 0, yandexbot: 0, otherBot: 0, human: 0 };
+      d.total += r.hits;
+      if (r.bucket === 'googlebot') d.googlebot += r.hits;
+      else if (r.bucket === 'bingbot') d.bingbot += r.hits;
+      else if (r.bucket === 'yandexbot') d.yandexbot += r.hits;
+      else if (r.bucket === 'human') d.human += r.hits;
+      else d.otherBot += r.hits;
+      map.set(r.day, d);
+    }
+    return Array.from(map.entries()).map(([day, v]) => ({ day, ...v }));
+  }, [rows]);
+  const max = useMemo(() => Math.max(1, ...days.map((d) => d.total)), [days]);
+  return (
+    <table className="w-full border-collapse text-[11px]">
+      <thead>
+        <tr>
+          <Th>{t('logAnalyzer.colDay', { defaultValue: 'Day' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colTotal', { defaultValue: 'Total' })}</Th>
+          <Th className="text-right">Googlebot</Th>
+          <Th className="text-right">Bingbot</Th>
+          <Th className="text-right">Yandex</Th>
+          <Th className="text-right">{t('logAnalyzer.colOtherBots', { defaultValue: 'Other bots' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colHuman', { defaultValue: 'Human' })}</Th>
+          <Th className="w-1/3">{t('logAnalyzer.colVolume', { defaultValue: 'Volume' })}</Th>
+        </tr>
+      </thead>
+      <tbody>
+        {days.map((d) => (
+          <tr key={d.day} className="border-b border-surface-800/60">
+            <td className="px-2 py-1 font-mono">{d.day}</td>
+            <td className="px-2 py-1 text-right font-mono">{fmtNum(d.total)}</td>
+            <td className="px-2 py-1 text-right font-mono text-blue-300">{fmtNum(d.googlebot)}</td>
+            <td className="px-2 py-1 text-right font-mono text-teal-300">{fmtNum(d.bingbot)}</td>
+            <td className="px-2 py-1 text-right font-mono text-red-300">{fmtNum(d.yandexbot)}</td>
+            <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(d.otherBot)}</td>
+            <td className="px-2 py-1 text-right font-mono text-surface-400">{fmtNum(d.human)}</td>
+            <td className="px-2 py-1">
+              <div className="h-2 rounded bg-blue-500/50" style={{ width: `${(d.total / max) * 100}%` }} />
+            </td>
+          </tr>
+        ))}
+        {days.length === 0 && <EmptyRow cols={8} />}
+      </tbody>
+    </table>
+  );
+}
+
+function BudgetTab({ version }: { version: number }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogCrawlBudgetRow[]>([]);
+  useEffect(() => {
+    void window.freecrawl.logCrawlBudget(500).then(setRows);
+  }, [version]);
+  return (
+    <table className="w-full border-collapse text-[11px]">
+      <thead>
+        <tr>
+          <Th>{t('logAnalyzer.colUrl', { defaultValue: 'URL' })}</Th>
+          <Th className="text-right">Googlebot</Th>
+          <Th className="text-right">{t('logAnalyzer.colBots', { defaultValue: 'Bots' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colTotal', { defaultValue: 'Total' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colStatus', { defaultValue: 'Status' })}</Th>
+          <Th>{t('logAnalyzer.colIndexability', { defaultValue: 'Indexability' })}</Th>
+          <Th className="text-right">{t('logAnalyzer.colDepth', { defaultValue: 'Depth' })}</Th>
+        </tr>
+      </thead>
+      <tbody>
+        {rows.map((r) => (
+          <tr key={r.url} className="border-b border-surface-800/60 hover:bg-surface-900/40">
+            <td className="max-w-[460px] truncate px-2 py-1 font-mono text-surface-200" title={r.url}>{r.url}</td>
+            <td className="px-2 py-1 text-right font-mono text-blue-300">{fmtNum(r.googlebotHits)}</td>
+            <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(r.botHits)}</td>
+            <td className="px-2 py-1 text-right font-mono">{fmtNum(r.totalHits)}</td>
+            <td className="px-2 py-1 text-right font-mono">{r.statusCode ?? '—'}</td>
+            <td className="px-2 py-1 text-surface-300">{r.indexability ?? '—'}</td>
+            <td className="px-2 py-1 text-right font-mono">{r.depth ?? '—'}</td>
+          </tr>
+        ))}
+        {rows.length === 0 && <EmptyRow cols={7} />}
+      </tbody>
+    </table>
+  );
+}
+
+function DiscoveryTab({ version, onToast }: { version: number; onToast: (m: string) => void }) {
+  const { t } = useTranslation();
+  const [rows, setRows] = useState<LogDiscoveryRow[]>([]);
+  const [seeding, setSeeding] = useState(false);
+  useEffect(() => {
+    void window.freecrawl.logDiscovery(500).then(setRows);
+  }, [version]);
+  const onSeed = useCallback(async () => {
+    setSeeding(true);
+    try {
+      const r = await window.freecrawl.logSeedDiscovery(500);
+      if (r.enqueued > 0) {
+        onToast(t('logAnalyzer.seeded', { defaultValue: 'Seeded {{n}} URL(s) into the active crawl', n: r.enqueued }));
+      } else if (r.reason === 'no-active-crawl') {
+        onToast(t('logAnalyzer.seedNoActive', { defaultValue: 'No active crawl — start a crawl first, then seed.' }));
+      } else if (r.reason === 'no-base-origin') {
+        onToast(t('logAnalyzer.seedNoOrigin', { defaultValue: 'No crawled URLs yet to derive a base origin from.' }));
+      } else {
+        onToast(t('logAnalyzer.seedNone', { defaultValue: 'Nothing new to seed.' }));
+      }
+    } finally {
+      setSeeding(false);
+    }
+  }, [onToast, t]);
+  return (
+    <div className="flex h-full flex-col">
+      <div className="flex shrink-0 items-center gap-3 border-b border-surface-800 px-3 py-1.5 text-[11px] text-surface-400">
+        <span>
+          {t('logAnalyzer.discoveryHint', {
+            defaultValue: 'URLs seen in the log but never reached by the crawl. Seed them into a running crawl.',
+          })}
+        </span>
+        <button
+          onClick={() => void onSeed()}
+          disabled={seeding || rows.length === 0}
+          className="ml-auto rounded bg-blue-600 px-3 py-1 font-medium text-white hover:bg-blue-500 disabled:opacity-50"
+        >
+          {t('logAnalyzer.seedBtn', { defaultValue: 'Seed into active crawl' })}
+        </button>
+      </div>
+      <div className="min-h-0 flex-1 overflow-auto">
+        <table className="w-full border-collapse text-[11px]">
+          <thead>
+            <tr>
+              <Th>{t('logAnalyzer.colPath', { defaultValue: 'Path' })}</Th>
+              <Th className="text-right">{t('logAnalyzer.colBots', { defaultValue: 'Bots' })}</Th>
+              <Th className="text-right">{t('logAnalyzer.colTotal', { defaultValue: 'Total' })}</Th>
+            </tr>
+          </thead>
+          <tbody>
+            {rows.map((r) => (
+              <tr key={r.path} className="border-b border-surface-800/60">
+                <td className="max-w-[560px] truncate px-2 py-1 font-mono text-surface-200" title={r.path}>{r.path}</td>
+                <td className="px-2 py-1 text-right font-mono text-surface-300">{fmtNum(r.botHits)}</td>
+                <td className="px-2 py-1 text-right font-mono">{fmtNum(r.totalHits)}</td>
+              </tr>
+            ))}
+            {rows.length === 0 && <EmptyRow cols={3} />}
+          </tbody>
+        </table>
+      </div>
+    </div>
+  );
+}
+
+function EmptyRow({ cols }: { cols: number }) {
+  const { t } = useTranslation();
+  return (
+    <tr>
+      <td colSpan={cols} className="px-3 py-6 text-center text-surface-500">
+        {t('logAnalyzer.noRows', { defaultValue: 'No data.' })}
+      </td>
+    </tr>
+  );
+}

apps/desktop/src/renderer/src/i18n/locales/en.json

@@ -1180,5 +1180,86 @@
       "telemetryHelp": "Off by default. When on, aggregated counts of feature use (e.g. \"export ran\", \"report opened\") may be sent — never URLs, project contents, or crawl results.",
       "statusNote": "No telemetry endpoint is currently wired up. Toggling this preference is recorded but nothing is transmitted in the current build. The toggle is exposed now so consent state persists across upgrades when a future build adds an endpoint."
     }
+  },
+  "logAnalyzer": {
+    "title": "Log File Analyzer",
+    "windowHint": "Apache / Nginx / IIS access logs — bot activity, crawl budget, crawl × log join",
+    "format": "Format",
+    "fmtAuto": "Auto-detect",
+    "fmtApacheCombined": "Apache / Nginx Combined",
+    "fmtApacheCommon": "Apache Common (CLF)",
+    "fmtNginx": "Nginx",
+    "fmtIis": "IIS W3C Extended",
+    "fmtCustom": "Custom (regex)",
+    "customRegexPlaceholder": "Named-group regex: (?<ip>…) (?<ts>…) (?<method>…) (?<path>…) (?<status>…) (?<ua>…)",
+    "verifyBots": "Verify bots (rDNS)",
+    "verifyBotsHint": "Reverse-DNS verify a sample of bot IPs (slower; catches spoofed user-agents)",
+    "import": "Import Log File…",
+    "analyzing": "Analyzing…",
+    "clear": "Clear",
+    "clearConfirm": "Clear all ingested log data from this project?",
+    "export": "Export",
+    "exportCsv": "CSV (all tables)",
+    "exportXlsx": "Excel (.xlsx, one sheet each)",
+    "exported": "Exported {{size}} KB → {{path}}",
+    "imported": "{{file}} — parsed {{parsed}}/{{total}} lines ({{format}})",
+    "importError": "Import failed: {{err}}",
+    "noData": "No log data yet",
+    "noDataHint": "Import a server access log (Apache, Nginx, or IIS W3C) to see which URLs search-engine bots crawl, how response codes are distributed in the log, and which logged URLs your crawl never reached.",
+    "cardTotalHits": "Total Hits",
+    "cardBotHits": "Bot Hits",
+    "cardHumanHits": "Human Hits",
+    "cardVerified": "Verified Bot Hits",
+    "cardUrls": "Distinct URLs",
+    "cardRange": "Date Range",
+    "filesIngested": "{{n}} file(s) ingested",
+    "tabUrls": "URL Hits",
+    "tabBots": "Bots",
+    "tabStatus": "Status Codes",
+    "tabTrend": "Trend",
+    "tabBudget": "Crawl Budget",
+    "tabOrphans": "Orphans",
+    "tabDiscovery": "Discovery",
+    "searchPath": "Filter by path…",
+    "filterAll": "All URLs",
+    "filterBots": "Bot-touched",
+    "filterOrphans": "Orphans (not crawled)",
+    "filterCrawled": "Crawled",
+    "filterByBot": "By bot",
+    "sortTotal": "Sort: Total hits",
+    "sortBots": "Sort: Bot hits",
+    "sortGoogle": "Sort: Googlebot hits",
+    "sortRecent": "Sort: Most recent",
+    "colPath": "Path",
+    "colTotal": "Total",
+    "colBots": "Bots",
+    "colStatus": "Status",
+    "colLast": "Last Hit",
+    "colCrawl": "In Crawl",
+    "orphan": "orphan",
+    "noRows": "No data.",
+    "pageRange": "{{from}}–{{to}} of {{total}}",
+    "colBot": "Bot",
+    "colFamily": "Family",
+    "colHits": "Hits",
+    "colIps": "IPs",
+    "colVerified": "Verified IPs",
+    "naVerify": "n/a",
+    "colCount": "Count",
+    "colBotCount": "Bot Count",
+    "colShare": "Share",
+    "colDay": "Day",
+    "colOtherBots": "Other bots",
+    "colHuman": "Human",
+    "colVolume": "Volume",
+    "colUrl": "URL",
+    "colIndexability": "Indexability",
+    "colDepth": "Depth",
+    "discoveryHint": "URLs seen in the log but never reached by the crawl. Seed them into a running crawl.",
+    "seedBtn": "Seed into active crawl",
+    "seeded": "Seeded {{n}} URL(s) into the active crawl",
+    "seedNoActive": "No active crawl — start a crawl first, then seed.",
+    "seedNoOrigin": "No crawled URLs yet to derive a base origin from.",
+    "seedNone": "Nothing new to seed."
   }
 }

apps/desktop/src/renderer/src/i18n/locales/tr.json

@@ -1179,5 +1179,86 @@
       "telemetryHelp": "Varsayılan olarak kapalı. Açıkken, özellik kullanımının toplulaştırılmış sayıları (örn. \"dışa aktarım çalıştı\", \"rapor açıldı\") gönderilebilir — URL, proje içeriği veya crawl sonuçları asla gönderilmez.",
       "statusNote": "Şu anda herhangi bir telemetri uç noktası kablolanmamıştır. Bu tercihi açıp kapatmak kaydedilir ancak mevcut sürümde hiçbir şey iletilmez. İlerideki bir sürüm uç nokta eklediğinde rıza durumunun korunması için bu toggle şimdi gösteriliyor."
     }
+  },
+  "logAnalyzer": {
+    "title": "Log Dosyası Analizi",
+    "windowHint": "Apache / Nginx / IIS erişim logları — bot etkinliği, crawl bütçesi, crawl × log birleşimi",
+    "format": "Biçim",
+    "fmtAuto": "Otomatik algıla",
+    "fmtApacheCombined": "Apache / Nginx Combined",
+    "fmtApacheCommon": "Apache Common (CLF)",
+    "fmtNginx": "Nginx",
+    "fmtIis": "IIS W3C Extended",
+    "fmtCustom": "Özel (regex)",
+    "customRegexPlaceholder": "Adlandırılmış grup regex: (?<ip>…) (?<ts>…) (?<method>…) (?<path>…) (?<status>…) (?<ua>…)",
+    "verifyBots": "Botları doğrula (rDNS)",
+    "verifyBotsHint": "Bot IP'lerinin bir örneğini ters DNS ile doğrula (daha yavaş; sahte user-agent'ları yakalar)",
+    "import": "Log Dosyası İçe Aktar…",
+    "analyzing": "Analiz ediliyor…",
+    "clear": "Temizle",
+    "clearConfirm": "Bu projedeki tüm log verileri temizlensin mi?",
+    "export": "Dışa Aktar",
+    "exportCsv": "CSV (tüm tablolar)",
+    "exportXlsx": "Excel (.xlsx, her tablo ayrı sayfa)",
+    "exported": "{{size}} KB dışa aktarıldı → {{path}}",
+    "imported": "{{file}} — {{parsed}}/{{total}} satır ayrıştırıldı ({{format}})",
+    "importError": "İçe aktarım başarısız: {{err}}",
+    "noData": "Henüz log verisi yok",
+    "noDataHint": "Hangi URL'leri arama motoru botlarının taradığını, logdaki yanıt kodu dağılımını ve crawl'ın hiç ulaşmadığı URL'leri görmek için bir sunucu erişim logu (Apache, Nginx veya IIS W3C) içe aktarın.",
+    "cardTotalHits": "Toplam İsabet",
+    "cardBotHits": "Bot İsabeti",
+    "cardHumanHits": "İnsan İsabeti",
+    "cardVerified": "Doğrulanmış Bot İsabeti",
+    "cardUrls": "Benzersiz URL",
+    "cardRange": "Tarih Aralığı",
+    "filesIngested": "{{n}} dosya içe aktarıldı",
+    "tabUrls": "URL İsabetleri",
+    "tabBots": "Botlar",
+    "tabStatus": "Durum Kodları",
+    "tabTrend": "Trend",
+    "tabBudget": "Crawl Bütçesi",
+    "tabOrphans": "Yetimler",
+    "tabDiscovery": "Keşif",
+    "searchPath": "Yola göre filtrele…",
+    "filterAll": "Tüm URL'ler",
+    "filterBots": "Bot dokunmuş",
+    "filterOrphans": "Yetimler (taranmamış)",
+    "filterCrawled": "Taranmış",
+    "filterByBot": "Bota göre",
+    "sortTotal": "Sırala: Toplam isabet",
+    "sortBots": "Sırala: Bot isabeti",
+    "sortGoogle": "Sırala: Googlebot isabeti",
+    "sortRecent": "Sırala: En yeni",
+    "colPath": "Yol",
+    "colTotal": "Toplam",
+    "colBots": "Botlar",
+    "colStatus": "Durum",
+    "colLast": "Son İsabet",
+    "colCrawl": "Crawl'da",
+    "orphan": "yetim",
+    "noRows": "Veri yok.",
+    "pageRange": "{{total}} kaydın {{from}}–{{to}}",
+    "colBot": "Bot",
+    "colFamily": "Aile",
+    "colHits": "İsabet",
+    "colIps": "IP'ler",
+    "colVerified": "Doğrulanmış IP",
+    "naVerify": "yok",
+    "colCount": "Sayı",
+    "colBotCount": "Bot Sayısı",
+    "colShare": "Pay",
+    "colDay": "Gün",
+    "colOtherBots": "Diğer botlar",
+    "colHuman": "İnsan",
+    "colVolume": "Hacim",
+    "colUrl": "URL",
+    "colIndexability": "İndekslenebilirlik",
+    "colDepth": "Derinlik",
+    "discoveryHint": "Logda görülen ama crawl'ın hiç ulaşmadığı URL'ler. Çalışan bir crawl'a tohum olarak ekleyin.",
+    "seedBtn": "Aktif crawl'a ekle",
+    "seeded": "{{n}} URL aktif crawl'a eklendi",
+    "seedNoActive": "Aktif crawl yok — önce bir crawl başlatın, sonra ekleyin.",
+    "seedNoOrigin": "Temel origin türetmek için henüz taranmış URL yok.",
+    "seedNone": "Eklenecek yeni bir şey yok."
   }
 }

apps/desktop/src/renderer/src/main.tsx

@@ -3,6 +3,7 @@ import { createRoot } from 'react-dom/client';
 import { App } from './App.js';
 import { LogsView } from './LogsView.js';
 import { VisualizationView } from './VisualizationView.js';
+import { LogAnalyzerView } from './LogAnalyzerView.js';
 import './styles.css';
 import './i18n/index.js';
 
@@ -12,6 +13,7 @@ import './i18n/index.js';
 const search = typeof window !== 'undefined' ? window.location.search : '';
 const isLogsWindow = /[?&]logs=1\b/.test(search);
 const isVisualizationWindow = /[?&]visualization=1\b/.test(search);
+const isLogAnalyzerWindow = /[?&]loganalyzer=1\b/.test(search);
 
 const root = document.getElementById('root');
 if (root) {
@@ -21,6 +23,8 @@ if (root) {
         <LogsView />
       ) : isVisualizationWindow ? (
         <VisualizationView />
+      ) : isLogAnalyzerWindow ? (
+        <LogAnalyzerView />
       ) : (
         <App />
       )}

apps/mcp-server/src/tools.ts

@@ -1792,5 +1792,88 @@ export function buildTools(): Tool[] {
       handler: async (args) =>
         bridgeRequest<unknown>('POST', '/v1/action/fetch-cancel', args),
     },
+
+    // ---- Faz 0.5 closeout — bulk export + project-file actions (the
+    // last `[⏸]` items). Dialog-driven in the desktop; here every path is
+    // explicit because an agent can't drive a save/open dialog. ----
+
+    {
+      requiresDb: false,
+      name: 'export_bulk',
+      description:
+        'Write one CSV per category/issue family (all URLs, internal HTML, 2xx/3xx/4xx/5xx, every issue filter, redirects/canonicals/hreflang/duplicates/security/AMP, etc.) into a folder — the same dump File → Bulk Export produces. Zero-row categories are skipped. Pass an absolute `outputDir`. Returns `{outputDir, files:[{filePath,label,category,rowsWritten}], errors}`.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          outputDir: { type: 'string', description: 'Absolute output folder. Created files land directly inside it.' },
+        },
+        required: ['outputDir'],
+      },
+      handler: async (args) =>
+        bridgeRequest<unknown>('POST', '/v1/action/export-bulk', args),
+    },
+
+    {
+      requiresDb: false,
+      name: 'project_save_as',
+      description:
+        'Snapshot the desktop\'s active project to a new `.seoproject` file via SQLite VACUUM INTO (atomic, WAL-safe). Unlike the desktop "Save Project As", this does NOT switch the active project — it just writes a copy. Pass an absolute `filePath`. Returns `{filePath, bytesWritten}`.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          filePath: { type: 'string', description: 'Absolute destination `.seoproject` path.' },
+        },
+        required: ['filePath'],
+      },
+      handler: async (args) =>
+        bridgeRequest<{ filePath: string; bytesWritten: number }>(
+          'POST',
+          '/v1/action/project-save-as',
+          args,
+        ),
+    },
+
+    {
+      requiresDb: false,
+      name: 'project_save_encrypted',
+      description:
+        'Export the active project as an AES-256-GCM-encrypted `.seoproject.enc` snapshot protected by a password. Keep the password safe — it cannot be recovered. Pass absolute `filePath` + `password`. Returns `{filePath, bytesWritten}`.',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          filePath: { type: 'string', description: 'Absolute destination `.seoproject.enc` path.' },
+          password: { type: 'string', description: 'Encryption password.' },
+        },
+        required: ['filePath', 'password'],
+      },
+      handler: async (args) =>
+        bridgeRequest<{ filePath: string; bytesWritten: number }>(
+          'POST',
+          '/v1/action/project-save-encrypted',
+          args,
+        ),
+    },
+
+    {
+      requiresDb: false,
+      name: 'project_open_encrypted',
+      description:
+        'Decrypt a `.seoproject.enc` snapshot to a destination `.seoproject` and make it the desktop\'s ACTIVE project (this DOES switch the UI). Pass absolute `filePath` (source `.enc`), `destPath` (output `.seoproject`), and `password`. Returns `{filePath}` (the opened destination).',
+      inputSchema: {
+        type: 'object',
+        properties: {
+          filePath: { type: 'string', description: 'Absolute source `.seoproject.enc` path.' },
+          destPath: { type: 'string', description: 'Absolute output `.seoproject` path for the decrypted project.' },
+          password: { type: 'string', description: 'Decryption password.' },
+        },
+        required: ['filePath', 'destPath', 'password'],
+      },
+      handler: async (args) =>
+        bridgeRequest<{ filePath: string }>(
+          'POST',
+          '/v1/action/project-open-encrypted',
+          args,
+        ),
+    },
   ];
 }

packages/core/src/bot-detect.ts

@@ -0,0 +1,166 @@
+/**
+ * V2 Faz 2 — bot/user-agent classification + reverse-DNS verification.
+ *
+ * `detectBot` maps a raw User-Agent string to a named crawler. The order
+ * of the table matters: more specific signatures (Googlebot Smartphone)
+ * are tested before their generic parents (Googlebot).
+ *
+ * `verifyBotByRdns` implements the canonical "is this really Googlebot?"
+ * check the search engines themselves document: reverse-DNS the source IP,
+ * confirm the PTR hostname ends with an official suffix, then forward-DNS
+ * that hostname and confirm it resolves back to the original IP. This
+ * catches user-agents that merely *claim* to be Googlebot/Bingbot/etc.
+ */
+
+import { promises as dns } from 'node:dns';
+
+export type BotFamily =
+  | 'googlebot'
+  | 'bingbot'
+  | 'yandexbot'
+  | 'search'
+  | 'ai'
+  | 'social'
+  | 'seo-tool'
+  | 'other';
+
+export interface BotInfo {
+  /** Display name, e.g. "Googlebot Smartphone". */
+  name: string;
+  family: BotFamily;
+  /** Official reverse-DNS suffixes (empty = not verifiable by rDNS). */
+  rdnsSuffixes: string[];
+}
+
+interface BotPattern {
+  re: RegExp;
+  name: string;
+  family: BotFamily;
+  rdnsSuffixes: string[];
+}
+
+const GOOGLE_SUFFIXES = ['.googlebot.com', '.google.com', '.googleusercontent.com'];
+const BING_SUFFIXES = ['.search.msn.com'];
+const YANDEX_SUFFIXES = ['.yandex.com', '.yandex.net', '.yandex.ru'];
+const APPLE_SUFFIXES = ['.applebot.apple.com'];
+const BAIDU_SUFFIXES = ['.baidu.com', '.baidu.jp'];
+
+// Ordered most-specific-first.
+const PATTERNS: BotPattern[] = [
+  // --- Google family ---
+  { re: /Googlebot[^)]*Mobile|Mobile[^)]*Googlebot/i, name: 'Googlebot Smartphone', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Googlebot-Image/i, name: 'Googlebot Image', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Googlebot-Video/i, name: 'Googlebot Video', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Googlebot-News/i, name: 'Googlebot News', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /AdsBot-Google-Mobile/i, name: 'AdsBot Google Mobile', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /AdsBot-Google/i, name: 'AdsBot Google', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Mediapartners-Google/i, name: 'Mediapartners (AdSense)', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Google-InspectionTool/i, name: 'Google InspectionTool', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Google-Extended/i, name: 'Google-Extended (Gemini)', family: 'ai', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /GoogleOther/i, name: 'GoogleOther', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /APIs-Google/i, name: 'APIs-Google', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /FeedFetcher-Google/i, name: 'FeedFetcher-Google', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+  { re: /Googlebot/i, name: 'Googlebot', family: 'googlebot', rdnsSuffixes: GOOGLE_SUFFIXES },
+
+  // --- Bing family ---
+  { re: /BingPreview/i, name: 'BingPreview', family: 'bingbot', rdnsSuffixes: BING_SUFFIXES },
+  { re: /adidxbot/i, name: 'AdIdxBot (Bing Ads)', family: 'bingbot', rdnsSuffixes: BING_SUFFIXES },
+  { re: /bingbot/i, name: 'Bingbot', family: 'bingbot', rdnsSuffixes: BING_SUFFIXES },
+  { re: /msnbot/i, name: 'MSNBot', family: 'bingbot', rdnsSuffixes: BING_SUFFIXES },
+
+  // --- Yandex family ---
+  { re: /YandexMobileBot/i, name: 'YandexMobileBot', family: 'yandexbot', rdnsSuffixes: YANDEX_SUFFIXES },
+  { re: /YandexImages/i, name: 'YandexImages', family: 'yandexbot', rdnsSuffixes: YANDEX_SUFFIXES },
+  { re: /YandexBot/i, name: 'YandexBot', family: 'yandexbot', rdnsSuffixes: YANDEX_SUFFIXES },
+  { re: /Yandex(?:Accessibility|Metrika|Webmaster|[A-Za-z]+)?/i, name: 'Yandex (other)', family: 'yandexbot', rdnsSuffixes: YANDEX_SUFFIXES },
+
+  // --- Other search engines ---
+  { re: /DuckDuckBot|DuckDuckGo-Favicons-Bot/i, name: 'DuckDuckBot', family: 'search', rdnsSuffixes: [] },
+  { re: /Baiduspider/i, name: 'Baiduspider', family: 'search', rdnsSuffixes: BAIDU_SUFFIXES },
+  { re: /Sogou web spider/i, name: 'Sogou', family: 'search', rdnsSuffixes: [] },
+  { re: /Exabot/i, name: 'Exabot', family: 'search', rdnsSuffixes: [] },
+  { re: /Applebot/i, name: 'Applebot', family: 'search', rdnsSuffixes: APPLE_SUFFIXES },
+  { re: /SeznamBot/i, name: 'SeznamBot', family: 'search', rdnsSuffixes: [] },
+  { re: /PetalBot/i, name: 'PetalBot', family: 'search', rdnsSuffixes: [] },
+
+  // --- AI crawlers ---
+  { re: /GPTBot/i, name: 'GPTBot (OpenAI)', family: 'ai', rdnsSuffixes: [] },
+  { re: /OAI-SearchBot/i, name: 'OAI-SearchBot (OpenAI)', family: 'ai', rdnsSuffixes: [] },
+  { re: /ChatGPT-User/i, name: 'ChatGPT-User', family: 'ai', rdnsSuffixes: [] },
+  { re: /ClaudeBot|Claude-Web|anthropic-ai/i, name: 'ClaudeBot (Anthropic)', family: 'ai', rdnsSuffixes: [] },
+  { re: /PerplexityBot/i, name: 'PerplexityBot', family: 'ai', rdnsSuffixes: [] },
+  { re: /CCBot/i, name: 'CCBot (Common Crawl)', family: 'ai', rdnsSuffixes: [] },
+  { re: /Bytespider/i, name: 'Bytespider (ByteDance)', family: 'ai', rdnsSuffixes: [] },
+  { re: /Amazonbot/i, name: 'Amazonbot', family: 'ai', rdnsSuffixes: [] },
+
+  // --- Social / link unfurlers ---
+  { re: /facebookexternalhit|facebookcatalog|meta-externalagent/i, name: 'Facebook', family: 'social', rdnsSuffixes: [] },
+  { re: /Twitterbot/i, name: 'Twitterbot', family: 'social', rdnsSuffixes: [] },
+  { re: /LinkedInBot/i, name: 'LinkedInBot', family: 'social', rdnsSuffixes: [] },
+  { re: /Slackbot/i, name: 'Slackbot', family: 'social', rdnsSuffixes: [] },
+  { re: /WhatsApp/i, name: 'WhatsApp', family: 'social', rdnsSuffixes: [] },
+  { re: /TelegramBot/i, name: 'TelegramBot', family: 'social', rdnsSuffixes: [] },
+  { re: /Discordbot/i, name: 'Discordbot', family: 'social', rdnsSuffixes: [] },
+  { re: /Pinterest(?:bot)?/i, name: 'Pinterestbot', family: 'social', rdnsSuffixes: [] },
+
+  // --- SEO / commercial crawlers ---
+  { re: /AhrefsBot/i, name: 'AhrefsBot', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /SemrushBot/i, name: 'SemrushBot', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /MJ12bot/i, name: 'MJ12bot (Majestic)', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /DotBot/i, name: 'DotBot (Moz)', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /rogerbot/i, name: 'rogerbot (Moz)', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /BLEXBot/i, name: 'BLEXBot', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /DataForSeoBot/i, name: 'DataForSeoBot', family: 'seo-tool', rdnsSuffixes: [] },
+  { re: /Screaming Frog SEO Spider/i, name: 'Screaming Frog', family: 'seo-tool', rdnsSuffixes: [] },
+
+  // --- Generic catch-all (last) ---
+  { re: /\b(bot|crawler|spider|crawl)\b/i, name: 'Other bot', family: 'other', rdnsSuffixes: [] },
+];
+
+/** Classify a User-Agent. Returns null for non-bot (human browser) UAs. */
+export function detectBot(userAgent: string | null | undefined): BotInfo | null {
+  if (!userAgent) return null;
+  for (const p of PATTERNS) {
+    if (p.re.test(userAgent)) {
+      return { name: p.name, family: p.family, rdnsSuffixes: p.rdnsSuffixes };
+    }
+  }
+  return null;
+}
+
+/**
+ * Forward-confirmed reverse-DNS check. Returns true only when:
+ *   1. the PTR record of `ip` ends with one of `suffixes`, AND
+ *   2. forward-resolving that hostname yields `ip` back.
+ * Any DNS error / mismatch → false. `suffixes` empty → false (the bot is
+ * not rDNS-verifiable, so we never claim verification).
+ */
+export async function verifyBotByRdns(
+  ip: string,
+  suffixes: string[],
+  timeoutMs = 3000,
+): Promise<boolean> {
+  if (!ip || suffixes.length === 0) return false;
+  try {
+    const hostnames = await withTimeout(dns.reverse(ip), timeoutMs);
+    const match = hostnames.find((h) =>
+      suffixes.some((s) => h.toLowerCase().endsWith(s)),
+    );
+    if (!match) return false;
+    const addrs = await withTimeout(dns.resolve(match), timeoutMs).catch(
+      () => [] as string[],
+    );
+    return addrs.includes(ip);
+  } catch {
+    return false;
+  }
+}
+
+function withTimeout<T>(p: Promise<T>, ms: number): Promise<T> {
+  return Promise.race([
+    p,
+    new Promise<T>((_, reject) =>
+      setTimeout(() => reject(new Error('dns-timeout')), ms),
+    ),
+  ]);
+}

packages/core/src/bulk-export.ts

@@ -0,0 +1,121 @@
+/**
+ * Bulk export — writes one CSV per category/issue family into a target
+ * folder. Shared by the desktop's File → Bulk Export (dialog-driven) and
+ * the MCP `export_bulk` action (path-driven) so the long task list lives
+ * in exactly one place.
+ *
+ * `import type` keeps `@freecrawl/db` a compile-time-only reference (core
+ * must not take a runtime dependency on db) — same pattern as csv-export.
+ */
+
+import { join } from 'node:path';
+import { unlink } from 'node:fs/promises';
+import type { ProjectDb } from '@freecrawl/db';
+import type {
+  BulkExportFile,
+  BulkExportResult,
+  UrlCategory,
+} from '@freecrawl/shared-types';
+import { exportUrlsToCsv } from './csv-export.js';
+
+export interface BulkExportTask {
+  label: string;
+  file: string;
+  category: UrlCategory;
+}
+
+/** Every category/issue family the bulk dump emits, in display order. */
+export const BULK_EXPORT_TASKS: BulkExportTask[] = [
+  { label: 'All URLs', file: 'all-urls.csv', category: 'all' },
+  { label: 'Internal HTML', file: 'internal-html.csv', category: 'internal:html' },
+  { label: 'Internal All', file: 'internal-all.csv', category: 'internal:all' },
+  { label: 'External All', file: 'external-all.csv', category: 'external:all' },
+  { label: '2xx Success', file: 'status-2xx.csv', category: 'status:2xx' },
+  { label: '3xx Redirects', file: 'status-3xx.csv', category: 'status:3xx' },
+  { label: '4xx Client Errors', file: 'status-4xx.csv', category: 'status:4xx' },
+  { label: '5xx Server Errors', file: 'status-5xx.csv', category: 'status:5xx' },
+  { label: 'Indexable', file: 'indexable.csv', category: 'indexability:indexable' },
+  { label: 'Non-Indexable', file: 'non-indexable.csv', category: 'indexability:non-indexable' },
+  { label: 'Title Issues — Missing', file: 'issues-title-missing.csv', category: 'issues:title-missing' },
+  { label: 'Title Issues — Duplicate', file: 'issues-title-duplicate.csv', category: 'issues:title-duplicate' },
+  { label: 'Meta Description Issues — Missing', file: 'issues-meta-missing.csv', category: 'issues:meta-missing' },
+  { label: 'H1 Issues — Missing', file: 'issues-h1-missing.csv', category: 'issues:h1-missing' },
+  { label: 'Canonical Issues — Missing', file: 'issues-canonical-missing.csv', category: 'issues:canonical-missing' },
+  { label: 'Pagination Broken', file: 'issues-pagination-broken.csv', category: 'issues:pagination-broken' },
+  { label: 'Mixed Content', file: 'issues-mixed-content.csv', category: 'issues:mixed-content' },
+  { label: 'Insecure Form Action', file: 'issues-insecure-form-action.csv', category: 'issues:insecure-form-action' },
+  { label: 'Hreflang — Reciprocity Missing', file: 'hreflang-reciprocity-missing.csv', category: 'issues:hreflang-reciprocity-missing' },
+  { label: 'Sitemap — Crawled, Not Listed', file: 'sitemap-crawled-not-in-sitemap.csv', category: 'issues:crawled-not-in-sitemap' },
+  { label: 'Image Missing Alt', file: 'issues-image-missing-alt.csv', category: 'issues:image-missing-alt' },
+  { label: 'Near-Duplicate Content', file: 'issues-near-duplicate.csv', category: 'issues:near-duplicate' },
+  { label: 'All Redirects', file: 'all-redirects.csv', category: 'tab:redirects' },
+  { label: 'Redirect Chains (Long)', file: 'redirect-chains-long.csv', category: 'issues:redirect-chain-long' },
+  { label: 'Redirect Loops', file: 'redirect-loops.csv', category: 'issues:redirect-loop' },
+  { label: 'Self-Redirects', file: 'redirect-self.csv', category: 'issues:redirect-self' },
+  { label: 'All Canonicals', file: 'all-canonicals.csv', category: 'tab:canonicals' },
+  { label: 'Canonical — Non Self-Referencing', file: 'canonical-non-self.csv', category: 'issues:canonical-non-self' },
+  { label: 'Canonical — To Non-200', file: 'canonical-to-non-200.csv', category: 'issues:canonical-to-non-200' },
+  { label: 'Canonical — To Redirect', file: 'canonical-to-redirect.csv', category: 'issues:canonical-to-redirect' },
+  { label: 'Canonical — To Noindex', file: 'canonical-to-noindex.csv', category: 'issues:canonical-to-noindex' },
+  { label: 'Canonical — Multi-Hop Chain', file: 'canonical-chain-multi-hop.csv', category: 'issues:canonical-chain-multi-hop' },
+  { label: 'Multiple Canonicals', file: 'canonical-multiple.csv', category: 'issues:multiple-canonicals' },
+  { label: 'All Pagination', file: 'all-pagination.csv', category: 'tab:pagination' },
+  { label: 'Pagination — Sequence Break', file: 'pagination-sequence-break.csv', category: 'issues:pagination-sequence-break' },
+  { label: 'Pagination — Canonical Conflict', file: 'pagination-canonical-conflict.csv', category: 'issues:pagination-canonical-conflict' },
+  { label: 'Hreflang — All', file: 'hreflang-all.csv', category: 'tab:hreflang' },
+  { label: 'Hreflang — x-default Missing', file: 'hreflang-x-default-missing.csv', category: 'issues:hreflang-x-default-missing' },
+  { label: 'Hreflang — Invalid Language Code', file: 'hreflang-invalid-code.csv', category: 'issues:hreflang-invalid-code' },
+  { label: 'Hreflang — Self-Ref Missing', file: 'hreflang-self-ref-missing.csv', category: 'issues:hreflang-self-ref-missing' },
+  { label: 'Hreflang — Target Issues', file: 'hreflang-target-issues.csv', category: 'issues:hreflang-target-issues' },
+  { label: 'Hreflang — Inconsistent Language', file: 'hreflang-inconsistent-lang.csv', category: 'issues:hreflang-inconsistent-lang' },
+  { label: 'Duplicate Pages', file: 'duplicate-pages.csv', category: 'tab:duplicates' },
+  { label: 'Duplicate Content — Exact', file: 'duplicate-content-exact.csv', category: 'issues:duplicate-content-exact' },
+  { label: 'Custom Extraction Results', file: 'custom-extraction.csv', category: 'tab:custom-extraction' },
+  { label: 'Custom Search Hits', file: 'custom-search.csv', category: 'tab:custom-search' },
+  { label: 'Structured Data — Missing', file: 'structured-data-missing.csv', category: 'issues:structured-data-missing' },
+  { label: 'Structured Data — Invalid', file: 'structured-data-invalid.csv', category: 'issues:structured-data-invalid' },
+  { label: 'Structured Data — Duplicate @id', file: 'structured-data-duplicate-id.csv', category: 'issues:schema-duplicate-id' },
+  { label: 'Structured Data — Missing Required Property', file: 'structured-data-missing-required.csv', category: 'issues:schema-missing-required' },
+  { label: 'AMP Pages', file: 'amp-pages.csv', category: 'tab:amp' },
+  { label: 'Image — Missing Alt', file: 'image-missing-alt.csv', category: 'issues:image-missing-alt' },
+  { label: 'Image — Empty Alt', file: 'image-empty-alt.csv', category: 'issues:image-empty-alt' },
+  { label: 'Image — Too Large', file: 'image-too-large.csv', category: 'issues:image-too-large' },
+  { label: 'Image — Broken Src', file: 'image-broken-src.csv', category: 'issues:image-broken-src' },
+  { label: 'Security Audit', file: 'security-audit.csv', category: 'tab:security' },
+  { label: 'Mixed Content — Active', file: 'mixed-content-active.csv', category: 'issues:mixed-content-active' },
+  { label: 'Mixed Content — Passive', file: 'mixed-content-passive.csv', category: 'issues:mixed-content-passive' },
+  { label: 'HSTS Missing', file: 'security-hsts-missing.csv', category: 'issues:hsts-missing' },
+  { label: 'CSP Missing', file: 'security-csp-missing.csv', category: 'issues:csp-missing' },
+  { label: 'SERP Summary', file: 'serp-summary.csv', category: 'tab:serp' },
+  { label: 'Outlinks Zero', file: 'outlinks-zero.csv', category: 'issues:outlinks-zero' },
+];
+
+/**
+ * Write every {@link BULK_EXPORT_TASKS} category to a CSV under
+ * `outputDir`. Zero-row categories are written then deleted so the dump
+ * folder isn't littered with empty files. Per-task write failures are
+ * collected (never abort the whole run). Caller owns any UI / dialogs.
+ */
+export async function runBulkExport(
+  db: ProjectDb,
+  outputDir: string,
+): Promise<BulkExportResult> {
+  const files: BulkExportFile[] = [];
+  const errors: { label: string; error: string }[] = [];
+  for (const task of BULK_EXPORT_TASKS) {
+    const filePath = join(outputDir, task.file);
+    try {
+      const { rowsWritten } = await exportUrlsToCsv(db, filePath, {
+        category: task.category,
+      });
+      if (rowsWritten === 0) {
+        await unlink(filePath).catch(() => undefined);
+        continue;
+      }
+      files.push({ filePath, label: task.label, category: task.category, rowsWritten });
+    } catch (err) {
+      errors.push({ label: task.label, error: (err as Error).message });
+    }
+  }
+  return { outputDir, files, errors };
+}

packages/core/src/index.ts

@@ -85,3 +85,40 @@ export {
   type SitemapEntry,
   type SitemapDiscoveryResult,
 } from './sitemap.js';
+export {
+  runBulkExport,
+  BULK_EXPORT_TASKS,
+  type BulkExportTask,
+} from './bulk-export.js';
+export {
+  createLogParser,
+  detectLogFormat,
+  parseApacheTimestamp,
+  DIRECTIVE,
+  type LogParser,
+  type ParsedLogLine,
+} from './log-parser.js';
+export {
+  detectBot,
+  verifyBotByRdns,
+  type BotInfo,
+  type BotFamily,
+} from './bot-detect.js';
+export {
+  analyzeLogFile,
+  type LogAnalysisResult,
+  type LogUrlStat,
+  type LogBotAgg,
+  type LogAnalyzeOptions,
+} from './log-analyzer.js';
+export {
+  buildLogExportXlsx,
+  buildLogExportCsv,
+  type LogExportTables,
+} from './log-export.js';
+export {
+  buildXlsx,
+  buildCsvTable,
+  type SheetSpec,
+  type SheetColumn,
+} from './spreadsheet.js';

packages/core/src/log-export.ts

@@ -0,0 +1,204 @@
+/**
+ * V2 Faz 2 — Log Analyzer "export everything" builder.
+ *
+ * Gathers the seven analyzer tables (the same data the standalone window
+ * renders) into either a single multi-sheet XLSX workbook or a single
+ * concatenated CSV (one section per table). The column layout mirrors the
+ * UI tables so the export reads the same as what's on screen.
+ */
+
+import type {
+  LogBotRow,
+  LogCrawlBudgetRow,
+  LogDiscoveryRow,
+  LogOverview,
+  LogStatusRow,
+  LogTrendRow,
+  LogUrlStatRow,
+} from '@freecrawl/shared-types';
+import {
+  buildCsvTable,
+  buildXlsx,
+  type SheetColumn,
+  type SheetSpec,
+} from './spreadsheet.js';
+
+export interface LogExportTables {
+  overview: LogOverview;
+  urlStats: LogUrlStatRow[];
+  orphans: LogUrlStatRow[];
+  bots: LogBotRow[];
+  status: LogStatusRow[];
+  trend: LogTrendRow[];
+  crawlBudget: LogCrawlBudgetRow[];
+  discovery: LogDiscoveryRow[];
+}
+
+interface Section {
+  name: string;
+  columns: SheetColumn[];
+  rows: Array<Record<string, unknown>>;
+}
+
+function fmtTs(ts: number | null): string {
+  if (ts === null) return '';
+  try {
+    return new Date(ts).toISOString().replace('T', ' ').slice(0, 16);
+  } catch {
+    return '';
+  }
+}
+
+/** Pivot raw (day, bucket, hits) trend rows into per-day columns. */
+function pivotTrend(trend: LogTrendRow[]): Array<Record<string, unknown>> {
+  const map = new Map<
+    string,
+    { day: string; total: number; googlebot: number; bingbot: number; yandexbot: number; otherBot: number; human: number }
+  >();
+  for (const r of trend) {
+    const d =
+      map.get(r.day) ??
+      { day: r.day, total: 0, googlebot: 0, bingbot: 0, yandexbot: 0, otherBot: 0, human: 0 };
+    d.total += r.hits;
+    if (r.bucket === 'googlebot') d.googlebot += r.hits;
+    else if (r.bucket === 'bingbot') d.bingbot += r.hits;
+    else if (r.bucket === 'yandexbot') d.yandexbot += r.hits;
+    else if (r.bucket === 'human') d.human += r.hits;
+    else d.otherBot += r.hits;
+    map.set(r.day, d);
+  }
+  return Array.from(map.values());
+}
+
+function buildSections(tables: LogExportTables): Section[] {
+  const { overview } = tables;
+  return [
+    {
+      name: 'Summary',
+      columns: [
+        { key: 'metric', label: 'Metric' },
+        { key: 'value', label: 'Value' },
+      ],
+      rows: [
+        { metric: 'Total Hits', value: overview.totalHits },
+        { metric: 'Bot Hits', value: overview.botHits },
+        { metric: 'Human Hits', value: overview.humanHits },
+        { metric: 'Verified Bot Hits', value: overview.verifiedBotHits },
+        { metric: 'Distinct URLs', value: overview.distinctUrls },
+        { metric: 'Date Range Start', value: fmtTs(overview.minTs) },
+        { metric: 'Date Range End', value: fmtTs(overview.maxTs) },
+        { metric: 'Files Ingested', value: overview.files.map((f) => f.fileName).join('; ') },
+      ],
+    },
+    {
+      name: 'URL Hits',
+      columns: [
+        { key: 'path', label: 'Path' },
+        { key: 'totalHits', label: 'Total' },
+        { key: 'botHits', label: 'Bots' },
+        { key: 'googlebotHits', label: 'Googlebot' },
+        { key: 'bingbotHits', label: 'Bingbot' },
+        { key: 'yandexbotHits', label: 'Yandex' },
+        { key: 'lastStatus', label: 'Status' },
+        { key: 'lastHit', label: 'Last Hit' },
+        { key: 'inCrawl', label: 'In Crawl' },
+      ],
+      rows: tables.urlStats.map((r) => ({
+        ...r,
+        lastHit: fmtTs(r.lastHitAt),
+        inCrawl: r.inCrawl ? 'crawled' : 'orphan',
+      })),
+    },
+    {
+      name: 'Bots',
+      columns: [
+        { key: 'bot', label: 'Bot' },
+        { key: 'family', label: 'Family' },
+        { key: 'hits', label: 'Hits' },
+        { key: 'totalIps', label: 'IPs' },
+        { key: 'verifiedIps', label: 'Verified IPs' },
+        { key: 'verifiable', label: 'Verifiable' },
+      ],
+      rows: tables.bots.map((b) => ({ ...b, verifiable: b.verifiable ? 'yes' : 'no' })),
+    },
+    {
+      name: 'Status Codes',
+      columns: [
+        { key: 'status', label: 'Status' },
+        { key: 'count', label: 'Count' },
+        { key: 'botCount', label: 'Bot Count' },
+      ],
+      rows: tables.status.map((s) => ({ ...s })),
+    },
+    {
+      name: 'Trend',
+      columns: [
+        { key: 'day', label: 'Day' },
+        { key: 'total', label: 'Total' },
+        { key: 'googlebot', label: 'Googlebot' },
+        { key: 'bingbot', label: 'Bingbot' },
+        { key: 'yandexbot', label: 'Yandex' },
+        { key: 'otherBot', label: 'Other Bots' },
+        { key: 'human', label: 'Human' },
+      ],
+      rows: pivotTrend(tables.trend),
+    },
+    {
+      name: 'Crawl Budget',
+      columns: [
+        { key: 'url', label: 'URL' },
+        { key: 'googlebotHits', label: 'Googlebot' },
+        { key: 'botHits', label: 'Bots' },
+        { key: 'totalHits', label: 'Total' },
+        { key: 'statusCode', label: 'Status' },
+        { key: 'indexability', label: 'Indexability' },
+        { key: 'depth', label: 'Depth' },
+      ],
+      rows: tables.crawlBudget.map((r) => ({ ...r })),
+    },
+    {
+      name: 'Orphans',
+      columns: [
+        { key: 'path', label: 'Path' },
+        { key: 'totalHits', label: 'Total' },
+        { key: 'botHits', label: 'Bots' },
+        { key: 'lastStatus', label: 'Status' },
+        { key: 'lastHit', label: 'Last Hit' },
+      ],
+      rows: tables.orphans.map((r) => ({ ...r, lastHit: fmtTs(r.lastHitAt) })),
+    },
+    {
+      name: 'Discovery',
+      columns: [
+        { key: 'path', label: 'Path' },
+        { key: 'botHits', label: 'Bots' },
+        { key: 'totalHits', label: 'Total' },
+      ],
+      rows: tables.discovery.map((r) => ({ ...r })),
+    },
+  ];
+}
+
+/** Build a multi-sheet XLSX workbook with one sheet per analyzer table. */
+export function buildLogExportXlsx(tables: LogExportTables): Buffer {
+  const sections = buildSections(tables);
+  const sheets: SheetSpec[] = sections.map((s) => ({
+    name: s.name,
+    columns: s.columns,
+    rows: s.rows,
+  }));
+  return buildXlsx(sheets);
+}
+
+/**
+ * Build a single CSV holding every table, each under a `# <Table>`
+ * banner. Prefixed with a UTF-8 BOM so Excel-for-Windows opens it in the
+ * right charset.
+ */
+export function buildLogExportCsv(tables: LogExportTables): string {
+  const sections = buildSections(tables);
+  const blocks = sections.map(
+    (s) => `# ${s.name}\n${buildCsvTable(s.columns, s.rows)}`,
+  );
+  return '' + blocks.join('\n\n') + '\n';
+}

packages/core/src/log-parser.ts

@@ -0,0 +1,299 @@
+/**
+ * V2 Faz 2 — access-log line parser.
+ *
+ * Covers the four formats the Log File Analyzer ingests:
+ *   - Apache Common Log Format (CLF)
+ *   - Apache / Nginx Combined Log Format (CLF + "referer" "user-agent")
+ *   - IIS W3C Extended (field-directive driven, space-separated)
+ *   - Custom (user-supplied named-group regex)
+ *
+ * The parser is stateful only for IIS (it has to remember the most recent
+ * `#Fields:` directive), so it is exposed as a small {@link LogParser}
+ * object rather than a pure function. Each `feed(line)` returns a
+ * {@link ParsedLogLine}, `null` (unparseable data line), or the
+ * `DIRECTIVE` sentinel (a comment / header line that carries no hit).
+ */
+
+import type { LogFormat, LogFormatChoice } from '@freecrawl/shared-types';
+
+export interface ParsedLogLine {
+  ip: string | null;
+  /** Epoch ms, or null when the timestamp could not be parsed. */
+  ts: number | null;
+  method: string | null;
+  /** Request target (path + query), fragment stripped. */
+  path: string | null;
+  status: number | null;
+  bytes: number | null;
+  userAgent: string | null;
+  referer: string | null;
+}
+
+/** Returned by `feed()` for comment / `#Fields:` lines — not a hit. */
+export const DIRECTIVE = Symbol('log-directive');
+export type FeedResult = ParsedLogLine | null | typeof DIRECTIVE;
+
+const MONTHS: Record<string, number> = {
+  Jan: 0, Feb: 1, Mar: 2, Apr: 3, May: 4, Jun: 5,
+  Jul: 6, Aug: 7, Sep: 8, Oct: 9, Nov: 10, Dec: 11,
+};
+
+// `10/Oct/2000:13:55:36 -0700`
+const APACHE_TS_RE =
+  /^(\d{2})\/([A-Za-z]{3})\/(\d{4}):(\d{2}):(\d{2}):(\d{2})\s*([+-]\d{4})?$/;
+
+/** Parse an Apache/Nginx `[day/Mon/year:HH:MM:SS +zzzz]` stamp to epoch ms. */
+export function parseApacheTimestamp(raw: string): number | null {
+  const m = APACHE_TS_RE.exec(raw.trim());
+  if (!m) return null;
+  const [, dd, mon, yyyy, hh, mi, ss, tz] = m;
+  const month = MONTHS[mon!];
+  if (month === undefined) return null;
+  let ms = Date.UTC(
+    Number(yyyy),
+    month,
+    Number(dd),
+    Number(hh),
+    Number(mi),
+    Number(ss),
+  );
+  if (tz && tz.length === 5) {
+    const sign = tz[0] === '-' ? -1 : 1;
+    const offMin = Number(tz.slice(1, 3)) * 60 + Number(tz.slice(3, 5));
+    ms -= sign * offMin * 60_000;
+  }
+  return Number.isFinite(ms) ? ms : null;
+}
+
+/** Combine an IIS W3C `date`(YYYY-MM-DD) + `time`(HH:MM:SS) pair (UTC). */
+function parseIisTimestamp(date: string | undefined, time: string | undefined): number | null {
+  if (!date || !time) return null;
+  const ms = Date.parse(`${date}T${time}Z`);
+  return Number.isFinite(ms) ? ms : null;
+}
+
+/** Strip a trailing `#fragment` (logs rarely carry one, but be safe). */
+function cleanPath(raw: string | null | undefined): string | null {
+  if (!raw || raw === '-') return null;
+  const hash = raw.indexOf('#');
+  return hash >= 0 ? raw.slice(0, hash) : raw;
+}
+
+function toInt(raw: string | null | undefined): number | null {
+  if (raw == null || raw === '-' || raw === '') return null;
+  const n = Number.parseInt(raw, 10);
+  return Number.isFinite(n) ? n : null;
+}
+
+// Apache/Nginx combined: ip - - [ts] "METHOD path proto" status bytes "ref" "ua"
+// The request-line group tolerates a missing protocol and odd methods.
+const COMBINED_RE =
+  /^(\S+)\s+\S+\s+\S+\s+\[([^\]]+)\]\s+"([A-Z!]+)\s+(\S+)(?:\s+[^"]*)?"\s+(\d{3})\s+(\S+)\s+"([^"]*)"\s+"([^"]*)"/;
+// Apache common: same, minus the two trailing quoted fields.
+const COMMON_RE =
+  /^(\S+)\s+\S+\s+\S+\s+\[([^\]]+)\]\s+"([A-Z!]+)\s+(\S+)(?:\s+[^"]*)?"\s+(\d{3})\s+(\S+)/;
+
+/**
+ * Sniff the dialect from a handful of sample lines. Falls back to
+ * `apache-combined` (the most common web-server default).
+ */
+export function detectLogFormat(sampleLines: string[]): LogFormat {
+  let apacheCommon = 0;
+  let apacheCombined = 0;
+  for (const raw of sampleLines) {
+    const line = raw.trim();
+    if (!line) continue;
+    if (line.startsWith('#')) {
+      // `#Fields:` / `#Software: Microsoft Internet Information Services`
+      if (/^#(Fields|Software|Version|Date):/i.test(line)) return 'iis-w3c';
+      continue;
+    }
+    if (COMBINED_RE.test(line)) apacheCombined++;
+    else if (COMMON_RE.test(line)) apacheCommon++;
+  }
+  if (apacheCombined === 0 && apacheCommon === 0) {
+    // No Apache-shaped lines but data present → likely IIS data without a
+    // visible header in the sample, or an unknown shape. Prefer IIS when
+    // lines look space-delimited with an ISO date up front.
+    const isoFirst = sampleLines.some((l) =>
+      /^\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s/.test(l.trim()),
+    );
+    if (isoFirst) return 'iis-w3c';
+  }
+  return apacheCombined >= apacheCommon ? 'apache-combined' : 'apache-common';
+}
+
+export interface LogParser {
+  readonly format: LogFormat;
+  feed(line: string): FeedResult;
+}
+
+/**
+ * Build a parser for the chosen format. `auto` requires the caller to
+ * have resolved the concrete format first (via {@link detectLogFormat});
+ * passing `auto` here defaults to `apache-combined`.
+ */
+export function createLogParser(opts: {
+  format: LogFormatChoice;
+  customRegex?: string;
+}): LogParser {
+  const format: LogFormat = opts.format === 'auto' ? 'apache-combined' : opts.format;
+
+  if (format === 'iis-w3c') return new IisParser();
+  if (format === 'custom') return new CustomParser(opts.customRegex ?? '');
+  return new ApacheParser(format);
+}
+
+class ApacheParser implements LogParser {
+  readonly format: LogFormat;
+  constructor(format: LogFormat) {
+    this.format = format;
+  }
+  feed(raw: string): FeedResult {
+    const line = raw.trim();
+    if (!line) return null;
+    if (line.startsWith('#')) return DIRECTIVE;
+    // Try combined first (superset), fall back to common.
+    const c = COMBINED_RE.exec(line);
+    if (c) {
+      return {
+        ip: c[1] === '-' ? null : c[1]!,
+        ts: parseApacheTimestamp(c[2]!),
+        method: c[3]!,
+        path: cleanPath(c[4]),
+        status: toInt(c[5]),
+        bytes: toInt(c[6]),
+        referer: c[7] && c[7] !== '-' ? c[7] : null,
+        userAgent: c[8] && c[8] !== '-' ? c[8] : null,
+      };
+    }
+    const m = COMMON_RE.exec(line);
+    if (m) {
+      return {
+        ip: m[1] === '-' ? null : m[1]!,
+        ts: parseApacheTimestamp(m[2]!),
+        method: m[3]!,
+        path: cleanPath(m[4]),
+        status: toInt(m[5]),
+        bytes: toInt(m[6]),
+        referer: null,
+        userAgent: null,
+      };
+    }
+    return null;
+  }
+}
+
+/**
+ * IIS W3C Extended. Data lines are space-separated values whose meaning
+ * comes from the most recent `#Fields:` directive. Standard field tokens:
+ *   date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username
+ *   c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status
+ *   sc-bytes time-taken
+ */
+class IisParser implements LogParser {
+  readonly format: LogFormat = 'iis-w3c';
+  private fields: string[] | null = null;
+
+  feed(raw: string): FeedResult {
+    const line = raw.trim();
+    if (!line) return null;
+    if (line.startsWith('#')) {
+      const m = /^#Fields:\s*(.+)$/i.exec(line);
+      if (m) this.fields = m[1]!.trim().split(/\s+/);
+      return DIRECTIVE;
+    }
+    if (!this.fields) return null; // data before any #Fields header — unusable
+    const parts = line.split(/\s+/);
+    if (parts.length < this.fields.length) {
+      // Quoted UA can contain spaces; IIS escapes them as '+', so a short
+      // split is usually a malformed line. Skip it.
+      return null;
+    }
+    const get = (name: string): string | undefined => {
+      const idx = this.fields!.indexOf(name);
+      return idx >= 0 ? parts[idx] : undefined;
+    };
+    const stem = get('cs-uri-stem');
+    if (!stem) return null;
+    const query = get('cs-uri-query');
+    const path =
+      query && query !== '-' ? `${stem}?${query.replace(/^\?/, '')}` : stem;
+    const ua = get('cs(User-Agent)');
+    const referer = get('cs(Referer)');
+    // IIS encodes spaces in UA as '+'. Decode for matching.
+    const decodedUa = ua && ua !== '-' ? ua.replace(/\+/g, ' ') : null;
+    return {
+      // Prefer the client IP; fall back to server IP only if absent.
+      ip: pick(get('c-ip')) ?? pick(get('s-ip')),
+      ts: parseIisTimestamp(get('date'), get('time')),
+      method: pick(get('cs-method')),
+      path: cleanPath(path),
+      status: toInt(get('sc-status')),
+      bytes: toInt(get('sc-bytes')),
+      userAgent: decodedUa,
+      referer: referer && referer !== '-' ? referer : null,
+    };
+  }
+}
+
+function pick(v: string | undefined): string | null {
+  return v && v !== '-' ? v : null;
+}
+
+/**
+ * Custom format driven by a user-supplied named-group regex. Recognised
+ * groups: ip, ts, method, path, status, bytes, ua, referer. The `ts`
+ * group is parsed by the Apache stamp parser first, then `Date.parse`
+ * as a fallback (handles ISO-8601 / RFC-2822).
+ */
+class CustomParser implements LogParser {
+  readonly format: LogFormat = 'custom';
+  private re: RegExp | null = null;
+  private compileError: string | null = null;
+
+  constructor(pattern: string) {
+    try {
+      this.re = new RegExp(pattern);
+    } catch (err) {
+      this.compileError = err instanceof Error ? err.message : String(err);
+    }
+  }
+
+  feed(raw: string): FeedResult {
+    const line = raw.trim();
+    if (!line) return null;
+    if (line.startsWith('#')) return DIRECTIVE;
+    if (!this.re) {
+      // Surface the compile error once, then behave as "everything skips".
+      if (this.compileError) {
+        const e = this.compileError;
+        this.compileError = null;
+        throw new Error(`Invalid custom log regex: ${e}`);
+      }
+      return null;
+    }
+    const m = this.re.exec(line);
+    const g = m?.groups;
+    if (!g) return null;
+    const tsRaw = g['ts'];
+    let ts: number | null = null;
+    if (tsRaw) {
+      ts = parseApacheTimestamp(tsRaw);
+      if (ts === null) {
+        const p = Date.parse(tsRaw);
+        ts = Number.isFinite(p) ? p : null;
+      }
+    }
+    return {
+      ip: pick(g['ip']),
+      ts,
+      method: pick(g['method']),
+      path: cleanPath(g['path']),
+      status: toInt(g['status']),
+      bytes: toInt(g['bytes']),
+      userAgent: pick(g['ua']),
+      referer: pick(g['referer']),
+    };
+  }
+}

packages/core/src/spreadsheet.ts

@@ -0,0 +1,283 @@
+/**
+ * Generic CSV + XLSX builders for arbitrary tabular data.
+ *
+ * Unlike `tabular-export.ts` (which is bound to `CrawlUrlRow` with one
+ * shared column set), this module takes plain `Record<string, unknown>`
+ * rows and per-sheet column definitions — so a single workbook can hold
+ * several differently-shaped tables (used by the Log Analyzer export).
+ *
+ * The XLSX writer is a minimal, dependency-free Office Open XML
+ * SpreadsheetML packager (ZIP + DEFLATE, inline strings). It mirrors the
+ * proven writer in `tabular-export.ts` but generalised to per-sheet
+ * columns. Both share the OWASP formula-injection guard.
+ */
+
+import zlib from 'node:zlib';
+
+export interface SheetColumn {
+  /** Row object key. */
+  key: string;
+  /** Human header shown in the first row. */
+  label: string;
+}
+
+export interface SheetSpec {
+  name: string;
+  columns: SheetColumn[];
+  rows: Array<Record<string, unknown>>;
+}
+
+/**
+ * Excel / LibreOffice / Sheets execute cells beginning with `=`, `+`,
+ * `-`, `@`, TAB, or CR as formulas (CVE-class). Prepend a `'` to neutralise
+ * — the canonical OWASP CSV-injection mitigation.
+ */
+export function sanitizeFormulaPrefix(str: string): string {
+  if (!str) return str;
+  const first = str.charCodeAt(0);
+  if (
+    first === 0x3d ||
+    first === 0x2b ||
+    first === 0x2d ||
+    first === 0x40 ||
+    first === 0x09 ||
+    first === 0x0d
+  ) {
+    return `'${str}`;
+  }
+  return str;
+}
+
+export function escapeCsv(value: unknown): string {
+  if (value === null || value === undefined) return '';
+  const str = sanitizeFormulaPrefix(String(value));
+  if (/[",\n\r]/.test(str)) {
+    return `"${str.replace(/"/g, '""')}"`;
+  }
+  return str;
+}
+
+/** Build a single CSV table (header + rows). No BOM (caller prepends). */
+export function buildCsvTable(
+  columns: SheetColumn[],
+  rows: Array<Record<string, unknown>>,
+): string {
+  const lines: string[] = [];
+  lines.push(columns.map((c) => escapeCsv(c.label)).join(','));
+  for (const row of rows) {
+    lines.push(columns.map((c) => escapeCsv(row[c.key])).join(','));
+  }
+  return lines.join('\n');
+}
+
+// ── XLSX internals ───────────────────────────────────────────────────
+
+function xmlEscape(str: string): string {
+  return str
+    .replace(/&/g, '&amp;')
+    .replace(/</g, '&lt;')
+    .replace(/>/g, '&gt;')
+    .replace(/"/g, '&quot;')
+    // strip control chars XML 1.0 forbids
+    // eslint-disable-next-line no-control-regex
+    .replace(/[\x00-\x08\x0B\x0C\x0E-\x1F]/g, '');
+}
+
+function colLetter(idx: number): string {
+  let n = idx;
+  let s = '';
+  for (;;) {
+    const r = n % 26;
+    s = String.fromCharCode(65 + r) + s;
+    n = Math.floor(n / 26) - 1;
+    if (n < 0) break;
+  }
+  return s;
+}
+
+function sanitizeSheetName(label: string, used: Set<string>): string {
+  let base = label.replace(/[\\/?*[\]:]/g, ' ').trim().slice(0, 31);
+  if (!base) base = 'Sheet';
+  let candidate = base;
+  let i = 2;
+  while (used.has(candidate.toLowerCase())) {
+    const suffix = ` (${i++})`;
+    candidate = base.slice(0, 31 - suffix.length) + suffix;
+  }
+  used.add(candidate.toLowerCase());
+  return candidate;
+}
+
+function buildSheetXml(columns: SheetColumn[], rows: Array<Record<string, unknown>>): string {
+  const parts: string[] = [];
+  parts.push(
+    '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n' +
+      '<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">' +
+      '<sheetData>',
+  );
+  parts.push('<row r="1">');
+  columns.forEach((c, i) => {
+    parts.push(`<c r="${colLetter(i)}1" t="inlineStr"><is><t>${xmlEscape(c.label)}</t></is></c>`);
+  });
+  parts.push('</row>');
+  rows.forEach((row, rIdx) => {
+    const r = rIdx + 2;
+    parts.push(`<row r="${r}">`);
+    columns.forEach((col, i) => {
+      const v = row[col.key];
+      if (v === null || v === undefined || v === '') return;
+      const cellRef = `${colLetter(i)}${r}`;
+      if (typeof v === 'number' && Number.isFinite(v)) {
+        parts.push(`<c r="${cellRef}"><v>${v}</v></c>`);
+      } else if (typeof v === 'boolean') {
+        parts.push(`<c r="${cellRef}" t="b"><v>${v ? 1 : 0}</v></c>`);
+      } else {
+        const safe = sanitizeFormulaPrefix(String(v));
+        parts.push(`<c r="${cellRef}" t="inlineStr"><is><t>${xmlEscape(safe)}</t></is></c>`);
+      }
+    });
+    parts.push('</row>');
+  });
+  parts.push('</sheetData></worksheet>');
+  return parts.join('');
+}
+
+const CRC_TABLE = (() => {
+  const table = new Uint32Array(256);
+  for (let n = 0; n < 256; n++) {
+    let c = n;
+    for (let k = 0; k < 8; k++) {
+      c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
+    }
+    table[n] = c >>> 0;
+  }
+  return table;
+})();
+
+function crc32(buf: Buffer): number {
+  let c = 0xffffffff;
+  for (let i = 0; i < buf.length; i++) {
+    c = CRC_TABLE[(c ^ buf[i]!) & 0xff]! ^ (c >>> 8);
+  }
+  return (c ^ 0xffffffff) >>> 0;
+}
+
+function buildZip(files: { name: string; data: Buffer }[]): Buffer {
+  const localChunks: Buffer[] = [];
+  const entries: { name: string; raw: Buffer; compressed: Buffer; crc: number; offset: number }[] = [];
+  let offset = 0;
+  for (const f of files) {
+    const raw = f.data;
+    const compressed = zlib.deflateRawSync(raw);
+    const crc = crc32(raw);
+    const nameBuf = Buffer.from(f.name, 'utf8');
+    const localHeader = Buffer.alloc(30);
+    localHeader.writeUInt32LE(0x04034b50, 0);
+    localHeader.writeUInt16LE(20, 4);
+    localHeader.writeUInt16LE(0x0800, 6);
+    localHeader.writeUInt16LE(8, 8);
+    localHeader.writeUInt16LE(0, 10);
+    localHeader.writeUInt16LE(0, 12);
+    localHeader.writeUInt32LE(crc, 14);
+    localHeader.writeUInt32LE(compressed.length, 18);
+    localHeader.writeUInt32LE(raw.length, 22);
+    localHeader.writeUInt16LE(nameBuf.length, 26);
+    localHeader.writeUInt16LE(0, 28);
+    entries.push({ name: f.name, raw, compressed, crc, offset });
+    localChunks.push(localHeader, nameBuf, compressed);
+    offset += localHeader.length + nameBuf.length + compressed.length;
+  }
+  const centralChunks: Buffer[] = [];
+  let cdSize = 0;
+  for (const e of entries) {
+    const nameBuf = Buffer.from(e.name, 'utf8');
+    const cd = Buffer.alloc(46);
+    cd.writeUInt32LE(0x02014b50, 0);
+    cd.writeUInt16LE(20, 4);
+    cd.writeUInt16LE(20, 6);
+    cd.writeUInt16LE(0x0800, 8);
+    cd.writeUInt16LE(8, 10);
+    cd.writeUInt16LE(0, 12);
+    cd.writeUInt16LE(0, 14);
+    cd.writeUInt32LE(e.crc, 16);
+    cd.writeUInt32LE(e.compressed.length, 20);
+    cd.writeUInt32LE(e.raw.length, 24);
+    cd.writeUInt16LE(nameBuf.length, 28);
+    cd.writeUInt16LE(0, 30);
+    cd.writeUInt16LE(0, 32);
+    cd.writeUInt16LE(0, 34);
+    cd.writeUInt16LE(0, 36);
+    cd.writeUInt32LE(0, 38);
+    cd.writeUInt32LE(e.offset, 42);
+    centralChunks.push(cd, nameBuf);
+    cdSize += cd.length + nameBuf.length;
+  }
+  const eocd = Buffer.alloc(22);
+  eocd.writeUInt32LE(0x06054b50, 0);
+  eocd.writeUInt16LE(0, 4);
+  eocd.writeUInt16LE(0, 6);
+  eocd.writeUInt16LE(entries.length, 8);
+  eocd.writeUInt16LE(entries.length, 10);
+  eocd.writeUInt32LE(cdSize, 12);
+  eocd.writeUInt32LE(offset, 16);
+  eocd.writeUInt16LE(0, 20);
+  return Buffer.concat([...localChunks, ...centralChunks, eocd]);
+}
+
+/** Build a multi-sheet XLSX workbook (each sheet has its own columns). */
+export function buildXlsx(sheets: SheetSpec[]): Buffer {
+  const used = new Set<string>();
+  const named = sheets.map((s) => ({ ...s, name: sanitizeSheetName(s.name, used) }));
+
+  const contentTypes =
+    '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n' +
+    '<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">' +
+    '<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>' +
+    '<Default Extension="xml" ContentType="application/xml"/>' +
+    '<Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/>' +
+    named
+      .map(
+        (_, i) =>
+          `<Override PartName="/xl/worksheets/sheet${i + 1}.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/>`,
+      )
+      .join('') +
+    '</Types>';
+
+  const rels =
+    '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n' +
+    '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">' +
+    '<Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="xl/workbook.xml"/>' +
+    '</Relationships>';
+
+  const workbook =
+    '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n' +
+    '<workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships">' +
+    '<sheets>' +
+    named.map((s, i) => `<sheet name="${xmlEscape(s.name)}" sheetId="${i + 1}" r:id="rId${i + 1}"/>`).join('') +
+    '</sheets></workbook>';
+
+  const workbookRels =
+    '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n' +
+    '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">' +
+    named
+      .map(
+        (_, i) =>
+          `<Relationship Id="rId${i + 1}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet${i + 1}.xml"/>`,
+      )
+      .join('') +
+    '</Relationships>';
+
+  const files: { name: string; data: Buffer }[] = [
+    { name: '[Content_Types].xml', data: Buffer.from(contentTypes, 'utf8') },
+    { name: '_rels/.rels', data: Buffer.from(rels, 'utf8') },
+    { name: 'xl/workbook.xml', data: Buffer.from(workbook, 'utf8') },
+    { name: 'xl/_rels/workbook.xml.rels', data: Buffer.from(workbookRels, 'utf8') },
+  ];
+  named.forEach((s, i) => {
+    files.push({
+      name: `xl/worksheets/sheet${i + 1}.xml`,
+      data: Buffer.from(buildSheetXml(s.columns, s.rows), 'utf8'),
+    });
+  });
+  return buildZip(files);
+}

packages/db/src/migrations.ts

@@ -1696,6 +1696,87 @@ const MIGRATIONS: Migration[] = [
       add('pdf_probe_status', 'INTEGER');
     },
   },
+  {
+    version: 72,
+    name: 'log_file_analyzer',
+    // V2 Faz 2 — Log File Analyzer storage. Aggregate-on-ingest: the core
+    // analyzer streams an access log and rolls it up into these compact
+    // tables (no raw per-hit rows — a multi-GB log would be millions of
+    // rows). Cumulative across files: re-ingesting another log merges into
+    // the same aggregates. Everything is keyed on the request PATH; the
+    // crawl × log join (orphans / crawl-budget) is done in JS against the
+    // `urls` table since SQLite can't parse URLs.
+    up: `
+      CREATE TABLE IF NOT EXISTS log_ingests (
+        id            INTEGER PRIMARY KEY AUTOINCREMENT,
+        file_path     TEXT NOT NULL,
+        file_name     TEXT NOT NULL,
+        format        TEXT NOT NULL,
+        total_lines   INTEGER NOT NULL DEFAULT 0,
+        parsed_lines  INTEGER NOT NULL DEFAULT 0,
+        skipped_lines INTEGER NOT NULL DEFAULT 0,
+        min_ts        INTEGER,
+        max_ts        INTEGER,
+        ingested_at   TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
+      );
+
+      CREATE TABLE IF NOT EXISTS log_url_stats (
+        path            TEXT PRIMARY KEY,
+        total_hits      INTEGER NOT NULL DEFAULT 0,
+        bot_hits        INTEGER NOT NULL DEFAULT 0,
+        googlebot_hits  INTEGER NOT NULL DEFAULT 0,
+        bingbot_hits    INTEGER NOT NULL DEFAULT 0,
+        yandexbot_hits  INTEGER NOT NULL DEFAULT 0,
+        other_bot_hits  INTEGER NOT NULL DEFAULT 0,
+        last_status     INTEGER,
+        first_ts        INTEGER,
+        last_ts         INTEGER
+      );
+      CREATE INDEX IF NOT EXISTS idx_log_url_total ON log_url_stats(total_hits);
+      CREATE INDEX IF NOT EXISTS idx_log_url_bot ON log_url_stats(bot_hits);
+      CREATE INDEX IF NOT EXISTS idx_log_url_googlebot ON log_url_stats(googlebot_hits);
+
+      CREATE TABLE IF NOT EXISTS log_daily (
+        day    TEXT NOT NULL,
+        bucket TEXT NOT NULL,
+        hits   INTEGER NOT NULL DEFAULT 0,
+        PRIMARY KEY (day, bucket)
+      );
+
+      CREATE TABLE IF NOT EXISTS log_status (
+        status    INTEGER PRIMARY KEY,
+        count     INTEGER NOT NULL DEFAULT 0,
+        bot_count INTEGER NOT NULL DEFAULT 0
+      );
+
+      CREATE TABLE IF NOT EXISTS log_bots (
+        bot          TEXT PRIMARY KEY,
+        family       TEXT NOT NULL,
+        hits         INTEGER NOT NULL DEFAULT 0,
+        total_ips    INTEGER NOT NULL DEFAULT 0,
+        verified_ips INTEGER NOT NULL DEFAULT 0,
+        verifiable   INTEGER NOT NULL DEFAULT 0
+      );
+    `,
+  },
+  {
+    version: 73,
+    name: 'log_url_bot',
+    // V2 Faz 2 — per-(URL, bot) hit counts. The aggregate `log_url_stats`
+    // only carries family-level columns (googlebot/bing/yandex/other), so
+    // it can't answer "which pages did SemrushBot specifically hit?". This
+    // table backs the URL Hits tab's per-bot filter. Cumulative across
+    // ingests, same as the other log tables.
+    up: `
+      CREATE TABLE IF NOT EXISTS log_url_bot (
+        path TEXT NOT NULL,
+        bot  TEXT NOT NULL,
+        hits INTEGER NOT NULL DEFAULT 0,
+        PRIMARY KEY (path, bot)
+      );
+      CREATE INDEX IF NOT EXISTS idx_log_url_bot_bot ON log_url_bot(bot);
+    `,
+  },
 ];
 
 export function runMigrations(db: DatabaseSync): void {

packages/db/src/project-db.ts

@@ -32,6 +32,17 @@ import type {
   UrlAnalyticsDetail,
   UrlCategory,
   UrlDetail,
+  LogIngestInput,
+  LogImportSummary,
+  LogOverview,
+  LogUrlStatRow,
+  LogUrlStatsInput,
+  LogUrlStatsResult,
+  LogBotRow,
+  LogStatusRow,
+  LogTrendRow,
+  LogCrawlBudgetRow,
+  LogDiscoveryRow,
 } from '@freecrawl/shared-types';
 import { issueCategoriesBySeverity } from '@freecrawl/shared-types';
 import { runMigrations } from './migrations.js';
@@ -5492,6 +5503,405 @@ export class ProjectDb {
     };
   }
 
+  // ===================================================================
+  // V2 Faz 2 — Log File Analyzer. Aggregate-on-ingest storage + the
+  // crawl × log joins. The desktop main process flattens the core
+  // analyzer's Map aggregates into `LogIngestInput` before calling
+  // `ingestLogAnalysis`; everything else here is read-side.
+  // ===================================================================
+
+  /** Cumulatively merge one analyzed log file's aggregates into the DB. */
+  ingestLogAnalysis(input: LogIngestInput): LogImportSummary {
+    const f = input.file;
+    this.db.exec('BEGIN');
+    try {
+      const ins = this.db
+        .prepare(
+          `INSERT INTO log_ingests
+             (file_path, file_name, format, total_lines, parsed_lines, skipped_lines, min_ts, max_ts)
+           VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
+        )
+        .run(
+          f.filePath,
+          f.fileName,
+          f.format,
+          f.totalLines,
+          f.parsedLines,
+          f.skippedLines,
+          f.minTs,
+          f.maxTs,
+        );
+
+      const upUrl = this.db.prepare(
+        `INSERT INTO log_url_stats
+           (path, total_hits, bot_hits, googlebot_hits, bingbot_hits, yandexbot_hits, other_bot_hits, last_status, first_ts, last_ts)
+         VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+         ON CONFLICT(path) DO UPDATE SET
+           total_hits     = total_hits + excluded.total_hits,
+           bot_hits       = bot_hits + excluded.bot_hits,
+           googlebot_hits = googlebot_hits + excluded.googlebot_hits,
+           bingbot_hits   = bingbot_hits + excluded.bingbot_hits,
+           yandexbot_hits = yandexbot_hits + excluded.yandexbot_hits,
+           other_bot_hits = other_bot_hits + excluded.other_bot_hits,
+           last_status = CASE
+             WHEN excluded.last_ts IS NULL THEN last_status
+             WHEN last_ts IS NULL OR excluded.last_ts >= last_ts THEN excluded.last_status
+             ELSE last_status END,
+           first_ts = CASE
+             WHEN first_ts IS NULL THEN excluded.first_ts
+             WHEN excluded.first_ts IS NULL THEN first_ts
+             ELSE MIN(first_ts, excluded.first_ts) END,
+           last_ts = CASE
+             WHEN last_ts IS NULL THEN excluded.last_ts
+             WHEN excluded.last_ts IS NULL THEN last_ts
+             ELSE MAX(last_ts, excluded.last_ts) END`,
+      );
+      for (const u of input.urlStats) {
+        upUrl.run(
+          u.path,
+          u.totalHits,
+          u.botHits,
+          u.googlebotHits,
+          u.bingbotHits,
+          u.yandexbotHits,
+          u.otherBotHits,
+          u.lastStatus,
+          u.firstTs,
+          u.lastTs,
+        );
+      }
+
+      const upDaily = this.db.prepare(
+        `INSERT INTO log_daily (day, bucket, hits) VALUES (?, ?, ?)
+         ON CONFLICT(day, bucket) DO UPDATE SET hits = hits + excluded.hits`,
+      );
+      for (const d of input.daily) upDaily.run(d.day, d.bucket, d.hits);
+
+      const upStatus = this.db.prepare(
+        `INSERT INTO log_status (status, count, bot_count) VALUES (?, ?, ?)
+         ON CONFLICT(status) DO UPDATE SET
+           count = count + excluded.count,
+           bot_count = bot_count + excluded.bot_count`,
+      );
+      for (const s of input.status) upStatus.run(s.status, s.count, s.botCount);
+
+      const upBot = this.db.prepare(
+        `INSERT INTO log_bots (bot, family, hits, total_ips, verified_ips, verifiable)
+         VALUES (?, ?, ?, ?, ?, ?)
+         ON CONFLICT(bot) DO UPDATE SET
+           hits = hits + excluded.hits,
+           total_ips = total_ips + excluded.total_ips,
+           verified_ips = verified_ips + excluded.verified_ips,
+           verifiable = excluded.verifiable`,
+      );
+      for (const b of input.bots) {
+        upBot.run(b.bot, b.family, b.hits, b.totalIps, b.verifiedIps, b.verifiable ? 1 : 0);
+      }
+
+      const upUrlBot = this.db.prepare(
+        `INSERT INTO log_url_bot (path, bot, hits) VALUES (?, ?, ?)
+         ON CONFLICT(path, bot) DO UPDATE SET hits = hits + excluded.hits`,
+      );
+      for (const ub of input.urlBots) upUrlBot.run(ub.path, ub.bot, ub.hits);
+
+      this.db.exec('COMMIT');
+      const row = this.db
+        .prepare('SELECT ingested_at FROM log_ingests WHERE id = ?')
+        .get(ins.lastInsertRowid as number) as { ingested_at: string } | undefined;
+      return {
+        filePath: f.filePath,
+        fileName: f.fileName,
+        format: f.format,
+        totalLines: f.totalLines,
+        parsedLines: f.parsedLines,
+        skippedLines: f.skippedLines,
+        minTs: f.minTs,
+        maxTs: f.maxTs,
+        ingestedAt: row?.ingested_at ?? '',
+      };
+    } catch (err) {
+      this.db.exec('ROLLBACK');
+      throw err;
+    }
+  }
+
+  /** True when at least one log file has been ingested. */
+  hasLogData(): boolean {
+    const r = this.db.prepare('SELECT COUNT(*) AS n FROM log_ingests').get() as { n: number };
+    return r.n > 0;
+  }
+
+  /** Top-level roll-up across every ingested log file. */
+  getLogOverview(): LogOverview {
+    const files = (
+      this.db
+        .prepare(
+          `SELECT file_path, file_name, format, total_lines, parsed_lines, skipped_lines, min_ts, max_ts, ingested_at
+             FROM log_ingests ORDER BY id DESC`,
+        )
+        .all() as {
+        file_path: string;
+        file_name: string;
+        format: string;
+        total_lines: number;
+        parsed_lines: number;
+        skipped_lines: number;
+        min_ts: number | null;
+        max_ts: number | null;
+        ingested_at: string;
+      }[]
+    ).map((r) => ({
+      filePath: r.file_path,
+      fileName: r.file_name,
+      format: r.format as LogImportSummary['format'],
+      totalLines: r.total_lines,
+      parsedLines: r.parsed_lines,
+      skippedLines: r.skipped_lines,
+      minTs: r.min_ts,
+      maxTs: r.max_ts,
+      ingestedAt: r.ingested_at,
+    }));
+
+    const agg = this.db
+      .prepare(
+        `SELECT
+            COALESCE(SUM(total_hits), 0) AS total,
+            COALESCE(SUM(bot_hits), 0) AS bots,
+            COUNT(*) AS urls
+           FROM log_url_stats`,
+      )
+      .get() as { total: number; bots: number; urls: number };
+    const span = this.db
+      .prepare('SELECT MIN(min_ts) AS lo, MAX(max_ts) AS hi FROM log_ingests')
+      .get() as { lo: number | null; hi: number | null };
+
+    // Estimate verified bot hits: per bot, scale its hits by the
+    // verified-IP fraction (0 when no IPs / not verifiable).
+    const botRows = this.db
+      .prepare('SELECT hits, total_ips, verified_ips FROM log_bots')
+      .all() as { hits: number; total_ips: number; verified_ips: number }[];
+    let verifiedBotHits = 0;
+    for (const b of botRows) {
+      if (b.total_ips > 0 && b.verified_ips > 0) {
+        verifiedBotHits += Math.round((b.hits * b.verified_ips) / b.total_ips);
+      }
+    }
+
+    return {
+      hasData: files.length > 0,
+      files,
+      totalHits: agg.total,
+      botHits: agg.bots,
+      humanHits: Math.max(0, agg.total - agg.bots),
+      verifiedBotHits,
+      distinctUrls: agg.urls,
+      minTs: span.lo,
+      maxTs: span.hi,
+    };
+  }
+
+  /** Per-bot roll-up (item 5). */
+  getLogBots(): LogBotRow[] {
+    return (
+      this.db
+        .prepare(
+          `SELECT bot, family, hits, total_ips, verified_ips, verifiable
+             FROM log_bots ORDER BY hits DESC, bot ASC`,
+        )
+        .all() as {
+        bot: string;
+        family: string;
+        hits: number;
+        total_ips: number;
+        verified_ips: number;
+        verifiable: number;
+      }[]
+    ).map((r) => ({
+      bot: r.bot,
+      family: r.family,
+      hits: r.hits,
+      totalIps: r.total_ips,
+      verifiedIps: r.verified_ips,
+      verifiable: r.verifiable === 1,
+    }));
+  }
+
+  /** Response-code distribution from the log (item 8). */
+  getLogStatusDistribution(): LogStatusRow[] {
+    return this.db
+      .prepare('SELECT status, count, bot_count AS botCount FROM log_status ORDER BY status ASC')
+      .all() as { status: number; count: number; botCount: number }[];
+  }
+
+  /** Daily bot/human hit trend (item 10). */
+  getLogTrend(): LogTrendRow[] {
+    return this.db
+      .prepare('SELECT day, bucket, hits FROM log_daily ORDER BY day ASC, bucket ASC')
+      .all() as { day: string; bucket: string; hits: number }[];
+  }
+
+  /** Build a Set of crawled internal-URL paths (pathname + search). */
+  private crawlInternalPathSet(): Set<string> {
+    const rows = this.db
+      .prepare('SELECT url FROM urls WHERE is_external = 0')
+      .all() as { url: string }[];
+    const set = new Set<string>();
+    for (const r of rows) {
+      const p = urlToPath(r.url);
+      if (p) set.add(p);
+    }
+    return set;
+  }
+
+  /**
+   * Paginated per-URL log activity (items 6, 9, 11). `filter` `orphans`
+   * keeps only paths NOT crawled; `crawled` the inverse; `bots` only
+   * bot-touched paths. The crawl membership join is done in JS.
+   */
+  queryLogUrlStats(input: LogUrlStatsInput): LogUrlStatsResult {
+    const limit = Math.max(1, Math.min(input.limit, 1000));
+    const offset = Math.max(0, input.offset);
+    const filter = input.filter ?? 'all';
+    const sortCol =
+      input.sortBy === 'botHits'
+        ? 'bot_hits'
+        : input.sortBy === 'googlebotHits'
+          ? 'googlebot_hits'
+          : input.sortBy === 'lastHitAt'
+            ? 'last_ts'
+            : 'total_hits';
+    const where: string[] = [];
+    const args: (string | number)[] = [];
+    if (input.search && input.search.trim()) {
+      where.push('path LIKE ?');
+      args.push(`%${input.search.trim()}%`);
+    }
+    if (filter === 'bots') where.push('bot_hits > 0');
+    if (input.bot && input.bot.trim()) {
+      // Per-named-bot filter — only paths this specific bot hit.
+      where.push(
+        'EXISTS (SELECT 1 FROM log_url_bot b WHERE b.path = log_url_stats.path AND b.bot = ? AND b.hits > 0)',
+      );
+      args.push(input.bot.trim());
+    }
+    const whereSql = where.length ? `WHERE ${where.join(' AND ')}` : '';
+    const orderSql = `ORDER BY ${sortCol} DESC, path ASC`;
+
+    const needsJoin = filter === 'orphans' || filter === 'crawled';
+    // Always needed — even non-join filters set the per-row `inCrawl` flag.
+    const crawlPaths = this.crawlInternalPathSet();
+
+    if (!needsJoin) {
+      const total = (
+        this.db.prepare(`SELECT COUNT(*) AS n FROM log_url_stats ${whereSql}`).get(...args) as {
+          n: number;
+        }
+      ).n;
+      const rows = this.db
+        .prepare(`SELECT * FROM log_url_stats ${whereSql} ${orderSql} LIMIT ? OFFSET ?`)
+        .all(...args, limit, offset) as LogUrlStatDbRow[];
+      return { rows: rows.map((r) => toLogUrlStatRow(r, crawlPaths)), total };
+    }
+
+    // Membership filter requires the JS join — fetch a bounded candidate
+    // set (sorted), filter, then slice for pagination.
+    const JOIN_CAP = 100_000;
+    const candidates = this.db
+      .prepare(`SELECT * FROM log_url_stats ${whereSql} ${orderSql} LIMIT ?`)
+      .all(...args, JOIN_CAP) as LogUrlStatDbRow[];
+    const matched = candidates.filter((r) => {
+      const inCrawl = crawlPaths.has(r.path);
+      return filter === 'orphans' ? !inCrawl : inCrawl;
+    });
+    const page = matched.slice(offset, offset + limit);
+    return {
+      rows: page.map((r) => toLogUrlStatRow(r, crawlPaths)),
+      total: matched.length,
+    };
+  }
+
+  /** Crawl budget — crawled URLs ranked by Googlebot hit count (item 7). */
+  getLogCrawlBudget(limit = 200): LogCrawlBudgetRow[] {
+    const logMap = new Map<string, LogUrlStatDbRow>();
+    const logRows = this.db
+      .prepare('SELECT * FROM log_url_stats WHERE bot_hits > 0')
+      .all() as LogUrlStatDbRow[];
+    for (const r of logRows) logMap.set(r.path, r);
+
+    const crawlRows = this.db
+      .prepare(
+        `SELECT url, status_code, indexability, depth FROM urls
+          WHERE is_external = 0 AND content_kind = 'html'`,
+      )
+      .all() as {
+      url: string;
+      status_code: number | null;
+      indexability: string | null;
+      depth: number | null;
+    }[];
+
+    const out: LogCrawlBudgetRow[] = [];
+    for (const c of crawlRows) {
+      const p = urlToPath(c.url);
+      if (!p) continue;
+      const stat = logMap.get(p);
+      if (!stat || stat.googlebot_hits === 0) continue;
+      out.push({
+        url: c.url,
+        path: p,
+        googlebotHits: stat.googlebot_hits,
+        botHits: stat.bot_hits,
+        totalHits: stat.total_hits,
+        statusCode: c.status_code,
+        indexability: c.indexability,
+        depth: c.depth,
+      });
+    }
+    out.sort((a, b) => b.googlebotHits - a.googlebotHits || b.totalHits - a.totalHits);
+    return out.slice(0, Math.max(1, Math.min(limit, 5000)));
+  }
+
+  /** Log URLs absent from the crawl, eligible to be seeded (item 12). */
+  getLogDiscovery(limit = 500): LogDiscoveryRow[] {
+    const crawlPaths = this.crawlInternalPathSet();
+    const rows = this.db
+      .prepare(
+        `SELECT path, total_hits, bot_hits FROM log_url_stats
+          ORDER BY bot_hits DESC, total_hits DESC LIMIT ?`,
+      )
+      .all(Math.max(1, Math.min(limit, 50_000)) * 4) as {
+      path: string;
+      total_hits: number;
+      bot_hits: number;
+    }[];
+    const out: LogDiscoveryRow[] = [];
+    for (const r of rows) {
+      if (crawlPaths.has(r.path)) continue;
+      // Only seed real page paths (skip static assets / querystring noise
+      // is left to the crawler's own scope rules at enqueue time).
+      out.push({ path: r.path, totalHits: r.total_hits, botHits: r.bot_hits });
+      if (out.length >= Math.max(1, Math.min(limit, 50_000))) break;
+    }
+    return out;
+  }
+
+  /** Wipe every ingested log aggregate. */
+  clearLogData(): void {
+    this.db.exec('BEGIN');
+    try {
+      this.db.exec('DELETE FROM log_url_stats');
+      this.db.exec('DELETE FROM log_url_bot');
+      this.db.exec('DELETE FROM log_daily');
+      this.db.exec('DELETE FROM log_status');
+      this.db.exec('DELETE FROM log_bots');
+      this.db.exec('DELETE FROM log_ingests');
+      this.db.exec('COMMIT');
+    } catch (err) {
+      this.db.exec('ROLLBACK');
+      throw err;
+    }
+  }
+
   /**
    * Top anchor texts across all internal links, ranked by frequency.
    * Used by the Visualization tab's anchor-text word cloud.
@@ -8323,6 +8733,50 @@ function isValidHreflangCode(raw: string): boolean {
   return /^[a-z]{2,3}(-[a-z]{2}|-[0-9]{3})?$/.test(code);
 }
 
+/**
+ * V2 Faz 2 — raw `log_url_stats` row shape (snake_case columns). Declared
+ * as a `type` (not `interface`) so it gets an implicit index signature and
+ * stays assertion-compatible with node:sqlite's `Record<string, …>` rows.
+ */
+type LogUrlStatDbRow = {
+  path: string;
+  total_hits: number;
+  bot_hits: number;
+  googlebot_hits: number;
+  bingbot_hits: number;
+  yandexbot_hits: number;
+  other_bot_hits: number;
+  last_status: number | null;
+  first_ts: number | null;
+  last_ts: number | null;
+};
+
+/** Map a raw log_url_stats row to the camelCase IPC row + crawl-join flag. */
+function toLogUrlStatRow(r: LogUrlStatDbRow, crawlPaths: Set<string>): LogUrlStatRow {
+  return {
+    path: r.path,
+    totalHits: r.total_hits,
+    botHits: r.bot_hits,
+    googlebotHits: r.googlebot_hits,
+    bingbotHits: r.bingbot_hits,
+    yandexbotHits: r.yandexbot_hits,
+    otherBotHits: r.other_bot_hits,
+    lastStatus: r.last_status,
+    lastHitAt: r.last_ts,
+    inCrawl: crawlPaths.has(r.path),
+  };
+}
+
+/** Extract `pathname + search` from a full crawl URL for log path joins. */
+function urlToPath(rawUrl: string): string | null {
+  try {
+    const u = new URL(rawUrl);
+    return u.pathname + u.search;
+  } catch {
+    return null;
+  }
+}
+
 /**
  * Number of `/`-delimited path segments in the URL (e.g. `/a/b/c` → 3,
  * `/` → 0). Falls back to 0 on parse errors so the column is always a

packages/shared-types/src/index.ts

@@ -7,3 +7,4 @@ export * from './pagespeed.js';
 export * from './google.js';
 export * from './ai.js';
 export * from './seo.js';
+export * from './loganalyzer.js';

packages/shared-types/src/ipc.ts

@@ -36,6 +36,22 @@ import type {
 } from './google.js';
 import type { AiProvider, AiRow } from './ai.js';
 import type { SeoProvider, SeoRow } from './seo.js';
+import type {
+  LogAnalyzeInput,
+  LogAnalyzeResult,
+  LogOverview,
+  LogUrlStatsInput,
+  LogUrlStatsResult,
+  LogBotRow,
+  LogStatusRow,
+  LogTrendRow,
+  LogCrawlBudgetRow,
+  LogOrphanRow,
+  LogDiscoveryRow,
+  LogSeedDiscoveryResult,
+  LogExportInput,
+  LogExportResult,
+} from './loganalyzer.js';
 
 export const IPC = {
   crawlStart: 'crawl:start',
@@ -274,6 +290,23 @@ export const IPC = {
   /** Cross-source orphan-pages report — URLs seen in sitemap / GSC /
    *  GA4 but not in the crawled `urls` table. */
   reportsOrphanCrossSource: 'reports:orphan-cross-source',
+  /** V2 Faz 2 — Log File Analyzer. `logAnalyzerOpenWindow` opens the
+   *  standalone analysis window; `logAnalyze` ingests one access-log file
+   *  (parse, bot-detect, aggregate, persist); the remaining channels read
+   *  the persisted aggregates the window renders. */
+  logAnalyzerOpenWindow: 'loganalyzer:open-window',
+  logAnalyze: 'loganalyzer:analyze',
+  logOverview: 'loganalyzer:overview',
+  logUrlStats: 'loganalyzer:url-stats',
+  logBots: 'loganalyzer:bots',
+  logStatus: 'loganalyzer:status',
+  logTrend: 'loganalyzer:trend',
+  logCrawlBudget: 'loganalyzer:crawl-budget',
+  logOrphans: 'loganalyzer:orphans',
+  logDiscovery: 'loganalyzer:discovery',
+  logSeedDiscovery: 'loganalyzer:seed-discovery',
+  logExport: 'loganalyzer:export',
+  logClear: 'loganalyzer:clear',
 } as const;
 
 export type IpcChannel = (typeof IPC)[keyof typeof IPC];
@@ -388,6 +421,7 @@ export type MenuEvent =
   | 'open-robots-tester'
   | 'open-sitemap-validator'
   | 'open-reports'
+  | 'open-log-analyzer'
   | 'open-settings'
   | 'open-scheduled-crawl'
   | 'about';
@@ -1517,6 +1551,33 @@ export interface FreeCrawlApi {
   onLogEntry(cb: (entry: LogEntry) => void): () => void;
   onLogsBatch(cb: (entries: LogEntry[]) => void): () => void;
   onLogsBusy(cb: (busy: boolean) => void): () => void;
+  /** V2 Faz 2 — open (or focus) the standalone Log Analyzer window. */
+  logAnalyzerOpenWindow(): Promise<void>;
+  /** Ingest one access-log file: parse → bot-detect → aggregate → persist.
+   *  Omitting `filePath` opens a native file picker. */
+  logAnalyze(input: LogAnalyzeInput): Promise<LogAnalyzeResult>;
+  /** Top-level roll-up across every ingested log file. */
+  logOverview(): Promise<LogOverview>;
+  /** Paginated per-URL log activity, joined with the crawl. */
+  logUrlStats(input: LogUrlStatsInput): Promise<LogUrlStatsResult>;
+  /** Per-bot roll-up (hits + verified-IP counts). */
+  logBots(): Promise<LogBotRow[]>;
+  /** Response-code distribution from the log. */
+  logStatus(): Promise<LogStatusRow[]>;
+  /** Daily bot/human hit trend. */
+  logTrend(): Promise<LogTrendRow[]>;
+  /** Crawled URLs ranked by Googlebot hit count (crawl budget). */
+  logCrawlBudget(limit?: number): Promise<LogCrawlBudgetRow[]>;
+  /** Bot-hit log URLs the crawl never reached (orphan candidates). */
+  logOrphans(input: LogUrlStatsInput): Promise<LogUrlStatsResult>;
+  /** Log URLs absent from the crawl, eligible to be seeded. */
+  logDiscovery(limit?: number): Promise<LogDiscoveryRow[]>;
+  /** Inject discovered log URLs into the active crawl (item 12). */
+  logSeedDiscovery(limit?: number): Promise<LogSeedDiscoveryResult>;
+  /** Export every analyzer table to a single CSV or multi-sheet XLSX. */
+  logExport(input: LogExportInput): Promise<LogExportResult>;
+  /** Wipe every ingested log aggregate from the project. */
+  logClear(): Promise<void>;
   onProgress(cb: (p: CrawlProgress) => void): () => void;
   onDone(cb: (summary: CrawlSummary) => void): () => void;
   onError(cb: (message: string) => void): () => void;

packages/shared-types/src/loganalyzer.ts

@@ -0,0 +1,229 @@
+/**
+ * V2 Faz 2 — Log File Analyzer.
+ *
+ * Shared IPC + domain types for the access-log analysis subsystem. The
+ * parser/aggregation engine lives in `@freecrawl/core` (log-parser /
+ * bot-detect / log-analyzer); persistence + cross-source joins live in
+ * `@freecrawl/db`; the desktop main process drives ingestion and exposes
+ * the query surface here over IPC. The standalone Log Analyzer window
+ * (renderer `?loganalyzer=1`) and the CLI `analyze-logs` command both
+ * consume these shapes.
+ */
+
+/** Supported access-log dialects. `nginx` shares Apache's combined grammar. */
+export type LogFormat =
+  | 'apache-combined'
+  | 'apache-common'
+  | 'nginx'
+  | 'iis-w3c'
+  | 'custom';
+
+/** Format passed by the caller — `auto` lets the parser sniff the dialect. */
+export type LogFormatChoice = LogFormat | 'auto';
+
+export interface LogAnalyzeInput {
+  /** Absolute log file path. Omitted → the main process opens a picker. */
+  filePath?: string;
+  /** Dialect. Default `auto`. */
+  format?: LogFormatChoice;
+  /**
+   * Named-group regex for `custom` format. Recognised groups:
+   * `ip`, `ts`, `method`, `path`, `status`, `bytes`, `ua`, `referer`.
+   */
+  customRegex?: string;
+  /**
+   * Reverse-DNS verify a capped sample of IPs per declared search-engine
+   * bot (Googlebot / Bingbot / YandexBot / …). Slower (DNS round-trips)
+   * but catches spoofed user-agents.
+   */
+  verifyBots?: boolean;
+}
+
+/**
+ * Plain (array-based) ingest payload handed to the DB layer. The core
+ * analyzer produces Map-based aggregates; the desktop main process (which
+ * sees both `core` and `db`) flattens them into this shape so the `db`
+ * package stays free of any `core` dependency.
+ */
+export interface LogIngestInput {
+  file: {
+    filePath: string;
+    fileName: string;
+    format: LogFormat;
+    totalLines: number;
+    parsedLines: number;
+    skippedLines: number;
+    minTs: number | null;
+    maxTs: number | null;
+  };
+  urlStats: Array<{
+    path: string;
+    totalHits: number;
+    botHits: number;
+    googlebotHits: number;
+    bingbotHits: number;
+    yandexbotHits: number;
+    otherBotHits: number;
+    lastStatus: number | null;
+    firstTs: number | null;
+    lastTs: number | null;
+  }>;
+  daily: Array<{ day: string; bucket: string; hits: number }>;
+  status: Array<{ status: number; count: number; botCount: number }>;
+  bots: Array<{
+    bot: string;
+    family: string;
+    hits: number;
+    totalIps: number;
+    verifiedIps: number;
+    verifiable: boolean;
+  }>;
+  /** Per-(path, bot) hit counts — backs the URL Hits per-bot filter. */
+  urlBots: Array<{ path: string; bot: string; hits: number }>;
+}
+
+/** One ingested file's metadata — mirrors a `log_ingests` row. */
+export interface LogImportSummary {
+  filePath: string;
+  fileName: string;
+  format: LogFormat;
+  totalLines: number;
+  parsedLines: number;
+  skippedLines: number;
+  /** Epoch ms of the earliest / latest parsed entry. */
+  minTs: number | null;
+  maxTs: number | null;
+  ingestedAt: string;
+}
+
+export interface LogAnalyzeResult {
+  ok: boolean;
+  error?: string;
+  /** Null when the user cancelled the file picker. */
+  imported: LogImportSummary | null;
+  overview: LogOverview | null;
+}
+
+/** Top-level roll-up across every ingested log file in the project. */
+export interface LogOverview {
+  hasData: boolean;
+  files: LogImportSummary[];
+  totalHits: number;
+  botHits: number;
+  humanHits: number;
+  /** Hits attributed to reverse-DNS-verified bot IPs. */
+  verifiedBotHits: number;
+  distinctUrls: number;
+  minTs: number | null;
+  maxTs: number | null;
+}
+
+/** One URL's aggregated log activity (item 6 — bot hits per URL). */
+export interface LogUrlStatRow {
+  path: string;
+  totalHits: number;
+  botHits: number;
+  googlebotHits: number;
+  bingbotHits: number;
+  yandexbotHits: number;
+  otherBotHits: number;
+  lastStatus: number | null;
+  lastHitAt: number | null;
+  /** True when a crawled URL shares this path (crawl × log join). */
+  inCrawl: boolean;
+}
+
+export interface LogUrlStatsInput {
+  limit: number;
+  offset: number;
+  search?: string;
+  sortBy?: 'totalHits' | 'botHits' | 'googlebotHits' | 'lastHitAt';
+  /** `orphans` = seen in the log but NOT crawled; `crawled` = the inverse. */
+  filter?: 'all' | 'bots' | 'orphans' | 'crawled';
+  /** Restrict to URLs a specific named bot hit (e.g. "SemrushBot").
+   *  Composes with `search` / `filter`. */
+  bot?: string;
+}
+
+export interface LogUrlStatsResult {
+  rows: LogUrlStatRow[];
+  total: number;
+}
+
+/** Per-bot roll-up (item 5 — bot detection + IP verification). */
+export interface LogBotRow {
+  bot: string;
+  family: string;
+  hits: number;
+  /** Distinct IPs seen claiming this bot. */
+  totalIps: number;
+  /** Of those, how many reverse-DNS-verified (0 when verification was off). */
+  verifiedIps: number;
+  verifiable: boolean;
+}
+
+/** Response-code distribution from the log (item 8 — separate from crawl). */
+export interface LogStatusRow {
+  status: number;
+  count: number;
+  botCount: number;
+}
+
+/** One time-trend bucket (item 10 — daily bot-hit trend). */
+export interface LogTrendRow {
+  /** `YYYY-MM-DD` (UTC). */
+  day: string;
+  /** Bot family label (`googlebot` / `bingbot` / …) or `human`. */
+  bucket: string;
+  hits: number;
+}
+
+/** Crawl budget — crawled URLs ranked by how often Googlebot hit them. */
+export interface LogCrawlBudgetRow {
+  url: string;
+  path: string;
+  googlebotHits: number;
+  botHits: number;
+  totalHits: number;
+  statusCode: number | null;
+  indexability: string | null;
+  depth: number | null;
+}
+
+/** Orphan candidate — bot-hit in the log but never reached by the crawl. */
+export interface LogOrphanRow {
+  path: string;
+  totalHits: number;
+  botHits: number;
+  lastStatus: number | null;
+}
+
+/** A log path eligible to be seeded into a crawl (item 12). */
+export interface LogDiscoveryRow {
+  path: string;
+  totalHits: number;
+  botHits: number;
+}
+
+export interface LogSeedDiscoveryResult {
+  /** Full URLs handed to the active crawler. */
+  enqueued: number;
+  /** True when a crawl was running and accepted the seeds. */
+  hasActiveCrawl: boolean;
+  /** Set when nothing could be seeded (no base origin / no active crawl). */
+  reason?: string;
+}
+
+export interface LogExportInput {
+  /** `xlsx` = one workbook, one sheet per table; `csv` = one file, all
+   *  tables stacked under `# <Table>` banners. */
+  format: 'csv' | 'xlsx';
+  /** Optional pre-resolved output path (skips the save dialog). */
+  filePath?: string;
+}
+
+export interface LogExportResult {
+  /** Empty when the user cancelled the save dialog. */
+  filePath: string;
+  bytesWritten: number;
+}

samples/apache-access.txt

@@ -0,0 +1,180 @@
+24.5.66.10 - - [18/Mar/2024:00:02:09 +0000] "GET /old-pricing HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+81.2.69.142 - - [18/Mar/2024:00:27:45 +0000] "GET /products/widget-a HTTP/1.1" 200 4352 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+54.36.148.12 - - [18/Mar/2024:00:30:09 +0000] "GET /wp-login.php HTTP/1.1" 404 358 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+40.77.167.45 - - [18/Mar/2024:00:36:39 +0000] "GET /products/widget-b HTTP/1.1" 200 2254 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+203.0.113.7 - - [18/Mar/2024:00:49:31 +0000] "GET /sitemap.xml HTTP/1.1" 200 6405 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+5.45.207.88 - - [18/Mar/2024:01:04:47 +0000] "GET /products/widget-a HTTP/1.1" 200 2684 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+81.2.69.142 - - [18/Mar/2024:01:28:16 +0000] "GET /blog/seo-guide HTTP/1.1" 200 3292 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+198.51.100.23 - - [18/Mar/2024:01:42:32 +0000] "GET /products/widget-b HTTP/1.1" 200 3635 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+5.255.253.21 - - [18/Mar/2024:02:03:03 +0000] "GET /sitemap.xml HTTP/1.1" 200 6105 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+95.108.213.5 - - [18/Mar/2024:02:15:12 +0000] "GET /products/widget-a HTTP/1.1" 200 6154 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+66.249.66.83 - - [18/Mar/2024:02:40:12 +0000] "GET /legacy/admin HTTP/1.1" 404 314 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.83 - - [18/Mar/2024:02:52:01 +0000] "GET /products/widget-b HTTP/1.1" 200 4005 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+5.45.207.88 - - [18/Mar/2024:03:23:35 +0000] "GET / HTTP/1.1" 200 1615 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+66.249.79.12 - - [18/Mar/2024:03:33:18 +0000] "GET /robots.txt HTTP/1.1" 200 6225 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+192.0.2.55 - - [18/Mar/2024:04:05:15 +0000] "GET /images/logo.png HTTP/1.1" 200 473 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+203.0.113.7 - - [18/Mar/2024:04:15:17 +0000] "GET /blog/log-analysis HTTP/1.1" 200 1765 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.64.40 - - [18/Mar/2024:04:24:48 +0000] "GET /sitemap.xml HTTP/1.1" 200 7812 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+198.51.100.23 - - [18/Mar/2024:04:38:41 +0000] "GET /js/app.js HTTP/1.1" 200 6010 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
+198.51.100.23 - - [18/Mar/2024:05:11:09 +0000] "GET /css/main.css HTTP/1.1" 200 6786 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:05:38:10 +0000] "GET /contact HTTP/1.1" 200 422 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+40.77.167.45 - - [18/Mar/2024:05:54:05 +0000] "GET /wp-login.php HTTP/1.1" 404 364 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+203.0.113.7 - - [18/Mar/2024:06:09:01 +0000] "GET /blog/seo-guide HTTP/1.1" 200 2804 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:06:10:28 +0000] "GET /robots.txt HTTP/1.1" 200 1233 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+192.0.2.55 - - [18/Mar/2024:06:25:32 +0000] "GET /products/widget-a HTTP/1.1" 200 4993 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+81.2.69.142 - - [18/Mar/2024:06:29:17 +0000] "GET /contact HTTP/1.1" 200 7388 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.64.40 - - [18/Mar/2024:06:48:13 +0000] "GET /products HTTP/1.1" 200 4803 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.64.40 - - [18/Mar/2024:07:06:39 +0000] "GET /css/main.css HTTP/1.1" 200 5189 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.1 - - [18/Mar/2024:07:25:39 +0000] "GET /products HTTP/1.1" 200 3677 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+40.77.167.45 - - [18/Mar/2024:07:37:54 +0000] "GET /products/discontinued HTTP/1.1" 404 369 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+81.2.69.142 - - [18/Mar/2024:08:04:43 +0000] "GET /sitemap.xml HTTP/1.1" 200 2832 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+24.5.66.10 - - [18/Mar/2024:08:24:33 +0000] "GET /products/discontinued HTTP/1.1" 404 253 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.79.12 - - [18/Mar/2024:08:41:04 +0000] "GET /legacy/admin HTTP/1.1" 404 290 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.1; +https://openai.com/gptbot"
+24.5.66.10 - - [18/Mar/2024:08:56:37 +0000] "GET /blog/log-analysis HTTP/1.1" 200 3399 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+24.5.66.10 - - [18/Mar/2024:09:12:32 +0000] "GET /about HTTP/1.1" 200 8107 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.1 - - [18/Mar/2024:09:25:29 +0000] "GET /old-sitemap-1.xml HTTP/1.1" 200 6074 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.1; +https://openai.com/gptbot"
+66.249.66.83 - - [18/Mar/2024:09:31:58 +0000] "GET /css/main.css HTTP/1.1" 200 1196 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+24.5.66.10 - - [18/Mar/2024:10:00:22 +0000] "GET /products/widget-b HTTP/1.1" 200 7816 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+66.249.66.83 - - [18/Mar/2024:10:09:52 +0000] "GET /pricing HTTP/1.1" 200 8329 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.64.40 - - [18/Mar/2024:10:24:10 +0000] "GET /css/main.css HTTP/1.1" 200 2552 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+5.255.253.21 - - [18/Mar/2024:10:53:17 +0000] "GET /products/discontinued HTTP/1.1" 404 237 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+85.208.96.20 - - [18/Mar/2024:11:14:23 +0000] "GET /images/logo.png HTTP/1.1" 200 1689 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+66.249.66.1 - - [18/Mar/2024:11:41:00 +0000] "GET /missing-page HTTP/1.1" 404 355 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+81.2.69.142 - - [18/Mar/2024:11:53:52 +0000] "GET /contact HTTP/1.1" 200 3341 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+192.0.2.55 - - [18/Mar/2024:12:16:36 +0000] "GET /robots.txt HTTP/1.1" 200 3109 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.64.40 - - [18/Mar/2024:12:30:07 +0000] "GET /about HTTP/1.1" 200 770 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+24.5.66.10 - - [18/Mar/2024:12:46:43 +0000] "GET /blog/log-analysis HTTP/1.1" 200 3006 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.83 - - [18/Mar/2024:13:14:21 +0000] "GET /products/widget-b HTTP/1.1" 200 1065 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+40.77.167.45 - - [18/Mar/2024:13:31:20 +0000] "GET /blog/log-analysis HTTP/1.1" 200 1976 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+157.55.39.120 - - [18/Mar/2024:13:40:09 +0000] "GET /blog/log-analysis HTTP/1.1" 200 4267 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+24.5.66.10 - - [18/Mar/2024:14:07:51 +0000] "GET /products HTTP/1.1" 200 2813 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+66.249.79.12 - - [18/Mar/2024:14:38:47 +0000] "GET /blog/log-analysis HTTP/1.1" 200 1516 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+203.0.113.7 - - [18/Mar/2024:14:43:01 +0000] "GET /products HTTP/1.1" 200 7665 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.1 - - [18/Mar/2024:14:48:56 +0000] "GET /contact HTTP/1.1" 200 2021 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+192.0.2.55 - - [18/Mar/2024:14:52:56 +0000] "GET /blog/log-analysis HTTP/1.1" 200 7877 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+81.2.69.142 - - [18/Mar/2024:15:03:34 +0000] "GET /products/discontinued HTTP/1.1" 404 231 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+24.5.66.10 - - [18/Mar/2024:15:16:11 +0000] "GET /blog/seo-guide HTTP/1.1" 200 4100 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+203.0.113.7 - - [18/Mar/2024:15:30:23 +0000] "GET /products/widget-b HTTP/1.1" 200 5490 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:15:52:57 +0000] "GET /about HTTP/1.1" 200 3882 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+85.208.96.20 - - [18/Mar/2024:16:08:14 +0000] "GET /images/logo.png HTTP/1.1" 200 5910 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+198.51.100.23 - - [18/Mar/2024:16:16:08 +0000] "GET /wp-login.php HTTP/1.1" 404 265 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.66.1 - - [18/Mar/2024:16:29:51 +0000] "GET /js/app.js HTTP/1.1" 200 484 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+40.77.167.45 - - [18/Mar/2024:16:33:44 +0000] "GET /legacy/admin HTTP/1.1" 404 479 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+198.51.100.23 - - [18/Mar/2024:17:04:59 +0000] "GET /robots.txt HTTP/1.1" 200 2090 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+203.0.113.7 - - [18/Mar/2024:17:07:04 +0000] "GET /missing-page HTTP/1.1" 404 453 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+46.229.168.66 - - [18/Mar/2024:17:33:28 +0000] "GET /blog/seo-guide HTTP/1.1" 200 5820 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+81.2.69.142 - - [18/Mar/2024:18:02:38 +0000] "GET /products/discontinued HTTP/1.1" 404 227 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+198.51.100.23 - - [18/Mar/2024:18:31:55 +0000] "GET /blog/seo-guide HTTP/1.1" 200 7310 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+198.51.100.23 - - [18/Mar/2024:18:45:00 +0000] "GET /legacy/admin HTTP/1.1" 404 470 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:19:10:28 +0000] "GET /products HTTP/1.1" 200 7675 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+66.249.66.83 - - [18/Mar/2024:19:36:30 +0000] "GET /js/app.js HTTP/1.1" 200 4028 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.64.40 - - [18/Mar/2024:19:54:34 +0000] "GET /pricing HTTP/1.1" 200 5238 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.79.12 - - [18/Mar/2024:20:12:38 +0000] "GET /about HTTP/1.1" 200 7050 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+192.0.2.55 - - [18/Mar/2024:20:34:37 +0000] "GET /sitemap.xml HTTP/1.1" 200 452 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+66.249.64.40 - - [18/Mar/2024:20:44:47 +0000] "GET /sitemap.xml HTTP/1.1" 200 7364 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.83 - - [18/Mar/2024:21:16:55 +0000] "GET / HTTP/1.1" 200 5306 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+54.36.148.12 - - [18/Mar/2024:21:36:50 +0000] "GET /css/main.css HTTP/1.1" 200 3022 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+85.208.96.20 - - [18/Mar/2024:22:01:46 +0000] "GET / HTTP/1.1" 200 5983 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+40.77.167.45 - - [18/Mar/2024:22:21:20 +0000] "GET / HTTP/1.1" 200 4185 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+192.0.2.55 - - [18/Mar/2024:22:28:49 +0000] "GET /images/logo.png HTTP/1.1" 200 5895 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.64.40 - - [18/Mar/2024:22:31:27 +0000] "GET /js/app.js HTTP/1.1" 200 2169 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.83 - - [18/Mar/2024:22:38:05 +0000] "GET /wp-login.php HTTP/1.1" 404 319 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+192.0.2.55 - - [18/Mar/2024:22:47:27 +0000] "GET /css/main.css HTTP/1.1" 200 4168 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+54.36.148.12 - - [18/Mar/2024:22:55:05 +0000] "GET /wp-login.php HTTP/1.1" 404 306 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+24.5.66.10 - - [18/Mar/2024:23:15:50 +0000] "GET /contact HTTP/1.1" 200 5468 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:23:24:03 +0000] "GET /products/widget-a HTTP/1.1" 200 4234 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:23:36:55 +0000] "GET /old-pricing HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+198.51.100.23 - - [18/Mar/2024:23:58:05 +0000] "GET /images/logo.png HTTP/1.1" 200 8200 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+95.108.213.5 - - [19/Mar/2024:00:10:52 +0000] "GET /missing-page HTTP/1.1" 404 469 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+85.208.96.20 - - [19/Mar/2024:00:31:25 +0000] "GET / HTTP/1.1" 200 6798 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+95.108.213.5 - - [19/Mar/2024:00:44:43 +0000] "GET /products/discontinued HTTP/1.1" 404 315 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+66.249.66.1 - - [19/Mar/2024:00:54:58 +0000] "GET /contact HTTP/1.1" 200 4119 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.1; +https://openai.com/gptbot"
+81.2.69.142 - - [19/Mar/2024:01:03:04 +0000] "GET /blog/seo-guide HTTP/1.1" 200 1332 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.1 - - [19/Mar/2024:01:18:19 +0000] "GET /api/checkout HTTP/1.1" 500 326 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+203.0.113.7 - - [19/Mar/2024:01:27:59 +0000] "GET /images/logo.png HTTP/1.1" 200 2955 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.83 - - [19/Mar/2024:01:35:20 +0000] "GET /products/widget-a HTTP/1.1" 200 7133 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.66.1 - - [19/Mar/2024:01:50:26 +0000] "GET /images/logo.png HTTP/1.1" 200 2138 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+81.2.69.142 - - [19/Mar/2024:02:21:49 +0000] "GET /robots.txt HTTP/1.1" 200 2217 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+81.2.69.142 - - [19/Mar/2024:02:33:50 +0000] "GET /products/widget-b HTTP/1.1" 200 8356 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
+81.2.69.142 - - [19/Mar/2024:02:46:46 +0000] "GET /robots.txt HTTP/1.1" 200 5307 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.79.12 - - [19/Mar/2024:02:51:29 +0000] "GET /products/widget-b HTTP/1.1" 200 8347 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+81.2.69.142 - - [19/Mar/2024:03:21:58 +0000] "GET /old-pricing HTTP/1.1" 301 0 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
+81.2.69.142 - - [19/Mar/2024:03:32:57 +0000] "GET /contact HTTP/1.1" 200 4475 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+40.77.167.45 - - [19/Mar/2024:03:51:47 +0000] "GET /blog/log-analysis HTTP/1.1" 200 5629 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+198.51.100.23 - - [19/Mar/2024:04:15:35 +0000] "GET /images/logo.png HTTP/1.1" 200 6472 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+46.229.168.66 - - [19/Mar/2024:04:42:43 +0000] "GET / HTTP/1.1" 200 1066 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+66.249.66.1 - - [19/Mar/2024:05:00:33 +0000] "GET /products/widget-b HTTP/1.1" 200 7033 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+207.46.13.91 - - [19/Mar/2024:05:08:25 +0000] "GET /about HTTP/1.1" 200 3621 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+5.45.207.88 - - [19/Mar/2024:05:12:18 +0000] "GET /robots.txt HTTP/1.1" 200 6597 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+207.46.13.91 - - [19/Mar/2024:05:14:28 +0000] "GET /images/logo.png HTTP/1.1" 200 6091 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+66.249.66.1 - - [19/Mar/2024:05:31:26 +0000] "GET /legacy/admin HTTP/1.1" 404 440 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.66.1 - - [19/Mar/2024:05:42:37 +0000] "GET /css/main.css HTTP/1.1" 200 908 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+192.0.2.55 - - [19/Mar/2024:05:59:29 +0000] "GET /old-sitemap-1.xml HTTP/1.1" 200 2534 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+54.36.148.12 - - [19/Mar/2024:06:22:49 +0000] "GET /robots.txt HTTP/1.1" 200 7150 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+203.0.113.7 - - [19/Mar/2024:06:47:01 +0000] "GET /js/app.js HTTP/1.1" 200 8010 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.66.1 - - [19/Mar/2024:06:50:22 +0000] "GET /contact HTTP/1.1" 200 5840 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.79.12 - - [19/Mar/2024:07:03:18 +0000] "GET /blog/log-analysis HTTP/1.1" 200 7012 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.1 - - [19/Mar/2024:07:24:33 +0000] "GET /css/main.css HTTP/1.1" 200 5314 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+203.0.113.7 - - [19/Mar/2024:07:34:13 +0000] "GET /products/discontinued HTTP/1.1" 404 230 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+207.46.13.91 - - [19/Mar/2024:07:38:40 +0000] "GET /products HTTP/1.1" 200 1127 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+24.5.66.10 - - [19/Mar/2024:08:03:39 +0000] "GET /about HTTP/1.1" 200 8144 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+85.208.96.20 - - [19/Mar/2024:08:35:20 +0000] "GET /about HTTP/1.1" 200 3294 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+207.46.13.91 - - [19/Mar/2024:09:07:28 +0000] "GET /products/widget-b HTTP/1.1" 200 6833 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+81.2.69.142 - - [19/Mar/2024:09:32:43 +0000] "GET /products HTTP/1.1" 200 1902 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.64.40 - - [19/Mar/2024:09:40:54 +0000] "GET /pricing HTTP/1.1" 200 5811 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.66.1 - - [19/Mar/2024:09:58:28 +0000] "GET /legacy/admin HTTP/1.1" 404 239 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+24.5.66.10 - - [19/Mar/2024:10:27:45 +0000] "GET /products/widget-a HTTP/1.1" 200 6597 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+157.55.39.120 - - [19/Mar/2024:10:34:23 +0000] "GET /products/discontinued HTTP/1.1" 404 393 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+46.229.168.66 - - [19/Mar/2024:10:55:08 +0000] "GET /missing-page HTTP/1.1" 404 294 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+192.0.2.55 - - [19/Mar/2024:11:19:28 +0000] "GET /pricing HTTP/1.1" 200 559 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+203.0.113.7 - - [19/Mar/2024:11:28:46 +0000] "GET /api/checkout HTTP/1.1" 500 208 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+192.0.2.55 - - [19/Mar/2024:11:41:43 +0000] "GET / HTTP/1.1" 200 4756 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+198.51.100.23 - - [19/Mar/2024:11:45:55 +0000] "GET /api/checkout HTTP/1.1" 500 254 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.83 - - [19/Mar/2024:12:05:54 +0000] "GET /blog/log-analysis HTTP/1.1" 200 5738 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+54.36.148.12 - - [19/Mar/2024:12:07:36 +0000] "GET / HTTP/1.1" 200 3532 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+66.249.66.1 - - [19/Mar/2024:12:12:05 +0000] "GET /old-sitemap-1.xml HTTP/1.1" 200 8392 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+203.0.113.7 - - [19/Mar/2024:12:32:27 +0000] "GET /products/widget-a HTTP/1.1" 200 6239 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+81.2.69.142 - - [19/Mar/2024:12:40:47 +0000] "GET /old-sitemap-1.xml HTTP/1.1" 200 3330 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.1 - - [19/Mar/2024:13:00:03 +0000] "GET /js/app.js HTTP/1.1" 200 3810 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+54.36.148.12 - - [19/Mar/2024:13:03:31 +0000] "GET /about HTTP/1.1" 200 4174 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+54.36.148.12 - - [19/Mar/2024:13:33:48 +0000] "GET /products/widget-b HTTP/1.1" 200 6361 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+40.77.167.45 - - [19/Mar/2024:13:48:16 +0000] "GET /css/main.css HTTP/1.1" 200 5139 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+192.0.2.55 - - [19/Mar/2024:13:52:36 +0000] "GET /robots.txt HTTP/1.1" 200 3091 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
+66.249.79.12 - - [19/Mar/2024:14:13:28 +0000] "GET /products/widget-a HTTP/1.1" 200 5319 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.66.83 - - [19/Mar/2024:14:34:55 +0000] "GET /images/logo.png HTTP/1.1" 200 8093 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+207.46.13.91 - - [19/Mar/2024:14:38:11 +0000] "GET /pricing HTTP/1.1" 200 6924 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+5.255.253.21 - - [19/Mar/2024:14:48:43 +0000] "GET /products HTTP/1.1" 200 6882 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+5.45.207.88 - - [19/Mar/2024:15:00:56 +0000] "GET / HTTP/1.1" 200 2450 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+192.0.2.55 - - [19/Mar/2024:15:25:59 +0000] "GET /robots.txt HTTP/1.1" 200 958 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+207.46.13.91 - - [19/Mar/2024:15:37:31 +0000] "GET /blog/seo-guide HTTP/1.1" 200 1706 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+46.229.168.66 - - [19/Mar/2024:16:09:54 +0000] "GET /products/widget-a HTTP/1.1" 200 6902 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+157.55.39.120 - - [19/Mar/2024:16:25:29 +0000] "GET /blog/log-analysis HTTP/1.1" 200 803 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+46.229.168.66 - - [19/Mar/2024:16:51:37 +0000] "GET /wp-login.php HTTP/1.1" 404 213 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+81.2.69.142 - - [19/Mar/2024:17:09:01 +0000] "GET /images/logo.png HTTP/1.1" 200 3599 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+66.249.79.12 - - [19/Mar/2024:17:21:38 +0000] "GET /about HTTP/1.1" 200 1271 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+203.0.113.7 - - [19/Mar/2024:17:26:37 +0000] "GET /products/widget-b HTTP/1.1" 200 8367 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+54.36.148.12 - - [19/Mar/2024:17:37:02 +0000] "GET /css/main.css HTTP/1.1" 200 1212 "-" "Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)"
+66.249.64.40 - - [19/Mar/2024:17:48:46 +0000] "GET /robots.txt HTTP/1.1" 200 6967 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+66.249.64.40 - - [19/Mar/2024:18:02:12 +0000] "GET /legacy/admin HTTP/1.1" 404 445 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+24.5.66.10 - - [19/Mar/2024:18:04:06 +0000] "GET /about HTTP/1.1" 200 3123 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+24.5.66.10 - - [19/Mar/2024:18:11:48 +0000] "GET /old-sitemap-1.xml HTTP/1.1" 200 4726 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+198.51.100.23 - - [19/Mar/2024:18:25:05 +0000] "GET /robots.txt HTTP/1.1" 200 5756 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+81.2.69.142 - - [19/Mar/2024:18:32:11 +0000] "GET /api/checkout HTTP/1.1" 500 329 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+81.2.69.142 - - [19/Mar/2024:18:48:03 +0000] "GET /robots.txt HTTP/1.1" 200 5085 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+66.249.66.1 - - [19/Mar/2024:19:03:20 +0000] "GET /contact HTTP/1.1" 200 6648 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+66.249.79.12 - - [19/Mar/2024:19:11:43 +0000] "GET /products/widget-a HTTP/1.1" 200 5975 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+85.208.96.20 - - [19/Mar/2024:19:38:00 +0000] "GET / HTTP/1.1" 200 7314 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+81.2.69.142 - - [19/Mar/2024:19:47:14 +0000] "GET / HTTP/1.1" 200 1713 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+66.249.66.1 - - [19/Mar/2024:19:54:00 +0000] "GET /css/main.css HTTP/1.1" 200 443 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+46.229.168.66 - - [19/Mar/2024:20:13:53 +0000] "GET /contact HTTP/1.1" 200 1704 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+66.249.66.1 - - [19/Mar/2024:20:30:28 +0000] "GET /blog/log-analysis HTTP/1.1" 200 6529 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Googlebot/2.1; +http://www.google.com/bot.html"
+81.2.69.142 - - [19/Mar/2024:20:51:57 +0000] "GET /contact HTTP/1.1" 200 3579 "https://www.google.com/" "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1"
+95.108.213.5 - - [19/Mar/2024:21:04:55 +0000] "GET /images/logo.png HTTP/1.1" 200 3129 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+95.108.213.5 - - [19/Mar/2024:21:14:47 +0000] "GET /js/app.js HTTP/1.1" 200 5876 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+203.0.113.7 - - [19/Mar/2024:21:19:46 +0000] "GET /api/checkout HTTP/1.1" 500 397 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+192.0.2.55 - - [19/Mar/2024:21:40:40 +0000] "GET /robots.txt HTTP/1.1" 200 7032 "https://www.google.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
+81.2.69.142 - - [19/Mar/2024:21:59:17 +0000] "GET /products HTTP/1.1" 200 4665 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+46.229.168.66 - - [19/Mar/2024:22:07:43 +0000] "GET /blog/seo-guide HTTP/1.1" 200 3053 "-" "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
+5.45.207.88 - - [19/Mar/2024:22:31:58 +0000] "GET /images/logo.png HTTP/1.1" 200 3341 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+40.77.167.45 - - [19/Mar/2024:22:35:03 +0000] "GET /robots.txt HTTP/1.1" 200 8092 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
+95.108.213.5 - - [19/Mar/2024:22:40:13 +0000] "GET /robots.txt HTTP/1.1" 200 684 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

samples/iis-access.txt

@@ -0,0 +1,184 @@
+#Software: Microsoft Internet Information Services 10.0
+#Version: 1.0
+#Date: 2024-03-18 00:00:00
+#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes time-taken
+2024-03-18 00:02:09 10.0.0.5 GET /old-pricing - 443 - 24.5.66.10 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 301 0 0 0 42
+2024-03-18 00:27:45 10.0.0.5 GET /products/widget-a - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 4352 42
+2024-03-18 00:30:09 10.0.0.5 GET /wp-login.php - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 404 0 0 358 42
+2024-03-18 00:36:39 10.0.0.5 GET /products/widget-b - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 2254 42
+2024-03-18 00:49:31 10.0.0.5 GET /sitemap.xml - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 6405 42
+2024-03-18 01:04:47 10.0.0.5 GET /products/widget-a - 443 - 5.45.207.88 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 2684 42
+2024-03-18 01:28:16 10.0.0.5 GET /blog/seo-guide - 443 - 81.2.69.142 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 3292 42
+2024-03-18 01:42:32 10.0.0.5 GET /products/widget-b - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 3635 42
+2024-03-18 02:03:03 10.0.0.5 GET /sitemap.xml - 443 - 5.255.253.21 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 6105 42
+2024-03-18 02:15:12 10.0.0.5 GET /products/widget-a - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 6154 42
+2024-03-18 02:40:12 10.0.0.5 GET /legacy/admin - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 404 0 0 314 42
+2024-03-18 02:52:01 10.0.0.5 GET /products/widget-b - 443 - 66.249.66.83 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 4005 42
+2024-03-18 03:23:35 10.0.0.5 GET / - 443 - 5.45.207.88 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 1615 42
+2024-03-18 03:33:18 10.0.0.5 GET /robots.txt - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 6225 42
+2024-03-18 04:05:15 10.0.0.5 GET /images/logo.png - 443 - 192.0.2.55 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 473 42
+2024-03-18 04:15:17 10.0.0.5 GET /blog/log-analysis - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 1765 42
+2024-03-18 04:24:48 10.0.0.5 GET /sitemap.xml - 443 - 66.249.64.40 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 7812 42
+2024-03-18 04:38:41 10.0.0.5 GET /js/app.js - 443 - 198.51.100.23 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - 200 0 0 6010 42
+2024-03-18 05:11:09 10.0.0.5 GET /css/main.css - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 6786 42
+2024-03-18 05:38:10 10.0.0.5 GET /contact - 443 - 198.51.100.23 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 422 42
+2024-03-18 05:54:05 10.0.0.5 GET /wp-login.php - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 404 0 0 364 42
+2024-03-18 06:09:01 10.0.0.5 GET /blog/seo-guide - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 2804 42
+2024-03-18 06:10:28 10.0.0.5 GET /robots.txt - 443 - 198.51.100.23 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 1233 42
+2024-03-18 06:25:32 10.0.0.5 GET /products/widget-a - 443 - 192.0.2.55 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 4993 42
+2024-03-18 06:29:17 10.0.0.5 GET /contact - 443 - 81.2.69.142 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 7388 42
+2024-03-18 06:48:13 10.0.0.5 GET /products - 443 - 66.249.64.40 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 4803 42
+2024-03-18 07:06:39 10.0.0.5 GET /css/main.css - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5189 42
+2024-03-18 07:25:39 10.0.0.5 GET /products - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 3677 42
+2024-03-18 07:37:54 10.0.0.5 GET /products/discontinued - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 404 0 0 369 42
+2024-03-18 08:04:43 10.0.0.5 GET /sitemap.xml - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 2832 42
+2024-03-18 08:24:33 10.0.0.5 GET /products/discontinued - 443 - 24.5.66.10 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 404 0 0 253 42
+2024-03-18 08:41:04 10.0.0.5 GET /legacy/admin - 443 - 66.249.79.12 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+GPTBot/1.1;++https://openai.com/gptbot - 404 0 0 290 42
+2024-03-18 08:56:37 10.0.0.5 GET /blog/log-analysis - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 3399 42
+2024-03-18 09:12:32 10.0.0.5 GET /about - 443 - 24.5.66.10 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 8107 42
+2024-03-18 09:25:29 10.0.0.5 GET /old-sitemap-1.xml - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+GPTBot/1.1;++https://openai.com/gptbot - 200 0 0 6074 42
+2024-03-18 09:31:58 10.0.0.5 GET /css/main.css - 443 - 66.249.66.83 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 1196 42
+2024-03-18 10:00:22 10.0.0.5 GET /products/widget-b - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 7816 42
+2024-03-18 10:09:52 10.0.0.5 GET /pricing - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 8329 42
+2024-03-18 10:24:10 10.0.0.5 GET /css/main.css - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 2552 42
+2024-03-18 10:53:17 10.0.0.5 GET /products/discontinued - 443 - 5.255.253.21 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 404 0 0 237 42
+2024-03-18 11:14:23 10.0.0.5 GET /images/logo.png - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 1689 42
+2024-03-18 11:41:00 10.0.0.5 GET /missing-page - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 404 0 0 355 42
+2024-03-18 11:53:52 10.0.0.5 GET /contact - 443 - 81.2.69.142 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 3341 42
+2024-03-18 12:16:36 10.0.0.5 GET /robots.txt - 443 - 192.0.2.55 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 3109 42
+2024-03-18 12:30:07 10.0.0.5 GET /about - 443 - 66.249.64.40 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 770 42
+2024-03-18 12:46:43 10.0.0.5 GET /blog/log-analysis - 443 - 24.5.66.10 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 3006 42
+2024-03-18 13:14:21 10.0.0.5 GET /products/widget-b - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 1065 42
+2024-03-18 13:31:20 10.0.0.5 GET /blog/log-analysis - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 1976 42
+2024-03-18 13:40:09 10.0.0.5 GET /blog/log-analysis - 443 - 157.55.39.120 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 4267 42
+2024-03-18 14:07:51 10.0.0.5 GET /products - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 2813 42
+2024-03-18 14:38:47 10.0.0.5 GET /blog/log-analysis - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 1516 42
+2024-03-18 14:43:01 10.0.0.5 GET /products - 443 - 203.0.113.7 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 7665 42
+2024-03-18 14:48:56 10.0.0.5 GET /contact - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 2021 42
+2024-03-18 14:52:56 10.0.0.5 GET /blog/log-analysis - 443 - 192.0.2.55 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 7877 42
+2024-03-18 15:03:34 10.0.0.5 GET /products/discontinued - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 404 0 0 231 42
+2024-03-18 15:16:11 10.0.0.5 GET /blog/seo-guide - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 4100 42
+2024-03-18 15:30:23 10.0.0.5 GET /products/widget-b - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 5490 42
+2024-03-18 15:52:57 10.0.0.5 GET /about - 443 - 198.51.100.23 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 3882 42
+2024-03-18 16:08:14 10.0.0.5 GET /images/logo.png - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 5910 42
+2024-03-18 16:16:08 10.0.0.5 GET /wp-login.php - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 404 0 0 265 42
+2024-03-18 16:29:51 10.0.0.5 GET /js/app.js - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 484 42
+2024-03-18 16:33:44 10.0.0.5 GET /legacy/admin - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 404 0 0 479 42
+2024-03-18 17:04:59 10.0.0.5 GET /robots.txt - 443 - 198.51.100.23 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 2090 42
+2024-03-18 17:07:04 10.0.0.5 GET /missing-page - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 404 0 0 453 42
+2024-03-18 17:33:28 10.0.0.5 GET /blog/seo-guide - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 5820 42
+2024-03-18 18:02:38 10.0.0.5 GET /products/discontinued - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 404 0 0 227 42
+2024-03-18 18:31:55 10.0.0.5 GET /blog/seo-guide - 443 - 198.51.100.23 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 7310 42
+2024-03-18 18:45:00 10.0.0.5 GET /legacy/admin - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 404 0 0 470 42
+2024-03-18 19:10:28 10.0.0.5 GET /products - 443 - 198.51.100.23 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 7675 42
+2024-03-18 19:36:30 10.0.0.5 GET /js/app.js - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 4028 42
+2024-03-18 19:54:34 10.0.0.5 GET /pricing - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5238 42
+2024-03-18 20:12:38 10.0.0.5 GET /about - 443 - 66.249.79.12 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 7050 42
+2024-03-18 20:34:37 10.0.0.5 GET /sitemap.xml - 443 - 192.0.2.55 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 452 42
+2024-03-18 20:44:47 10.0.0.5 GET /sitemap.xml - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 7364 42
+2024-03-18 21:16:55 10.0.0.5 GET / - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5306 42
+2024-03-18 21:36:50 10.0.0.5 GET /css/main.css - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 3022 42
+2024-03-18 22:01:46 10.0.0.5 GET / - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 5983 42
+2024-03-18 22:21:20 10.0.0.5 GET / - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 4185 42
+2024-03-18 22:28:49 10.0.0.5 GET /images/logo.png - 443 - 192.0.2.55 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 5895 42
+2024-03-18 22:31:27 10.0.0.5 GET /js/app.js - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 2169 42
+2024-03-18 22:38:05 10.0.0.5 GET /wp-login.php - 443 - 66.249.66.83 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 404 0 0 319 42
+2024-03-18 22:47:27 10.0.0.5 GET /css/main.css - 443 - 192.0.2.55 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 4168 42
+2024-03-18 22:55:05 10.0.0.5 GET /wp-login.php - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 404 0 0 306 42
+2024-03-18 23:15:50 10.0.0.5 GET /contact - 443 - 24.5.66.10 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 5468 42
+2024-03-18 23:24:03 10.0.0.5 GET /products/widget-a - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 4234 42
+2024-03-18 23:36:55 10.0.0.5 GET /old-pricing - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 301 0 0 0 42
+2024-03-18 23:58:05 10.0.0.5 GET /images/logo.png - 443 - 198.51.100.23 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 8200 42
+2024-03-19 00:10:52 10.0.0.5 GET /missing-page - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 404 0 0 469 42
+2024-03-19 00:31:25 10.0.0.5 GET / - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 6798 42
+2024-03-19 00:44:43 10.0.0.5 GET /products/discontinued - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 404 0 0 315 42
+2024-03-19 00:54:58 10.0.0.5 GET /contact - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+GPTBot/1.1;++https://openai.com/gptbot - 200 0 0 4119 42
+2024-03-19 01:03:04 10.0.0.5 GET /blog/seo-guide - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 1332 42
+2024-03-19 01:18:19 10.0.0.5 GET /api/checkout - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 500 0 0 326 42
+2024-03-19 01:27:59 10.0.0.5 GET /images/logo.png - 443 - 203.0.113.7 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 2955 42
+2024-03-19 01:35:20 10.0.0.5 GET /products/widget-a - 443 - 66.249.66.83 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 7133 42
+2024-03-19 01:50:26 10.0.0.5 GET /images/logo.png - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 2138 42
+2024-03-19 02:21:49 10.0.0.5 GET /robots.txt - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 2217 42
+2024-03-19 02:33:50 10.0.0.5 GET /products/widget-b - 443 - 81.2.69.142 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - 200 0 0 8356 42
+2024-03-19 02:46:46 10.0.0.5 GET /robots.txt - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 5307 42
+2024-03-19 02:51:29 10.0.0.5 GET /products/widget-b - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 8347 42
+2024-03-19 03:21:58 10.0.0.5 GET /old-pricing - 443 - 81.2.69.142 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - 301 0 0 0 42
+2024-03-19 03:32:57 10.0.0.5 GET /contact - 443 - 81.2.69.142 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 4475 42
+2024-03-19 03:51:47 10.0.0.5 GET /blog/log-analysis - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 5629 42
+2024-03-19 04:15:35 10.0.0.5 GET /images/logo.png - 443 - 198.51.100.23 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 - 200 0 0 6472 42
+2024-03-19 04:42:43 10.0.0.5 GET / - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 1066 42
+2024-03-19 05:00:33 10.0.0.5 GET /products/widget-b - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 7033 42
+2024-03-19 05:08:25 10.0.0.5 GET /about - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 3621 42
+2024-03-19 05:12:18 10.0.0.5 GET /robots.txt - 443 - 5.45.207.88 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 6597 42
+2024-03-19 05:14:28 10.0.0.5 GET /images/logo.png - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 6091 42
+2024-03-19 05:31:26 10.0.0.5 GET /legacy/admin - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 404 0 0 440 42
+2024-03-19 05:42:37 10.0.0.5 GET /css/main.css - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 908 42
+2024-03-19 05:59:29 10.0.0.5 GET /old-sitemap-1.xml - 443 - 192.0.2.55 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 2534 42
+2024-03-19 06:22:49 10.0.0.5 GET /robots.txt - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 7150 42
+2024-03-19 06:47:01 10.0.0.5 GET /js/app.js - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 8010 42
+2024-03-19 06:50:22 10.0.0.5 GET /contact - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 5840 42
+2024-03-19 07:03:18 10.0.0.5 GET /blog/log-analysis - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 7012 42
+2024-03-19 07:24:33 10.0.0.5 GET /css/main.css - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 5314 42
+2024-03-19 07:34:13 10.0.0.5 GET /products/discontinued - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 404 0 0 230 42
+2024-03-19 07:38:40 10.0.0.5 GET /products - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 1127 42
+2024-03-19 08:03:39 10.0.0.5 GET /about - 443 - 24.5.66.10 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 8144 42
+2024-03-19 08:35:20 10.0.0.5 GET /about - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 3294 42
+2024-03-19 09:07:28 10.0.0.5 GET /products/widget-b - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 6833 42
+2024-03-19 09:32:43 10.0.0.5 GET /products - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 1902 42
+2024-03-19 09:40:54 10.0.0.5 GET /pricing - 443 - 66.249.64.40 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 5811 42
+2024-03-19 09:58:28 10.0.0.5 GET /legacy/admin - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 404 0 0 239 42
+2024-03-19 10:27:45 10.0.0.5 GET /products/widget-a - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 6597 42
+2024-03-19 10:34:23 10.0.0.5 GET /products/discontinued - 443 - 157.55.39.120 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 404 0 0 393 42
+2024-03-19 10:55:08 10.0.0.5 GET /missing-page - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 404 0 0 294 42
+2024-03-19 11:19:28 10.0.0.5 GET /pricing - 443 - 192.0.2.55 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 - 200 0 0 559 42
+2024-03-19 11:28:46 10.0.0.5 GET /api/checkout - 443 - 203.0.113.7 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 500 0 0 208 42
+2024-03-19 11:41:43 10.0.0.5 GET / - 443 - 192.0.2.55 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 4756 42
+2024-03-19 11:45:55 10.0.0.5 GET /api/checkout - 443 - 198.51.100.23 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 500 0 0 254 42
+2024-03-19 12:05:54 10.0.0.5 GET /blog/log-analysis - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5738 42
+2024-03-19 12:07:36 10.0.0.5 GET / - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 3532 42
+2024-03-19 12:12:05 10.0.0.5 GET /old-sitemap-1.xml - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 8392 42
+2024-03-19 12:32:27 10.0.0.5 GET /products/widget-a - 443 - 203.0.113.7 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 6239 42
+2024-03-19 12:40:47 10.0.0.5 GET /old-sitemap-1.xml - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 3330 42
+2024-03-19 13:00:03 10.0.0.5 GET /js/app.js - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 3810 42
+2024-03-19 13:03:31 10.0.0.5 GET /about - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 4174 42
+2024-03-19 13:33:48 10.0.0.5 GET /products/widget-b - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 6361 42
+2024-03-19 13:48:16 10.0.0.5 GET /css/main.css - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 5139 42
+2024-03-19 13:52:36 10.0.0.5 GET /robots.txt - 443 - 192.0.2.55 facebookexternalhit/1.1+(+http://www.facebook.com/externalhit_uatext.php) - 200 0 0 3091 42
+2024-03-19 14:13:28 10.0.0.5 GET /products/widget-a - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5319 42
+2024-03-19 14:34:55 10.0.0.5 GET /images/logo.png - 443 - 66.249.66.83 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 8093 42
+2024-03-19 14:38:11 10.0.0.5 GET /pricing - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 6924 42
+2024-03-19 14:48:43 10.0.0.5 GET /products - 443 - 5.255.253.21 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 6882 42
+2024-03-19 15:00:56 10.0.0.5 GET / - 443 - 5.45.207.88 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 2450 42
+2024-03-19 15:25:59 10.0.0.5 GET /robots.txt - 443 - 192.0.2.55 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 958 42
+2024-03-19 15:37:31 10.0.0.5 GET /blog/seo-guide - 443 - 207.46.13.91 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 1706 42
+2024-03-19 16:09:54 10.0.0.5 GET /products/widget-a - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 6902 42
+2024-03-19 16:25:29 10.0.0.5 GET /blog/log-analysis - 443 - 157.55.39.120 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 803 42
+2024-03-19 16:51:37 10.0.0.5 GET /wp-login.php - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 404 0 0 213 42
+2024-03-19 17:09:01 10.0.0.5 GET /images/logo.png - 443 - 81.2.69.142 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 - 200 0 0 3599 42
+2024-03-19 17:21:38 10.0.0.5 GET /about - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 1271 42
+2024-03-19 17:26:37 10.0.0.5 GET /products/widget-b - 443 - 203.0.113.7 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 8367 42
+2024-03-19 17:37:02 10.0.0.5 GET /css/main.css - 443 - 54.36.148.12 Mozilla/5.0+(compatible;+AhrefsBot/7.0;++http://ahrefs.com/robot/) - 200 0 0 1212 42
+2024-03-19 17:48:46 10.0.0.5 GET /robots.txt - 443 - 66.249.64.40 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 6967 42
+2024-03-19 18:02:12 10.0.0.5 GET /legacy/admin - 443 - 66.249.64.40 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 404 0 0 445 42
+2024-03-19 18:04:06 10.0.0.5 GET /about - 443 - 24.5.66.10 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 3123 42
+2024-03-19 18:11:48 10.0.0.5 GET /old-sitemap-1.xml - 443 - 24.5.66.10 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 4726 42
+2024-03-19 18:25:05 10.0.0.5 GET /robots.txt - 443 - 198.51.100.23 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 5756 42
+2024-03-19 18:32:11 10.0.0.5 GET /api/checkout - 443 - 81.2.69.142 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 500 0 0 329 42
+2024-03-19 18:48:03 10.0.0.5 GET /robots.txt - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 5085 42
+2024-03-19 19:03:20 10.0.0.5 GET /contact - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 6648 42
+2024-03-19 19:11:43 10.0.0.5 GET /products/widget-a - 443 - 66.249.79.12 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 5975 42
+2024-03-19 19:38:00 10.0.0.5 GET / - 443 - 85.208.96.20 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 7314 42
+2024-03-19 19:47:14 10.0.0.5 GET / - 443 - 81.2.69.142 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 1713 42
+2024-03-19 19:54:00 10.0.0.5 GET /css/main.css - 443 - 66.249.66.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/118.0.0.0+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 443 42
+2024-03-19 20:13:53 10.0.0.5 GET /contact - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 1704 42
+2024-03-19 20:30:28 10.0.0.5 GET /blog/log-analysis - 443 - 66.249.66.1 Mozilla/5.0+AppleWebKit/537.36+(KHTML,+like+Gecko);+compatible;+Googlebot/2.1;++http://www.google.com/bot.html - 200 0 0 6529 42
+2024-03-19 20:51:57 10.0.0.5 GET /contact - 443 - 81.2.69.142 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_2+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.2+Mobile/15E148+Safari/604.1 https://www.google.com/ 200 0 0 3579 42
+2024-03-19 21:04:55 10.0.0.5 GET /images/logo.png - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 3129 42
+2024-03-19 21:14:47 10.0.0.5 GET /js/app.js - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 5876 42
+2024-03-19 21:19:46 10.0.0.5 GET /api/checkout - 443 - 203.0.113.7 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 500 0 0 397 42
+2024-03-19 21:40:40 10.0.0.5 GET /robots.txt - 443 - 192.0.2.55 Mozilla/5.0+(X11;+Linux+x86_64;+rv:121.0)+Gecko/20100101+Firefox/121.0 https://www.google.com/ 200 0 0 7032 42
+2024-03-19 21:59:17 10.0.0.5 GET /products - 443 - 81.2.69.142 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/120.0.0.0+Safari/537.36 https://www.google.com/ 200 0 0 4665 42
+2024-03-19 22:07:43 10.0.0.5 GET /blog/seo-guide - 443 - 46.229.168.66 Mozilla/5.0+(compatible;+SemrushBot/7~bl;++http://www.semrush.com/bot.html) - 200 0 0 3053 42
+2024-03-19 22:31:58 10.0.0.5 GET /images/logo.png - 443 - 5.45.207.88 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 3341 42
+2024-03-19 22:35:03 10.0.0.5 GET /robots.txt - 443 - 40.77.167.45 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 200 0 0 8092 42
+2024-03-19 22:40:13 10.0.0.5 GET /robots.txt - 443 - 95.108.213.5 Mozilla/5.0+(compatible;+YandexBot/3.0;++http://yandex.com/bots) - 200 0 0 684 42