Compare commits

..
Author SHA1 Message Date
d32f271195 🐛 appsec: do not buffer unreadable (gRPC/HTTP2) request bodies (#332)
* 🐛 fix appsec silently 403-ing gRPC streams with unreadable body

A bidirectional gRPC stream is an HTTP/2 request with no Content-Length
whose body never reaches EOF. Since #321 removed the ContentLength guard,
appsecQuery buffered it with io.ReadAll, which blocked until the request
timed out and was turned into a 403 (issue #323). The backend was never
reached (OriginStatus:0).

Mirror the reference lua-cs-bouncer behaviour: detect an unreadable body
(ProtoMajor >= 2 && ContentLength < 0) and, instead of buffering it,
forward the request to Appsec with headers only. Add a new
CrowdsecAppsecDropUnreadableBody option (default false) that mirrors the
reference APPSEC_DROP_UNREADABLE_BODY: when true, such requests are
blocked outright instead of forwarded without their body.

Readable HTTP/1.1 bodies are still buffered and inspected, so the bypass
closed by #321 stays closed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* 🚨 appsec: satisfy linters (gocritic ifElseChain, misspell)

Rewrite the body-handling if/else chain in appsecQuery as a switch
(gocritic) and use US spelling "behavior" (misspell).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* 🔇 appsec: drop redundant unreadable-body debug log

Address review on #332: the caller (handleNextServeHTTP) already logs the
returned error with the request IP, so the inner Debug line duplicated it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* 🍱 increase gocyclo

* 🍱 fix lint

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2026-07-01 12:44:58 +02:00
Daniel Berteaudandmaxlerebourg 1c1672c856 Consider 502, 503 and 504 as unavaible for appsec (#338)
* Consider 502, 503 and 504 as unavaible for appsec

Fixes #337

*  add test and the function isReverseProxyError

---------

Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2026-06-30 22:03:06 +02:00
dependabot[bot] 21895fbb9d ⬆️ Bump actions/cache from 5 to 6 (#344)
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 10:11:22 +02:00
omerandmaxlerebourg 7c73cb38dd Add parameter to configure the ban page Content-Type response header (#325)
* Add parameter to configure Ban Response Content-Type

* Add testing for new BanResponseContentType parameter

* Ensure there is a fallback to default Content-Type is user provided empty value

* Set Content-Type even if banTemplate is nil

* Add more edge cases for testing ban response Content-Type

* Add CR/LF validation for BanResponseContentType

* Add CaptchaResponseContentType to allow separate Content-Type configuration for captcha responses

* Add testing for new CaptchaResponseContentType

* Update README

* Split nil and CR/LF response Content-Type value validation into separate function

* Throw error instead of setting the default in case of empty parameter declaration

* Update testing accordingly

*  remove HTML from var name, add tests and infer content type from filePath

* 🍱 fix lint ?

* 🍱 fix lint

* 🍱 fix lint

* 🍱 fix lint + naming

* 🍱 fix lint

* 🍱 fuck lint

---------

Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2026-06-28 22:30:48 +02:00
mathieuHaandClaude Opus 4.8 f4dcd933c8 🐛 fall back to system trust store when no custom TLS CA is set (#331)
* 🐛 fall back to system trust store when no custom TLS CA is set

Closes #327.

Until now, configuring `crowdsecLapiScheme=https` forced the operator
to either provide `crowdsecLapiTLSCertificateAuthority` (a custom CA)
or set `crowdsecLapiTLSInsecureVerify=true` — there was no way to rely
on the host's system trust store, which is the expected setup when the
LAPI sits behind a reverse proxy with a publicly trusted (e.g. Let's
Encrypt) certificate.

Two contributing bugs:
  - `validateParamsTLS` rejected an empty CA up-front.
  - `getTLSConfig` always allocated an empty `tls.Config.RootCAs`,
    which silently disabled the standard library's fall-back to
    `x509.SystemCertPool()`.

Fix: drop the validation error for the empty-CA case and only allocate
`RootCAs` when a custom CA is actually provided. Same change applies
symmetrically to the AppSec path since the helper is shared.

Add unit tests covering the four meaningful states (HTTP, HTTPS with
system CA, HTTPS with custom CA, HTTPS with insecure verify) plus the
malformed-PEM rejection. README updated to document the system trust
store as an explicit option for both LAPI and AppSec HTTPS.

* 📝 fix gofmt alignment in TLS test struct

*  update existing test: https without CA is now accepted

* ♻️ tests: hoist shared validPEM to package level, rename cfgGarbage

Address review on #331:
- the self-signed validPEM block was duplicated in two test funcs; declare it
  once at package level and drop both local copies.
- rename cfgGarbage -> cfgInvalidCA (and its test case) for a descriptive name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

*  e2e mock: add tls-system-ca scenario (HTTPS LAPI via system trust store)

CI regression coverage for this PR: with no custom CA configured, the bouncer
must fall back to the OS/system trust store for an HTTPS LAPI.

In the binary suite the "system trust store" is whatever Go's
x509.SystemCertPool() reads, which honours SSL_CERT_FILE on the Traefik process.
The scenario mints a throwaway CA, serves the mock LAPI over HTTPS with a cert
signed by it, and runs the stack twice:
  - positive: SSL_CERT_FILE = our CA       -> LAPI trusted    -> 200
  - negative: SSL_CERT_FILE = empty bundle  -> not trusted     -> 403 (fail-closed)
The negative run proves the patch still VERIFIES (not an insecure skip).

- mocklapi: optional --lapi-tls-cert/--lapi-tls-key to serve the LAPI over TLS.
- common.sh: opt-in LAPI_TLS_CERT/KEY (HTTPS mock) and TRAEFIK_SSL_CERT_FILE
  (inject SSL_CERT_FILE into Traefik); both default-empty, other scenarios
  unaffected.
- adds openssl as a scenario-only dependency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 09:27:32 +02:00
dependabot[bot] 67b33dcf13 ⬆️ Bump actions/checkout from 6 to 7 (#341)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-22 08:56:27 +02:00
dependabot[bot] 661a89ea9c ⬆️ Bump actions/upload-artifact from 4 to 7 (#336)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-15 08:49:57 +02:00
mathieuHaandClaude Opus 4.8 14b9c47ab2 tests: CI end-to-end suite — Traefik binary + mock LAPI (#329)
*  tests: end-to-end suite scaffold + stream-mode scenario

Add tests/e2e/ structure with shared bash helpers and the first
scenario (stream-mode): spin up real Traefik + Crowdsec via docker
compose, mount the repo as a local plugin, add a ban via cscli, verify
the bouncer blocks the matching X-Forwarded-For, then delete the
decision and verify pass-through.

Adds .github/workflows/e2e.yml with one matrix job per scenario
(stream-mode for now), and Makefile targets `e2e` and `e2e_<scenario>`
for local runs.

Refs #328

*  tests: revert to docker provider + bump versions + add 6 scenarios

Reverts the stream-mode scenario to the Traefik docker provider (the
file provider was a workaround for a local docker daemon API version
mismatch, irrelevant in CI). Bumps Traefik to v3.7.1 and Crowdsec to
v1.7.8 across all scenarios.

Adds six new E2E scenarios:
- live-mode: short defaultDecisionSeconds, verifies cache-then-recheck
- none-mode: verifies LAPI is queried per request, no caching
- trusted-ips: clientTrustedIPs bypass even when the trusted IP is banned
- custom-ban-page: BanHTMLFilePath body + Content-Type validation
- captcha: captcha decision serves the captcha page (HTTP 200)
- appsec: SQLi probe blocked by appsec-virtual-patching

Each scenario uses an isolated compose project, mounts the repo as a
local plugin, and asserts behavior via curl. Workflow matrix and
Makefile E2E_SCENARIOS updated accordingly.

Refs #328

* 🐛 tests: set CROWDSEC_BYPASS_DB_VOLUME_CHECK for v1.7+

Crowdsec v1.7 refuses to start without an explicit volume mount on
/var/lib/crowdsec/data (or the bypass env var). For E2E we don't need
db persistence — set the bypass everywhere so the stack boots.

* 🐛 tests: appsec scenario uses OWASP CRS inband collection

appsec-virtual-patching only ships CVE-specific rules, not generic
SQLi. Switch to crowdsecurity/appsec-crs-inband (the blocking OWASP
Core Rule Set) and use a SQLi probe that CRS paranoia level 1
matches (rule 942100/942130).

* ♻️ tests: run e2e suite in a single sequential job

The matrix spawned one runner per scenario, so the Traefik, Crowdsec and
whoami images were pulled — and Crowdsec booted — once per scenario. Run
the whole suite in a single job with `make -k e2e` instead: Docker caches
the images locally so they are pulled only once, and `-k` keeps the
remaining scenarios running after a failure (make still exits non-zero).

Scenarios already share the canonical `crowdsec` container name and the
8000 port, so they were meant to run sequentially anyway.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

*  tests: add binary e2e suite (Traefik binary + Go mock LAPI), run it in CI

Add a second e2e suite that runs Traefik as a downloaded binary with the
plugin loaded from source, and replaces Crowdsec with a small stdlib-only Go
LAPI mock driven via /admin endpoints. No Docker, no real Crowdsec.

The mock lives in its own nested Go module (tests/e2e/mock/mocklapi) so it
stays out of the plugin module's build, lint, test and vendor.

This suite validates the plugin's own behaviour (live/none/stream modes,
caching, trusted-IP bypass, ban/captcha rendering). Crowdsec and AppSec
correctness are out of scope on purpose — they are validated upstream by the
maintainer — so the AppSec scenario is intentionally absent and the README
says so to avoid misfiled issues.

CI now runs this suite only (`make e2e_mock`), since it needs neither Docker
nor a real Crowdsec. The Docker suite (tests/e2e/scenarios) is kept for local
debugging (`make e2e`).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* 🔧 e2e mock: address review — clarify backend flag, move ignore to root

- Document the --backend-addr flag: it is the stub upstream service Traefik
  proxies allowed requests to (the traefik/whoami equivalent), not AppSec.
- Move the .cache/ ignore rule from the per-suite .gitignore to the repo root
  .gitignore, and make the wording accurate: the cache persists across local
  runs but is recreated on every (fresh-runner) CI run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ♻️ tests: move local Docker e2e suite to its own PR (#333)

Per review, split the e2e work so each PR is focused. CI runs the binary +
mock-LAPI suite (this PR); the heavier, local-only Docker suite (real Traefik
+ Crowdsec, incl. appsec) now lives in #333.

Removes tests/e2e/scenarios, tests/e2e/lib and the Docker-suite README, and
drops the `e2e` Make target here (kept in #333). The binary/mock suite and its
`e2e_mock` target are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* 🔥 e2e mock: simplify the Crowdsec LAPI mock

Per review, trim the mock to the minimum the plugin actually exercises:

- Drop the stream delta bookkeeping (startup flag + "already streamed" set).
  The plugin re-Sets/Deletes its cache on every poll, so reporting the whole
  active set as "new" and removed ones as "deleted" is enough.
- Shrink the Decision struct to the three fields the plugin reads
  (value/type/duration); drop id/origin/scope/scenario and the id counter.
- Drop API-key auth and the /admin/reset endpoint — no scenario exercises
  either. Also drop the now-unused lapi_reset helper.
- Replace the store struct + methods with two package-level maps + a mutex.

mocklapi/main.go: 234 -> 118 lines. All six scenarios still pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

*  e2e mock: replace fixed sleeps with condition polling

The `sleep 4` / `sleep 3` after a decision change were magic numbers tied to
updateIntervalSeconds / defaultDecisionSeconds. Replace them with waits on the
actual condition:

- After a ban/unban, poll with wait_for_status until the expected code shows up
  (stream propagation / live-mode cache TTL).
- Captcha keeps status 200 before and after, so gate on the body marker via a
  new wait_for_body_contains helper.
- Control assertions that must NOT change stay immediate (assert_status).

Self-documenting, faster on the happy path (returns on the first poll that
sees the change), and more robust under slow CI. No fixed sleeps remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ⬆️ e2e mock: bump module go directive to 1.23

Align the mock module with the project's Go version (CI uses 1.23). Part of
standardising the whole project on Go 1.23; the plugin module is bumped
separately.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ⬇️ e2e mock: keep Go floor at 1.22 (yaegi ceiling)

Revert the mock module back to go 1.22 and make the e2e workflow read the Go
version from go.mod (go-version-file) instead of hardcoding 1.23.

Rationale: the plugin is interpreted by yaegi, and even Traefik v3.7.1 ships
yaegi v0.16.1 (Go 1.22), so the project stays on 1.22. The earlier bump to 1.23
is dropped (plugin go.mod stays 1.22, see #330 for the Renovate cap + CI pin).
Mock + all six scenarios verified on Go 1.22.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

*  e2e mock: add AppSec scenario + custom remediation header assertion

Address review feedback on the binary e2e suite:

- mocklapi: add an AppSec WAF stand-in (--appsec-addr) that blocks any URI
  containing "rpc2" — the exact probe from examples/appsec-enabled — and allows
  the rest. Lets the suite exercise the plugin's AppSec wiring (header
  forwarding + allow/block enforcement) without the real CRS engine.
- new scenarios/appsec: benign request passes, /foo/rpc2 is 403.
- custom-ban-page: assert the banned response carries the custom remediation
  header (remediationHeadersCustomName), per review.
- README: drop the "don't open issues / AppSec intentionally absent" framing;
  describe what the suite actually covers, including AppSec wiring.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:49:18 +02:00
mathieuHaandClaude Opus 4.8 f5d580578c 📝 docs: fix Mermaid diagrams not rendering (lowercase destroy keyword) (#335)
The sequence diagrams used the keyword `Destroy` (capital D). Mermaid
keywords are case-sensitive, so the invalid token aborted parsing and
every diagram failed to render on GitHub.

Lowercase all `Destroy` -> `destroy` in README.md and the captcha
example README. The `create` keyword was already correct, and each
create/destroy is properly paired with an adjacent message.

Fixes #270

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:40:56 +02:00
27 changed files with 738 additions and 258 deletions
+2 -2
View File
@@ -17,7 +17,7 @@ jobs:
name: e2e (binary + mock LAPI) name: e2e (binary + mock LAPI)
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v6 - uses: actions/checkout@v7
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
@@ -35,7 +35,7 @@ jobs:
run: make -k e2e_mock run: make -k e2e_mock
- name: Upload logs on failure - name: Upload logs on failure
if: failure() if: failure()
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v7
with: with:
name: e2e-logs name: e2e-logs
path: /tmp/e2e-mock-*.log path: /tmp/e2e-mock-*.log
+2 -2
View File
@@ -33,14 +33,14 @@ jobs:
# https://github.com/marketplace/actions/checkout # https://github.com/marketplace/actions/checkout
- name: Check out code - name: Check out code
uses: actions/checkout@v6 uses: actions/checkout@v7
with: with:
path: go/src/github.com/${{ github.repository }} path: go/src/github.com/${{ github.repository }}
fetch-depth: 0 fetch-depth: 0
# https://github.com/marketplace/actions/cache # https://github.com/marketplace/actions/cache
- name: Cache Go modules - name: Cache Go modules
uses: actions/cache@v5 uses: actions/cache@v6
with: with:
path: ${{ github.workspace }}/go/pkg/mod path: ${{ github.workspace }}/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v6 uses: actions/checkout@v7
with: with:
ref: main ref: main
+1 -1
View File
@@ -7,7 +7,7 @@ linters-settings:
disable: disable:
- fieldalignment - fieldalignment
gocyclo: gocyclo:
min-complexity: 15 min-complexity: 20
goconst: goconst:
min-len: 5 min-len: 5
min-occurrences: 4 min-occurrences: 4
+1 -1
View File
@@ -4,7 +4,7 @@ export GO111MODULE=on
# Binary/mock suite (Traefik binary + mock LAPI). This is what CI runs. # Binary/mock suite (Traefik binary + mock LAPI). This is what CI runs.
# The local Docker suite (make e2e) lives in a separate PR/branch. # The local Docker suite (make e2e) lives in a separate PR/branch.
E2E_MOCK_SCENARIOS := stream-mode live-mode none-mode trusted-ips custom-ban-page captcha appsec E2E_MOCK_SCENARIOS := stream-mode live-mode none-mode trusted-ips custom-ban-page captcha appsec tls-system-ca
default: lint test default: lint test
+43 -36
View File
@@ -68,7 +68,7 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant CrowdsecLAPI create participant CrowdsecLAPI
TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a Crowdsec Decision ? TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a Crowdsec Decision ?
Destroy CrowdsecLAPI destroy CrowdsecLAPI
CrowdsecLAPI-->>TraefikPlugin: Yes a ban Decision CrowdsecLAPI-->>TraefikPlugin: Yes a ban Decision
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
@@ -82,9 +82,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant CrowdsecLAPI create participant CrowdsecLAPI
TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ? TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ?
Destroy CrowdsecLAPI destroy CrowdsecLAPI
CrowdsecLAPI-->>TraefikPlugin: Nothing, all good! CrowdsecLAPI-->>TraefikPlugin: Nothing, all good!
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -105,10 +105,10 @@ sequenceDiagram
PluginCache-->>TraefikPlugin: Nothing, all good! PluginCache-->>TraefikPlugin: Nothing, all good!
create participant CrowdsecLAPI create participant CrowdsecLAPI
TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ? TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ?
Destroy CrowdsecLAPI destroy CrowdsecLAPI
CrowdsecLAPI-->>TraefikPlugin: Yes a ban Decision CrowdsecLAPI-->>TraefikPlugin: Yes a ban Decision
TraefikPlugin-->>PluginCache: Store the information for this IP for DefaultDecisionSeconds TraefikPlugin-->>PluginCache: Store the information for this IP for DefaultDecisionSeconds
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
@@ -125,10 +125,10 @@ sequenceDiagram
PluginCache-->>TraefikPlugin: Nothing, all good! PluginCache-->>TraefikPlugin: Nothing, all good!
create participant CrowdsecLAPI create participant CrowdsecLAPI
TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ? TraefikPlugin-->>CrowdsecLAPI: Does the User IP has a crowdsec decision ?
Destroy CrowdsecLAPI destroy CrowdsecLAPI
CrowdsecLAPI-->>TraefikPlugin: Nothing, all good! CrowdsecLAPI-->>TraefikPlugin: Nothing, all good!
TraefikPlugin-->>PluginCache: Store the information for this IP for DefaultDecisionSeconds TraefikPlugin-->>PluginCache: Store the information for this IP for DefaultDecisionSeconds
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
@@ -145,11 +145,11 @@ sequenceDiagram
participant TraefikPlugin participant TraefikPlugin
participant CrowdsecLAPI participant CrowdsecLAPI
TraefikPlugin->>CrowdsecLAPI: What are the current decisions TraefikPlugin->>CrowdsecLAPI: What are the current decisions
Destroy CrowdsecLAPI destroy CrowdsecLAPI
CrowdsecLAPI->>TraefikPlugin: Here is the list CrowdsecLAPI->>TraefikPlugin: Here is the list
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Store this list TraefikPlugin-->>PluginCache: Store this list
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
``` ```
@@ -162,9 +162,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ? TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Yes a ban decision PluginCache-->>TraefikPlugin: Yes a ban decision
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
@@ -177,9 +177,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ? TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Nothing, all good! PluginCache-->>TraefikPlugin: Nothing, all good!
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -195,11 +195,11 @@ sequenceDiagram
participant TraefikPlugin participant TraefikPlugin
participant CrowdsecCAPI participant CrowdsecCAPI
TraefikPlugin->>CrowdsecCAPI: What are the current decisions from CAPI TraefikPlugin->>CrowdsecCAPI: What are the current decisions from CAPI
Destroy CrowdsecCAPI destroy CrowdsecCAPI
CrowdsecCAPI->>TraefikPlugin: Here is the list CrowdsecCAPI->>TraefikPlugin: Here is the list
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Store this list TraefikPlugin-->>PluginCache: Store this list
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
``` ```
@@ -212,9 +212,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ? TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Yes a ban decision PluginCache-->>TraefikPlugin: Yes a ban decision
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
@@ -227,9 +227,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ? TraefikPlugin-->>PluginCache: Does the User IP has a crowdsec decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Nothing, all good! PluginCache-->>TraefikPlugin: Nothing, all good!
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -247,9 +247,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant CrowdsecAppSec create participant CrowdsecAppSec
TraefikPlugin-->>CrowdsecAppSec: Is this request malicious ? TraefikPlugin-->>CrowdsecAppSec: Is this request malicious ?
Destroy CrowdsecAppSec destroy CrowdsecAppSec
CrowdsecAppSec-->>TraefikPlugin: Yes I think so CrowdsecAppSec-->>TraefikPlugin: Yes I think so
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
@@ -262,9 +262,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant CrowdsecAppSec create participant CrowdsecAppSec
TraefikPlugin-->>CrowdsecAppSec: Is this request malicious ? TraefikPlugin-->>CrowdsecAppSec: Is this request malicious ?
Destroy CrowdsecAppSec destroy CrowdsecAppSec
CrowdsecAppSec-->>TraefikPlugin: No I don't think so CrowdsecAppSec-->>TraefikPlugin: No I don't think so
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -285,12 +285,12 @@ sequenceDiagram
User->>TraefikPlugin: Fine, done! User->>TraefikPlugin: Fine, done!
create participant ProviderCaptcha create participant ProviderCaptcha
TraefikPlugin-->>ProviderCaptcha: Is the validation OK ? TraefikPlugin-->>ProviderCaptcha: Is the validation OK ?
Destroy ProviderCaptcha destroy ProviderCaptcha
ProviderCaptcha-->>TraefikPlugin: Yes ProviderCaptcha-->>TraefikPlugin: Yes
TraefikPlugin-->>PluginCache: Set the User IP Clean for captchaGracePeriodSeconds TraefikPlugin-->>PluginCache: Set the User IP Clean for captchaGracePeriodSeconds
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -362,7 +362,7 @@ make run
- CrowdsecAppsecTlsCertificateAuthority - CrowdsecAppsecTlsCertificateAuthority
- string - string
- default: "" - default: ""
- PEM-encoded Certificate Authority of Appsec - PEM-encoded Certificate Authority used to verify Appsec's server certificate. When empty (and `crowdsecAppsecTlsInsecureVerify` is `false`), the host's system trust store is used.
- CrowdsecAppsecScheme - CrowdsecAppsecScheme
- string - string
- default: value of `CrowdsecLapiScheme`, expected values are: `http`, `https` - default: value of `CrowdsecLapiScheme`, expected values are: `http`, `https`
@@ -382,6 +382,10 @@ make run
- int64 - int64
- default: 10485760 (= 10MB) - default: 10485760 (= 10MB)
- Transmit only the first number of bytes to Crowdsec Appsec Server. - Transmit only the first number of bytes to Crowdsec Appsec Server.
- CrowdsecAppsecUnreadableBodyBlock
- bool
- default: false
- Behaviour when the request body cannot be buffered for inspection (HTTP/2 or HTTP/3 request without a `Content-Length`, typically a bidirectional gRPC stream). When `false` (default) the request is forwarded to the Appsec Server with headers only (the body is left to stream through untouched). When `true` the request is blocked outright. Mirrors the reference bouncers' `APPSEC_DROP_UNREADABLE_BODY` option.
- CrowdsecAppsecKey - CrowdsecAppsecKey
- string - string
- default: value of `CrowdsecLapiKey` - default: value of `CrowdsecLapiKey`
@@ -408,7 +412,7 @@ make run
- CrowdsecLapiTlsCertificateAuthority - CrowdsecLapiTlsCertificateAuthority
- string - string
- default: "" - default: ""
- PEM-encoded Certificate Authority of the Crowdsec LAPI - PEM-encoded Certificate Authority used to verify the LAPI's server certificate. When empty (and `crowdsecLapiTlsInsecureVerify` is `false`), the host's system trust store is used.
- CrowdsecLapiTlsCertificateBouncer - CrowdsecLapiTlsCertificateBouncer
- string - string
- default: "" - default: ""
@@ -513,14 +517,14 @@ make run
- int64 - int64
- default: 1800 (= 30 minutes) - default: 1800 (= 30 minutes)
- Period after validation of a captcha before a new validation is required if Crowdsec decision is still valid - Period after validation of a captcha before a new validation is required if Crowdsec decision is still valid
- CaptchaHTMLFilePath - CaptchaFilePath
- string - string
- default: /captcha.html - default: /captcha.html
- Path where the captcha template is stored - Path where the captcha template is stored. The Content-Type header is automatically inferred from the file extension.
- BanHTMLFilePath - BanFilePath
- string - string
- default: "" - default: ""
- Path where the ban html file is stored (default empty ""=disabled) - Path where the ban file is stored (default empty ""=disabled). The Content-Type header is automatically inferred from the file extension.
- TraceHeadersCustomName - TraceHeadersCustomName
- string - string
- default: "" - default: ""
@@ -616,6 +620,7 @@ http:
crowdsecAppsecFailureBlock: true crowdsecAppsecFailureBlock: true
crowdsecAppsecUnreachableBlock: true crowdsecAppsecUnreachableBlock: true
crowdsecAppsecBodyLimit: 10485760 crowdsecAppsecBodyLimit: 10485760
crowdsecAppsecUnreadableBodyBlock: false
crowdsecLapiKey: privateKey-foo crowdsecLapiKey: privateKey-foo
crowdsecLapiScheme: http crowdsecLapiScheme: http
crowdsecLapiHost: crowdsec:8080 crowdsecLapiHost: crowdsec:8080
@@ -727,16 +732,18 @@ A script is available to generate certificates in `examples/tls-auth/gencerts.sh
#### Use HTTPS to communicate with the LAPI #### Use HTTPS to communicate with the LAPI
To communicate with the LAPI in HTTPS you need to either accept any certificates by setting the `crowdsecLapiTLSInsecureVerify` to true or add the CA used by the server certificate of Crowdsec using `crowdsecLapiTLSCertificateAuthority` or `crowdsecLapiTLSCertificateAuthorityFile`. Set `crowdsecLapiScheme` to `https`. The plugin then validates Crowdsec's server certificate. Three options:
Set the `crowdsecLapiScheme` to https.
- **Publicly trusted certificate** (e.g. Let's Encrypt behind a reverse proxy): leave `crowdsecLapiTLSCertificateAuthority` empty and `crowdsecLapiTLSInsecureVerify` `false`. The plugin falls back to the host's system trust store (the `traefik` image ships `ca-certificates`).
- **Private/self-signed CA**: set `crowdsecLapiTLSCertificateAuthority` (or `…File`) to the PEM-encoded CA that signed Crowdsec's server cert.
- **Skip verification entirely** (not recommended for production): set `crowdsecLapiTLSInsecureVerify` to `true`.
Crowdsec must be listening in HTTPS for this to work. Crowdsec must be listening in HTTPS for this to work.
Please see the [tls-auth example](https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/blob/main/examples/tls-auth/README.md) or the official documentation: [docs.crowdsec.net/docs/local_api/tls_auth/](https://docs.crowdsec.net/docs/local_api/tls_auth/) Please see the [tls-auth example](https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/blob/main/examples/tls-auth/README.md) or the official documentation: [docs.crowdsec.net/docs/local_api/tls_auth/](https://docs.crowdsec.net/docs/local_api/tls_auth/)
#### Use HTTPS to communicate with the Appsec #### Use HTTPS to communicate with the Appsec
To communicate with the Appsec in HTTPS you need to either accept any certificates by setting the `crowdsecAppsecTLSInsecureVerify` to true or add the CA used by the server certificate of Crowdsec using `crowdsecAppsecTLSCertificateAuthority` or `crowdsecAppsecTLSCertificateAuthorityFile`. Set `crowdsecAppsecScheme` to `https`. Same three options as for the LAPI, prefixed `crowdsecAppsec…` instead of `crowdsecLapi…`: empty CA + secure verify falls back to the system trust store, a custom CA pins to your private PKI, and `crowdsecAppsecTLSInsecureVerify=true` skips verification altogether.
Set the `crowdsecAppsecScheme` to https.
Currently AppSec does not support mTLS authentication for the AppSec Component. Currently AppSec does not support mTLS authentication for the AppSec Component.
+113 -82
View File
@@ -9,7 +9,6 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
htmltemplate "html/template"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
@@ -84,49 +83,59 @@ type Bouncer struct {
name string name string
template *template.Template template *template.Template
enabled bool enabled bool
appsecEnabled bool appsecEnabled bool
appsecScheme string appsecScheme string
appsecHost string appsecHost string
appsecPath string appsecPath string
appsecKey string appsecKey string
appsecFailureBlock bool appsecFailureBlock bool
appsecUnreachableBlock bool appsecUnreachableBlock bool
appsecBodyLimit int64 appsecUnreadableBodyBlock bool
crowdsecScheme string appsecBodyLimit int64
crowdsecHost string crowdsecScheme string
crowdsecPath string crowdsecHost string
crowdsecKey string crowdsecPath string
crowdsecMode string crowdsecKey string
crowdsecMachineID string crowdsecMode string
crowdsecPassword string crowdsecMachineID string
crowdsecScenarios []string crowdsecPassword string
updateInterval int64 crowdsecScenarios []string
updateMaxFailure int64 updateInterval int64
defaultDecisionTimeout int64 updateMaxFailure int64
remediationStatusCode int defaultDecisionTimeout int64
remediationCustomHeader string remediationStatusCode int
forwardedCustomHeader string remediationCustomHeader string
crowdsecStreamRoute string forwardedCustomHeader string
crowdsecHeader string crowdsecStreamRoute string
redisUnreachableBlock bool crowdsecHeader string
banTemplate *htmltemplate.Template redisUnreachableBlock bool
traceCustomHeader string banTemplate *template.Template
clientPoolStrategy *ip.PoolStrategy banTemplateContentType string
serverPoolStrategy *ip.PoolStrategy traceCustomHeader string
httpClient *http.Client clientPoolStrategy *ip.PoolStrategy
httpAppsecClient *http.Client serverPoolStrategy *ip.PoolStrategy
cacheClient *cache.Client httpClient *http.Client
captchaClient *captcha.Client httpAppsecClient *http.Client
log *slog.Logger cacheClient *cache.Client
captchaClient *captcha.Client
log *slog.Logger
} }
// New creates the crowdsec bouncer plugin. // New creates the crowdsec bouncer plugin.
// //
//nolint:nestif,gocyclo,gocognit //nolint:nestif,gocyclo,gocognit,funlen,maintidx
func New(_ context.Context, next http.Handler, config *configuration.Config, name string) (http.Handler, error) { func New(_ context.Context, next http.Handler, config *configuration.Config, name string) (http.Handler, error) {
config.LogLevel = strings.ToUpper(config.LogLevel) config.LogLevel = strings.ToUpper(config.LogLevel)
log := logger.NewWithFormat(config.LogLevel, config.LogFilePath, config.LogFormat) log := logger.NewWithFormat(config.LogLevel, config.LogFilePath, config.LogFormat)
if config.BanFilePath == "" && config.BanHTMLFilePath != "" {
config.BanFilePath = config.BanHTMLFilePath
}
if config.CaptchaHTMLFilePath != "" {
config.CaptchaFilePath = config.CaptchaHTMLFilePath
}
err := configuration.ValidateParams(config, log) err := configuration.ValidateParams(config, log)
if err != nil { if err != nil {
log.Error("New:validateParams " + err.Error()) log.Error("New:validateParams " + err.Error())
@@ -184,9 +193,10 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
} }
} }
var banTemplate *htmltemplate.Template var banTemplate *template.Template
if config.BanHTMLFilePath != "" { var banTemplateContentType string
banTemplate, _ = configuration.GetHTMLTemplate(config.BanHTMLFilePath) if config.BanFilePath != "" {
banTemplate, banTemplateContentType, _ = configuration.GetTemplate(config.BanFilePath)
} }
bouncer := &Bouncer{ bouncer := &Bouncer{
@@ -194,35 +204,37 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
name: name, name: name,
template: template.New("CrowdsecBouncer").Delims("[[", "]]"), template: template.New("CrowdsecBouncer").Delims("[[", "]]"),
enabled: config.Enabled, enabled: config.Enabled,
crowdsecMode: config.CrowdsecMode, crowdsecMode: config.CrowdsecMode,
appsecEnabled: config.CrowdsecAppsecEnabled, appsecEnabled: config.CrowdsecAppsecEnabled,
appsecScheme: config.CrowdsecAppsecScheme, appsecScheme: config.CrowdsecAppsecScheme,
appsecHost: config.CrowdsecAppsecHost, appsecHost: config.CrowdsecAppsecHost,
appsecPath: config.CrowdsecAppsecPath, appsecPath: config.CrowdsecAppsecPath,
appsecKey: config.CrowdsecAppsecKey, appsecKey: config.CrowdsecAppsecKey,
appsecFailureBlock: config.CrowdsecAppsecFailureBlock, appsecFailureBlock: config.CrowdsecAppsecFailureBlock,
appsecUnreachableBlock: config.CrowdsecAppsecUnreachableBlock, appsecUnreachableBlock: config.CrowdsecAppsecUnreachableBlock,
appsecBodyLimit: config.CrowdsecAppsecBodyLimit, appsecUnreadableBodyBlock: config.CrowdsecAppsecUnreadableBodyBlock,
crowdsecScheme: config.CrowdsecLapiScheme, appsecBodyLimit: config.CrowdsecAppsecBodyLimit,
crowdsecHost: config.CrowdsecLapiHost, crowdsecScheme: config.CrowdsecLapiScheme,
crowdsecPath: config.CrowdsecLapiPath, crowdsecHost: config.CrowdsecLapiHost,
crowdsecKey: config.CrowdsecLapiKey, crowdsecPath: config.CrowdsecLapiPath,
crowdsecMachineID: config.CrowdsecCapiMachineID, crowdsecKey: config.CrowdsecLapiKey,
crowdsecPassword: config.CrowdsecCapiPassword, crowdsecMachineID: config.CrowdsecCapiMachineID,
crowdsecScenarios: config.CrowdsecCapiScenarios, crowdsecPassword: config.CrowdsecCapiPassword,
updateInterval: config.UpdateIntervalSeconds, crowdsecScenarios: config.CrowdsecCapiScenarios,
updateMaxFailure: config.UpdateMaxFailure, updateInterval: config.UpdateIntervalSeconds,
remediationCustomHeader: config.RemediationHeadersCustomName, updateMaxFailure: config.UpdateMaxFailure,
forwardedCustomHeader: config.ForwardedHeadersCustomName, remediationCustomHeader: config.RemediationHeadersCustomName,
defaultDecisionTimeout: config.DefaultDecisionSeconds, forwardedCustomHeader: config.ForwardedHeadersCustomName,
remediationStatusCode: config.RemediationStatusCode, defaultDecisionTimeout: config.DefaultDecisionSeconds,
redisUnreachableBlock: config.RedisCacheUnreachableBlock, remediationStatusCode: config.RemediationStatusCode,
banTemplate: banTemplate, redisUnreachableBlock: config.RedisCacheUnreachableBlock,
traceCustomHeader: config.TraceHeadersCustomName, banTemplate: banTemplate,
crowdsecStreamRoute: crowdsecStreamRoute, banTemplateContentType: banTemplateContentType,
crowdsecHeader: crowdsecHeader, traceCustomHeader: config.TraceHeadersCustomName,
log: log, crowdsecStreamRoute: crowdsecStreamRoute,
crowdsecHeader: crowdsecHeader,
log: log,
serverPoolStrategy: &ip.PoolStrategy{ serverPoolStrategy: &ip.PoolStrategy{
Checker: serverChecker, Checker: serverChecker,
}, },
@@ -276,7 +288,7 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
config.CaptchaSiteKey, config.CaptchaSiteKey,
config.CaptchaSecretKey, config.CaptchaSecretKey,
config.RemediationHeadersCustomName, config.RemediationHeadersCustomName,
config.CaptchaHTMLFilePath, config.CaptchaFilePath,
config.CaptchaGracePeriodSeconds, config.CaptchaGracePeriodSeconds,
) )
if err != nil { if err != nil {
@@ -317,7 +329,7 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
// ServeHTTP principal function of plugin. // ServeHTTP principal function of plugin.
// //
//nolint:nestif,gocyclo //nolint:nestif
func (bouncer *Bouncer) ServeHTTP(rw http.ResponseWriter, req *http.Request) { func (bouncer *Bouncer) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
if !bouncer.enabled { if !bouncer.enabled {
bouncer.next.ServeHTTP(rw, req) bouncer.next.ServeHTTP(rw, req)
@@ -433,14 +445,9 @@ func (bouncer *Bouncer) handleBanServeHTTP(rw http.ResponseWriter, req *http.Req
if bouncer.remediationCustomHeader != "" { if bouncer.remediationCustomHeader != "" {
rw.Header().Set(bouncer.remediationCustomHeader, "ban") rw.Header().Set(bouncer.remediationCustomHeader, "ban")
} }
if bouncer.banTemplate == nil { rw.Header().Set("Content-Type", bouncer.banTemplateContentType)
rw.WriteHeader(bouncer.remediationStatusCode)
return
}
rw.Header().Set("Content-Type", "text/html; charset=utf-8")
rw.WriteHeader(bouncer.remediationStatusCode) rw.WriteHeader(bouncer.remediationStatusCode)
if bouncer.banTemplate == nil || req.Method == http.MethodHead {
if req.Method == http.MethodHead {
return return
} }
templateData := map[string]string{ templateData := map[string]string{
@@ -672,6 +679,12 @@ func handleStreamCache(bouncer *Bouncer) error {
return nil return nil
} }
func isReverseProxyError(statusCode int) bool {
return statusCode == http.StatusBadGateway ||
statusCode == http.StatusServiceUnavailable ||
statusCode == http.StatusGatewayTimeout
}
func crowdsecQuery(bouncer *Bouncer, stringURL string, data []byte) ([]byte, error) { func crowdsecQuery(bouncer *Bouncer, stringURL string, data []byte) ([]byte, error) {
var req *http.Request var req *http.Request
if len(data) > 0 { if len(data) > 0 {
@@ -683,7 +696,7 @@ func crowdsecQuery(bouncer *Bouncer, stringURL string, data []byte) ([]byte, err
req.Header.Set("User-Agent", "Crowdsec-Bouncer-Traefik-Plugin/"+pluginVersion) req.Header.Set("User-Agent", "Crowdsec-Bouncer-Traefik-Plugin/"+pluginVersion)
res, err := bouncer.httpClient.Do(req) res, err := bouncer.httpClient.Do(req)
if err != nil { if err != nil || isReverseProxyError(res.StatusCode) {
return nil, fmt.Errorf("crowdsecQuery:unreachable url:%s %w", stringURL, err) return nil, fmt.Errorf("crowdsecQuery:unreachable url:%s %w", stringURL, err)
} }
defer func() { defer func() {
@@ -711,6 +724,17 @@ func crowdsecQuery(bouncer *Bouncer, stringURL string, data []byte) ([]byte, err
return body, nil return body, nil
} }
// isBodyUnreadable reports whether the request body cannot be buffered before
// forwarding it to the Appsec component. An HTTP/2 or HTTP/3 request without a
// Content-Length (typically a bidirectional gRPC stream) keeps its body open
// for the whole life of the stream and never reaches EOF, so reading it with
// io.ReadAll would block until the request times out and is wrongly turned into
// a 403. This mirrors the reference lua-cs-bouncer behavior, which refuses to
// read the body of an HTTP/2+ request that has no Content-Length.
func isBodyUnreadable(httpReq *http.Request) bool {
return httpReq.Body != nil && httpReq.ProtoMajor >= 2 && httpReq.ContentLength < 0
}
func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error { func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
routeURL := url.URL{ routeURL := url.URL{
Scheme: bouncer.appsecScheme, Scheme: bouncer.appsecScheme,
@@ -718,7 +742,14 @@ func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
Path: bouncer.appsecPath, Path: bouncer.appsecPath,
} }
var req *http.Request var req *http.Request
if bouncer.appsecBodyLimit > 0 && httpReq.Body != nil { switch {
case isBodyUnreadable(httpReq):
if bouncer.appsecUnreadableBodyBlock {
// The caller (handleNextServeHTTP) logs this returned error with the IP.
return errors.New("appsecQuery:unreadableBody dropped")
}
req, _ = http.NewRequest(http.MethodGet, routeURL.String(), nil)
case bouncer.appsecBodyLimit > 0 && httpReq.Body != nil:
var bodyBuffer bytes.Buffer var bodyBuffer bytes.Buffer
limitedReader := io.LimitReader(httpReq.Body, bouncer.appsecBodyLimit) limitedReader := io.LimitReader(httpReq.Body, bouncer.appsecBodyLimit)
teeReader := io.TeeReader(limitedReader, &bodyBuffer) teeReader := io.TeeReader(limitedReader, &bodyBuffer)
@@ -729,7 +760,7 @@ func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
// Conserve body intact after reading it for other middlewares and service // Conserve body intact after reading it for other middlewares and service
httpReq.Body = io.NopCloser(io.MultiReader(&bodyBuffer, httpReq.Body)) httpReq.Body = io.NopCloser(io.MultiReader(&bodyBuffer, httpReq.Body))
req, _ = http.NewRequest(http.MethodPost, routeURL.String(), bytes.NewBuffer(bodyBytes)) req, _ = http.NewRequest(http.MethodPost, routeURL.String(), bytes.NewBuffer(bodyBytes))
} else { default:
req, _ = http.NewRequest(http.MethodGet, routeURL.String(), nil) req, _ = http.NewRequest(http.MethodGet, routeURL.String(), nil)
} }
@@ -747,7 +778,7 @@ func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
req.Header.Set("User-Agent", "Crowdsec-Bouncer-Traefik-Plugin/"+pluginVersion) req.Header.Set("User-Agent", "Crowdsec-Bouncer-Traefik-Plugin/"+pluginVersion)
res, err := bouncer.httpAppsecClient.Do(req) res, err := bouncer.httpAppsecClient.Do(req)
if err != nil { if err != nil || isReverseProxyError(res.StatusCode) {
bouncer.log.Error("appsecQuery:unreachable") bouncer.log.Error("appsecQuery:unreachable")
if bouncer.appsecUnreachableBlock { if bouncer.appsecUnreachableBlock {
return fmt.Errorf("appsecQuery:unreachable %w", err) return fmt.Errorf("appsecQuery:unreachable %w", err)
+2 -2
View File
@@ -32,13 +32,13 @@ func getTestConfig() *configuration.Config {
ForwardedHeadersTrustedIPs: []string{"127.0.0.1"}, ForwardedHeadersTrustedIPs: []string{"127.0.0.1"},
ForwardedHeadersCustomName: "", ForwardedHeadersCustomName: "",
RemediationStatusCode: 403, RemediationStatusCode: 403,
BanHTMLFilePath: "", BanFilePath: "",
RemediationHeadersCustomName: "", RemediationHeadersCustomName: "",
CaptchaProvider: "", CaptchaProvider: "",
CaptchaSiteKey: "", CaptchaSiteKey: "",
CaptchaSecretKey: "", CaptchaSecretKey: "",
CaptchaGracePeriodSeconds: 1, CaptchaGracePeriodSeconds: 1,
CaptchaHTMLFilePath: "", CaptchaFilePath: "",
RedisCacheEnabled: false, RedisCacheEnabled: false,
RedisCacheHost: "", RedisCacheHost: "",
RedisCachePassword: "", RedisCachePassword: "",
+184 -3
View File
@@ -2,16 +2,19 @@ package crowdsec_bouncer_traefik_plugin //nolint:revive,stylecheck
import ( import (
"context" "context"
htmltemplate "html/template" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url"
"reflect" "reflect"
"testing" "testing"
"text/template" "text/template"
"time"
cache "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/cache" cache "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/cache"
configuration "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration" configuration "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration"
ip "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip" ip "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip"
logger "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger"
) )
func TestServeHTTP(t *testing.T) { func TestServeHTTP(t *testing.T) {
@@ -190,11 +193,11 @@ func Test_crowdsecQuery(t *testing.T) {
func TestHandleBanServeHTTPWithDifferentMethods(t *testing.T) { func TestHandleBanServeHTTPWithDifferentMethods(t *testing.T) {
html := "<html>You are banned</html>" html := "<html>You are banned</html>"
banTemplate, _ := htmltemplate.New("html").Parse(html) banTemplate, _ := template.New("html").Delims("{{", "}}").Parse(html)
tests := []struct { tests := []struct {
name string name string
method string method string
banTemplate *htmltemplate.Template banTemplate *template.Template
expectBodyContent bool expectBodyContent bool
}{ }{
{ {
@@ -235,6 +238,7 @@ func TestHandleBanServeHTTPWithDifferentMethods(t *testing.T) {
remediationStatusCode: http.StatusForbidden, remediationStatusCode: http.StatusForbidden,
remediationCustomHeader: "X-Test-Remediation", remediationCustomHeader: "X-Test-Remediation",
banTemplate: tt.banTemplate, banTemplate: tt.banTemplate,
banTemplateContentType: "text/html; charset=utf-8",
} }
rw := httptest.NewRecorder() rw := httptest.NewRecorder()
@@ -269,6 +273,50 @@ func TestHandleBanServeHTTPWithDifferentMethods(t *testing.T) {
} }
} }
func TestHandleBanServeHTTPContentType(t *testing.T) {
html := "<html>You are banned</html>"
banTemplate, _ := template.New("html").Delims("{{", "}}").Parse(html)
tests := []struct {
name string
banTemplate *template.Template
banTemplateContentType string
}{
{
name: "Default HTML content type",
banTemplate: banTemplate,
banTemplateContentType: "text/html; charset=utf-8",
},
{
name: "Custom JSON content type",
banTemplate: banTemplate,
banTemplateContentType: "application/json",
},
{
name: "Content type set even when banTemplate is nil",
banTemplate: nil,
banTemplateContentType: "application/json",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
bouncer := &Bouncer{
remediationStatusCode: http.StatusForbidden,
banTemplate: tt.banTemplate,
banTemplateContentType: tt.banTemplateContentType,
}
rw := httptest.NewRecorder()
req := &http.Request{Method: http.MethodGet}
bouncer.handleBanServeHTTP(rw, req, "0.0.0.0", "TEST")
if got := rw.Header().Get("Content-Type"); got != tt.banTemplateContentType {
t.Errorf("Expected Content-Type %q, got %q", tt.banTemplateContentType, got)
}
})
}
}
func TestCaptchaMethodBasedLogic(t *testing.T) { func TestCaptchaMethodBasedLogic(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -332,3 +380,136 @@ func TestCaptchaMethodBasedLogic(t *testing.T) {
}) })
} }
} }
// blockingBody simulates a request body that never reaches EOF, like a
// bidirectional gRPC stream that keeps its body open for the whole life of
// the connection. Reading from it blocks until the test is done.
type blockingBody struct {
done <-chan struct{}
}
func (b blockingBody) Read(_ []byte) (int, error) {
<-b.done
return 0, io.EOF
}
func (blockingBody) Close() error { return nil }
func Test_isBodyUnreadable(t *testing.T) {
tests := []struct {
name string
protoMajor int
contentLength int64
hasBody bool
want bool
}{
{name: "http2 grpc stream without content-length", protoMajor: 2, contentLength: -1, hasBody: true, want: true},
{name: "http3 stream without content-length", protoMajor: 3, contentLength: -1, hasBody: true, want: true},
{name: "http2 with content-length", protoMajor: 2, contentLength: 42, hasBody: true, want: false},
{name: "http1.1 chunked without content-length", protoMajor: 1, contentLength: -1, hasBody: true, want: false},
{name: "http2 without body", protoMajor: 2, contentLength: -1, hasBody: false, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, "http://localhost", nil)
req.ProtoMajor = tt.protoMajor
req.ContentLength = tt.contentLength
if tt.hasBody {
req.Body = http.NoBody
} else {
req.Body = nil
}
if got := isBodyUnreadable(req); got != tt.want {
t.Errorf("isBodyUnreadable() = %v, want %v", got, tt.want)
}
})
}
}
// newStreamingRequest builds an HTTP/2 request whose body never reaches EOF,
// like a bidirectional gRPC stream (issue #323).
func newStreamingRequest(done <-chan struct{}) *http.Request {
req, _ := http.NewRequest(http.MethodPost, "http://localhost/signalexchange.SignalExchange/ConnectStream", blockingBody{done: done})
req.Header.Set("Content-Type", "application/grpc")
req.ProtoMajor = 2
req.ContentLength = -1
return req
}
// Test_appsecQuery_streamingDoesNotBlock is a regression test for issue #323:
// a gRPC streaming request whose body never reaches EOF must not be buffered
// (io.ReadAll would block until timeout and wrongly produce a 403). The appsec
// query must complete promptly, inspecting headers only.
func Test_appsecQuery_streamingDoesNotBlock(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreachableBlock: true,
appsecFailureBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
finished <- appsecQuery(bouncer, "1.2.3.4", newStreamingRequest(done))
}()
select {
case err := <-finished:
if err != nil {
t.Errorf("appsecQuery() on streaming request returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("appsecQuery() blocked on a streaming request body (issue #323 regression)")
}
}
// Test_appsecQuery_dropUnreadableBody verifies that, when configured to do so,
// a request with an unreadable body is dropped (blocked) instead of forwarded
// without its body, mirroring the reference APPSEC_DROP_UNREADABLE_BODY option.
func Test_appsecQuery_dropUnreadableBody(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreadableBodyBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
finished <- appsecQuery(bouncer, "1.2.3.4", newStreamingRequest(done))
}()
select {
case err := <-finished:
if err == nil {
t.Error("appsecQuery() expected an error to block the request, got nil")
}
case <-time.After(2 * time.Second):
t.Fatal("appsecQuery() blocked on a streaming request body (issue #323 regression)")
}
}
+6 -6
View File
@@ -100,9 +100,9 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the user IP has a crowdsec decision ? TraefikPlugin-->>PluginCache: Does the user IP has a crowdsec decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Nothing, all good! PluginCache-->>TraefikPlugin: Nothing, all good!
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -121,12 +121,12 @@ sequenceDiagram
User->>TraefikPlugin: Fine, done! User->>TraefikPlugin: Fine, done!
create participant ProviderCaptcha create participant ProviderCaptcha
TraefikPlugin-->>ProviderCaptcha: Is the validation OK ? TraefikPlugin-->>ProviderCaptcha: Is the validation OK ?
Destroy ProviderCaptcha destroy ProviderCaptcha
ProviderCaptcha-->>TraefikPlugin: Yes ProviderCaptcha-->>TraefikPlugin: Yes
TraefikPlugin-->>PluginCache: Set the User IP Clean for captchaGracePeriodSeconds TraefikPlugin-->>PluginCache: Set the User IP Clean for captchaGracePeriodSeconds
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Done PluginCache-->>TraefikPlugin: Done
Destroy TraefikPlugin destroy TraefikPlugin
TraefikPlugin->>Webserver: Forwarding this HTTP Request from User TraefikPlugin->>Webserver: Forwarding this HTTP Request from User
Webserver->>User: HTTP Response Webserver->>User: HTTP Response
``` ```
@@ -140,7 +140,7 @@ sequenceDiagram
User->>TraefikPlugin: Can I access that webpage User->>TraefikPlugin: Can I access that webpage
create participant PluginCache create participant PluginCache
TraefikPlugin-->>PluginCache: Does the User IP has a Crowdsec Decision ? TraefikPlugin-->>PluginCache: Does the User IP has a Crowdsec Decision ?
Destroy PluginCache destroy PluginCache
PluginCache-->>TraefikPlugin: Yes a ban Decision PluginCache-->>TraefikPlugin: Yes a ban Decision
TraefikPlugin->>User: No, HTTP 403 TraefikPlugin->>User: No, HTTP 403
``` ```
+3 -3
View File
@@ -9,11 +9,11 @@ This can be usefull as some browser (Firefox for instance) return a 403 blank we
```yaml ```yaml
labels: labels:
# Define ban HTML file path # Define ban file path
- "traefik.http.middlewares.crowdsec.plugin.bouncer.banHtmlFilePath=/ban.html" - "traefik.http.middlewares.crowdsec.plugin.bouncer.banFilePath=/ban.html"
``` ```
The ban HTML file must be present in the Traefik container (bind mounted or added during a custom build). The ban file must be present in the Traefik container (bind mounted or added during a custom build).
It is not directly accessible from Traefik even when importing the plugin, so [download](https://raw.githubusercontent.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/master/ban.html) it locally to expose it to Traefik. It is not directly accessible from Traefik even when importing the plugin, so [download](https://raw.githubusercontent.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/master/ban.html) it locally to expose it to Traefik.
```yaml ```yaml
+2 -2
View File
@@ -42,8 +42,8 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.enabled=true" - "traefik.http.middlewares.crowdsec.plugin.bouncer.enabled=true"
- "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapikey=40796d93c2958f9e58345514e67740e5" - "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapikey=40796d93c2958f9e58345514e67740e5"
- "traefik.http.middlewares.crowdsec.plugin.bouncer.loglevel=DEBUG" - "traefik.http.middlewares.crowdsec.plugin.bouncer.loglevel=DEBUG"
# Define ban HTML file path # Define ban file path
- "traefik.http.middlewares.crowdsec.plugin.bouncer.banHtmlFilePath=/ban.html" - "traefik.http.middlewares.crowdsec.plugin.bouncer.banFilePath=/ban.html"
crowdsec: crowdsec:
image: crowdsecurity/crowdsec:v1.6.1-2 image: crowdsecurity/crowdsec:v1.6.1-2
+8 -6
View File
@@ -4,11 +4,11 @@ package captcha
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"html/template"
"log/slog" "log/slog"
"net/http" "net/http"
"net/url" "net/url"
"strings" "strings"
"text/template"
cache "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/cache" cache "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/cache"
configuration "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration" configuration "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration"
@@ -21,7 +21,8 @@ type Client struct {
secretKey string secretKey string
remediationCustomHeader string remediationCustomHeader string
gracePeriodSeconds int64 gracePeriodSeconds int64
captchaTemplate *template.Template templateContentType string
template *template.Template
cacheClient *cache.Client cacheClient *cache.Client
httpClient *http.Client httpClient *http.Client
log *slog.Logger log *slog.Logger
@@ -74,8 +75,9 @@ func (c *Client) New(log *slog.Logger, cacheClient *cache.Client, httpClient *ht
c.siteKey = siteKey c.siteKey = siteKey
c.secretKey = secretKey c.secretKey = secretKey
c.remediationCustomHeader = remediationCustomHeader c.remediationCustomHeader = remediationCustomHeader
html, _ := configuration.GetHTMLTemplate(captchaTemplatePath) template, contentType, _ := configuration.GetTemplate(captchaTemplatePath)
c.captchaTemplate = html c.template = template
c.templateContentType = contentType
c.gracePeriodSeconds = gracePeriodSeconds c.gracePeriodSeconds = gracePeriodSeconds
c.log = log c.log = log
c.httpClient = httpClient c.httpClient = httpClient
@@ -100,12 +102,12 @@ func (c *Client) ServeHTTP(rw http.ResponseWriter, r *http.Request, remoteIP str
http.Redirect(rw, r, r.URL.String(), http.StatusFound) http.Redirect(rw, r, r.URL.String(), http.StatusFound)
return return
} }
rw.Header().Set("Content-Type", "text/html; charset=utf-8") rw.Header().Set("Content-Type", c.templateContentType)
if c.remediationCustomHeader != "" { if c.remediationCustomHeader != "" {
rw.Header().Set(c.remediationCustomHeader, "captcha") rw.Header().Set(c.remediationCustomHeader, "captcha")
} }
rw.WriteHeader(http.StatusOK) rw.WriteHeader(http.StatusOK)
err = c.captchaTemplate.Execute(rw, map[string]string{ err = c.template.Execute(rw, map[string]string{
"SiteKey": c.siteKey, "SiteKey": c.siteKey,
"FrontendJS": c.infoProvider.js, "FrontendJS": c.infoProvider.js,
"FrontendKey": c.infoProvider.key, "FrontendKey": c.infoProvider.key,
+100 -69
View File
@@ -6,7 +6,6 @@ import (
"crypto/x509" "crypto/x509"
"errors" "errors"
"fmt" "fmt"
"html/template"
"log/slog" "log/slog"
"net/http" "net/http"
"net/url" "net/url"
@@ -15,6 +14,7 @@ import (
"reflect" "reflect"
"regexp" "regexp"
"strings" "strings"
"text/template"
ip "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip" ip "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip"
) )
@@ -63,6 +63,7 @@ type Config struct {
CrowdsecAppsecTLSCertificateBouncerKeyFile string `json:"crowdsecAppsecTlsCertificateBouncerKeyFile,omitempty"` CrowdsecAppsecTLSCertificateBouncerKeyFile string `json:"crowdsecAppsecTlsCertificateBouncerKeyFile,omitempty"`
CrowdsecAppsecFailureBlock bool `json:"crowdsecAppsecFailureBlock,omitempty"` CrowdsecAppsecFailureBlock bool `json:"crowdsecAppsecFailureBlock,omitempty"`
CrowdsecAppsecUnreachableBlock bool `json:"crowdsecAppsecUnreachableBlock,omitempty"` CrowdsecAppsecUnreachableBlock bool `json:"crowdsecAppsecUnreachableBlock,omitempty"`
CrowdsecAppsecUnreadableBodyBlock bool `json:"crowdsecAppsecUnreadableBodyBlock,omitempty"`
CrowdsecAppsecBodyLimit int64 `json:"crowdsecAppsecBodyLimit,omitempty"` CrowdsecAppsecBodyLimit int64 `json:"crowdsecAppsecBodyLimit,omitempty"`
CrowdsecLapiScheme string `json:"crowdsecLapiScheme,omitempty"` CrowdsecLapiScheme string `json:"crowdsecLapiScheme,omitempty"`
CrowdsecLapiHost string `json:"crowdsecLapiHost,omitempty"` CrowdsecLapiHost string `json:"crowdsecLapiHost,omitempty"`
@@ -99,8 +100,10 @@ type Config struct {
RedisCachePasswordFile string `json:"redisCachePasswordFile,omitempty"` RedisCachePasswordFile string `json:"redisCachePasswordFile,omitempty"`
RedisCacheDatabase string `json:"redisCacheDatabase,omitempty"` RedisCacheDatabase string `json:"redisCacheDatabase,omitempty"`
RedisCacheUnreachableBlock bool `json:"redisCacheUnreachableBlock,omitempty"` RedisCacheUnreachableBlock bool `json:"redisCacheUnreachableBlock,omitempty"`
BanHTMLFilePath string `json:"banHtmlFilePath,omitempty"` BanHTMLFilePath string `json:"banHtmlFilePath,omitempty"` // Deprecated: Keep it for historical compatibility
CaptchaHTMLFilePath string `json:"captchaHtmlFilePath,omitempty"` BanFilePath string `json:"banFilePath,omitempty"`
CaptchaHTMLFilePath string `json:"captchaHtmlFilePath,omitempty"` // Deprecated: Keep it for historical compatibility
CaptchaFilePath string `json:"captchaFilePath,omitempty"`
CaptchaProvider string `json:"captchaProvider,omitempty"` CaptchaProvider string `json:"captchaProvider,omitempty"`
CaptchaCustomJsURL string `json:"captchaCustomJsUrl,omitempty"` CaptchaCustomJsURL string `json:"captchaCustomJsUrl,omitempty"`
CaptchaCustomValidateURL string `json:"captchaCustomValidateUrl,omitempty"` CaptchaCustomValidateURL string `json:"captchaCustomValidateUrl,omitempty"`
@@ -125,52 +128,53 @@ func contains(source []string, target string) bool {
// New creates the default plugin configuration. // New creates the default plugin configuration.
func New() *Config { func New() *Config {
return &Config{ return &Config{
Enabled: false, Enabled: false,
LogLevel: LogINFO, LogLevel: LogINFO,
LogFormat: "common", LogFormat: "common",
LogFilePath: "", LogFilePath: "",
CrowdsecMode: LiveMode, CrowdsecMode: LiveMode,
CrowdsecAppsecEnabled: false, CrowdsecAppsecEnabled: false,
CrowdsecAppsecFailureBlock: true, CrowdsecAppsecFailureBlock: true,
CrowdsecAppsecUnreachableBlock: true, CrowdsecAppsecUnreachableBlock: true,
CrowdsecAppsecBodyLimit: 10485760, CrowdsecAppsecUnreadableBodyBlock: true,
CrowdsecAppsecScheme: "", CrowdsecAppsecBodyLimit: 10485760,
CrowdsecAppsecHost: "crowdsec:7422", CrowdsecAppsecScheme: "",
CrowdsecAppsecPath: "/", CrowdsecAppsecHost: "crowdsec:7422",
CrowdsecAppsecKey: "", CrowdsecAppsecPath: "/",
CrowdsecAppsecTLSInsecureVerify: false, CrowdsecAppsecKey: "",
CrowdsecLapiScheme: HTTP, CrowdsecAppsecTLSInsecureVerify: false,
CrowdsecLapiHost: "crowdsec:8080", CrowdsecLapiScheme: HTTP,
CrowdsecLapiPath: "/", CrowdsecLapiHost: "crowdsec:8080",
CrowdsecLapiKey: "", CrowdsecLapiPath: "/",
CrowdsecLapiTLSInsecureVerify: false, CrowdsecLapiKey: "",
UpdateIntervalSeconds: 60, CrowdsecLapiTLSInsecureVerify: false,
MetricsUpdateIntervalSeconds: 600, UpdateIntervalSeconds: 60,
UpdateMaxFailure: 0, MetricsUpdateIntervalSeconds: 600,
StreamStartupBlock: true, UpdateMaxFailure: 0,
DefaultDecisionSeconds: 60, StreamStartupBlock: true,
RemediationStatusCode: http.StatusForbidden, DefaultDecisionSeconds: 60,
HTTPTimeoutSeconds: 10, RemediationStatusCode: http.StatusForbidden,
CaptchaProvider: "", HTTPTimeoutSeconds: 10,
CaptchaCustomJsURL: "", CaptchaProvider: "",
CaptchaCustomValidateURL: "", CaptchaCustomJsURL: "",
CaptchaCustomKey: "", CaptchaCustomValidateURL: "",
CaptchaCustomResponse: "", CaptchaCustomKey: "",
CaptchaSiteKey: "", CaptchaCustomResponse: "",
CaptchaSecretKey: "", CaptchaSiteKey: "",
CaptchaGracePeriodSeconds: 1800, CaptchaSecretKey: "",
CaptchaHTMLFilePath: "/captcha.html", CaptchaGracePeriodSeconds: 1800,
BanHTMLFilePath: "", CaptchaFilePath: "/captcha.html",
TraceHeadersCustomName: "", BanFilePath: "",
RemediationHeadersCustomName: "", TraceHeadersCustomName: "",
ForwardedHeadersCustomName: "X-Forwarded-For", RemediationHeadersCustomName: "",
ForwardedHeadersTrustedIPs: []string{}, ForwardedHeadersCustomName: "X-Forwarded-For",
ClientTrustedIPs: []string{}, ForwardedHeadersTrustedIPs: []string{},
RedisCacheEnabled: false, ClientTrustedIPs: []string{},
RedisCacheHost: "redis:6379", RedisCacheEnabled: false,
RedisCachePassword: "", RedisCacheHost: "redis:6379",
RedisCacheDatabase: "", RedisCachePassword: "",
RedisCacheUnreachableBlock: true, RedisCacheDatabase: "",
RedisCacheUnreachableBlock: true,
} }
} }
@@ -201,28 +205,50 @@ func GetVariable(config *Config, key string) (string, error) {
return strings.TrimSpace(value), nil return strings.TrimSpace(value), nil
} }
// GetHTMLTemplate get compiled HTML template. func getContentTypeFromPath(path string) string {
func GetHTMLTemplate(path string) (*template.Template, error) {
var err error
if path == "" { if path == "" {
return nil, errors.New("no html template provided") return ""
} }
ext := strings.ToLower(filepath.Ext(path))
contentTypeMap := map[string]string{
".html": "text/html; charset=utf-8",
".htm": "text/html; charset=utf-8",
".json": "application/json",
".txt": "text/plain",
".xml": "application/xml",
".js": "application/javascript",
".css": "text/css",
}
if contentType, ok := contentTypeMap[ext]; ok {
return contentType
}
// Default to HTML for backward compatibility
return "text/html; charset=utf-8"
}
// GetTemplate get compiled template with {{ and }} delimiters.
// Uses text/template for all file types to avoid HTML escaping issues.
func GetTemplate(path string) (*template.Template, string, error) {
if path == "" {
return nil, "", errors.New("no template file provided")
}
contentType := getContentTypeFromPath(path)
//nolint:gosec //nolint:gosec
b, err := os.ReadFile(path) b, err := os.ReadFile(path)
if err != nil { if err != nil {
return nil, err return nil, "", err
} }
html := string(b) content := string(b)
compiledTemplate, err := template.New("html").Parse(html) compiledTemplate, err := template.New(filepath.Base(path)).Delims("{{", "}}").Parse(content)
if err != nil { if err != nil {
return nil, fmt.Errorf("impossible to compile html template: %w", err) return nil, "", fmt.Errorf("impossible to compile template %s: %w", path, err)
} }
return compiledTemplate, nil return compiledTemplate, contentType, nil
} }
// ValidateParams validate all the param gave by user. // ValidateParams validate all the param gave by user.
// //
//nolint:gocyclo,gocognit //nolint:gocyclo,gocognit,nestif
func ValidateParams(config *Config, log *slog.Logger) error { func ValidateParams(config *Config, log *slog.Logger) error {
if err := validateParamsRequired(config); err != nil { if err := validateParamsRequired(config); err != nil {
return err return err
@@ -260,12 +286,14 @@ func ValidateParams(config *Config, log *slog.Logger) error {
if _, err := GetVariable(config, "CaptchaSecretKey"); err != nil { if _, err := GetVariable(config, "CaptchaSecretKey"); err != nil {
return err return err
} }
if _, err := GetHTMLTemplate(config.CaptchaHTMLFilePath); err != nil { if config.CaptchaFilePath != "" {
return err if _, _, err := GetTemplate(config.CaptchaFilePath); err != nil {
return err
}
} }
} }
if config.BanHTMLFilePath != "" { if config.BanFilePath != "" {
if _, err := GetHTMLTemplate(config.BanHTMLFilePath); err != nil { if _, _, err := GetTemplate(config.BanFilePath); err != nil {
return err return err
} }
} }
@@ -361,7 +389,8 @@ func validateParamsTLS(config *Config) error {
return err return err
} }
if certAuth == "" { if certAuth == "" {
return errors.New("CrowdsecLapiTLSCertificateAuthority must be specified when CrowdsecLapiScheme='https' and CrowdsecLapiTLSInsecureVerify=false") // No custom CA — runtime will fall back to the system trust store.
return nil
} }
tlsConfig := new(tls.Config) tlsConfig := new(tls.Config)
tlsConfig.RootCAs = x509.NewCertPool() tlsConfig.RootCAs = x509.NewCertPool()
@@ -453,29 +482,31 @@ func validateParamsRequired(config *Config) error {
func getTLSConfig(config *Config, log *slog.Logger, prefix, scheme string, insecureVerify bool) (*tls.Config, error) { func getTLSConfig(config *Config, log *slog.Logger, prefix, scheme string, insecureVerify bool) (*tls.Config, error) {
tlsConfig := new(tls.Config) tlsConfig := new(tls.Config)
tlsConfig.RootCAs = x509.NewCertPool()
if scheme != HTTPS { if scheme != HTTPS {
log.Debug("getTLSConfig:" + prefix + "Scheme https:no") log.Debug("getTLSConfig:" + prefix + "Scheme https:no")
return tlsConfig, nil return tlsConfig, nil
} }
// RootCAs is intentionally left nil unless a custom CA is provided:
// crypto/tls then falls back to x509.SystemCertPool(), which is what we
// want when the LAPI is exposed behind a reverse proxy with a publicly
// trusted certificate (e.g. Let's Encrypt).
//nolint:nestif //nolint:nestif
if insecureVerify { if insecureVerify {
tlsConfig.InsecureSkipVerify = true tlsConfig.InsecureSkipVerify = true
log.Debug("getTLSConfig:" + prefix + "TLSInsecureVerify tlsInsecure:true") log.Debug("getTLSConfig:" + prefix + "TLSInsecureVerify tlsInsecure:true")
// If we return here and still want to use client auth this won't work
// return tlsConfig, nil
} else { } else {
certAuthority, err := GetVariable(config, prefix+"TLSCertificateAuthority") certAuthority, err := GetVariable(config, prefix+"TLSCertificateAuthority")
if err != nil { if err != nil {
return nil, err return nil, err
} }
if len(certAuthority) > 0 { if len(certAuthority) > 0 {
tlsConfig.RootCAs = x509.NewCertPool()
if !tlsConfig.RootCAs.AppendCertsFromPEM([]byte(certAuthority)) { if !tlsConfig.RootCAs.AppendCertsFromPEM([]byte(certAuthority)) {
// here we return because if CrowdsecLapiTLSInsecureVerify is false
// and CA not load, we can't communicate with https
return nil, errors.New("getTLSConfig:" + prefix + " cannot load CA and verify cert is enabled") return nil, errors.New("getTLSConfig:" + prefix + " cannot load CA and verify cert is enabled")
} }
log.Debug("getTLSConfig:" + prefix + "TLSCertificateAuthority CA added successfully") log.Debug("getTLSConfig:" + prefix + "TLSCertificateAuthority CA added successfully")
} else {
log.Debug("getTLSConfig:" + prefix + " no CA provided, using system trust store")
} }
} }
certBouncer, err := GetVariable(config, prefix+"TLSCertificateBouncer") certBouncer, err := GetVariable(config, prefix+"TLSCertificateBouncer")
+90 -21
View File
@@ -1,13 +1,25 @@
package configuration package configuration
import ( import (
"crypto/tls"
"reflect"
"testing" "testing"
logger "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger" logger "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger"
) )
// validPEM is a minimal self-signed certificate accepted by AppendCertsFromPEM,
// shared by the TLS tests below.
const validPEM = `-----BEGIN CERTIFICATE-----
MIIBhTCCASugAwIBAgIQIRi6zePL6mKjOipn+dNuaTAKBggqhkjOPQQDAjASMRAw
DgYDVQQKEwdBY21lIENvMB4XDTE3MTAyMDE5NDMwNloXDTE4MTAyMDE5NDMwNlow
EjEQMA4GA1UEChMHQWNtZSBDbzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABD0d
7VNhbWvZLWPuj/RtHFjvtJBEwOkhbN/BnnE8rnZR8+sbwnc/KhCk3FhnpHZnQz7B
5aETbbIgmuvewdjvSBSjYzBhMA4GA1UdDwEB/wQEAwICpDATBgNVHSUEDDAKBggr
BgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MCkGA1UdEQQiMCCCDmxvY2FsaG9zdDo1
NDUzgg4xMjcuMC4wLjE6NTQ1MzAKBggqhkjOPQQDAgNIADBFAiEA2zpJEPQyz6/l
Wf86aX6PepsntZv2GYlA5UpabfT2EZICICpJ5h/iI+i341gBmLiAFQOyTDT+/wQc
6MF9+Yw1Yy0t
-----END CERTIFICATE-----`
func getMinimalConfig() *Config { func getMinimalConfig() *Config {
cfg := New() cfg := New()
cfg.CrowdsecLapiKey = "test" cfg.CrowdsecLapiKey = "test"
@@ -111,7 +123,7 @@ func Test_ValidateParams(t *testing.T) {
{name: "Not validate a bad clients ips", args: args{config: cfg5}, wantErr: true}, {name: "Not validate a bad clients ips", args: args{config: cfg5}, wantErr: true},
// HTTPS enabled // HTTPS enabled
{name: "Validate https config with insecure verify", args: args{config: cfg6}, wantErr: false}, {name: "Validate https config with insecure verify", args: args{config: cfg6}, wantErr: false},
{name: "Not validate https without cert authority", args: args{config: cfg7}, wantErr: true}, {name: "Validate https without cert authority (falls back to system trust store)", args: args{config: cfg7}, wantErr: false},
{name: "Valid log level uppercase INFO", args: args{config: cfg8}, wantErr: false}, {name: "Valid log level uppercase INFO", args: args{config: cfg8}, wantErr: false},
{name: "Valid log level lowercase info", args: args{config: cfg9}, wantErr: false}, {name: "Valid log level lowercase info", args: args{config: cfg9}, wantErr: false},
{name: "Invalid log level Warning", args: args{config: cfg10}, wantErr: true}, {name: "Invalid log level Warning", args: args{config: cfg10}, wantErr: true},
@@ -126,19 +138,24 @@ func Test_ValidateParams(t *testing.T) {
} }
func Test_validateParamsTLS(t *testing.T) { func Test_validateParamsTLS(t *testing.T) {
type args struct { cfgEmpty := getMinimalConfig()
config *Config cfgValid := getMinimalConfig()
} cfgValid.CrowdsecLapiTLSCertificateAuthority = validPEM
cfgInvalidCA := getMinimalConfig()
cfgInvalidCA.CrowdsecLapiTLSCertificateAuthority = "not a pem"
tests := []struct { tests := []struct {
name string name string
args args config *Config
wantErr bool wantErr bool
}{ }{
// TODO: Add test cases. {name: "Empty CA is accepted (system trust store used at runtime)", config: cfgEmpty, wantErr: false},
{name: "Valid PEM CA is accepted", config: cfgValid, wantErr: false},
{name: "Invalid CA is rejected", config: cfgInvalidCA, wantErr: true},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
if err := validateParamsTLS(tt.args.config); (err != nil) != tt.wantErr { if err := validateParamsTLS(tt.config); (err != nil) != tt.wantErr {
t.Errorf("validateParamsTLS() error = %v, wantErr %v", err, tt.wantErr) t.Errorf("validateParamsTLS() error = %v, wantErr %v", err, tt.wantErr)
} }
}) })
@@ -233,26 +250,78 @@ func Test_validateParamsAPIKey(t *testing.T) {
func Test_GetTLSConfigCrowdsec(t *testing.T) { func Test_GetTLSConfigCrowdsec(t *testing.T) {
log := logger.New("INFO", "") log := logger.New("INFO", "")
type args struct {
config *Config httpCfg := getMinimalConfig()
} httpCfg.CrowdsecLapiScheme = HTTP
httpsSystemCA := getMinimalConfig()
httpsSystemCA.CrowdsecLapiScheme = HTTPS
httpsCustomCA := getMinimalConfig()
httpsCustomCA.CrowdsecLapiScheme = HTTPS
httpsCustomCA.CrowdsecLapiTLSCertificateAuthority = validPEM
httpsInsecure := getMinimalConfig()
httpsInsecure.CrowdsecLapiScheme = HTTPS
httpsInsecure.CrowdsecLapiTLSInsecureVerify = true
httpsBadCA := getMinimalConfig()
httpsBadCA.CrowdsecLapiScheme = HTTPS
httpsBadCA.CrowdsecLapiTLSCertificateAuthority = "not a pem"
tests := []struct { tests := []struct {
name string name string
args args config *Config
want *tls.Config wantErr bool
wantErr bool wantRootCAsNil bool
wantInsecureSkip bool
}{ }{
// TODO: Add test cases. {name: "HTTP scheme returns empty tls.Config", config: httpCfg, wantRootCAsNil: true},
{name: "HTTPS without CA leaves RootCAs nil (system trust store)", config: httpsSystemCA, wantRootCAsNil: true},
{name: "HTTPS with custom CA populates RootCAs", config: httpsCustomCA, wantRootCAsNil: false},
{name: "HTTPS with insecure verify sets InsecureSkipVerify", config: httpsInsecure, wantRootCAsNil: true, wantInsecureSkip: true},
{name: "HTTPS with garbage CA is rejected", config: httpsBadCA, wantErr: true},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
got, err := GetTLSConfigCrowdsec(tt.args.config, log, false) got, err := GetTLSConfigCrowdsec(tt.config, log, false)
if (err != nil) != tt.wantErr { if (err != nil) != tt.wantErr {
t.Errorf("getTLSConfigCrowdsec() error = %v, wantErr %v", err, tt.wantErr) t.Errorf("GetTLSConfigCrowdsec() error = %v, wantErr %v", err, tt.wantErr)
return return
} }
if !reflect.DeepEqual(got, tt.want) { if tt.wantErr {
t.Errorf("getTLSConfigCrowdsec() = %v, want %v", got, tt.want) return
}
if (got.RootCAs == nil) != tt.wantRootCAsNil {
t.Errorf("GetTLSConfigCrowdsec() RootCAs nil = %v, want nil = %v", got.RootCAs == nil, tt.wantRootCAsNil)
}
if got.InsecureSkipVerify != tt.wantInsecureSkip {
t.Errorf("GetTLSConfigCrowdsec() InsecureSkipVerify = %v, want %v", got.InsecureSkipVerify, tt.wantInsecureSkip)
}
})
}
}
func Test_getContentTypeFromPath(t *testing.T) {
tests := []struct {
name string
path string
expected string
}{
{name: "HTML file with .html extension", path: "/ban.html", expected: "text/html; charset=utf-8"},
{name: "JSON file", path: "/ban.json", expected: "application/json"},
{name: "Text file", path: "/ban.txt", expected: "text/plain"},
{name: "File with mixed case extension", path: "/ban.HtMl", expected: "text/html; charset=utf-8"},
{name: "Unknown extension defaults to HTML", path: "/ban.xyz", expected: "text/html; charset=utf-8"},
{name: "File without extension", path: "/ban", expected: "text/html; charset=utf-8"},
{name: "Empty path", path: "", expected: ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := getContentTypeFromPath(tt.path)
if got != tt.expected {
t.Errorf("GetContentTypeFromPath(%q) = %q, want %q", tt.path, got, tt.expected)
} }
}) })
} }
+11 -4
View File
@@ -30,7 +30,7 @@ that lives upstream in Crowdsec.
|-----------|-----| |-----------|-----|
| Traefik | Binary `v3.7.1`, downloaded into `.cache/` (reused across local runs; re-downloaded on fresh CI runners) | | Traefik | Binary `v3.7.1`, downloaded into `.cache/` (reused across local runs; re-downloaded on fresh CI runners) |
| Plugin | Loaded via `experimental.localPlugins` from the repo root (symlinked into `plugins-local/`) | | Plugin | Loaded via `experimental.localPlugins` from the repo root (symlinked into `plugins-local/`) |
| LAPI | `mocklapi` — a stdlib-only Go command (its own nested module), compiled and cached under `.cache/`, driven through `/admin` endpoints instead of `cscli` | | LAPI | `mocklapi` — a stdlib-only Go command (its own nested module), compiled and cached under `.cache/`, driven through `/admin` endpoints instead of `cscli`. Serves plain HTTP, or HTTPS when `--lapi-tls-cert/--lapi-tls-key` are passed (the `tls-system-ca` scenario) |
| AppSec | WAF stand-in built into the mock — blocks URIs containing `rpc2`, allows the rest | | AppSec | WAF stand-in built into the mock — blocks URIs containing `rpc2`, allows the rest |
| Backend | A plain HTTP responder built into the mock | | Backend | A plain HTTP responder built into the mock |
@@ -39,9 +39,16 @@ backend `8091`, AppSec `8092`.
## Running locally ## Running locally
Prerequisites: `bash`, `curl`, `go`, `tar`. On first use the Traefik binary is Prerequisites: `bash`, `curl`, `go`, `tar` (plus `openssl` for the
fetched and the mock is compiled into `.cache/`. That cache is reused across `tls-system-ca` scenario, which mints a throwaway CA at runtime). On first use
local runs; CI runs on fresh runners, so both are recreated on every CI run. the Traefik binary is fetched and the mock is compiled into `.cache/`. That
cache is reused across local runs; CI runs on fresh runners, so both are
recreated on every CI run.
The `tls-system-ca` scenario verifies that, with no custom CA configured, the
bouncer falls back to the OS/system trust store for an HTTPS LAPI: it serves the
mock over TLS and points the Traefik process's `SSL_CERT_FILE` at the test CA
(trusted → 200) or an empty bundle (untrusted → 403, proving it still verifies).
```bash ```bash
# one scenario # one scenario
+16 -3
View File
@@ -190,16 +190,29 @@ start_stack() {
-e "s|@@SCENARIO_DIR@@|${scenario_dir}|g" \ -e "s|@@SCENARIO_DIR@@|${scenario_dir}|g" \
"$scenario_dir/dynamic.yml" > "$WORKDIR/dynamic.yml" "$scenario_dir/dynamic.yml" > "$WORKDIR/dynamic.yml"
# Opt-in HTTPS LAPI: a scenario exports LAPI_TLS_CERT/LAPI_TLS_KEY to serve the
# LAPI over TLS (used by tls-system-ca). Default empty -> plaintext as before.
local mock_tls_args=() lapi_scheme=http lapi_curl=()
if [[ -n "${LAPI_TLS_CERT:-}" && -n "${LAPI_TLS_KEY:-}" ]]; then
mock_tls_args=(--lapi-tls-cert "$LAPI_TLS_CERT" --lapi-tls-key "$LAPI_TLS_KEY")
lapi_scheme=https
lapi_curl=(-k) # the readiness probe ignores trust; the bouncer's trust is what we test
fi
"$mock_bin" \ "$mock_bin" \
--lapi-addr "127.0.0.1:${LAPI_PORT}" \ --lapi-addr "127.0.0.1:${LAPI_PORT}" \
--backend-addr "127.0.0.1:${BACKEND_PORT}" \ --backend-addr "127.0.0.1:${BACKEND_PORT}" \
--appsec-addr "127.0.0.1:${APPSEC_PORT}" >"$WORKDIR/mock.log" 2>&1 & --appsec-addr "127.0.0.1:${APPSEC_PORT}" \
"${mock_tls_args[@]}" >"$WORKDIR/mock.log" 2>&1 &
MOCK_PID=$! MOCK_PID=$!
( cd "$WORKDIR" && exec "$traefik_bin" --configfile=traefik.yml ) >"$WORKDIR/traefik.log" 2>&1 & # Opt-in trust store for the Traefik process: a scenario exports
# TRAEFIK_SSL_CERT_FILE to point Go's x509.SystemCertPool() at a specific CA
# bundle. Empty -> Go's default system store (unchanged behaviour).
( cd "$WORKDIR" && SSL_CERT_FILE="${TRAEFIK_SSL_CERT_FILE:-}" exec "$traefik_bin" --configfile=traefik.yml ) >"$WORKDIR/traefik.log" 2>&1 &
TRAEFIK_PID=$! TRAEFIK_PID=$!
wait_for_status "http://127.0.0.1:${LAPI_PORT}/health" 200 30 wait_for_status "${lapi_scheme}://127.0.0.1:${LAPI_PORT}/health" 200 30 "${lapi_curl[@]}"
# AppSec stand-in: a bare GET carries no "rpc2" URI, so it answers 200 (allow). # AppSec stand-in: a bare GET carries no "rpc2" URI, so it answers 200 (allow).
wait_for_status "http://127.0.0.1:${APPSEC_PORT}/" 200 30 wait_for_status "http://127.0.0.1:${APPSEC_PORT}/" 200 30
# /ping is served by Traefik itself once it is up (plugin compilation included). # /ping is served by Traefik itself once it is up (plugin compilation included).
+28 -1
View File
@@ -13,6 +13,7 @@ package main
import ( import (
"encoding/json" "encoding/json"
"flag" "flag"
"io"
"log" "log"
"net/http" "net/http"
"strings" "strings"
@@ -52,6 +53,11 @@ func main() {
backendAddr := flag.String("backend-addr", "127.0.0.1:8091", "address for the stub upstream service") backendAddr := flag.String("backend-addr", "127.0.0.1:8091", "address for the stub upstream service")
// AppSec WAF stand-in (the real engine listens on :7422). Not a CRS engine. // AppSec WAF stand-in (the real engine listens on :7422). Not a CRS engine.
appsecAddr := flag.String("appsec-addr", "127.0.0.1:8092", "address for the AppSec mock") appsecAddr := flag.String("appsec-addr", "127.0.0.1:8092", "address for the AppSec mock")
// Optional TLS for the LAPI: when both are set the LAPI is served over HTTPS
// (cert signed by the scenario's throwaway CA) so the suite can exercise the
// bouncer's system-trust-store path. Backend and AppSec stay plaintext.
lapiTLSCert := flag.String("lapi-tls-cert", "", "PEM cert to serve the LAPI over HTTPS (optional)")
lapiTLSKey := flag.String("lapi-tls-key", "", "PEM key for --lapi-tls-cert")
flag.Parse() flag.Parse()
go func() { go func() {
@@ -67,9 +73,26 @@ func main() {
// exercised without standing up the real WAF. // exercised without standing up the real WAF.
go func() { go func() {
log.Fatal(http.ListenAndServe(*appsecAddr, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log.Fatal(http.ListenAndServe(*appsecAddr, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.Contains(r.Header.Get("X-Crowdsec-Appsec-Uri"), "rpc2") { if strings.Contains(r.Header.Get("X-Crowdsec-Appsec-Uri"), "403") {
w.WriteHeader(http.StatusForbidden) w.WriteHeader(http.StatusForbidden)
} }
if strings.Contains(r.Header.Get("X-Crowdsec-Appsec-Uri"), "500") {
w.WriteHeader(http.StatusInternalServerError)
}
if strings.Contains(r.Header.Get("X-Crowdsec-Appsec-Uri"), "502") {
w.WriteHeader(http.StatusBadGateway)
}
// Read body
body, err := io.ReadAll(r.Body)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
defer r.Body.Close()
if strings.Contains(string(body), "a=0") {
w.WriteHeader(http.StatusForbidden)
return
}
}))) })))
}() }()
@@ -130,6 +153,10 @@ func main() {
} }
}) })
if *lapiTLSCert != "" && *lapiTLSKey != "" {
log.Printf("mocklapi: LAPI on %s (TLS), backend on %s, appsec on %s", *lapiAddr, *backendAddr, *appsecAddr)
log.Fatal(http.ListenAndServeTLS(*lapiAddr, *lapiTLSCert, *lapiTLSKey, mux))
}
log.Printf("mocklapi: LAPI on %s, backend on %s, appsec on %s", *lapiAddr, *backendAddr, *appsecAddr) log.Printf("mocklapi: LAPI on %s, backend on %s, appsec on %s", *lapiAddr, *backendAddr, *appsecAddr)
log.Fatal(http.ListenAndServe(*lapiAddr, mux)) log.Fatal(http.ListenAndServe(*lapiAddr, mux))
} }
@@ -23,6 +23,9 @@ http:
crowdsecLapiHost: "@@LAPI_HOST@@" crowdsecLapiHost: "@@LAPI_HOST@@"
crowdsecLapiKey: "@@APIKEY@@" crowdsecLapiKey: "@@APIKEY@@"
crowdsecAppsecEnabled: "true" crowdsecAppsecEnabled: "true"
crowdsecAppsecFailureBlock: "true"
crowdsecAppsecBodyLimit: 4
crowdsecAppsecUnreachableBlock: "false"
crowdsecAppsecScheme: http crowdsecAppsecScheme: http
crowdsecAppsecHost: "@@APPSEC_HOST@@" crowdsecAppsecHost: "@@APPSEC_HOST@@"
forwardedHeadersTrustedIps: forwardedHeadersTrustedIps:
+15 -3
View File
@@ -13,11 +13,23 @@ SCENARIO=appsec
# plugin's AppSec path end to end (header forwarding + allow/block handling); it # plugin's AppSec path end to end (header forwarding + allow/block handling); it
# does not test the real WAF's detection accuracy. # does not test the real WAF's detection accuracy.
body() { body() {
echo "[$SCENARIO] benign request must pass (AppSec allows)" echo "[$SCENARIO] benign request must pass (AppSec 200)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4" assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] request whose URI contains 'rpc2' must be blocked (AppSec 403)" echo "[$SCENARIO] request that return 403 must be blocked (AppSec 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo/rpc2" 403 -H "X-Forwarded-For: 1.2.3.4" assert_status "http://127.0.0.1:${WEB_PORT}/foo/403" 403 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] request that return 500 must be blocked (because CrowdsecAppsecFailureBlock = true) (AppSec 500)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo/500" 403 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] request that return 502 must pass (because CrowdsecAppsecUnreachableBlock = false) (Proxy error 502)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo/502" 200 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] request that send bad body after crowdsecAppsecBodyLimit must pass (AppSec 200)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4" -X POST -d "______&a=0"
echo "[$SCENARIO] request that send bad body before crowdsecAppsecBodyLimit must pass (AppSec 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.4" -X POST -d "a=0&______"
} }
run_scenario "$SCENARIO" "$HERE" body run_scenario "$SCENARIO" "$HERE" body
+3
View File
@@ -16,6 +16,9 @@ body() {
echo "[$SCENARIO] captcha page must be served once the decision is polled (200 + marker)" echo "[$SCENARIO] captcha page must be served once the decision is polled (200 + marker)"
wait_for_body_contains "http://127.0.0.1:${WEB_PORT}/foo" "E2E_CAPTCHA_PAGE_MARKER" 15 -H "X-Forwarded-For: 1.2.3.4" wait_for_body_contains "http://127.0.0.1:${WEB_PORT}/foo" "E2E_CAPTCHA_PAGE_MARKER" 15 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] captcha response Content-Type is HTML"
assert_header "http://127.0.0.1:${WEB_PORT}/foo" Content-Type "text/html; charset=utf-8" -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] captcha response is HTTP 200 (the captcha page itself, not a 403)" echo "[$SCENARIO] captcha response is HTTP 200 (the captcha page itself, not a 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4" assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4"
@@ -1,8 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>E2E ban marker</title></head>
<body>
<h1 id="e2e-ban-marker">E2E_CUSTOM_BAN_PAGE_MARKER</h1>
<p>IP: {{ .ClientIP }} reason: {{ .RemediationReason }}</p>
</body>
</html>
@@ -0,0 +1,4 @@
{
"marker": "E2E_CUSTOM_BAN_PAGE_MARKER",
"body": "IP: {{ .ClientIP }}, reason: {{ .RemediationReason }}, trace: {{ .TraceID }}"
}
@@ -24,5 +24,6 @@ http:
crowdsecLapiKey: "@@APIKEY@@" crowdsecLapiKey: "@@APIKEY@@"
forwardedHeadersTrustedIps: forwardedHeadersTrustedIps:
- "127.0.0.1/32" - "127.0.0.1/32"
banHtmlFilePath: "@@SCENARIO_DIR@@/ban.html" banFilePath: "@@SCENARIO_DIR@@/ban.json"
remediationHeadersCustomName: "X-E2E-Remediation" remediationHeadersCustomName: "X-E2E-Remediation"
traceHeadersCustomName: x-trace
@@ -15,11 +15,14 @@ body() {
wait_for_status "http://127.0.0.1:${WEB_PORT}/foo" 403 15 -H "X-Forwarded-For: 1.2.3.4" wait_for_status "http://127.0.0.1:${WEB_PORT}/foo" 403 15 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] banned response Content-Type is HTML" echo "[$SCENARIO] banned response Content-Type is HTML"
assert_header "http://127.0.0.1:${WEB_PORT}/foo" Content-Type "text/html; charset=utf-8" -H "X-Forwarded-For: 1.2.3.4" assert_header "http://127.0.0.1:${WEB_PORT}/foo" Content-Type "application/json" -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] banned response body contains the custom marker" echo "[$SCENARIO] banned response body contains the custom marker"
assert_body_contains "http://127.0.0.1:${WEB_PORT}/foo" "E2E_CUSTOM_BAN_PAGE_MARKER" -H "X-Forwarded-For: 1.2.3.4" assert_body_contains "http://127.0.0.1:${WEB_PORT}/foo" "E2E_CUSTOM_BAN_PAGE_MARKER" -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] banned response body contains the IP and reason from templating"
assert_body_contains "http://127.0.0.1:${WEB_PORT}/foo" "IP: 1.2.3.4, reason: LAPI, trace: 0123456789" -H "X-Forwarded-For: 1.2.3.4" -H "X-Trace: 0123456789"
echo "[$SCENARIO] banned response carries the custom remediation header (remediationHeadersCustomName)" echo "[$SCENARIO] banned response carries the custom remediation header (remediationHeadersCustomName)"
assert_header "http://127.0.0.1:${WEB_PORT}/foo" X-E2E-Remediation "ban" -H "X-Forwarded-For: 1.2.3.4" assert_header "http://127.0.0.1:${WEB_PORT}/foo" X-E2E-Remediation "ban" -H "X-Forwarded-For: 1.2.3.4"
} }
@@ -0,0 +1,28 @@
http:
routers:
r:
rule: "PathPrefix(`/foo`)"
entryPoints:
- web
service: backend
middlewares:
- bouncer
services:
backend:
loadBalancer:
servers:
- url: "@@BACKEND_URL@@"
middlewares:
bouncer:
plugin:
bouncer:
enabled: "true"
crowdsecMode: live
defaultDecisionSeconds: "2"
# HTTPS LAPI with NO custom CA configured: the bouncer must fall back to
# the OS/system trust store (which the scenario controls via SSL_CERT_FILE).
crowdsecLapiScheme: https
crowdsecLapiHost: "@@LAPI_HOST@@"
crowdsecLapiKey: "@@APIKEY@@"
forwardedHeadersTrustedIps:
- "127.0.0.1/32"
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Scenario: HTTPS LAPI with no custom CA configured -> the bouncer must fall back
# to the OS/system trust store (PR #331). In the binary suite the "system trust
# store" is whatever Go's x509.SystemCertPool() reads, which honours SSL_CERT_FILE
# on the Traefik process. We mint a throwaway CA, serve the mock LAPI over HTTPS
# with a cert signed by it, and run the stack twice:
#
# positive: SSL_CERT_FILE = our CA -> LAPI trusted -> 200
# negative: SSL_CERT_FILE = empty bundle -> LAPI not trusted -> 403
#
# live mode is fail-closed, so a TLS error becomes a 403. The negative run proves
# the patch still VERIFIES (it is not an insecure skip).
#
# Extra dependency vs other scenarios: openssl.
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
# shellcheck source=../../lib/common.sh
source "$HERE/../../lib/common.sh"
SCENARIO=tls-system-ca
SCENARIO_NAME="$SCENARIO"
SCENARIO_LOG="/tmp/e2e-mock-${SCENARIO}.log"
CERT_DIR="$(mktemp -d)"
cleanup() {
local rc=$?
if (( rc != 0 )); then
dump_diagnostics > "$SCENARIO_LOG" 2>&1 || true
echo "[$SCENARIO] failed. Logs written to $SCENARIO_LOG" >&2
fi
stop_stack
rm -rf "$CERT_DIR"
exit $rc
}
trap cleanup EXIT
echo "[$SCENARIO] minting throwaway CA + LAPI cert (SAN=IP:127.0.0.1)..."
openssl ecparam -name prime256v1 -genkey -noout -out "$CERT_DIR/ca.key" 2>/dev/null
openssl req -x509 -new -key "$CERT_DIR/ca.key" -sha256 -days 3650 \
-subj "/CN=crowdsec-bouncer e2e test CA" -out "$CERT_DIR/ca.crt" 2>/dev/null
openssl ecparam -name prime256v1 -genkey -noout -out "$CERT_DIR/lapi.key" 2>/dev/null
openssl req -new -key "$CERT_DIR/lapi.key" -subj "/CN=lapi" -out "$CERT_DIR/lapi.csr" 2>/dev/null
openssl x509 -req -in "$CERT_DIR/lapi.csr" -CA "$CERT_DIR/ca.crt" -CAkey "$CERT_DIR/ca.key" \
-CAcreateserial -days 3650 -sha256 -out "$CERT_DIR/lapi.crt" \
-extfile <(printf "subjectAltName=IP:127.0.0.1\nbasicConstraints=CA:FALSE\nkeyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth") 2>/dev/null
: > "$CERT_DIR/empty.crt" # an empty bundle = a system store that trusts nothing
# The mock serves the same CA-signed cert in both runs; only Traefik's trust differs.
export LAPI_TLS_CERT="$CERT_DIR/lapi.crt" LAPI_TLS_KEY="$CERT_DIR/lapi.key"
echo "[$SCENARIO] === positive: CA in the system trust store ==="
export TRAEFIK_SSL_CERT_FILE="$CERT_DIR/ca.crt"
start_stack "$HERE"
echo "[$SCENARIO] HTTPS LAPI verifies via system trust store -> request passes (200)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4"
stop_stack
echo "[$SCENARIO] === negative: CA absent from the system trust store ==="
export TRAEFIK_SSL_CERT_FILE="$CERT_DIR/empty.crt"
start_stack "$HERE"
echo "[$SCENARIO] LAPI cert not trusted -> TLS fails, fail-closed (403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.4"
stop_stack
echo "[$SCENARIO] OK"