710e88808b
* add the bot detection feature to the crowdsec traefik bouncer
* fix the return value
* add http status normalization
* fix appsec query test return values
* add boundaries for appsec response
* 🎨 satisfy golangci-lint on the appsec response path
gofmt on bouncer_test.go (merge artifact), a doc comment revive requires
on the exported AppSecResponse, and nolint directives for the linters the
new appsecQuery signature trips: tagliatelle on the snake_case json tags
that are Appsec's wire format, and nilnil/gocognit/gocyclo/funlen on
appsecQuery itself, matching how New is already handled.
No feature code changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* 🐛 keep serving the configured ban page for appsec bans
An appsec "ban" with a structured body was served straight from appsec,
so an operator with banFilePath configured silently lost their ban page
as soon as Crowdsec 1.8 started returning structured JSON — and got a
blank 403 when appsec sent no user_body_content at all. Route the ban
action back through handleBanServeHTTP, which keeps the template, the
content type, the remediation header value and the trace templating.
Only actions that must carry their own content, challenge among them,
are relayed from appsec. Those now fall back to banTemplateContentType
when appsec sends no Content-Type, instead of letting Go sniff it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* 🍱 reduce test and logic in appsec query
* 🍱 use same appsec response limit as defined for user
* 🍱 fix lint
* 🐛 fix tests
* 🍱 revert to 1Mo appsec response body
* 📝 examples: add a bot detection example
The PR relays the whole AppSec remediation rather than collapsing it into a
403, which is what makes the bot challenge possible: a ban is a status code,
a challenge is a status code plus a body, cookies and headers. Nothing
demonstrated that, so this adds an example alongside the existing ones.
Verified end to end against CrowdSec 1.8.1: the challenge page is served, the
browser solves the proof of work, the submission is accepted and the request
reaches the backend.
Three things break this setup silently, and all three cost time here, so the
README names them:
- The router must also match /crowdsec-internal. The challenge page loads its
fingerprint script and its proof-of-work worker from there by absolute path,
so a router scoped to the application prefix alone returns 404 and the
challenge can never be solved, with no error logged anywhere.
- The acquisition must load the config that calls SendChallenge(), which is
appsec-bot-challenge-scoring; the -balanced variant only carries the
rejection threshold. Loading the latter alone processes requests and
challenges nothing. The wildcard form from the CrowdSec enable guide is used
instead of a hand-written list, since it also picks up the exclusion configs
the collection ships.
- Referencing an appsec-config that is not installed is fatal rather than
degraded. CrowdSec refuses to start and, because crowdsecAppsecFailureBlock
defaults to true, every request then returns a 403 that reads like a WAF
decision. appsec-default needs appsec-virtual-patching and
appsec-generic-rules, so both are in COLLECTIONS.
The routing requirement is in the main README too, since it applies to anyone
enabling bot detection and not just to this example.
The example runs the plugin from the working tree, as relaying the remediation
is not in a release yet; the catalog lines are present and commented out.
* 🍱 reduce complexity and set the remediationHeader to 'challenge' instead of decision.Action
* 📝 examples: add a bot detection example
The PR relays the whole AppSec remediation rather than collapsing it into a
403, which is what makes the bot challenge possible: a ban is a status code,
a challenge is a status code plus a body, cookies and headers. Nothing
demonstrated that, so this adds an example alongside the existing ones.
Verified end to end against CrowdSec 1.8.1: the challenge page is served, the
browser solves the proof of work, the submission is accepted and the request
reaches the backend.
Two things break this setup silently, and both cost time here, so the README
names them:
- The router must also match /crowdsec-internal. The challenge page loads its
fingerprint script and its proof-of-work worker from there by absolute path,
so a router scoped to the application prefix alone returns 404 and the
challenge can never be solved, with no error logged anywhere.
- The acquisition must load the config that calls SendChallenge(), which is
appsec-bot-challenge-scoring; the -balanced variant only carries the
rejection threshold. Loading the latter alone processes requests and
challenges nothing. The wildcard form from the CrowdSec enable guide is used
instead of a hand-written list, since it also picks up the exclusion configs
the collection ships.
The routing requirement is in the main README too, since it applies to anyone
enabling bot detection and not just to this example, along with the supported
versions stated the same way as the existing Appsec line.
* ✅ tests: drop the challenge Content-Type fallback test
c8d7751 removed the fallback that set Content-Type from banTemplateContentType
when the appsec response carried none, but left the test asserting it, so the
suite has been red since.
Removing the test rather than restoring the fallback, per maxlerebourg.
* 🍱 harmonize appsecServer vs appsec in test
* 🍱 reduce loc and go closer to Lua implementation
* 💄 fix log statusCode 401 -> 403
* 🍱 fix lint
* 🍱 add guard on appsec response
* 🍱 fix lint
---------
Co-authored-by: mhx <mathieu@hanotaux.fr>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2.8 KiB
Example
Bot detection with the AppSec challenge
AppSec bot detection, added in CrowdSec 1.8.0, answers a request with a challenge page instead of a plain block. The browser solves a proof of work and gets a cookie that lets the following requests through.
That needs the bouncer to relay what AppSec sends back: a ban is a status code, a challenge is a status code plus a body, cookies and headers.
Docs: intro, enable, challenge protocol.
Two things that fail silently
The router must also match /crowdsec-internal. The challenge page loads its fingerprint
script and proof-of-work worker from there by absolute path. Scope the router to your
application prefix alone and they 404, so the challenge can never be solved and nothing is
logged:
- "traefik.http.routers.router-foo.rule=PathPrefix(`/foo`) || PathPrefix(`/crowdsec-internal`)"
The acquisition must load the config that calls SendChallenge(), which is
appsec-bot-challenge-scoring; the -balanced variant only carries the rejection threshold.
The wildcard below is the form from the enable guide, and it also picks up the exclusion
configs the collection ships:
appsec_configs:
- crowdsecurity/appsec-default
- crowdsecurity/appsec-bot-*
Warning: leave
crowdsecAppsecBodyLimitabove 0. At 0 the browser's solved challenge is never forwarded to AppSec, so it can never be validated and the client loops on the page.
WAF challenge runtime initialized in the CrowdSec logs means the configuration is right.
Running it
make run_bot_detection
Open http://localhost:8000/foo in a browser: challenge page, then whoami. curl gets the same page but cannot solve it, which is the point.
A client that fails the fingerprint scoring is told so rather than silently blocked:
Relaying the AppSec remediation landed in plugin 1.8.0, so that is the version the compose
file pins. To run it against the working tree instead, swap in the localplugins lines that
are there commented out.
Troubleshooting
docker exec crowdsec cscli metrics show appsec
Ch. Requested at 0 while Processed climbs means requests reach AppSec but nothing
challenges them, so the acquisition config is wrong rather than the bouncer.
Output of cscli metrics appsec
Failed challenge Crowdsec bot detected
Bot detection is flagged Alpha by CrowdSec. The challenge needs SSE4.1 and writable-executable memory, so hardened or older clients cannot solve it.