Files
crowdsec-bouncer-traefik-pl…/bouncer_test.go
T
710e88808b add the bot detection feature to the crowdsec traefik bouncer (#343)
* add the bot detection feature to the crowdsec traefik bouncer

* fix the return value

* add http status normalization

* fix appsec query test return values

* add boundaries for appsec response

* 🎨 satisfy golangci-lint on the appsec response path

gofmt on bouncer_test.go (merge artifact), a doc comment revive requires
on the exported AppSecResponse, and nolint directives for the linters the
new appsecQuery signature trips: tagliatelle on the snake_case json tags
that are Appsec's wire format, and nilnil/gocognit/gocyclo/funlen on
appsecQuery itself, matching how New is already handled.

No feature code changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* 🐛 keep serving the configured ban page for appsec bans

An appsec "ban" with a structured body was served straight from appsec,
so an operator with banFilePath configured silently lost their ban page
as soon as Crowdsec 1.8 started returning structured JSON — and got a
blank 403 when appsec sent no user_body_content at all. Route the ban
action back through handleBanServeHTTP, which keeps the template, the
content type, the remediation header value and the trace templating.

Only actions that must carry their own content, challenge among them,
are relayed from appsec. Those now fall back to banTemplateContentType
when appsec sends no Content-Type, instead of letting Go sniff it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* 🍱 reduce test and logic in appsec query

* 🍱 use same appsec response limit as defined for user

* 🍱 fix lint

* 🐛 fix tests

* 🍱 revert to 1Mo appsec response body

* 📝 examples: add a bot detection example

The PR relays the whole AppSec remediation rather than collapsing it into a
403, which is what makes the bot challenge possible: a ban is a status code,
a challenge is a status code plus a body, cookies and headers. Nothing
demonstrated that, so this adds an example alongside the existing ones.

Verified end to end against CrowdSec 1.8.1: the challenge page is served, the
browser solves the proof of work, the submission is accepted and the request
reaches the backend.

Three things break this setup silently, and all three cost time here, so the
README names them:

- The router must also match /crowdsec-internal. The challenge page loads its
  fingerprint script and its proof-of-work worker from there by absolute path,
  so a router scoped to the application prefix alone returns 404 and the
  challenge can never be solved, with no error logged anywhere.
- The acquisition must load the config that calls SendChallenge(), which is
  appsec-bot-challenge-scoring; the -balanced variant only carries the
  rejection threshold. Loading the latter alone processes requests and
  challenges nothing. The wildcard form from the CrowdSec enable guide is used
  instead of a hand-written list, since it also picks up the exclusion configs
  the collection ships.
- Referencing an appsec-config that is not installed is fatal rather than
  degraded. CrowdSec refuses to start and, because crowdsecAppsecFailureBlock
  defaults to true, every request then returns a 403 that reads like a WAF
  decision. appsec-default needs appsec-virtual-patching and
  appsec-generic-rules, so both are in COLLECTIONS.

The routing requirement is in the main README too, since it applies to anyone
enabling bot detection and not just to this example.

The example runs the plugin from the working tree, as relaying the remediation
is not in a release yet; the catalog lines are present and commented out.

* 🍱 reduce complexity and set the remediationHeader to 'challenge' instead of decision.Action

* 📝 examples: add a bot detection example

The PR relays the whole AppSec remediation rather than collapsing it into a
403, which is what makes the bot challenge possible: a ban is a status code,
a challenge is a status code plus a body, cookies and headers. Nothing
demonstrated that, so this adds an example alongside the existing ones.

Verified end to end against CrowdSec 1.8.1: the challenge page is served, the
browser solves the proof of work, the submission is accepted and the request
reaches the backend.

Two things break this setup silently, and both cost time here, so the README
names them:

- The router must also match /crowdsec-internal. The challenge page loads its
  fingerprint script and its proof-of-work worker from there by absolute path,
  so a router scoped to the application prefix alone returns 404 and the
  challenge can never be solved, with no error logged anywhere.
- The acquisition must load the config that calls SendChallenge(), which is
  appsec-bot-challenge-scoring; the -balanced variant only carries the
  rejection threshold. Loading the latter alone processes requests and
  challenges nothing. The wildcard form from the CrowdSec enable guide is used
  instead of a hand-written list, since it also picks up the exclusion configs
  the collection ships.

The routing requirement is in the main README too, since it applies to anyone
enabling bot detection and not just to this example, along with the supported
versions stated the same way as the existing Appsec line.

* ✅ tests: drop the challenge Content-Type fallback test

c8d7751 removed the fallback that set Content-Type from banTemplateContentType
when the appsec response carried none, but left the test asserting it, so the
suite has been red since.

Removing the test rather than restoring the fallback, per maxlerebourg.

* 🍱 harmonize appsecServer vs appsec in test

* 🍱 reduce loc and go closer to Lua implementation

* 💄 fix log statusCode 401 -> 403

* 🍱 fix lint

* 🍱 add guard on appsec response

* 🍱 fix lint

---------

Co-authored-by: mhx <mathieu@hanotaux.fr>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2026-09-12 14:58:48 +02:00

888 lines
27 KiB
Go

package crowdsec_bouncer_traefik_plugin //nolint:revive,stylecheck
import (
"context"
"io"
"net/http"
"net/http/httptest"
"net/url"
"reflect"
"strings"
"sync"
"testing"
"text/template"
"time"
cache "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/cache"
configuration "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration"
ip "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip"
logger "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger"
)
func TestServeHTTP(t *testing.T) {
cfg := CreateConfig()
cfg.CrowdsecLapiKey = "test"
ctx := context.Background()
next := http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {})
handler, err := New(ctx, next, cfg, "demo-plugin")
if err != nil {
t.Fatal(err)
}
recorder := httptest.NewRecorder()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://localhost", nil)
if err != nil {
t.Fatal(err)
}
handler.ServeHTTP(recorder, req)
}
func TestNew(t *testing.T) {
type args struct {
ctx context.Context //nolint:containedctx
next http.Handler
config *configuration.Config
name string
}
tests := []struct {
name string
args args
want http.Handler
wantErr bool
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := New(tt.args.ctx, tt.args.next, tt.args.config, tt.args.name)
if (err != nil) != tt.wantErr {
t.Errorf("New() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("New() = %v, want %v", got, tt.want)
}
})
}
}
func TestBouncer_ServeHTTP(t *testing.T) {
type fields struct {
next http.Handler
name string
template *template.Template
enabled bool
crowdsecScheme string
crowdsecHost string
crowdsecKey string
crowdsecMode string
updateInterval int64
defaultDecisionTimeout int64
forwardedCustomHeader string
clientPoolStrategy *ip.PoolStrategy
serverPoolStrategy *ip.PoolStrategy
httpClient *http.Client
cacheClient *cache.Client
}
type args struct {
rw http.ResponseWriter
req *http.Request
}
tests := []struct {
name string
fields fields
args args
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(_ *testing.T) {
bouncer := &Bouncer{
next: tt.fields.next,
name: tt.fields.name,
template: tt.fields.template,
enabled: tt.fields.enabled,
crowdsecScheme: tt.fields.crowdsecScheme,
crowdsecHost: tt.fields.crowdsecHost,
crowdsecKey: tt.fields.crowdsecKey,
crowdsecMode: tt.fields.crowdsecMode,
updateInterval: tt.fields.updateInterval,
defaultDecisionTimeout: tt.fields.defaultDecisionTimeout,
forwardedCustomHeader: tt.fields.forwardedCustomHeader,
clientPoolStrategy: tt.fields.clientPoolStrategy,
serverPoolStrategy: tt.fields.serverPoolStrategy,
httpClient: tt.fields.httpClient,
cacheClient: tt.fields.cacheClient,
}
bouncer.ServeHTTP(tt.args.rw, tt.args.req)
})
}
}
func Test_handleNoStreamCache(t *testing.T) {
newBouncer := func(mode string, updateMaxFailure int64) *Bouncer {
log := logger.New("ERROR", "")
cacheClient := &cache.Client{}
cacheClient.New(log, false, "", nil, "", "")
return &Bouncer{
crowdsecScheme: "http",
crowdsecHost: "127.0.0.1:1",
crowdsecPath: "/",
crowdsecMode: mode,
updateMaxFailure: updateMaxFailure,
defaultDecisionTimeout: 60,
httpClient: &http.Client{Timeout: time.Second},
cacheClient: cacheClient,
log: log,
}
}
tests := []struct {
name string
mode string
updateMaxFailure int64
want string
wantErr bool
}{
{
name: "live mode blocks by default when crowdsec is unreachable",
mode: configuration.LiveMode,
updateMaxFailure: 0,
want: cache.BannedValue,
wantErr: true,
},
{
name: "live mode blocks for any positive updateMaxFailure",
mode: configuration.LiveMode,
updateMaxFailure: 3,
want: cache.BannedValue,
wantErr: true,
},
{
name: "live mode lets the request through when updateMaxFailure is -1",
mode: configuration.LiveMode,
updateMaxFailure: -1,
want: cache.NoBannedValue,
wantErr: true,
},
{
name: "none mode lets the request through when updateMaxFailure is -1",
mode: configuration.NoneMode,
updateMaxFailure: -1,
want: cache.NoBannedValue,
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := handleNoStreamCache(newBouncer(tt.mode, tt.updateMaxFailure), "1.2.3.4")
if (err != nil) != tt.wantErr {
t.Errorf("handleNoStreamCache() error = %v, wantErr %v", err, tt.wantErr)
}
if got != tt.want {
t.Errorf("handleNoStreamCache() = %q, want %q", got, tt.want)
}
})
}
}
func Test_handleStreamCache(t *testing.T) {
type args struct {
bouncer *Bouncer
}
tests := []struct {
name string
args args
wantErr bool
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := handleStreamCache(tt.args.bouncer)
if (err != nil) != tt.wantErr {
t.Errorf("handleStreamCache() error = %v, wantErr %v", err, tt.wantErr)
return
}
})
}
}
func Test_crowdsecQuery(t *testing.T) {
type args struct {
bouncer *Bouncer
stringURL string
data []byte
}
tests := []struct {
name string
args args
want []byte
wantErr bool
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := crowdsecQuery(tt.args.bouncer, tt.args.stringURL, tt.args.data)
if (err != nil) != tt.wantErr {
t.Errorf("crowdsecQuery() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("crowdsecQuery() = %v, want %v", got, tt.want)
}
})
}
}
func TestHandleBanServeHTTPWithDifferentMethods(t *testing.T) {
html := "<html>You are banned</html>"
banTemplate, _ := template.New("html").Delims("{{", "}}").Parse(html)
tests := []struct {
name string
method string
banTemplate *template.Template
expectBodyContent bool
}{
{
name: "GET request should have body with template",
method: http.MethodGet,
banTemplate: banTemplate,
expectBodyContent: true,
},
{
name: "HEAD request should NOT have body even with template",
method: http.MethodHead,
banTemplate: banTemplate,
expectBodyContent: false,
},
{
name: "POST request should have body with template",
method: http.MethodPost,
banTemplate: banTemplate,
expectBodyContent: true,
},
{
name: "PUT request should have body with template",
method: http.MethodPut,
banTemplate: banTemplate,
expectBodyContent: true,
},
{
name: "DELETE request should have body with template",
method: http.MethodDelete,
banTemplate: banTemplate,
expectBodyContent: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
bouncer := &Bouncer{
remediationStatusCode: http.StatusForbidden,
remediationCustomHeader: "X-Test-Remediation",
banTemplate: tt.banTemplate,
banTemplateContentType: "text/html; charset=utf-8",
}
rw := httptest.NewRecorder()
req := &http.Request{Method: tt.method}
bouncer.handleBanServeHTTP(rw, req, "0.0.0.0", "TEST")
// Check status code
if rw.Code != http.StatusForbidden {
t.Errorf("Expected status code 403, got %d", rw.Code)
}
// Check custom header
headerValue := rw.Header().Get("X-Test-Remediation")
if headerValue != "ban" {
t.Errorf("Expected header X-Test-Remediation to be 'ban', got %s", headerValue)
}
// Check body content
body := rw.Body.String()
hasBodyContent := len(body) > 0
if hasBodyContent != tt.expectBodyContent {
t.Errorf("Method %s: expected body content: %v, got body content: %v (body: %q)",
tt.method, tt.expectBodyContent, hasBodyContent, body)
}
// If we expect body content, verify it matches template
if tt.expectBodyContent && body != html {
t.Errorf("Expected body %q, got %q", html, body)
}
})
}
}
func TestHandleBanServeHTTPContentType(t *testing.T) {
html := "<html>You are banned</html>"
banTemplate, _ := template.New("html").Delims("{{", "}}").Parse(html)
tests := []struct {
name string
banTemplate *template.Template
banTemplateContentType string
}{
{
name: "Default HTML content type",
banTemplate: banTemplate,
banTemplateContentType: "text/html; charset=utf-8",
},
{
name: "Custom JSON content type",
banTemplate: banTemplate,
banTemplateContentType: "application/json",
},
{
name: "Content type set even when banTemplate is nil",
banTemplate: nil,
banTemplateContentType: "application/json",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
bouncer := &Bouncer{
remediationStatusCode: http.StatusForbidden,
banTemplate: tt.banTemplate,
banTemplateContentType: tt.banTemplateContentType,
}
rw := httptest.NewRecorder()
req := &http.Request{Method: http.MethodGet}
bouncer.handleBanServeHTTP(rw, req, "0.0.0.0", "TEST")
if got := rw.Header().Get("Content-Type"); got != tt.banTemplateContentType {
t.Errorf("Expected Content-Type %q, got %q", tt.banTemplateContentType, got)
}
})
}
}
func TestCaptchaMethodBasedLogic(t *testing.T) {
tests := []struct {
name string
method string
remediation string
expectBanFallback bool
}{
{
name: "GET with captcha remediation should allow captcha",
method: http.MethodGet,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
{
name: "HEAD with captcha remediation should fallback to ban",
method: http.MethodHead,
remediation: cache.CaptchaValue,
expectBanFallback: true,
},
{
name: "POST with captcha remediation should allow captcha",
method: http.MethodPost,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
{
name: "PUT with captcha remediation should allow captcha",
method: http.MethodPut,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
{
name: "DELETE with captcha remediation should allow captcha",
method: http.MethodDelete,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
{
name: "PATCH with captcha remediation should allow captcha",
method: http.MethodPatch,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
{
name: "OPTIONS with captcha remediation should allow captcha",
method: http.MethodOptions,
remediation: cache.CaptchaValue,
expectBanFallback: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Test the core logic: captcha is served for all methods except HEAD
shouldUseCaptcha := tt.remediation == cache.CaptchaValue && tt.method != http.MethodHead
if shouldUseCaptcha == tt.expectBanFallback {
t.Errorf("Method %s with %s remediation: expected ban fallback %v, but logic would use captcha %v",
tt.method, tt.remediation, tt.expectBanFallback, shouldUseCaptcha)
}
})
}
}
// blockingBody simulates a request body that never reaches EOF, like a
// bidirectional gRPC stream that keeps its body open for the whole life of
// the connection. Reading from it blocks until the test is done.
type blockingBody struct {
done <-chan struct{}
}
func (b blockingBody) Read(_ []byte) (int, error) {
<-b.done
return 0, io.EOF
}
func (blockingBody) Close() error { return nil }
func Test_isBodyUnreadable(t *testing.T) {
realBody := func() io.ReadCloser { return io.NopCloser(strings.NewReader("data")) }
tests := []struct {
name string
protoMajor int
contentLength int64
body io.ReadCloser
want bool
}{
{name: "http2 grpc stream without content-length", protoMajor: 2, contentLength: -1, body: realBody(), want: true},
{name: "http3 stream without content-length", protoMajor: 3, contentLength: -1, body: realBody(), want: true},
{name: "http2 with content-length", protoMajor: 2, contentLength: 42, body: realBody(), want: false},
{name: "http1.1 chunked without content-length", protoMajor: 1, contentLength: -1, body: realBody(), want: false},
{name: "http2 without body", protoMajor: 2, contentLength: -1, body: nil, want: false},
{name: "http2 with http.NoBody", protoMajor: 2, contentLength: -1, body: http.NoBody, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, "http://localhost", nil)
req.ProtoMajor = tt.protoMajor
req.ContentLength = tt.contentLength
req.Body = tt.body
if got := isBodyUnreadable(req); got != tt.want {
t.Errorf("isBodyUnreadable() = %v, want %v", got, tt.want)
}
})
}
}
func newUnreadableRequest(method string, done <-chan struct{}) *http.Request {
req, _ := http.NewRequest(method, "http://localhost/api/admin/reservations/8fff14a2", blockingBody{done: done})
req.ProtoMajor = 3
req.ContentLength = -1
return req
}
// newStreamingRequest builds an HTTP/2 request whose body never reaches EOF,
// like a bidirectional gRPC stream (issue #323).
func newStreamingRequest(done <-chan struct{}) *http.Request {
req, _ := http.NewRequest(http.MethodPost, "http://localhost/signalexchange.SignalExchange/ConnectStream", blockingBody{done: done})
req.Header.Set("Content-Type", "application/grpc")
req.ProtoMajor = 2
req.ContentLength = -1
return req
}
// Test_appsecQuery_streamingDoesNotBlock is a regression test for issue #323:
// a gRPC streaming request whose body never reaches EOF must not be buffered
// (io.ReadAll would block until timeout and wrongly produce a 403). The appsec
// query must complete promptly, inspecting headers only.
func Test_appsecQuery_streamingDoesNotBlock(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreachableBlock: true,
appsecFailureBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
_, err := appsecQuery(bouncer, "1.2.3.4", newStreamingRequest(done))
finished <- err
}()
select {
case err := <-finished:
if err != nil {
t.Errorf("appsecQuery() on streaming request returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("appsecQuery() blocked on a streaming request body (issue #323 regression)")
}
}
// Test_appsecQuery_dropUnreadableBody verifies that, when configured to do so,
// a request with an unreadable body is dropped (blocked) instead of forwarded
// without its body, mirroring the reference APPSEC_DROP_UNREADABLE_BODY option.
func Test_appsecQuery_dropUnreadableBody(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreadableBodyBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
type bodyReq func(done <-chan struct{}) *http.Request
tests := []struct {
name string
req bodyReq
wantErr bool
}{
// a streaming body that never reaches EOF is unreadable: the request is dropped.
{name: "streaming request is dropped", req: newStreamingRequest, wantErr: true},
// issue #351 regression: an HTTP/3 GET without content-length has no body to read, so it must not be dropped.
{name: "bodyless HTTP/3 GET is not dropped", req: func(done <-chan struct{}) *http.Request { return newUnreadableRequest(http.MethodGet, done) }},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
_, err := appsecQuery(bouncer, "1.2.3.4", tc.req(done))
finished <- err
}()
select {
case err := <-finished:
if tc.wantErr && err == nil {
t.Error("appsecQuery() expected the request to be dropped, got nil")
}
if !tc.wantErr && err != nil {
t.Errorf("appsecQuery() returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("appsecQuery() blocked while handling the request body")
}
})
}
}
func Test_appsecQuery_oversizedResponse(t *testing.T) {
testCases := []struct {
name string
statusCode int
wantErr bool
}{
{name: "OK passes", statusCode: http.StatusOK, wantErr: false},
{name: "Forbidden blocks", statusCode: http.StatusForbidden, wantErr: true},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(tc.statusCode)
_, _ = io.WriteString(rw, strings.Repeat("x", int(appsecResponseBodyLimit)+1))
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 1000,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
decision, err := appsecQuery(bouncer, "1.2.3.4", httptest.NewRequest(http.MethodGet, "http://localhost/", nil))
if tc.wantErr && err == nil {
t.Fatal("appsecQuery() expected error, got nil")
}
if !tc.wantErr && err != nil {
t.Fatalf("appsecQuery() returned error: %v", err)
}
if decision != nil {
t.Fatalf("appsecQuery() returned decision: %v", decision)
}
})
}
}
func TestHandleNextServeHTTPRelaysStructuredAppsecChallenge(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{
"action":"challenge",
"http_status":200,
"user_body_content":"<html>challenge</html>",
"user_cookies":["__crowdsec_challenge=value; Path=/; HttpOnly"],
"user_headers":{
"Content-Type":["text/html"],
"Cache-Control":["no-store"]
}
}`))
}))
defer appsecServer.Close()
appsecURL, err := url.Parse(appsecServer.URL)
if err != nil {
t.Fatal(err)
}
nextCalled := false
bouncer := &Bouncer{
next: http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
nextCalled = true
}),
appsecEnabled: true,
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
httpAppsecClient: appsecServer.Client(),
remediationStatusCode: http.StatusForbidden,
remediationCustomHeader: "X-Remediation",
log: logger.New("DEBUG", ""),
}
recorder := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "http://example.com/protected", nil)
bouncer.handleNextServeHTTP(recorder, req, "192.0.2.10")
if nextCalled {
t.Fatal("next handler should not be called for appsec challenge")
}
if recorder.Code != http.StatusOK {
t.Fatalf("expected challenge status 200, got %d", recorder.Code)
}
if got := recorder.Body.String(); got != "<html>challenge</html>" {
t.Fatalf("expected appsec challenge body, got %q", got)
}
if got := recorder.Header().Get("Content-Type"); got != "text/html" {
t.Fatalf("expected Content-Type relayed, got %q", got)
}
if got := recorder.Header().Get("Set-Cookie"); got != "__crowdsec_challenge=value; Path=/; HttpOnly" {
t.Fatalf("expected Set-Cookie relayed, got %q", got)
}
if got := recorder.Header().Get("X-Remediation"); got != "challenge" {
t.Fatalf("expected custom remediation header challenge, got %q", got)
}
}
func TestHandleNextServeHTTPLegacyAppsecForbiddenFallsBackToBan(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
}))
defer appsecServer.Close()
appsecURL, err := url.Parse(appsecServer.URL)
if err != nil {
t.Fatal(err)
}
nextCalled := false
bouncer := &Bouncer{
next: http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
nextCalled = true
}),
appsecEnabled: true,
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
httpAppsecClient: appsecServer.Client(),
remediationStatusCode: http.StatusForbidden,
remediationCustomHeader: "X-Remediation",
log: logger.New("DEBUG", ""),
}
recorder := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "http://example.com/protected", nil)
bouncer.handleNextServeHTTP(recorder, req, "192.0.2.10")
if nextCalled {
t.Fatal("next handler should not be called for appsec forbidden")
}
if recorder.Code != http.StatusForbidden {
t.Fatalf("expected fallback ban status 403, got %d", recorder.Code)
}
if got := recorder.Header().Get("X-Remediation"); got != "ban" {
t.Fatalf("expected fallback remediation header ban, got %q", got)
}
}
// Test_appsecQuery_reusesConnection is a regression test for issue #384: the appsec response
// body must be drained for non-200 status response so net/http can return the conn to the idle pool.
func Test_appsecQuery_reusesConnection(t *testing.T) {
for _, status := range []int{http.StatusOK, http.StatusForbidden, http.StatusInternalServerError} {
t.Run(http.StatusText(status), func(t *testing.T) {
var mu sync.Mutex
conns := map[string]bool{}
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
mu.Lock()
conns[r.RemoteAddr] = true
mu.Unlock()
rw.WriteHeader(status)
if _, errWrite := rw.Write([]byte(`{"action":"allow"}`)); errWrite != nil {
t.Errorf("appsec stub write: %v", errWrite)
}
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecFailureBlock: false,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
const calls = 10
for i := 0; i < calls; i++ { //nolint:intrange
req, _ := http.NewRequest(http.MethodGet, "http://localhost/", nil)
_, _ = appsecQuery(bouncer, "1.2.3.4", req)
}
mu.Lock()
defer mu.Unlock()
if len(conns) != 1 {
t.Errorf("appsecQuery() opened %d connections for %d calls, want 1 (response body not drained?)", len(conns), calls)
}
})
}
}
func TestHandleNextServeHTTPStructuredBanKeepsBanTemplate(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{"action":"ban","http_status":403,"user_body_content":"appsec default page"}`))
}))
defer appsecServer.Close()
appsecURL, err := url.Parse(appsecServer.URL)
if err != nil {
t.Fatal(err)
}
banTemplate, err := template.New("ban").Parse("<html>BAN ip={{.ClientIP}} reason={{.RemediationReason}}</html>")
if err != nil {
t.Fatal(err)
}
nextCalled := false
bouncer := &Bouncer{
next: http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
nextCalled = true
}),
appsecEnabled: true,
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
httpAppsecClient: appsecServer.Client(),
remediationStatusCode: http.StatusForbidden,
remediationCustomHeader: "X-Remediation",
banTemplate: banTemplate,
banTemplateContentType: "text/html; charset=utf-8",
log: logger.New("ERROR", ""),
}
recorder := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "http://example.com/protected", nil)
bouncer.handleNextServeHTTP(recorder, req, "192.0.2.10")
if nextCalled {
t.Fatal("next handler should not be called for an appsec ban")
}
if recorder.Code != http.StatusForbidden {
t.Fatalf("expected 403, got %d", recorder.Code)
}
want := "<html>BAN ip=192.0.2.10 reason=APPSEC</html>"
if got := recorder.Body.String(); got != want {
t.Fatalf("appsec ban must keep the configured ban template, got %q want %q", got, want)
}
if got := recorder.Header().Get("Content-Type"); got != "text/html; charset=utf-8" {
t.Fatalf("expected banTemplateContentType, got %q", got)
}
if got := recorder.Header().Get("X-Remediation"); got != "ban" {
t.Fatalf("expected remediation header ban, got %q", got)
}
}
func Test_appsecQuery_unreadableBodyMethods(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
tests := []struct {
method string
wantDropped bool
}{
{method: http.MethodGet, wantDropped: false},
{method: http.MethodHead, wantDropped: false},
{method: http.MethodOptions, wantDropped: false},
{method: http.MethodDelete, wantDropped: false},
{method: http.MethodPost, wantDropped: true},
{method: http.MethodPut, wantDropped: true},
{method: http.MethodPatch, wantDropped: true},
}
for _, tt := range tests {
t.Run(tt.method, func(t *testing.T) {
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreadableBodyBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
_, err := appsecQuery(bouncer, "1.2.3.4", newUnreadableRequest(tt.method, done))
finished <- err
}()
select {
case err := <-finished:
if tt.wantDropped && err == nil {
t.Errorf("appsecQuery() on an unreadable-body %s: expected the request to be dropped, got nil", tt.method)
}
if !tt.wantDropped && err != nil {
t.Errorf("appsecQuery() on a bodyless HTTP/3 %s returned error: %v", tt.method, err)
}
case <-time.After(2 * time.Second):
t.Fatalf("appsecQuery() blocked on an unreadable %s body", tt.method)
}
})
}
}