Compare commits

..
Author SHA1 Message Date
maxlerebourgandRenovate Bot 9daba9739c ⬆️ renovate: Update all (#350)
Co-authored-by: Renovate Bot <22881669+maxlerebourg@users.noreply.github.com>
2026-07-26 17:06:02 +02:00
maxlerebourg e98b8ed5ba Renovate update version.go (#360)
*  Renovate update version.go

* 🍱 renovate every day
2026-07-26 17:05:43 +02:00
bb44aef718 feat: Allow cache reading from replicas (#342)
* feat: Allow cache reading from replicas

* 🍱 fix logic

*  add testing for redis with mock

* 🍱 fix permission

* 📝 test(e2e/redis): fix swapped IP→verdict comments

The mock returns "f" (not banned) for 1.2.3.4 and "t" (banned) for
1.2.3.5, and the run.sh assertions match that. Both doc comments
described the opposite mapping; correct them to match the code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

*  test(e2e/redis): exercise read-from-replica path

The redis scenario only set redisCacheHost, so it validated the writer
but never the round-robin reader path this feature adds. Split the mock
into two roles: the primary (--redis-addr) now answers every GET with a
miss, while the replica (--redis-read-addr) serves the hardcoded
verdicts. The scenario points redisCacheReadHosts at the replica (twice,
to drive round-robin), so the banned-IP-blocked assertion only passes if
the plugin actually reads decisions from the replica rather than the
primary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* 📝 docs: note replicas don't fall back to primary on outage

When RedisCacheReadHosts is set, reads are not retried against the
primary if the replicas are unreachable. Document that this, combined
with the default RedisCacheUnreachableBlock=true, means a replica outage
can block traffic while the primary is healthy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* 🐛 fix(cache): avoid nil-pointer panic on empty redis read

redisCache.get fell through to `switch err.Error()` when Get returned a
nil error with an empty value, panicking on the nil error. simpleredis
never returns that combination today (a miss yields RedisMiss), so it
was unreachable in practice — but the read path is safer treating an
empty, error-free read as a cache miss, which also guarantees err is
non-nil before err.Error() is called.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* 🍱 add test for rotation

* 🐛 readd redis/run.sh

* 🐛 fix redis/run.sh

* 🐛 fix test label; remove log

* 🍱 add tests for roundRobin

* 🐛 fix test

---------

Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
Co-authored-by: mhx <mathieu@hanotaux.fr>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-26 13:51:22 +02:00
31874263f2 🐛 Do not consider body unreadable when it's http.NoBody (#352)
* Do not consider body unreadable when it's http.NoBody

Fix #351

* 🐛 appsec: only drop unreadable bodies for methods that carry one

Gate the crowdsecAppsecUnreadableBodyBlock drop on the request method,
mirroring the reference lua-cs-bouncer METHODS_WITH_BODY list
(POST/PUT/PATCH/DELETE). Browser GETs over HTTP/3 never carry a
Content-Length (quic-go always wraps the stream in a non-nil body with
ContentLength -1), so they were all wrongly dropped; they are now
forwarded to appsec headers-only, while gRPC streams (always POST) stay
droppable.

Fix #351

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* 🍱 add test and remove useless comment

* 🍱 add test for GET with unreadable body

* 🍱 fix log

* 🍱 fix test

---------

Co-authored-by: mhx <mathieu@hanotaux.fr>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: maxlerebourg <maxlerebourg@gmail.com>
2026-07-24 20:49:43 +02:00
maxlerebourgandRenovate Bot 1c98c70f14 ⬆️ renovate: Update all (#349)
Co-authored-by: Renovate Bot <22881669+maxlerebourg@users.noreply.github.com>
2026-07-05 19:56:15 +02:00
mathieuHaandClaude Fable 5 be13c49144 🐛 ci(renovate): fix OOM crash and stop using Mend's default gitAuthor (#347)
The 2026-07-03 runs surfaced two config problems:

- The grouped "renovate/all" branch made Renovate fetch and hold the
  changelog of every upgrade instance separately (Traefik v3.0.0->v3.7.6
  was fetched ~10x, once per compose file), which blew the default 4GB
  V8 heap: "FATAL ERROR: ... JavaScript heap out of memory" (exit 134),
  killing the run after the branch was pushed but before the PR was
  opened. Disable changelog fetching (release notes were truncated in
  the grouped PR body anyway) and raise the Node heap to 8GB as a
  safety net.

- No gitAuthor was set, so commits were authored as Mend's
  renovate@whitesourcesoftware.com, which GitHub flags "Unverified"
  (Vigilant Mode) and Renovate warns about on every run. Use the token
  owner's noreply address instead.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 19:36:44 +02:00
33 changed files with 346 additions and 136 deletions
-46
View File
@@ -1,46 +0,0 @@
name: Release Version Update
on:
release:
types: [published]
permissions:
contents: write
jobs:
update-version:
name: Update version in source
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
ref: main
- name: Extract version from tag
id: get_version
run: |
TAG="${{ github.event.release.tag_name }}"
VERSION="${TAG#v}"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
- name: Update version in version.go
run: |
sed -i 's/pluginVersion = "[^"]*"/pluginVersion = "'"${{ steps.get_version.outputs.version }}"'"/' version.go
cat version.go
- name: Commit, push, and retag
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add version.go
if git diff --cached --quiet; then
echo "Version already up to date, nothing to commit"
exit 0
fi
git commit -m "⬆️ chore: bump version to ${{ steps.get_version.outputs.version }}"
git push origin main
# Move the release tag to include the version update
git tag -f "${{ steps.get_version.outputs.tag }}"
git push -f origin "${{ steps.get_version.outputs.tag }}"
+3 -3
View File
@@ -1,6 +1,6 @@
name: Renovate
# Self-hosted Renovate: opens dependency-update PRs on a weekly schedule.
# Self-hosted Renovate: opens dependency-update PRs on a daily schedule.
# Config lives in /renovate.json. Requires a repo/org secret RENOVATE_TOKEN
# (a PAT with `repo` + `workflow` scope, or a fine-grained token with
# contents:write + pull-requests:write) so Renovate can push branches and open
@@ -8,7 +8,7 @@ name: Renovate
on:
schedule:
- cron: "0 4 * * 1" # every Monday at 04:00 UTC
- cron: "0 4 * * *" # every day at 04:00 UTC
workflow_dispatch:
inputs:
logLevel:
@@ -28,7 +28,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Run Renovate
uses: renovatebot/github-action@v46.1.14
uses: renovatebot/github-action@v46.1.20
with:
token: ${{ secrets.RENOVATE_TOKEN }}
env:
+1
View File
@@ -41,6 +41,7 @@ linters-settings:
- $test
allow:
- $gostd
- github.com/maxlerebourg/simpleredis
- github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger
- github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip
- github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/configuration
+2 -3
View File
@@ -4,7 +4,7 @@ export GO111MODULE=on
# Binary/mock suite (Traefik binary + mock LAPI). This is what CI runs.
# The local Docker suite (make e2e) lives in a separate PR/branch.
E2E_MOCK_SCENARIOS := stream-mode live-mode none-mode trusted-ips custom-ban-page captcha appsec tls-system-ca
E2E_MOCK_SCENARIOS := $(notdir $(wildcard tests/e2e/mock/scenarios/*))
default: lint test
@@ -20,7 +20,7 @@ yaegi_test:
e2e_mock: $(addprefix e2e_mock_,$(E2E_MOCK_SCENARIOS))
e2e_mock_%:
./tests/e2e/mock/scenarios/$*/run.sh
bash ./tests/e2e/mock/scenarios/$*/run.sh
vendor:
go mod vendor
@@ -124,4 +124,3 @@ show_metrics:
show_decisions:
docker exec crowdsec cscli decisions list
+10 -2
View File
@@ -444,7 +444,12 @@ make run
- RedisCacheHost
- string
- default: "redis:6379"
- hostname and port for the Redis service
- hostname and port for the Redis write host (primary)
- RedisCacheReadHosts
- []string
- default: []
- List of Redis replica hostnames (host:port) to use for read operations. Reads are distributed round-robin across replicas. Falls back to RedisCacheHost when empty.
- Note: when set, reads are not retried against RedisCacheHost (the primary) if the replicas are unreachable. With RedisCacheUnreachableBlock at its default (true), a replica outage will therefore block/delay requests even though the primary is healthy.
- RedisCachePassword
- string
- default: ""
@@ -640,7 +645,10 @@ http:
forwardedHeadersCustomName: X-Custom-Header
remediationHeadersCustomName: cs-remediation
redisCacheEnabled: false
redisCacheHost: "redis:6379"
redisCacheHost: "redis-primary:6379"
redisCacheReadHosts:
- "redis-replica-1:6379"
- "redis-replica-2:6379"
redisCachePassword: password
redisCacheDatabase: "5"
redisCacheUnreachableBlock: true
+13 -3
View File
@@ -268,6 +268,7 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
log,
config.RedisCacheEnabled,
config.RedisCacheHost,
config.RedisCacheReadHosts,
config.RedisCachePassword,
config.RedisCacheDatabase,
)
@@ -732,7 +733,17 @@ func crowdsecQuery(bouncer *Bouncer, stringURL string, data []byte) ([]byte, err
// a 403. This mirrors the reference lua-cs-bouncer behavior, which refuses to
// read the body of an HTTP/2+ request that has no Content-Length.
func isBodyUnreadable(httpReq *http.Request) bool {
return httpReq.Body != nil && httpReq.ProtoMajor >= 2 && httpReq.ContentLength < 0
return httpReq.Body != nil && httpReq.Body != http.NoBody && httpReq.ProtoMajor >= 2 && httpReq.ContentLength < 0
}
// isMethodWithBody used only when isBodyUnreadable returns true but the request method can't have body.
func isMethodWithBody(method string) bool {
switch method {
case http.MethodPost, http.MethodPut, http.MethodPatch, http.MethodDelete:
return true
default:
return false
}
}
func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
@@ -744,8 +755,7 @@ func appsecQuery(bouncer *Bouncer, ip string, httpReq *http.Request) error {
var req *http.Request
switch {
case isBodyUnreadable(httpReq):
if bouncer.appsecUnreadableBodyBlock {
// The caller (handleNextServeHTTP) logs this returned error with the IP.
if bouncer.appsecUnreadableBodyBlock && isMethodWithBody(httpReq.Method) {
return errors.New("appsecQuery:unreadableBody dropped")
}
req, _ = http.NewRequest(http.MethodGet, routeURL.String(), nil)
+53 -11
View File
@@ -7,6 +7,7 @@ import (
"net/http/httptest"
"net/url"
"reflect"
"strings"
"testing"
"text/template"
"time"
@@ -396,29 +397,27 @@ func (b blockingBody) Read(_ []byte) (int, error) {
func (blockingBody) Close() error { return nil }
func Test_isBodyUnreadable(t *testing.T) {
realBody := func() io.ReadCloser { return io.NopCloser(strings.NewReader("data")) }
tests := []struct {
name string
protoMajor int
contentLength int64
hasBody bool
body io.ReadCloser
want bool
}{
{name: "http2 grpc stream without content-length", protoMajor: 2, contentLength: -1, hasBody: true, want: true},
{name: "http3 stream without content-length", protoMajor: 3, contentLength: -1, hasBody: true, want: true},
{name: "http2 with content-length", protoMajor: 2, contentLength: 42, hasBody: true, want: false},
{name: "http1.1 chunked without content-length", protoMajor: 1, contentLength: -1, hasBody: true, want: false},
{name: "http2 without body", protoMajor: 2, contentLength: -1, hasBody: false, want: false},
{name: "http2 grpc stream without content-length", protoMajor: 2, contentLength: -1, body: realBody(), want: true},
{name: "http3 stream without content-length", protoMajor: 3, contentLength: -1, body: realBody(), want: true},
{name: "http2 with content-length", protoMajor: 2, contentLength: 42, body: realBody(), want: false},
{name: "http1.1 chunked without content-length", protoMajor: 1, contentLength: -1, body: realBody(), want: false},
{name: "http2 without body", protoMajor: 2, contentLength: -1, body: nil, want: false},
{name: "http2 with http.NoBody", protoMajor: 2, contentLength: -1, body: http.NoBody, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, "http://localhost", nil)
req.ProtoMajor = tt.protoMajor
req.ContentLength = tt.contentLength
if tt.hasBody {
req.Body = http.NoBody
} else {
req.Body = nil
}
req.Body = tt.body
if got := isBodyUnreadable(req); got != tt.want {
t.Errorf("isBodyUnreadable() = %v, want %v", got, tt.want)
}
@@ -513,3 +512,46 @@ func Test_appsecQuery_dropUnreadableBody(t *testing.T) {
t.Fatal("appsecQuery() blocked on a streaming request body (issue #323 regression)")
}
}
func newUnreadableGetRequest(done <-chan struct{}) *http.Request {
req, _ := http.NewRequest(http.MethodGet, "http://localhost/", blockingBody{done: done})
req.ProtoMajor = 3
req.ContentLength = -1
return req
}
// Test_appsecQuery_unreadableBodyGetNotDropped is a regression test for issue #351
func Test_appsecQuery_unreadableBodyGetNotDropped(t *testing.T) {
appsecServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
rw.WriteHeader(http.StatusOK)
}))
defer appsecServer.Close()
appsecURL, _ := url.Parse(appsecServer.URL)
bouncer := &Bouncer{
appsecScheme: appsecURL.Scheme,
appsecHost: appsecURL.Host,
appsecPath: "/",
appsecBodyLimit: 10485760,
appsecUnreadableBodyBlock: true,
httpAppsecClient: appsecServer.Client(),
log: logger.New("INFO", ""),
}
done := make(chan struct{})
defer close(done)
finished := make(chan error, 1)
go func() {
finished <- appsecQuery(bouncer, "1.2.3.4", newUnreadableGetRequest(done))
}()
select {
case err := <-finished:
if err != nil {
t.Errorf("appsecQuery() on an HTTP/3 GET without content-length returned error: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("appsecQuery() blocked on an HTTP/3 GET request body (issue #351 regression)")
}
}
+2 -2
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.5.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -80,7 +80,7 @@ services:
- "traefik.http.routers.router-bar3.entrypoints=web"
- "traefik.http.routers.router-bar3.middlewares=crowdsec2@docker"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -12,7 +12,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
# - './ban.html:/ban.html:ro'
@@ -59,7 +59,7 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.forwardedheaderstrustedips=172.21.0.5"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.5.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.5.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -47,7 +47,7 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdsecappsechost=crowdsec:7422"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+4 -4
View File
@@ -1,6 +1,6 @@
services:
cloudflare:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "cloudflare"
restart: unless-stopped
command:
@@ -19,7 +19,7 @@ services:
- 8080:8080
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -33,7 +33,7 @@ services:
- "--entrypoints.web.forwardedheaders.trustedips=172.21.0.5"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- logs-traefik:/var/log/traefik
@@ -79,7 +79,7 @@ services:
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -55,7 +55,7 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.captchaHTMLFilePath=/captcha.html"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -46,7 +46,7 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.banFilePath=/ban.html"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.5.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -14,7 +14,7 @@ services:
- "--entrypoints.web.forwardedheaders.trustedips=172.18.0.0/24"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.4.5"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -59,7 +59,7 @@ services:
- "traefik.http.middlewares.crowdsec.plugin.bouncer.captchaHTMLFilePath=/captcha.html"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+1 -1
View File
@@ -1,5 +1,5 @@
image:
tag: v1.6.1-2
tag: v1.7.8-2
agent:
acquisition:
+2 -2
View File
@@ -1,5 +1,5 @@
image:
tag: v3.0.0
tag: v3.7.8
logs:
general:
@@ -15,4 +15,4 @@ experimental:
plugins:
bouncer:
moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
version: "v1.3.0"
version: "v1.6.0"
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -71,7 +71,7 @@ services:
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+2 -2
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.5.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.5.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -71,7 +71,7 @@ services:
# Define AppSec host and port informations
- "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdsecappsechost=crowdsec:7422"
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+3 -3
View File
@@ -1,6 +1,6 @@
services:
traefik:
image: "traefik:v3.0.0"
image: "traefik:v3.7.8"
container_name: "traefik"
restart: unless-stopped
command:
@@ -13,7 +13,7 @@ services:
- "--entrypoints.web.address=:80"
- "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
- "--experimental.plugins.bouncer.version=v1.3.0"
- "--experimental.plugins.bouncer.version=v1.6.0"
# - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
@@ -65,7 +65,7 @@ services:
crowdsec:
image: crowdsecurity/crowdsec:v1.6.8
image: crowdsecurity/crowdsec:v1.7.8
container_name: "crowdsec"
restart: unless-stopped
environment:
+1 -1
View File
@@ -1,6 +1,6 @@
module github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
go 1.22
go 1.22.12
require (
github.com/leprosus/golang-ttl-map v1.1.7
+43 -24
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"log/slog"
"sync/atomic"
ttl_map "github.com/leprosus/golang-ttl-map"
simpleredis "github.com/maxlerebourg/simpleredis"
@@ -27,10 +28,7 @@ const (
)
//nolint:gochecknoglobals
var (
redis simpleredis.SimpleRedis
cache = ttl_map.New()
)
var cache = ttl_map.New()
type localCache struct{}
@@ -52,33 +50,48 @@ func (localCache) delete(key string) {
}
type redisCache struct {
log *slog.Logger
log *slog.Logger
writer simpleredis.SimpleRedis
readers []simpleredis.SimpleRedis
counter atomic.Uint64
}
func (redisCache) get(key string) (string, error) {
value, err := redis.Get(key)
func (rc *redisCache) nextReader() *simpleredis.SimpleRedis {
n := len(rc.readers)
if n == 0 {
return &rc.writer
}
idx := rc.counter.Add(1) % uint64(n)
return &rc.readers[idx]
}
func (rc *redisCache) get(key string) (string, error) {
value, err := rc.nextReader().Get(key)
if err != nil {
switch err.Error() {
case simpleredis.RedisMiss:
return "", errors.New(CacheMiss)
case simpleredis.RedisUnreachable:
return "", errors.New(CacheUnreachable)
default:
return "", err
}
}
valueString := string(value)
if err == nil && len(valueString) > 0 {
if len(valueString) > 0 {
return valueString, nil
}
errRedisMessage := err.Error()
if errRedisMessage == simpleredis.RedisMiss {
return "", errors.New(CacheMiss)
}
if errRedisMessage == simpleredis.RedisUnreachable {
return "", errors.New(CacheUnreachable)
}
return "", err
return "", errors.New(CacheMiss)
}
func (rc redisCache) set(key, value string, duration int64) {
if err := redis.Set(key, []byte(value), duration); err != nil {
func (rc *redisCache) set(key, value string, duration int64) {
if err := rc.writer.Set(key, []byte(value), duration); err != nil {
rc.log.Error("cache:setDecisionRedisCache" + err.Error())
}
}
func (rc redisCache) delete(key string) {
if err := redis.Del(key); err != nil {
func (rc *redisCache) delete(key string) {
if err := rc.writer.Del(key); err != nil {
rc.log.Error("cache:deleteDecisionRedisCache " + err.Error())
}
}
@@ -96,15 +109,21 @@ type Client struct {
}
// New Initialize cache client.
func (c *Client) New(log *slog.Logger, isRedis bool, host, pass, database string) {
func (c *Client) New(log *slog.Logger, isRedis bool, writeHost string, readHosts []string, pass, database string) {
c.log = log
if isRedis {
redis.Init(host, pass, database)
c.cache = &redisCache{log: log}
rc := &redisCache{log: log}
rc.writer.Init(writeHost, pass, database)
for _, h := range readHosts {
var r simpleredis.SimpleRedis
r.Init(h, pass, database)
rc.readers = append(rc.readers, r)
}
c.cache = rc
} else {
c.cache = &localCache{}
}
c.log.Debug(fmt.Sprintf("cache:New initialized isRedis:%v", isRedis))
c.log.Debug(fmt.Sprintf("cache:New initialized isRedis:%v writeHost:%v readHosts:%v", isRedis, writeHost, readHosts))
}
// Delete delete decision in cache.
+38
View File
@@ -6,6 +6,7 @@ import (
"testing"
logger "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/logger"
simpleredis "github.com/maxlerebourg/simpleredis"
)
func Test_Get(t *testing.T) {
@@ -122,3 +123,40 @@ func Test_Delete(t *testing.T) {
})
}
}
// indexOfReader returns the position of r inside rc.readers, or -1 when r is the writer (the no-readers fallback).
func indexOfReader(rc *redisCache, r *simpleredis.SimpleRedis) int {
if r == &rc.writer {
return -1
}
for i := range rc.readers {
if r == &rc.readers[i] {
return i
}
}
return -2
}
func Test_nextReader(t *testing.T) {
// The counter starts at 0, so the first Add(1) yields index 1, then 2, 0, 1, ... over n readers.
tests := []struct {
name string
readers int
want []int
}{
{name: "round-robin over three readers", readers: 3, want: []int{1, 2, 0, 1, 2, 0, 1}},
{name: "single reader always selected", readers: 1, want: []int{0, 0, 0, 0, 0}},
{name: "no readers fall back to writer", readers: 0, want: []int{-1, -1, -1}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rc := &redisCache{log: logger.New("INFO", "")}
rc.readers = make([]simpleredis.SimpleRedis, tt.readers)
for call, want := range tt.want {
if got := indexOfReader(rc, rc.nextReader()); got != want {
t.Errorf("call %d: nextReader() -> reader[%d], want reader[%d]", call, got, want)
}
}
})
}
}
+2
View File
@@ -96,6 +96,7 @@ type Config struct {
ClientTrustedIPs []string `json:"clientTrustedIps,omitempty"`
RedisCacheEnabled bool `json:"redisCacheEnabled,omitempty"`
RedisCacheHost string `json:"redisCacheHost,omitempty"`
RedisCacheReadHosts []string `json:"redisCacheReadHosts,omitempty"`
RedisCachePassword string `json:"redisCachePassword,omitempty"`
RedisCachePasswordFile string `json:"redisCachePasswordFile,omitempty"`
RedisCacheDatabase string `json:"redisCacheDatabase,omitempty"`
@@ -172,6 +173,7 @@ func New() *Config {
ClientTrustedIPs: []string{},
RedisCacheEnabled: false,
RedisCacheHost: "redis:6379",
RedisCacheReadHosts: []string{},
RedisCachePassword: "",
RedisCacheDatabase: "",
RedisCacheUnreachableBlock: true,
+10
View File
@@ -25,6 +25,16 @@
}
],
"customManagers": [
{
"description": "Plugin self-pin in version.go (pluginVersion)",
"customType": "regex",
"managerFilePatterns": ["/^version\\.go$/"],
"matchStrings": [
"pluginVersion\\s*=\\s*\"(?<currentValue>v[0-9]+\\.[0-9]+\\.[0-9]+)\""
],
"depNameTemplate": "maxlerebourg/crowdsec-bouncer-traefik-plugin",
"datasourceTemplate": "github-tags"
},
{
"description": "Plugin self-pin in docker-compose CLI args (--experimental.plugins.bouncer.version=vX)",
"customType": "regex",
+7 -1
View File
@@ -14,12 +14,14 @@
set -euo pipefail
# Pinned to match the Docker suite (tests/e2e/scenarios/*/docker-compose.yml).
TRAEFIK_VERSION="${TRAEFIK_VERSION:-v3.7.1}"
TRAEFIK_VERSION="${TRAEFIK_VERSION:-v3.7.8}"
WEB_PORT="${WEB_PORT:-8000}"
LAPI_PORT="${LAPI_PORT:-8090}"
BACKEND_PORT="${BACKEND_PORT:-8091}"
APPSEC_PORT="${APPSEC_PORT:-8092}"
REDIS_PORT="${REDIS_PORT:-8093}"
REDIS_READ_PORT="${REDIS_READ_PORT:-8094}"
LAPI_KEY="${LAPI_KEY:-e2e-mock-key}"
MOCK_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -187,6 +189,8 @@ start_stack() {
-e "s|@@LAPI_HOST@@|127.0.0.1:${LAPI_PORT}|g" \
-e "s|@@APPSEC_HOST@@|127.0.0.1:${APPSEC_PORT}|g" \
-e "s|@@BACKEND_URL@@|http://127.0.0.1:${BACKEND_PORT}|g" \
-e "s|@@REDIS_HOST@@|127.0.0.1:${REDIS_PORT}|g" \
-e "s|@@REDIS_READ_HOST@@|127.0.0.1:${REDIS_READ_PORT}|g" \
-e "s|@@SCENARIO_DIR@@|${scenario_dir}|g" \
"$scenario_dir/dynamic.yml" > "$WORKDIR/dynamic.yml"
@@ -203,6 +207,8 @@ start_stack() {
--lapi-addr "127.0.0.1:${LAPI_PORT}" \
--backend-addr "127.0.0.1:${BACKEND_PORT}" \
--appsec-addr "127.0.0.1:${APPSEC_PORT}" \
--redis-addr "127.0.0.1:${REDIS_PORT}" \
--redis-read-addr "127.0.0.1:${REDIS_READ_PORT}" \
"${mock_tls_args[@]}" >"$WORKDIR/mock.log" 2>&1 &
MOCK_PID=$!
+1 -1
View File
@@ -3,4 +3,4 @@
// golangci-lint and `go mod vendor`. Stdlib only no dependencies.
module mocklapi
go 1.22
go 1.22.12
+59 -3
View File
@@ -2,7 +2,8 @@
// suite. It answers only the few LAPI routes the plugin calls — live/none
// decision lookups, the stream poll and the usage-metrics push — and lets the
// test drive decisions through /admin instead of `cscli`. It also serves the
// stub upstream that Traefik proxies allowed requests to.
// stub upstream that Traefik proxies allowed requests to, and a hardcoded Redis
// stand-in for exercising the redis cache path.
//
// It is NOT a Crowdsec/AppSec conformance harness — the real WAF engine (OWASP
// CRS, virtual patching) is out of scope. The AppSec endpoint here emulates a
@@ -11,10 +12,12 @@
package main
import (
"bufio"
"encoding/json"
"flag"
"io"
"log"
"net"
"net/http"
"strings"
"sync"
@@ -46,6 +49,50 @@ func list(m map[string]Decision) []Decision {
return out
}
// --- Redis mock (inline-command wire format, as spoken by simpleredis) ---
// serveRedis is a hardcoded stand-in. When verdicts is true it plays a replica
// that holds decisions: every line is scanned for known IPs, 1.2.3.4 → "f"
// (clean), 1.2.3.5 → "t" (banned); any other GET is a miss ($-1). When verdicts
// is false it plays the primary and answers every GET with a miss, so a
// scenario can prove reads are served from the replica and not the primary.
// SET, DEL, AUTH, SELECT get +OK (they don't read the response anyway).
func serveRedis(addr string, verdicts bool) {
ln, err := net.Listen("tcp", addr)
if err != nil {
log.Fatal(err)
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
continue
}
go func(conn net.Conn) {
defer conn.Close()
rd := bufio.NewReader(conn)
for {
line, _, err := rd.ReadLine()
if err != nil {
return
}
s := string(line)
switch {
case verdicts && strings.Contains(s, "1.2.3.4"):
conn.Write([]byte("$1\r\nf\r\n"))
case verdicts && strings.Contains(s, "1.2.3.5"):
conn.Write([]byte("$1\r\nt\r\n"))
case strings.HasPrefix(strings.ToUpper(s), "GET "):
conn.Write([]byte("$-1\r\n"))
default:
conn.Write([]byte("+OK\r\n"))
}
}
}(conn)
}
}
func main() {
lapiAddr := flag.String("lapi-addr", "127.0.0.1:8090", "address for the LAPI mock")
// The stub upstream Traefik proxies allowed requests to — the binary-suite
@@ -53,6 +100,12 @@ func main() {
backendAddr := flag.String("backend-addr", "127.0.0.1:8091", "address for the stub upstream service")
// AppSec WAF stand-in (the real engine listens on :7422). Not a CRS engine.
appsecAddr := flag.String("appsec-addr", "127.0.0.1:8092", "address for the AppSec mock")
// Redis stand-ins on plain TCP ports, enough to exercise the plugin's redis
// cache path. The primary answers every GET with a miss; the replica serves
// the hardcoded verdicts, so a scenario pointing redisCacheReadHosts at the
// replica proves reads are offloaded to replicas.
redisAddr := flag.String("redis-addr", "127.0.0.1:8093", "address for the Redis primary mock (writes; GET always misses)")
redisReadAddr := flag.String("redis-read-addr", "127.0.0.1:8094", "address for the Redis replica mock (serves cached verdicts)")
// Optional TLS for the LAPI: when both are set the LAPI is served over HTTPS
// (cert signed by the scenario's throwaway CA) so the suite can exercise the
// bouncer's system-trust-store path. Backend and AppSec stay plaintext.
@@ -96,6 +149,9 @@ func main() {
})))
}()
go serveRedis(*redisAddr, false)
go serveRedis(*redisReadAddr, true)
mux := http.NewServeMux()
// Readiness probe for the test harness (empty body, 200).
@@ -154,9 +210,9 @@ func main() {
})
if *lapiTLSCert != "" && *lapiTLSKey != "" {
log.Printf("mocklapi: LAPI on %s (TLS), backend on %s, appsec on %s", *lapiAddr, *backendAddr, *appsecAddr)
log.Printf("mocklapi: LAPI on %s (TLS), backend on %s, appsec on %s, redis on %s (read %s)", *lapiAddr, *backendAddr, *appsecAddr, *redisAddr, *redisReadAddr)
log.Fatal(http.ListenAndServeTLS(*lapiAddr, *lapiTLSCert, *lapiTLSKey, mux))
}
log.Printf("mocklapi: LAPI on %s, backend on %s, appsec on %s", *lapiAddr, *backendAddr, *appsecAddr)
log.Printf("mocklapi: LAPI on %s, backend on %s, appsec on %s, redis on %s (read %s)", *lapiAddr, *backendAddr, *appsecAddr, *redisAddr, *redisReadAddr)
log.Fatal(http.ListenAndServe(*lapiAddr, mux))
}
+1 -1
View File
@@ -18,7 +18,7 @@ http:
bouncer:
enabled: "true"
# IP bouncing disabled — this scenario exercises AppSec only.
crowdsecMode: none
crowdsecMode: appsec
crowdsecLapiScheme: http
crowdsecLapiHost: "@@LAPI_HOST@@"
crowdsecLapiKey: "@@APIKEY@@"
+9
View File
@@ -30,6 +30,15 @@ body() {
echo "[$SCENARIO] request that send bad body before crowdsecAppsecBodyLimit must pass (AppSec 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.4" -X POST -d "a=0&______"
echo "[$SCENARIO] request http2 that send no body GET (AppSec 200)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4" --http2-prior-knowledge -H "Content-Length:"
echo "[$SCENARIO] request http2 that send unreadable body GET (AppSec 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.4" --http2-prior-knowledge -H "Content-Length:" -d "test"
echo "[$SCENARIO] request http2 that send unreadable body POST (AppSec 403)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.4" --http2-prior-knowledge -H "Content-Length:" -X POST -d "test"
}
run_scenario "$SCENARIO" "$HERE" body
@@ -0,0 +1,30 @@
http:
routers:
r:
rule: "PathPrefix(`/foo`)"
entryPoints:
- web
service: backend
middlewares:
- bouncer
services:
backend:
loadBalancer:
servers:
- url: "@@BACKEND_URL@@"
middlewares:
bouncer:
plugin:
bouncer:
enabled: "true"
crowdsecMode: live
crowdsecLapiScheme: http
crowdsecLapiHost: "@@LAPI_HOST@@"
crowdsecLapiKey: "@@APIKEY@@"
redisCacheEnabled: "true"
redisCacheHost: "@@REDIS_HOST@@"
redisCacheReadHosts:
- "@@REDIS_READ_HOST@@"
- "@@REDIS_HOST@@"
forwardedHeadersTrustedIps:
- "127.0.0.1/32"
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
# shellcheck source=../../lib/common.sh
source "$HERE/../../lib/common.sh"
SCENARIO=redis
# The replica mock returns "f" (not banned) for 1.2.3.4 and "t" (banned) for 1.2.3.5.
# The primary mock always misses.
body() {
echo "[$SCENARIO] cached banned IP must not be blocked because call for primary (test rotation)"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.5"
echo "[$SCENARIO] cached banned IP must be blocked"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 403 -H "X-Forwarded-For: 1.2.3.5"
echo "[$SCENARIO] cached clean IP must pass"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.4"
echo "[$SCENARIO] unknown IP (redis miss) must fall through to LAPI and pass"
assert_status "http://127.0.0.1:${WEB_PORT}/foo" 200 -H "X-Forwarded-For: 1.2.3.6"
}
run_scenario "$SCENARIO" "$HERE" body
+2 -2
View File
@@ -1,4 +1,4 @@
package crowdsec_bouncer_traefik_plugin //nolint:revive,stylecheck
// pluginVersion is updated automatically by the release workflow.
var pluginVersion = "1.6.X" //nolint:gochecknoglobals
// pluginVersion is updated automatically by the release workflow and Renovate.
var pluginVersion = "v1.6.0" //nolint:gochecknoglobals