Files
crowdsec-bouncer-traefik-pl…/pkg/cache/cache.go
T
mathieuHaandClaude Opus 5 955391671c ♻️ review fixes for #368: CIDR key correctness, lookup cost, tests (#372)
* ♻️ cidr: build keys through net.IPNet instead of hand-masking bytes

CIDRKeys masked the address byte by byte and formatted the result with
string concatenation, while SetCIDR/DeleteCIDR format their keys with
net.IPNet.String() via NormalizeCIDR. The two agreed only by coincidence:
any divergence in formatting silently stops every range decision from
matching, with no test covering the invariant.

Mask with net.IP.Mask and format through net.IPNet.String() so both sides
go through the same formatter. Output is byte for byte identical to the
previous implementation (checked against a golden dump of both IPv4 and
IPv6 keys, including ::ffff: forms).

Dropping the inner byte loops also removes the only intrange violation in
the tree, so the linter exclusion added for them is no longer needed, and
the redundant import alias on pkg/ip goes away with it.

*  cidr: only probe the prefix lengths that have a decision

In stream mode nothing caches a negative result per IP, so the exact IP
lookup misses on every legitimate request and each one fell through to
GetCIDR, which probed every possible prefix length: 33 cache reads for an
IPv4 client, 129 for an IPv6 one, even when no range decision existed at
all. On the local cache that is wasted work on the request path; with
redis it is 33 to 129 sequential round trips per request.

Keep the set of prefix lengths that have at least one decision under a
single key, written before the decision itself, and probe only those.
Measured cache reads per request: 1 with no range decision (was 33 / 129),
2 with a single /24 in use, 4 with four prefix lengths in use.

The set only grows, so a deleted or expired decision leaves a length
behind that costs one extra read rather than risking an unmatched
decision, and it is written with an effectively infinite duration since it
has to outlive every decision it describes. If it is ever missing while
decisions live (a redis eviction under maxmemory), range decisions stop
matching until the next one arrives; it is the hottest key of the
namespace, so an LRU policy evicts it last.

* 🔊 cidr: log the decisions dropped for an unparsable CIDR

SetCIDR and DeleteCIDR returned silently when NormalizeCIDR rejected the
value, so a range decision the plugin does not understand is not enforced
and nothing says why. Every other operation of the package logs, and this
one fails open, which is the direction worth shouting about.

Log at Error with the raw value and what the consequence is, so an
unexpected decision format shows up in the logs instead of looking like a
decision that was applied.

*  cidr: cover the invariant the range matching rests on

The helpers were tested in isolation but nothing tied them together, and
what actually has to hold is that the key SetCIDR writes for a decision is
one of the keys GetCIDR looks up for an IP that decision covers. A
formatting change on either side would have silently stopped every range
decision from matching with all tests green.

TestCIDRKeys_MatchNormalizeCIDR pins that both ways, including the cases
worth being explicit about: a decision that is not on a network address,
IPv4 mapped clients against an IPv4 range, and that the two families do not
mix. Dropping the mask in cidrKey fails 9 of its cases.

Also covers CIDRLookupKeys against CIDRKeys length by length, the IPv6 side
of the network address test, and CIDRPrefixLen.

*  cache: cover the CIDR operations and what a lookup costs

pkg/cache had tests for Get, Set and Delete but none for their CIDR
counterparts, so the range keyspace was only exercised end to end by the
e2e scenario.

Test_GetCIDR covers hits, the boundaries of a range, IPv6, IPv4 mapped
clients and invalid input. Test_GetCIDR_MostSpecific pins the precedence
between overlapping decisions, which is deliberate behaviour that nothing
was holding in place: a captcha on a /24 is not overruled by a ban on its
/8. Test_DeleteCIDR checks the wider decision survives a narrower one being
removed, and Test_SetCIDR_InvalidIsNotStored that a rejected value stores
nothing at all.

Test_GetCIDR_Reads counts cache reads through an isolated cacheInterface,
so the cost of a lookup is part of the contract: probing every prefix
length again turns it into 34 reads for IPv4 and 130 for IPv6 and fails.

* 🐛 e2e: stop racing the deadline in the stream failure check

handleStreamTicker compares updateFailure to updateMaxFailure before
incrementing it, so with updateMaxFailure 2 and a 1s interval the bouncer
gives up on the third consecutive failed poll, roughly 3s after the
endpoint starts failing. The check slept 2s and then polled for a 200 for
up to 15s, leaving about a second of margin, and once that window closes it
never reopens: a runner under load turns this into a 15s wait followed by a
failure.

Assert the 200 immediately after the endpoint starts failing, which is
always inside the window, and keep polling for the 403 that follows.

* 🐛 e2e: do not fail a scenario on a single slow response

The new -m 1 is right for the polling helpers, where a timed out request is
just another attempt, but the assertions and the mock control plane have no
retry: one request that takes over a second on a loaded runner fails the
scenario, and since common.sh runs under set -euo pipefail a timed out
lapi_add_decision aborts it before the decision even exists.

Bound the connect at 1s instead and give the whole request 5s in the seven
places that get a single attempt. Nothing waits longer on the happy path.

*  e2e: restore the stream-mode failure check

Revert 99673b1. The check is not ours to change: it must stay as it was.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* 🔥 e2e: drop the comment above wait_for_status

The timeout change in 3f78887 stands; only the comment goes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 21:10:44 +02:00

234 lines
6.9 KiB
Go

// Package cache implements utility routines for manipulating cache.
// It supports currently local file and redis cache.
package cache
import (
"errors"
"fmt"
"log/slog"
"strconv"
"strings"
"sync/atomic"
ttl_map "github.com/leprosus/golang-ttl-map"
simpleredis "github.com/maxlerebourg/simpleredis"
"github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/pkg/ip"
)
const (
// BannedValue Banned string.
BannedValue = "t"
// NoBannedValue No banned string.
NoBannedValue = "f"
// CaptchaValue Need captcha string.
CaptchaValue = "c"
// CaptchaDoneValue Captcha done string.
CaptchaDoneValue = "d"
// CacheMiss error string when cache is miss.
CacheMiss = "cache:miss"
// CacheUnreachable error string when cache is unreachable.
CacheUnreachable = "cache:unreachable"
// cidrPrefix namespaces a CIDR decision, one cache key per CIDR.
cidrPrefix = "cidr:"
// cidrPrefixLensKey holds the prefix lengths that have a decision, so a lookup probes
// only those. Its absence means no CIDR decision was ever stored.
cidrPrefixLensKey = "cidrprefixlens"
// cidrPrefixLensSeparator separates the prefix lengths in cidrPrefixLensKey.
cidrPrefixLensSeparator = ","
// cidrPrefixLensDuration has to outlive every decision it describes, so it is
// effectively infinite. It cannot be zero: the local cache ignores a zero duration
// and redis rejects a non positive EX.
cidrPrefixLensDuration = 10 * 365 * 24 * 60 * 60
)
//nolint:gochecknoglobals
var cache = ttl_map.New()
type localCache struct{}
func (localCache) get(key string) (string, error) {
value, isCached := cache.Get(key)
valueString, isValid := value.(string)
if isCached && isValid && len(valueString) > 0 {
return valueString, nil
}
return "", errors.New(CacheMiss)
}
func (localCache) set(key, value string, duration int64) {
cache.Set(key, value, duration)
}
func (localCache) delete(key string) {
cache.Del(key)
}
type redisCache struct {
log *slog.Logger
writer simpleredis.SimpleRedis
readers []simpleredis.SimpleRedis
counter atomic.Uint64
}
func (rc *redisCache) nextReader() *simpleredis.SimpleRedis {
n := len(rc.readers)
if n == 0 {
return &rc.writer
}
idx := rc.counter.Add(1) % uint64(n)
return &rc.readers[idx]
}
func (rc *redisCache) get(key string) (string, error) {
value, err := rc.nextReader().Get(key)
if err != nil {
switch err.Error() {
case simpleredis.RedisMiss:
return "", errors.New(CacheMiss)
case simpleredis.RedisUnreachable:
return "", errors.New(CacheUnreachable)
default:
return "", err
}
}
valueString := string(value)
if len(valueString) > 0 {
return valueString, nil
}
return "", errors.New(CacheMiss)
}
func (rc *redisCache) set(key, value string, duration int64) {
if err := rc.writer.Set(key, []byte(value), duration); err != nil {
rc.log.Error("cache:setDecisionRedisCache" + err.Error())
}
}
func (rc *redisCache) delete(key string) {
if err := rc.writer.Del(key); err != nil {
rc.log.Error("cache:deleteDecisionRedisCache " + err.Error())
}
}
type cacheInterface interface {
set(key, value string, duration int64)
get(key string) (string, error)
delete(key string)
}
// Client Cache client.
type Client struct {
cache cacheInterface
log *slog.Logger
}
// New Initialize cache client.
func (c *Client) New(log *slog.Logger, isRedis bool, writeHost string, readHosts []string, pass, database string) {
c.log = log
if isRedis {
rc := &redisCache{log: log}
rc.writer.Init(writeHost, pass, database)
for _, h := range readHosts {
var r simpleredis.SimpleRedis
r.Init(h, pass, database)
rc.readers = append(rc.readers, r)
}
c.cache = rc
} else {
c.cache = &localCache{}
}
c.log.Debug(fmt.Sprintf("cache:New initialized isRedis:%v writeHost:%v readHosts:%v", isRedis, writeHost, readHosts))
}
// Delete delete decision in cache.
func (c *Client) Delete(key string) {
c.log.Debug(fmt.Sprintf("cache:Delete key:%v", key))
c.cache.delete(key)
}
// Get check in the cache if the IP has the banned / not banned value.
// Otherwise return with an error to add the IP in cache if we are on.
func (c *Client) Get(key string) (string, error) {
c.log.Debug(fmt.Sprintf("cache:Get key:%v", key))
return c.cache.get(key)
}
// Set update the cache with the IP as key and the value banned / not banned.
func (c *Client) Set(key string, value string, duration int64) {
c.log.Debug(fmt.Sprintf("cache:Set key:%v value:%v duration:%vs", key, value, duration))
c.cache.set(key, value, duration)
}
// DeleteCIDR removes a CIDR decision from the cache.
func (c *Client) DeleteCIDR(cidr string) {
normalized := ip.NormalizeCIDR(cidr)
if normalized == "" {
c.log.Error(fmt.Sprintf("cache:DeleteCIDR:invalidCIDR cidr:%v decision is left in cache", cidr))
return
}
cidr = normalized
c.cache.delete(cidrPrefix + cidr)
c.log.Debug(fmt.Sprintf("cache:DeleteCIDR cidr:%v", cidr))
}
// GetCIDR checks if an IP matches a CIDR decision in the cache.
// Only probes the prefix lengths that have a decision: it is on the request path.
func (c *Client) GetCIDR(ipStr string) (string, error) {
prefixLens, err := c.cache.get(cidrPrefixLensKey)
if err != nil {
return "", err
}
for _, key := range ip.CIDRLookupKeys(ipStr, parsePrefixLens(prefixLens)) {
value, getErr := c.cache.get(cidrPrefix + key)
if getErr == nil {
return value, nil
}
}
return "", errors.New(CacheMiss)
}
// SetCIDR stores a CIDR decision in the cache.
func (c *Client) SetCIDR(cidr, value string, duration int64) {
normalized := ip.NormalizeCIDR(cidr)
prefixLen := ip.CIDRPrefixLen(cidr)
if normalized == "" || prefixLen < 0 {
c.log.Error(fmt.Sprintf("cache:SetCIDR:invalidCIDR cidr:%v value:%v decision is not enforced", cidr, value))
return
}
// Publish the length first, or a concurrent lookup misses the decision.
c.addCIDRPrefixLen(prefixLen)
c.cache.set(cidrPrefix+normalized, value, duration)
c.log.Debug(fmt.Sprintf("cache:SetCIDR cidr:%v value:%v duration:%vs", normalized, value, duration))
}
// addCIDRPrefixLen records a prefix length in the set probed on lookup. The set only grows:
// a stale length costs one extra read, dropping one too early leaves decisions unmatched.
func (c *Client) addCIDRPrefixLen(prefixLen int) {
prefixLens, err := c.cache.get(cidrPrefixLensKey)
if err == nil {
for _, known := range parsePrefixLens(prefixLens) {
if known == prefixLen {
return
}
}
prefixLens += cidrPrefixLensSeparator + strconv.Itoa(prefixLen)
} else {
prefixLens = strconv.Itoa(prefixLen)
}
c.cache.set(cidrPrefixLensKey, prefixLens, cidrPrefixLensDuration)
c.log.Debug(fmt.Sprintf("cache:addCIDRPrefixLen prefixLens:%v", prefixLens))
}
// parsePrefixLens decodes the set of prefix lengths stored in cidrPrefixLensKey.
func parsePrefixLens(value string) []int {
fields := strings.Split(value, cidrPrefixLensSeparator)
prefixLens := make([]int, 0, len(fields))
for _, field := range fields {
if prefixLen, err := strconv.Atoi(field); err == nil {
prefixLens = append(prefixLens, prefixLen)
}
}
return prefixLens
}