Architecture

Component overview, storage backends, and deployment patterns

Anubis is designed as a modular reverse proxy with pluggable components for storage, policy evaluation, and external integrations.

System Components

Core Components

Reverse Proxy

Location: lib/anubis.go:Server

The main HTTP handler that intercepts all requests:

type Server struct {
    next        http.Handler      // Upstream application
    store       store.Interface   // Storage backend
    mux         *http.ServeMux    // Internal routes
    policy      *policy.ParsedConfig
    OGTags      *ogtags.OGTagCache
    logger      *slog.Logger
    opts        Options
    ed25519Priv ed25519.PrivateKey
    hs512Secret []byte
}

Responsibilities:

• Cookie validation and JWT parsing
• Request routing to policy engine
• Static asset serving (JS/CSS for challenges)
• OpenGraph tag caching
• Proxying validated requests to upstream

Key Methods:

• ServeHTTP(): Main request handler
• maybeReverseProxy(): Core request flow logic
• check(): Policy evaluation orchestration

Policy Engine

Location: lib/policy/

Evaluates incoming requests against configured rules:

type ParsedConfig struct {
    Store             store.Interface
    Bots              []Bot
    Thresholds        []*Threshold
    DefaultDifficulty int
    DNSBL             bool
    DnsCache          *dns.DnsCache
    Dns               *dns.Dns
    Logger            *slog.Logger
}

Checker Implementations:

RemoteAddrChecker

CIDR-based IP matching using gaissmai/bart prefix tables.

Location: lib/policy/checker.go:20

HeaderMatchesChecker

Regex matching against HTTP headers.

Location: lib/policy/checker.go:66

PathChecker

Regex matching against request paths.

Location: lib/policy/checker.go:96

CELChecker

Common Expression Language for complex conditions.

Location: lib/policy/celchecker.go

Flow:

1. Load and parse YAML policy configuration
2. Compile regex patterns and CEL expressions
3. Build checker chain for each bot rule
4. Evaluate rules sequentially on each request
5. Return CheckResult with action and weight

Challenge Engine

Location: lib/challenge/

Pluggable challenge system with multiple implementations:

type Impl interface {
    Setup(mux *http.ServeMux)
    Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in *IssueInput) (templ.Component, error)
    Validate(r *http.Request, lg *slog.Logger, in *ValidateInput) error
}

Registered Implementations:

Algorithm	File	Description
fast	lib/challenge/proofofwork/	SHA256-based proof-of-work
slow	lib/challenge/proofofwork/	Deprecated alias for fast
metarefresh	lib/challenge/metarefresh/	HTML meta refresh redirect
preact	lib/challenge/preact/	Interactive Preact component

Challenge Lifecycle:

1. issueChallenge(): Generate UUID, random data, store metadata
2. Issue(): Render challenge page with embedded solver
3. Client solves challenge and submits solution
4. Validate(): Verify solution, mark as spent
5. signJWT(): Generate authentication token

Storage Backends

Location: lib/store/

Unified interface for data persistence:

type Interface interface {
    Delete(ctx context.Context, key string) error
    Get(ctx context.Context, key string) ([]byte, error)
    Set(ctx context.Context, key string, value []byte, expiry time.Duration) error
    IsPersistent() bool
}

Implementations:

Memory (memory)

Location: lib/store/memory/

In-memory map with TTL support using decaymap.

Pros: Zero dependencies, instant access

Cons: Not persistent, single-instance only

Use case: Development, testing, single-server deployments

BBolt (bbolt)

Location: lib/store/bbolt/

Embedded key-value database.

Pros: Persistent, ACID transactions, no network

Cons: File locking (single process), slower than memory

Use case: Single-instance production with persistence

store:
  backend: bbolt
  parameters:
    path: /var/lib/anubis/anubis.db

Valkey/Redis (valkey)

Location: lib/store/valkey/

Network-based key-value store.

Pros: Shared state across instances, high performance, native TTL

Cons: External dependency, network latency

Use case: Multi-instance production deployments

store:
  backend: valkey
  parameters:
    addr: redis.example.com:6379
    password: ${REDIS_PASSWORD}
    db: 0

S3 API (s3api)

Location: lib/store/s3api/

S3-compatible object storage.

Pros: Unlimited capacity, durable, shared state

Cons: High latency, not optimized for small objects

Use case: Very high-volume or compliance-driven deployments

store:
  backend: s3api
  parameters:
    endpoint: https://s3.amazonaws.com
    bucket: anubis-challenges
    region: us-east-1

Stored Data:

Key Pattern	Type	TTL	Purpose
challenge:<uuid>	challenge.Challenge	30m	Challenge metadata and spent status
dronebl:<ip>	dnsbl.DroneBLResponse	24h	DNSBL lookup results
ogtags:allow:<host><path>	bool	Variable	OpenGraph tag asset allowlist
dns:forward:<ip>	[]string	Configurable	Forward DNS cache
dns:reverse:<hostname>	[]string	Configurable	Reverse DNS cache

JWT Validator

Location: lib/anubis.go:getTokenKeyfunc()

Supports two signing algorithms:

Ed25519 (Default)

func (s *Server) getTokenKeyfunc() jwt.Keyfunc {
    if len(s.hs512Secret) == 0 {
        return func(token *jwt.Token) (interface{}, error) {
            return s.ed25519Priv.Public().(ed25519.PublicKey), nil
        }
    }
    // ...
}

JWT Claims:

{
  "challenge": "01JXXXXXXXXXXXXXXXXXXXXXX",
  "method": "fast",
  "policyRule": "abc123...def",
  "action": "CHALLENGE",
  "restriction": "sha256(header_value)",
  "difficulty": 3,
  "iat": 1234567890,
  "nbf": 1234567830,
  "exp": 1234654290
}

Validation Checks:

• Signature integrity
• Expiration time (exp)
• Not-before time (nbf)
• Policy rule hash match (detects config changes)
• Optional restriction header binding

Thoth Integration

Location: lib/thoth/

Optional gRPC service for enriched IP intelligence:

type Client struct {
    conn    *grpc.ClientConn
    health  healthv1.HealthClient
    IPToASN iptoasnv1.IpToASNServiceClient
}

Features:

• ASN lookup and matching
• GeoIP country detection
• IP reputation data
• Cached responses (configurable TTL)

Configuration:

ANUBIS_THOTH_URL=thoth.example.com:443
ANUBIS_THOTH_API_TOKEN=your-token-here
ANUBIS_THOTH_PLAINTEXT=false

Policy Integration:

bots:
  - name: cloud-providers
    asns:
      match: [16509, 15169, 8075]
    action: CHALLENGE
  
  - name: high-risk-countries
    geoip:
      countries: [CN, RU, KP]
    action: DENY

Thoth is optional. ASN/GeoIP rules are skipped (with warnings) if Thoth is not configured.

Deployment Patterns

Single Instance

Configuration:

store:
  backend: bbolt
  parameters:
    path: /var/lib/anubis/anubis.db

Pros: Simple, no external dependencies

Cons: Single point of failure, limited scalability

Multi-Instance (Shared State)

Configuration:

store:
  backend: valkey
  parameters:
    addr: redis.prod.internal:6379
    password: ${REDIS_PASSWORD}

Pros: High availability, horizontal scaling

Cons: Redis as dependency, network latency

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: anubis
spec:
  replicas: 3
  selector:
    matchLabels:
      app: anubis
  template:
    metadata:
      labels:
        app: anubis
    spec:
      containers:
      - name: anubis
        image: ghcr.io/techarohq/anubis:latest
        env:
        - name: ANUBIS_UPSTREAM
          value: http://backend-service:8080
        - name: ANUBIS_STORE_BACKEND
          value: valkey
        - name: ANUBIS_STORE_ADDR
          value: redis-service:6379
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: anubis
spec:
  selector:
    app: anubis
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

Nginx Integration

Nginx as TLS terminator:

upstream anubis {
    server localhost:8080;
}

server {
    listen 443 ssl http2;
    server_name example.com;

    ssl_certificate /etc/ssl/certs/example.com.crt;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        proxy_pass http://anubis;
        proxy_set_header Host $host;
        proxy_set_header X-Real-Ip $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Uri $request_uri;
    }
}

Required Headers: Anubis depends on X-Real-Ip being set. Without it, all requests will fail with misconfiguration errors.

Caddy Integration

example.com {
    reverse_proxy localhost:8080 {
        header_up X-Real-Ip {remote_host}
        header_up X-Forwarded-For {remote_host}
        header_up X-Forwarded-Proto {scheme}
        header_up X-Forwarded-Host {host}
    }
}

Request Headers

Anubis requires from upstream proxy:

Header	Required	Purpose
X-Real-Ip	Yes	Client IP for policy evaluation
X-Forwarded-For	Recommended	IP chain for logging
X-Forwarded-Proto	For redirects	Protocol for redirect URLs
X-Forwarded-Host	For redirects	Host for redirect URLs
X-Forwarded-Uri	For redirects	Original URI for redirects

Anubis adds to upstream requests:

Header	Value	Purpose
X-Anubis-Rule	bot/rule-name	Which rule matched
X-Anubis-Action	ALLOW/CHALLENGE/etc	What action was taken
X-Anubis-Status	PASS	Challenge validation status

Performance Characteristics

Latency Impact

Cold path (no cookie):

• Policy evaluation: ~1-5ms
• Challenge rendering: ~10-50ms
• Total: 11-55ms overhead

Warm path (valid cookie):

• JWT validation: ~0.5-2ms
• Total: less than 2ms overhead

Challenge solving:

• Difficulty 3: ~500ms client-side
• Difficulty 5: ~60s client-side

Memory Usage

Baseline:

• Anubis process: ~50-100 MB
• Per challenge (memory store): ~2 KB
• Per JWT: 0 bytes (stateless)

With 10,000 active challenges:

• Memory store: ~120 MB total
• BBolt: ~30 MB on disk
• Valkey: Negligible (external)

Throughput

Benchmark results (Intel i7, 8 cores):

Scenario	Requests/sec	Latency p50	Latency p99
Valid JWT (proxy)	25,000	1.2ms	4.5ms
Policy eval (ALLOW)	18,000	2.1ms	8.2ms
Challenge render	5,000	12ms	35ms

Use ANUBIS_LOG_LEVEL=warn in production to reduce logging overhead.

Monitoring and Observability

Prometheus Metrics

Anubis exposes metrics at /.within.website/metrics:

# Challenge lifecycle
anubis_challenges_issued{method="embedded"}
anubis_challenges_validated{method="fast"}
anubis_failed_validations{method="fast"}
anubis_challenge_time_taken_seconds{algorithm="fast"}

# Policy decisions
anubis_policy_results{rule="bot/name",action="ALLOW"}

# Proxy traffic
anubis_proxied_requests_total{host="example.com"}

# DroneBL
anubis_dronebl_hits{status="all_good"}

Structured Logging

Anubis uses log/slog for structured logging:

{
  "time": "2026-03-03T10:15:30Z",
  "level": "INFO",
  "msg": "new challenge issued",
  "challenge": "01JXXXXXXXXXXXXXXXXXXXXXX",
  "rule": "bot/suspicious-crawler",
  "difficulty": 3,
  "remote_addr": "203.0.113.42"
}

Log Levels:

• DEBUG: Policy evaluation details, JWT validation
• INFO: Challenge issuance, successful validations
• WARN: Configuration warnings, deprecated features
• ERROR: Validation failures, store errors

Health Checks

Anubis doesn’t expose a dedicated health endpoint, but you can:

1. Check metrics endpoint: GET /.within.website/metrics
2. Use upstream health (after ALLOW rule)
3. Monitor store connectivity

Security Considerations

Constant-Time Operations

JWT signature validation and challenge hash comparison use crypto/subtle.ConstantTimeCompare to prevent timing attacks.

Cookie Security

• HttpOnly: Not set (needs JS access for challenge solving)
• Secure: Configurable via ANUBIS_COOKIE_SECURE
• SameSite: Configurable (Lax, Strict, None)
• Partitioned: Optional for CHIPS support

Double-Spend Prevention

Challenges marked as Spent=true after first successful validation. Store atomicity prevents reuse.

Policy Invalidation

JWT contains policy rule hash. Config changes automatically invalidate all existing tokens.

XSS Protection

Redirect URL validation prevents javascript:, data:, and other dangerous schemes.

Troubleshooting

X-Real-Ip not set

Error: [misconfiguration] X-Real-Ip header is not set

Fix: Configure your reverse proxy to set this header from client IP.

Policy doesn't match

Issue: Requests not triggering expected rules

Debug: Enable ANUBIS_LOG_LEVEL=debug and check check_result logs.

Cookies not persisting

Issue: Challenges loop indefinitely

Causes:

• Cookie domain mismatch
• Third-party cookie blocking
• SameSite=None without Secure

Store connectivity

Issue: can't fetch challenge errors

Check: Store backend health, network connectivity, credentials.

Next Steps

Configuration Guide

Learn about all configuration options and environment variables

Deployment Guide

Production deployment best practices and examples