Custom Storage Backends

Anubis uses the store.Interface abstraction for all persistence operations. This allows you to implement custom storage backends for Redis, databases, cloud storage, or any key-value system.

Store Interface

The core storage interface defines four methods:

type Interface interface {
	// Delete removes a value from the store by key.
	Delete(ctx context.Context, key string) error

	// Get returns the value of a key assuming that value exists and has not expired.
	Get(ctx context.Context, key string) ([]byte, error)

	// Set puts a value into the store that expires according to its expiry.
	Set(ctx context.Context, key string, value []byte, expiry time.Duration) error

	// IsPersistent returns true if this storage backend persists data across
	// service restarts (e.g., bbolt, valkey). Returns false for volatile storage
	// like in-memory backends.
	IsPersistent() bool
}

Source: lib/store/interface.go:31-45

Standard Errors

Implementations should return these sentinel errors:

var (
	// ErrNotFound is returned when the store implementation cannot find the value
	// for a given key.
	ErrNotFound = errors.New("store: key not found")

	// ErrCantDecode is returned when a store adaptor cannot decode the store format
	// to a value used by the code.
	ErrCantDecode = errors.New("store: can't decode value")

	// ErrCantEncode is returned when a store adaptor cannot encode the value into
	// the format that the store uses.
	ErrCantEncode = errors.New("store: can't encode value")

	// ErrBadConfig is returned when a store adaptor's configuration is invalid.
	ErrBadConfig = errors.New("store: configuration is invalid")
)

Source: lib/store/interface.go:11-26

Implementation Example: In-Memory Store

The memory backend demonstrates a minimal implementation:

package memory

import (
	"context"
	"fmt"
	"time"

	"github.com/TecharoHQ/anubis/decaymap"
	"github.com/TecharoHQ/anubis/lib/store"
)

type impl struct {
	store *decaymap.Impl[string, []byte]
}

func (i *impl) Delete(_ context.Context, key string) error {
	if !i.store.Delete(key) {
		return fmt.Errorf("%w: %q", store.ErrNotFound, key)
	}
	return nil
}

func (i *impl) Get(_ context.Context, key string) ([]byte, error) {
	result, ok := i.store.Get(key)
	if !ok {
		return nil, fmt.Errorf("%w: %q", store.ErrNotFound, key)
	}
	return result, nil
}

func (i *impl) Set(_ context.Context, key string, value []byte, expiry time.Duration) error {
	i.store.Set(key, value, expiry)
	return nil
}

func (i *impl) IsPersistent() bool {
	return false
}

func New(ctx context.Context) store.Interface {
	result := &impl{
		store: decaymap.New[string, []byte](),
	}
	go result.cleanupThread(ctx)
	return result
}

Source: lib/store/memory/memory.go:25-78

Implementation Example: Valkey/Redis

The Valkey backend shows integration with external systems:

package valkey

import (
	"context"
	"time"

	"github.com/TecharoHQ/anubis/lib/store"
	valkey "github.com/redis/go-redis/v9"
)

type Store struct {
	client redisClient
}

var _ store.Interface = (*Store)(nil)

func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
	cmd := s.client.Get(ctx, key)
	if err := cmd.Err(); err != nil {
		if err == valkey.Nil {
			return nil, store.ErrNotFound
		}
		return nil, err
	}
	return cmd.Bytes()
}

func (s *Store) Set(ctx context.Context, key string, value []byte, expiry time.Duration) error {
	return s.client.Set(ctx, key, value, expiry).Err()
}

func (s *Store) Delete(ctx context.Context, key string) error {
	res := s.client.Del(ctx, key)
	if err := res.Err(); err != nil {
		return err
	}
	if n, _ := res.Result(); n == 0 {
		return store.ErrNotFound
	}
	return nil
}

func (s *Store) IsPersistent() bool {
	return true
}

Source: lib/store/valkey/valkey.go

Implementation Example: bbolt

The bbolt backend demonstrates persistent on-disk storage with expiry management:

// Store implements store.Interface backed by bbolt.
//
// In essence, bbolt is a hierarchical key/value store with a twist: every value
// needs to belong to a bucket. Each value in the store is given its own bucket 
// with two keys:
//
// 1. data - The raw data, usually in JSON
// 2. expiry - The expiry time formatted as a time.RFC3339Nano timestamp string
type Store struct {
	bdb *bbolt.DB
}

func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
	var result []byte

	if err := s.bdb.View(func(tx *bbolt.Tx) error {
		itemBucket := tx.Bucket([]byte(key))
		if itemBucket == nil {
			return fmt.Errorf("%w: %q", store.ErrNotFound, key)
		}

		expiryStr := itemBucket.Get([]byte("expiry"))
		if expiryStr == nil {
			return fmt.Errorf("[unexpected] %w: %q (expiry is nil)", store.ErrNotFound, key)
		}

		expiry, err := time.Parse(time.RFC3339Nano, string(expiryStr))
		if err != nil {
			return fmt.Errorf("[unexpected] %w: %w", store.ErrCantDecode, err)
		}

		if time.Now().After(expiry) {
			go s.Delete(context.Background(), key)
			return fmt.Errorf("%w: %q", store.ErrNotFound, key)
		}

		dataStr := itemBucket.Get([]byte("data"))
		if dataStr == nil {
			return fmt.Errorf("[unexpected] %w: %q (data is nil)", store.ErrNotFound, key)
		}

		result = make([]byte, len(dataStr))
		copy(result, dataStr)
		return nil
	); err != nil {
		return nil, err
	}

	return result, nil
}

func (s *Store) Set(ctx context.Context, key string, value []byte, expiry time.Duration) error {
	expires := time.Now().Add(expiry)

	return s.bdb.Update(func(tx *bbolt.Tx) error {
		valueBkt, err := tx.CreateBucketIfNotExists([]byte(key))
		if err != nil {
			return fmt.Errorf("%w: %w: %q (create bucket)", store.ErrCantEncode, err, key)
		}

		if err := valueBkt.Put([]byte("expiry"), []byte(expires.Format(time.RFC3339Nano))); err != nil {
			return fmt.Errorf("%w: %q (expiry)", store.ErrCantEncode, key)
		}

		if err := valueBkt.Put([]byte("data"), value); err != nil {
			return fmt.Errorf("%w: %q (data)", store.ErrCantEncode, key)
		}

		return nil
	)
}

Source: lib/store/bbolt/bbolt.go:60-122

Factory Pattern

Store backends use a factory pattern for registration:

type Factory interface {
	Build(ctx context.Context, config json.RawMessage) (Interface, error)
	Valid(config json.RawMessage) error
}

func Register(name string, impl Factory) {
	regLock.Lock()
	defer regLock.Unlock()
	registry[name] = impl
}

Source: lib/store/registry.go:15-25

Example Factory

type factory struct{}

func (factory) Build(ctx context.Context, _ json.RawMessage) (store.Interface, error) {
	return New(ctx), nil
}

func (factory) Valid(json.RawMessage) error { 
	return nil 
}

func init() {
	store.Register("memory", factory{)
}

Source: lib/store/memory/memory.go:13-23

JSON Type Wrapper

Anubis provides a generic JSON wrapper for type-safe operations:

type JSON[T any] struct {
	Underlying Interface
	Prefix     string
}

func (j *JSON[T]) Get(ctx context.Context, key string) (T, error) {
	if j.Prefix != "" {
		key = j.Prefix + key
	}

	data, err := j.Underlying.Get(ctx, key)
	if err != nil {
		return z[T](), err
	}

	var result T
	if err := json.Unmarshal(data, &result); err != nil {
		return z[T](), fmt.Errorf("%w: %w", ErrCantDecode, err)
	}

	return result, nil
}

func (j *JSON[T]) Set(ctx context.Context, key string, value T, expiry time.Duration) error {
	if j.Prefix != "" {
		key = j.Prefix + key
	}

	data, err := json.Marshal(value)
	if err != nil {
		return fmt.Errorf("%w: %w", ErrCantEncode, err)
	}

	return j.Underlying.Set(ctx, key, data, expiry)
}

Source: lib/store/interface.go:49-95

Configuration

Reference your store backend in anubis.yaml:

store:
  backend: mycustom
  parameters:
    connection_string: "postgresql://..."
    pool_size: 10

Cleanup Threads

Implement background cleanup for expired keys:

func (s *Store) cleanupThread(ctx context.Context) {
	t := time.NewTicker(time.Hour)
	defer t.Stop()

	for {
		select {
		case <-ctx.Done():
			return
		case <-t.C:
			if err := s.cleanup(ctx); err != nil {
				slog.Error("error during cleanup", "err", err)
			}
		}
	}
}

Source: lib/store/bbolt/bbolt.go:156-170

Best Practices

Return standard errors: Use store.ErrNotFound, store.ErrCantDecode, etc.
Handle expiry: Implement automatic cleanup or lazy deletion of expired keys
Context awareness: Honor context.Context for cancellation and timeouts
IsPersistent accuracy: Return false for in-memory stores, true for persistent backends
Thread safety: Ensure concurrent access is safe
Configuration validation: Implement Factory.Valid() to catch config errors early

Available Backends

Query registered stores at runtime:

methods := store.Methods()  // Returns []string of backend names
factory, ok := store.Get("bbolt")  // Get a specific factory

Source: lib/store/registry.go:34-42

Development

Advanced

Custom Storage Backends

Store Interface

Standard Errors

Implementation Example: In-Memory Store

Implementation Example: Valkey/Redis

Implementation Example: bbolt

Factory Pattern

Example Factory

JSON Type Wrapper

Configuration

Cleanup Threads

Best Practices

Available Backends

​Store Interface

​Standard Errors

​Implementation Example: In-Memory Store

​Implementation Example: Valkey/Redis

​Implementation Example: bbolt

​Factory Pattern

​Example Factory

​JSON Type Wrapper

​Configuration

​Cleanup Threads

​Best Practices

​Available Backends

Store Interface

Standard Errors

Implementation Example: In-Memory Store

Implementation Example: Valkey/Redis

Implementation Example: bbolt

Factory Pattern

Example Factory

JSON Type Wrapper

Configuration

Cleanup Threads

Best Practices

Available Backends