Distributed key-value store as a pluggable Node subsystem. More...

#include "librats/util/rats_export.h"
#include "librats/node/peer_network.h"
#include "librats/peer/peer.h"
#include "librats/peer/peer_id.h"
#include "librats/core/bytes.h"
#include "librats/util/json.h"
#include <string>
#include <vector>
#include <functional>
#include <map>
#include <memory>
#include <mutex>
#include <unordered_map>
#include <atomic>
#include <chrono>
#include <thread>
#include <optional>
#include <condition_variable>

Include dependency graph for storage.h:

Classes
struct	librats::StorageEntry
	Storage entry - represents a single key-value pair in the database. More...

struct	librats::StorageChangeEvent
	Storage change event - passed to change callbacks. More...

struct	librats::StorageConfig
	Storage configuration. More...

struct	librats::StorageStatistics
	Storage statistics. More...

class	librats::StorageManager
	StorageManager - Distributed key-value storage with peer synchronization. More...

Namespaces
namespace	librats

Typedefs
using	librats::StorageChangeCallback = std::function< void(const StorageChangeEvent &)>
	Callback function types for storage events.

using	librats::StorageSyncCompleteCallback = std::function< void(bool success, const std::string &error_message)>

Enumerations
enum class	librats::StorageValueType : uint8_t { librats::BINARY = 0x01 , librats::STRING = 0x02 , librats::INT64 = 0x03 , librats::DOUBLE = 0x04 , librats::JSON = 0x05 }
	Value types supported by the distributed storage. More...

enum class	librats::StorageOperation : uint8_t { librats::OP_PUT = 0x01 , librats::OP_DELETE = 0x02 }
	Storage operation types for change events. More...

enum class	librats::StorageSyncStatus { librats::NOT_STARTED , librats::IN_PROGRESS , librats::COMPLETED , librats::FAILED }
	Storage synchronization status. More...

Functions
uint32_t	librats::storage_calculate_crc32 (const void *data, size_t length)

std::string	librats::storage_value_type_to_string (StorageValueType type)

StorageValueType	librats::string_to_storage_value_type (const std::string &str)

Detailed Description

Distributed key-value store as a pluggable Node subsystem.

A replicated key-value database with Last-Write-Wins (LWW) conflict resolution, typed values, binary on-disk persistence, and peer synchronization. It is a Subsystem: it reaches the mesh only through PeerNetwork (never the Node), exactly like PubSub/FileTransfer.

Replication model — an epidemic LWW broadcast:

A local put/remove builds a StorageEntry (a delete is a tombstone entry with deleted=true) and sends it to every connected peer.
On receiving an entry, a node applies it under LWW. It re-forwards the entry to its other peers ONLY if the entry actually won (carried new information). A duplicate loses LWW and is not forwarded, so flooding terminates naturally — no separate dedup table needed.
On peer connect, both sides ask each other for a full snapshot (anti-entropy) so a late joiner catches up. LWW makes the merge order-independent.

Backpressure — why a snapshot is a stream, not a message: A database is unbounded, a peer's send queue is not. One message carrying the whole store stops being sendable at the connection's low-water mark, gets the peer dropped as a slow consumer past its high-water mark, and is not even representable past the block-size ceiling — and since both ends request a snapshot on connect, two of them cross on every single link. So a snapshot is served as a sequence of bounded SYNC_CHUNK messages walking the (ordered) key space, one chunk at a time, and the next chunk is only produced while PeerNetwork::peer_writable() says there is room. Every send() return value is honoured: a peer that fills up stops receiving individual entries and is instead owed a fresh snapshot, which the sync thread starts once on_peer_writable says the link has drained. Losing live updates to a congested peer is safe precisely because the store is LWW — the snapshot that follows carries the winning state either way.

All of that runs on this module's own sync thread. Serialising even one chunk happens off the reactor thread, so a reactor never spends time (or holds storage_mutex_) proportional to the size of the database.

Wire format (MessageType::Storage payload, opcode in byte 0): ENTRY: [1][StorageEntry.serialize()] SYNC_REQUEST: [2] SYNC_CHUNK: [3][flags:u8][count:u32][StorageEntry.serialize()] * count flags bit0 (LAST) marks the final chunk of a snapshot; a snapshot of an empty store is one chunk with count 0.

The class is also usable standalone (no network attached) as a local, persistent key-value store; all network operations no-op until attach().

Definition in file storage.h.

Classes

Namespaces

Typedefs

Enumerations

Functions

Detailed Description