Skip to content

logicblocks.event.processing

class EventConsumerStateConverter[E: Event]

Bases: ABC

Converts between events and the consumer state that tracks them.

Implementations define what state is recorded when an event is processed and how that state is turned into a query constraint that resumes consumption after the last processed event.

def event_to_state(event: E) -> JsonObject

Return the consumer state to record for a processed event.

Parameters:

  • event (E) — The event that has been processed.
def initial_state() -> JsonObject

Return the consumer state to use when no events are processed.

def state_to_query_constraint(
    state: JsonObject,
) -> constraints.QueryConstraint | None

Convert consumer state into a constraint for resuming consumption.

Parameters:

  • state (JsonObject) — The consumer state to convert.

Returns:

  • constraints.QueryConstraint | None — a constraint selecting events after those already processed, or None if all events should be consumed.
class ContinueErrorHandler[T](value_factory: Callable[[BaseException], T])

Bases: ErrorHandler[T]

Error handler that decides to continue, returning a value.

Exceptions that are not Exception subclasses, such as asyncio.CancelledError, are raised instead.

Parameters:

  • value_factory (Callable[[BaseException], T]) — Returns the value to continue with for an exception.
value_factory: Callable[[BaseException], T]
def handle(exception: BaseException) -> ErrorHandlerDecision[T]
class ContinueErrorHandlerDecision[T](value: T)

Bases: ErrorHandlerDecision[T]

Decision to continue, returning a value in place of a result.

Attributes:

  • value (T) — The value to return.
value: T
class DistributedEventBroker[E: Event](
    event_subscriber_manager: EventSubscriberManager[E],
    event_subscription_coordinator: EventSubscriptionCoordinator,
    event_subscription_observer: EventSubscriptionObserver[E],
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
)

Bases: EventBroker[E]

An event broker that coordinates event source allocation across nodes.

While running, the broker concurrently:

  • maintains the registration of this node’s subscribers in shared state, sending heartbeats for healthy subscribers and purging stale ones, via the subscriber manager;
  • competes for leadership and, while leader, distributes the event sources requested by each subscriber group across the live subscribers in that group, via the subscription coordinator;
  • applies the resulting allocations to this node’s subscribers, calling accept and withdraw as they change, via the subscription observer.

Subscribers are unregistered from shared state when execution ends.

Parameters:

  • event_subscriber_manager (EventSubscriberManager[E]) — Manages this node’s subscribers.
  • event_subscription_coordinator (EventSubscriptionCoordinator) — Allocates event sources to subscribers when this node is leader.
  • event_subscription_observer (EventSubscriptionObserver[E]) — Applies allocations to this node’s subscribers.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution. Defaults to a RetryErrorHandler.
property status: ProcessStatus

The combined status of the coordinator and observer.

async def register(subscriber: EventSubscriber[E]) -> None
async def execute() -> None

Run the broker until cancelled.

Errors are handled by the configured error handler.

class DistributedEventBrokerSettings(
    subscriber_manager_heartbeat_interval: timedelta = timedelta(seconds=10),
    subscriber_manager_purge_interval: timedelta = timedelta(minutes=1),
    subscriber_manager_subscriber_max_age: timedelta = timedelta(minutes=10),
    coordinator_subscriber_max_time_since_last_seen: timedelta = timedelta(seconds=60),
    coordinator_distribution_interval: timedelta = timedelta(seconds=20),
    coordinator_leadership_max_duration: timedelta = timedelta(minutes=15),
    coordinator_leadership_attempt_interval: timedelta = timedelta(seconds=5),
    observer_synchronisation_interval: timedelta = timedelta(seconds=20),
)

Settings for a distributed event broker.

Attributes:

  • subscriber_manager_heartbeat_interval (timedelta) — How often heartbeats are sent for this node’s healthy subscribers.
  • subscriber_manager_purge_interval (timedelta) — How often subscribers not seen recently are purged from shared state.
  • subscriber_manager_subscriber_max_age (timedelta) — How long since last seen before a subscriber is purged.
  • coordinator_subscriber_max_time_since_last_seen (timedelta) — How long since last seen before the coordinator stops allocating event sources to a subscriber.
  • coordinator_distribution_interval (timedelta) — How often the leader redistributes event sources.
  • coordinator_leadership_max_duration (timedelta) — The maximum time a node holds leadership before releasing it.
  • coordinator_leadership_attempt_interval (timedelta) — How often a node that is not leader attempts to acquire leadership.
  • observer_synchronisation_interval (timedelta) — How often allocations are synchronised to this node’s subscribers.
subscriber_manager_heartbeat_interval: timedelta = timedelta(seconds=10)
subscriber_manager_purge_interval: timedelta = timedelta(minutes=1)
subscriber_manager_subscriber_max_age: timedelta = timedelta(minutes=10)

coordinator_subscriber_max_time_since_last_seen

Section titled “coordinator_subscriber_max_time_since_last_seen”
coordinator_subscriber_max_time_since_last_seen: timedelta = timedelta(seconds=60)
coordinator_distribution_interval: timedelta = timedelta(seconds=20)
coordinator_leadership_max_duration: timedelta = timedelta(minutes=15)
coordinator_leadership_attempt_interval: timedelta = timedelta(seconds=5)
observer_synchronisation_interval: timedelta = timedelta(seconds=20)
class ErrorHandler[T]

Bases: ABC

Base class for handlers deciding how to respond to an exception.

def handle(exception: BaseException) -> ErrorHandlerDecision[T]

Decide how to respond to the given exception.

Parameters:

  • exception (BaseException) — The exception to handle.

Returns:

class ErrorHandlerDecision[T]

Base class for decisions made by an ErrorHandler.

Provides factory methods for each kind of decision.

@staticmethod
def exit_fatally(*, exit_code: int = 1) -> ExitErrorHandlerDecision

Return a decision to exit the process.

Parameters:

  • exit_code (int) — The exit code to exit with.
@staticmethod
def raise_exception(exception: BaseException) -> RaiseErrorHandlerDecision

Return a decision to raise an exception.

Parameters:

  • exception (BaseException) — The exception to raise.
@staticmethod
def continue_execution[R](*, value: R) -> ContinueErrorHandlerDecision[R]

Return a decision to continue, returning a value as the result.

Parameters:

  • value (R) — The value to return.
@staticmethod
def retry_execution(
    *,
    wait_before_retry: timedelta | None = None,
) -> RetryErrorHandlerDecision

Return a decision to retry the failed execution.

Parameters:

  • wait_before_retry (timedelta | None) — The time to wait before retrying. If None, retries immediately.
class ErrorHandlingService[T = Any](
    service: Service[T] | CallableServiceCallable[T] | None = None,
    *,
    callable: CallableServiceCallable[T] | None = None,
    error_handler: ErrorHandler[T],
    sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
)

Bases: Service[T]

Service that applies an error handler to a wrapped service.

Parameters:

  • service (Service[T] | CallableServiceCallable[T] | None) — The service to execute. Passing a bare callable is deprecated; wrap it in a CallableService.
  • callable (CallableServiceCallable[T] | None) — Deprecated; use service instead.
  • error_handler (ErrorHandler[T]) — The error handler to apply to any exception raised.
  • sleep (Callable[[float], Awaitable[None]]) — The function used to wait before retrying.

Raises:

  • TypeError — If neither service nor callable is provided.
@classmethod
async def apply_error_handling(
    run: Callable[[], Awaitable[T]],
    error_handler: ErrorHandler[T],
    *,
    sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
) -> T

Run a callable, responding to any exception as the handler decides.

Retry decisions run the callable again, after waiting if required; continue decisions return their value; raise decisions raise their exception; and exit decisions raise SystemExit.

Parameters:

  • run (Callable[[], Awaitable[T]]) — The callable to run.
  • error_handler (ErrorHandler[T]) — Decides how to respond to each exception.
  • sleep (Callable[[float], Awaitable[None]]) — The function used to wait before retrying.

Returns:

  • T — the result of the callable, or the value of a continue decision.

Raises:

  • SystemExit — If the handler decides to exit.
  • ValueError — If the handler returns an unknown decision.
async def execute() -> T
class ErrorHandlingServiceMixin[T = Any](error_handler: ErrorHandler[T])

Bases: Service[T], ABC

Mixin applying error handling to a service’s _do_execute method.

Deprecated: use composition with ErrorHandlingService instead.

Parameters:

  • error_handler (ErrorHandler[T]) — The error handler to apply.
async def execute() -> T
class CallableService[T = Any](callable: CallableServiceCallable[T])

Bases: Service[T]

Service that executes an asynchronous callable.

Parameters:

@overload
@classmethod
def from_maybe_callable[S: Service](service_or_callable: S) -> S
@overload
@classmethod
def from_maybe_callable(
    service_or_callable: CallableServiceCallable[T],
) -> Self

Wrap a callable as a service, passing services through unchanged.

Parameters:

Returns:

  • the service if one was given, otherwise a CallableService wrapping the callable.
async def execute() -> T
class EventBroker[E: Event]

Bases: Service[NoneType], Process, ABC

A long-running service that allocates event sources to subscribers.

Subscribers are registered with the broker, which then hands each of them the event sources they requested, via EventSubscriber.accept, while it executes. Brokers expose a ProcessStatus so their health can be monitored.

Use make_event_broker to construct a broker of the required type.

async def register(subscriber: EventSubscriber[E]) -> None

Register a subscriber to receive event sources from the broker.

Subscribers must be registered before the broker is executed, as the distributed broker never allocates sources to later ones.

Parameters:

class EventBrokerType

Bases: ABC

Marker types selecting the broker strategy for make_event_broker.

Attributes:

  • Singleton (type[_SingletonEventBrokerType]) — Selects a broker that passes each registered subscriber on the local node the event sources it requested.
  • Distributed (type[_DistributedEventBrokerType]) — Selects a broker that coordinates across nodes, sharing the requested event sources between the subscribers in each subscriber group.
Singleton: type[_SingletonEventBrokerType]
Distributed: type[_DistributedEventBrokerType]
class EventConsumer

Bases: ABC

Base class for consumers that consume events from some source.

async def consume_all() -> None

Consume all events currently available to this consumer.

class DeferredFuture[T](name: str)

A handle to the future result of a service not yet scheduled.

Supports a subset of the asyncio.Future interface, delegating to the underlying future once the service has been scheduled. It can be awaited.

Parameters:

  • name (str) — The name of the service, used in error messages.
def result() -> T

Return the result of the service.

Raises:

  • BaseException — Any exception raised by the service.
  • asyncio.CancelledError — If the service was cancelled.
  • RuntimeError — If the service has not been scheduled yet.
  • asyncio.InvalidStateError — If the service has not finished.
def exception() -> BaseException | None

Return the exception raised by the service, if any.

Raises:

  • RuntimeError — If the service has not been scheduled yet.
  • asyncio.InvalidStateError — If the service has not finished.
def done() -> bool

Return whether the service has been scheduled and finished.

def __await__()
class EventConsumerState(state: JsonObject)

Bases: JsonValueConvertible

The persisted state of an event consumer.

Attributes:

  • state (JsonObject) — The state as produced by an EventConsumerStateConverter.
state: JsonObject
@classmethod
def deserialise(
    value: JsonValue,
    fallback: Callable[[Any, JsonValue], Any] = default_deserialisation_fallback,
) -> Self

Deserialise consumer state from a JSON value.

For backwards compatibility, a top-level last_sequence_number is merged into the state and a missing state is treated as empty.

Parameters:

  • value (JsonValue) — The JSON value to deserialise.
  • fallback (Callable[[Any, JsonValue], Any]) — Called with the class and value if the value is not a valid consumer state.

Returns:

  • Self — the deserialised consumer state, or the result of the fallback.
def serialise(
    fallback: Callable[[object], JsonValue] = default_serialisation_fallback,
) -> JsonValue
class EventConsumerStateStore[E: Event](
    category: EventCategory,
    converter: EventConsumerStateConverter[E],
    persistence_interval: EventCount | None = EventCount(100),
)

Records and persists event consumer state in an event category.

State is held in memory per partition and persisted as state-changed events in the stream named after the partition.

Parameters:

  • category (EventCategory) — The category in which to store consumer state.
  • converter (EventConsumerStateConverter[E]) — Converts between events and consumer state.
  • persistence_interval (EventCount | None) — The number of recorded events after which save_if_needed persists state. If None, save_if_needed never saves.
def record_processed(
    event: E,
    *,
    extra_state: JsonObject | None = None,
    partition: str = 'default',
) -> EventConsumerState

Record that an event has been processed.

The state is held in memory until saved.

Parameters:

  • event (E) — The processed event.
  • extra_state (JsonObject | None) — Additional state to merge into the converted state.
  • partition (str) — The partition to record the state against.

Returns:

def reset(*, extra_state: JsonObject | None = None, partition: str = 'default')

Reset a partition’s state to the converter’s initial state.

The reset state is held in memory only and is never saved.

Parameters:

  • extra_state (JsonObject | None) — Additional state to merge into the initial state.
  • partition (str) — The partition to reset.
async def save_if_needed(*, partition: str = 'default') -> None

Save a partition’s state if the persistence interval is reached.

Parameters:

  • partition (str) — The partition to save.
async def save(partition: str | None = None) -> None

Persist unsaved state.

Partitions with no state or no changes since the last save are skipped.

Parameters:

  • partition (str | None) — The partition to save. If None, all partitions with recorded state are saved.

Raises:

  • UnmetWriteConditionError — If the partition’s stream has been written to by another writer since it was last read.
async def load(*, partition: str = 'default') -> EventConsumerState | None

Load a partition’s state.

State held in memory is returned if present; otherwise the latest persisted state is read from the partition’s stream.

Parameters:

  • partition (str) — The partition to load.

Returns:

async def load_to_query_constraint(
    *,
    partition: str = 'default',
) -> constraints.QueryConstraint | None

Load a partition’s state as a constraint for resuming consumption.

Parameters:

  • partition (str) — The partition to load.

Returns:

  • constraints.QueryConstraint | None — a constraint selecting events after those already processed, or None if there is no state or all events should be consumed.
class EventCount

Bases: int

A count of events.

def increment() -> Self

Return a new count one greater than this one.

class EventProcessor[E: Event]

Bases: ABC

Base class for processors that handle events one at a time.

When used with an EventSourceConsumer, each event is acknowledged automatically once process_event returns successfully.

async def process_event(event: E) -> None

Process a single event.

Parameters:

  • event (E) — The event to process.
class EventSourceConsumer[I: EventSourceIdentifier, E: Event](
    *,
    source: EventSource[I, E],
    processor: SupportedProcessors[E],
    state_store: EventConsumerStateStore[E],
    logger: FilteringBoundLogger = default_logger,
    save_state_after_consumption: bool = True,
)

Bases: EventConsumer

Consumes events from a single event source, tracking progress.

On each call to consume_all, the consumer loads its last recorded state, iterates the source from that point onwards and passes events to the processor, recording progress in the state store.

How events are acknowledged depends on the processor type:

  • EventProcessor: each event is acknowledged after process_event returns, with state saved whenever the persistence interval is reached. If save_state_after_consumption is True, state is also saved once all events have been consumed.
  • AutoCommitEventIteratorProcessor: each event is acknowledged when the processor requests the next event.
  • ManagedEventIteratorProcessor: the processor acknowledges events and commits state itself via the provided manager.

Parameters:

  • source (EventSource[I, E]) — The event source to consume from.
  • processor (SupportedProcessors[E]) — The processor to pass events to.
  • state_store (EventConsumerStateStore[E]) — The store used to load and save consumer state.
  • logger (FilteringBoundLogger) — The logger to use.
  • save_state_after_consumption (bool) — Whether to force a save of consumer state after all events have been consumed. Only applies to EventProcessor processors.
async def consume_all() -> None

Consume all events in the source after the last recorded position.

Raises:

  • TypeError — If the processor is not of a supported type.
class EventBrokerStorageType

Bases: ABC

Marker types selecting the broker storage for make_event_broker.

Attributes:

  • InMemory (type[_InMemoryEventBrokerStorageType]) — Selects in-memory storage, suitable for tests and single process deployments.
  • Postgres (type[_PostgresEventBrokerStorageType]) — Selects PostgreSQL storage, required by the distributed strategy to coordinate between nodes.
InMemory: type[_InMemoryEventBrokerStorageType]
Postgres: type[_PostgresEventBrokerStorageType]
InMemoryEventBrokerStorageTypeType = type[_InMemoryEventBrokerStorageType]
PostgresEventBrokerStorageTypeType = type[_PostgresEventBrokerStorageType]
class EventSubscriber[E: Event]

Bases: ABC

A consumer of event sources allocated by an event broker.

Each subscriber declares the event sources it would like to consume via subscription_requests. The broker then calls accept with each event source allocated to the subscriber and withdraw with each event source that is no longer allocated to it.

Subscribers sharing a group are treated as instances of the same logical consumer, so that a distributed broker can spread the requested event sources across them.

property group: str

The group to which the subscriber belongs.

property id: str

The ID of the subscriber, unique within its group.

property key: EventSubscriberKey

The key identifying the subscriber, from its group and ID.

property subscription_requests: Sequence[EventSourceIdentifier]

The identifiers of the event sources the subscriber requests.

def health() -> EventSubscriberHealth

Return the current health of the subscriber.

A distributed broker only sends heartbeats for healthy subscribers, so an unhealthy subscriber eventually stops receiving allocations.

Returns:

async def accept(source: EventSource[EventSourceIdentifier, E]) -> None

Accept an event source allocated to the subscriber.

Implementations should tolerate being passed a source they have already accepted, as some brokers reallocate sources periodically.

Parameters:

async def withdraw(source: EventSource[EventSourceIdentifier, E]) -> None

Withdraw an event source no longer allocated to the subscriber.

Parameters:

class EventSubscriberHealth

Bases: StrEnum

The health of an event subscriber, as reported to the broker.

HEALTHY = 'healthy'
UNHEALTHY = 'unhealthy'
class EventSubscriptionConsumer[E: Event](
    group: str,
    id: str,
    subscription_requests: Sequence[EventSourceIdentifier],
    delegate_factory: Callable[[EventSource[EventSourceIdentifier, Any]], EventConsumer],
    logger: FilteringBoundLogger = default_logger,
)

Bases: EventConsumer, EventSubscriber[E]

An event subscriber that consumes from the sources it is allocated.

An event broker allocates sources to the subscriber via accept and removes them via withdraw. Each accepted source is consumed by a delegate consumer created by the delegate factory.

Parameters:

  • group (str) — The group the subscriber belongs to.
  • id (str) — The unique identifier of the subscriber.
  • subscription_requests (Sequence[EventSourceIdentifier]) — The identifiers of the event sources the subscriber requests.
  • delegate_factory (Callable[[EventSource[EventSourceIdentifier, Any]], EventConsumer]) — Creates the consumer for an accepted source.
  • logger (FilteringBoundLogger) — The logger to use.
property group: str
property id: str
def health() -> EventSubscriberHealth

Return the health of the subscriber, which is always healthy.

property subscription_requests: Sequence[EventSourceIdentifier]
async def accept(source: EventSource[EventSourceIdentifier, E]) -> None

Accept a source, creating a delegate consumer for it.

If the source has already been accepted, the existing delegate consumer is retained.

Parameters:

async def withdraw(source: EventSource[EventSourceIdentifier, E]) -> None

Withdraw a source, discarding its delegate consumer.

Withdrawing a source that has not been accepted logs a warning.

Parameters:

async def consume_all() -> None

Consume all available events from each accepted source in turn.

class ExecutionMode

Bases: Enum

Whether a ServiceManager waits for a service when starting.

Attributes:

  • FOREGROUND — ServiceManager.start waits for the service to finish.
  • BACKGROUND — The service runs without ServiceManager.start waiting for it.
FOREGROUND = auto()
BACKGROUND = auto()
class ExitErrorHandler(
    exit_code_factory: Callable[[BaseException], int] = default_exit_code_factory,
)

Bases: ErrorHandler[Any]

Error handler that decides to exit the process.

Parameters:

  • exit_code_factory (Callable[[BaseException], int]) — Returns the exit code for an exception. Defaults to always returning 1.
exit_code_factory
def handle(exception: BaseException) -> ErrorHandlerDecision[Any]
class ExitErrorHandlerDecision(exit_code: int)

Bases: ErrorHandlerDecision[Any]

Decision to exit the process by raising SystemExit.

Attributes:

  • exit_code (int) — The exit code to exit with.
exit_code: int
class InMemoryLockManager

Bases: LockManager

Lock manager using asyncio locks, local to a single event loop.

async def try_lock(lock_name: str) -> AsyncGenerator[Lock, None]
async def wait_for_lock(
    lock_name: str,
    *,
    timeout: timedelta | None = None,
) -> AsyncGenerator[Lock, None]
class IsolationMode

Bases: Enum

Which thread and event loop a ServiceManager runs a service on.

Attributes:

  • MAIN_THREAD — The event loop that started the service manager.
  • SHARED_THREAD — A separate thread and event loop shared by all services with this isolation mode.
  • DEDICATED_THREAD — A separate thread and event loop for this service alone.
MAIN_THREAD = auto()
SHARED_THREAD = auto()
DEDICATED_THREAD = auto()
class Lock(
    name: str,
    locked: bool,
    timed_out: bool = False,
    wait_time: timedelta = timedelta(seconds=0),
)

The outcome of an attempt to acquire a named lock.

Attributes:

  • name (str) — The name of the lock.
  • locked (bool) — Whether the lock was acquired.
  • timed_out (bool) — Whether the attempt timed out waiting for the lock.
  • wait_time (timedelta) — How long was spent waiting for the lock.
name: str
locked: bool
timed_out: bool = False
wait_time: timedelta = timedelta(seconds=0)
class LockManager

Bases: ABC

Base class for managers of named mutual exclusion locks.

Both methods are asynchronous context managers yielding a Lock describing whether the lock was acquired. If acquired, the lock is held until the context exits.

def try_lock(lock_name: str) -> AsyncGenerator[Lock, None]

Attempt to acquire a lock without waiting.

Parameters:

  • lock_name (str) — The name of the lock to acquire.

Yields:

  • AsyncGenerator[Lock, None] — the lock, with locked set to False if it is already held.
def wait_for_lock(
    lock_name: str,
    *,
    timeout: timedelta | None = None,
) -> AsyncGenerator[Lock, None]

Wait to acquire a lock, optionally up to a timeout.

Parameters:

  • lock_name (str) — The name of the lock to acquire.
  • timeout (timedelta | None) — The maximum time to wait. If None, waits indefinitely.

Yields:

  • AsyncGenerator[Lock, None] — the lock, with timed_out set to True if the timeout elapsed before the lock was acquired.
class OverrideRetryStrategyDecision[T](
    error_handler_decision: ErrorHandlerDecision[T],
)

Bases: RetryStrategyDecision

Decision to use an error handler decision in place of retrying.

Attributes:

error_handler_decision: ErrorHandlerDecision[T]
class PollingService(
    service: Service[Any] | CallableServiceCallable[Any] | None = None,
    *,
    callable: CallableServiceCallable[Any] | None = None,
    poll_interval: timedelta = timedelta(milliseconds=200),
)

Bases: Service[Never]

Service that executes another service repeatedly, forever.

The poll interval is the delay between one execution finishing and the next starting. Any exception raised by the wrapped service stops polling and propagates.

Parameters:

  • service (Service[Any] | CallableServiceCallable[Any] | None) — The service to execute on each poll. Passing a bare callable is deprecated; wrap it in a CallableService.
  • callable (CallableServiceCallable[Any] | None) — Deprecated; use service instead.
  • poll_interval (timedelta) — The delay between executions.

Raises:

  • TypeError — If neither service nor callable is provided.
async def execute() -> Never
class PostgresLockManager(connection_settings: ConnectionSettings)

Bases: LockManager

Lock manager using PostgreSQL transaction-level advisory locks.

Each lock attempt opens a new connection, and the lock is held for the duration of that connection’s transaction.

Parameters:

  • connection_settings (ConnectionSettings) — The settings used to connect to PostgreSQL.
connection_settings
async def try_lock(lock_name: str) -> AsyncGenerator[Lock, None]
async def wait_for_lock(
    lock_name: str,
    *,
    timeout: timedelta | None = None,
) -> AsyncGenerator[Lock, None]
class Process

Bases: ABC

Base class for processes that report a status.

property status: ProcessStatus

The current status of the process.

class ProcessStatus

Bases: StrEnum

The lifecycle status of a process.

INITIALISED = 'initialised'
STARTING = 'starting'
WAITING = 'waiting'
RUNNING = 'running'
STOPPING = 'stopping'
STOPPED = 'stopped'
ERRORED = 'errored'
UNKNOWN = 'unknown'
class HasProcessStatus

Bases: Protocol

Protocol for objects that report a process status.

property status: ProcessStatus
class ProjectionEventProcessor[State: JsonPersistable = JsonValue, Metadata: JsonPersistable = JsonValue](
    projector: Projector[StreamIdentifier, State, Metadata],
    projection_store: ProjectionStore,
    state_type: type[State] = JsonValueType,
    metadata_type: type[Metadata] = JsonValueType,
)

Bases: EventProcessor[StoredEvent]

Event processor that updates projections as events are processed.

For each event, the processor locates the current projection for the event’s stream, applies the event using the projector and saves the updated projection.

Parameters:

  • projector (Projector[StreamIdentifier, State, Metadata]) — The projector used to apply events to projections.
  • projection_store (ProjectionStore) — The store in which projections are kept.
  • state_type (type[State]) — The type to deserialise projection state into.
  • metadata_type (type[Metadata]) — The type to deserialise projection metadata into.
async def process_event(event: StoredEvent) -> None

Apply the event to its stream’s projection and save the result.

Parameters:

class RaiseErrorHandler(
    exception_factory: Callable[[BaseException], BaseException] = default_exception_factory,
)

Bases: ErrorHandler[Any]

Error handler that decides to raise an exception.

Parameters:

  • exception_factory (Callable[[BaseException], BaseException]) — Returns the exception to raise for a handled exception. Defaults to returning the handled exception.
exception_factory
def handle(exception: BaseException) -> ErrorHandlerDecision[Any]
class RaiseErrorHandlerDecision(exception: BaseException)

Bases: ErrorHandlerDecision[Any]

Decision to raise an exception.

Attributes:

  • exception (BaseException) — The exception to raise.
exception: BaseException
class RetryErrorHandler(retry_strategy: RetryStrategy | None = None)

Bases: ErrorHandler[Any]

Error handler that decides to retry the failed execution.

Exceptions that are not Exception subclasses, such as asyncio.CancelledError, are raised instead.

Parameters:

  • retry_strategy (RetryStrategy | None) — Decides how to retry for each exception. If None, retries immediately.
def handle(exception: BaseException) -> ErrorHandlerDecision[Any]
class RetryErrorHandlerDecision(wait_before_retry: timedelta | None = None)

Bases: ErrorHandlerDecision[Any]

Decision to retry the failed execution.

Attributes:

  • wait_before_retry (timedelta | None) — The time to wait before retrying. If None, retries immediately.
wait_before_retry: timedelta | None = None
class RetryImmediatelyDecision

Bases: RetryStrategyDecision

Decision to retry without waiting.

class RetryStrategyDecision

Base class for decisions made by a RetryStrategy.

Provides factory methods for each kind of decision.

@staticmethod
def retry_immediately() -> RetryImmediatelyDecision

Return a decision to retry without waiting.

@staticmethod
def wait(duration: timedelta) -> WaitRetryStrategyDecision

Return a decision to wait before retrying.

Parameters:

  • duration (timedelta) — The time to wait before retrying.
@staticmethod
def override[R](
    error_handler_decision: ErrorHandlerDecision[R],
) -> OverrideRetryStrategyDecision[R]

Return a decision to use an error handler decision instead.

Allows a retry strategy to stop retrying, for example by raising or exiting.

Parameters:

class Service[T = Any]

Bases: ABC

Base class for long-running or one-off units of work.

Services are typically registered with and run by a ServiceManager.

async def execute() -> T

Run the service.

Returns:

  • T — the result of the service, if any.
class ServiceManager

Registers, starts and stops a collection of services.

Services are registered with an execution mode and an isolation mode, and are all scheduled when the manager starts. The manager can be used as an asynchronous context manager, starting on entry and stopping on exit.

property services: Mapping[str, ManagedServiceState[Any]]

The state of each registered service, keyed by name.

def service(name: str) -> ManagedServiceState[Any] | None

Return the state of the named service, or None if not found.

Parameters:

  • name (str) — The name of the service.
def register[T](
    service: Service[T],
    *,
    name: str | None = None,
    execution_mode: ExecutionMode = ExecutionMode.BACKGROUND,
    isolation_mode: IsolationMode = IsolationMode.MAIN_THREAD,
) -> Self

Register a service to be run when the manager starts.

Parameters:

  • service (Service[T]) — The service to register.
  • name (str | None) — The unique name of the service. Defaults to a random identifier.
  • execution_mode (ExecutionMode) — Whether start waits for the service to finish.
  • isolation_mode (IsolationMode) — Which thread and event loop to run the service on.

Returns:

  • Self — this manager, for chaining.

Raises:

  • ValueError — If a service with the same name is already registered.
def stop_on(signals: Sequence[int]) -> Self

Stop the manager when any of the given signals is received.

Signal handlers are installed on the event loop when the manager starts.

Parameters:

  • signals (Sequence[int]) — The signal numbers to stop on.

Returns:

  • Self — this manager, for chaining.
async def __aenter__() -> Self
async def __aexit__(
    exc_type: type[BaseException] | None = None,
    exc_value: BaseException | None = None,
    traceback: TracebackType | None = None,
) -> bool
async def start() -> Self

Start all registered services.

Waits until all FOREGROUND services have finished; exceptions they raise are not propagated but are available via each service’s future. BACKGROUND services continue running after this returns.

Returns:

  • Self — this manager.
async def stop() -> Self

Cancel all running services and shut down their threads.

Returns:

  • Self — this manager.
class ManagedServiceState[T]

Bases: ABC

The state of a service registered with a ServiceManager.

property service: Service[T]
property name: str
property execution_mode: ExecutionMode
property isolation_mode: IsolationMode
property service_status: ProcessStatus

The service’s status, or UNKNOWN if it does not report one.

property future: DeferredFuture[T]

A future for the result of the service once it is scheduled.

class StatusTrackingService[T = Any](service: Service[T])

Bases: Service[T]

Service that tracks the process status of a wrapped service.

The status is RUNNING while the wrapped service executes, STOPPED once it returns or is cancelled, and ERRORED if it raises any other exception.

Parameters:

  • service (Service[T]) — The service to execute and track.
property status: ProcessStatus
async def execute() -> T
class DelegateServiceStatusTrackingMixin

Bases: ABC

Mixin reporting the status of the service held in _service.

The status is UNKNOWN if the service does not report one.

property status: ProcessStatus
class SingletonEventBroker[E: Event](
    node_id: str,
    event_subscriber_store: EventSubscriberStore[E],
    event_source_factory: EventSourceFactory[E],
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
    logger: FilteringBoundLogger = default_logger,
    distribution_interval: timedelta = timedelta(seconds=30),
)

Bases: EventBroker[E]

An event broker that allocates event sources on a single node.

While running, the broker repeatedly passes every registered subscriber each of the event sources it requested, waiting for the distribution interval between rounds. No coordination takes place between nodes, so each subscriber receives all of its requested event sources, and receives them again on every round. Sources are never withdrawn.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • event_subscriber_store (EventSubscriberStore[E]) — The store holding registered subscribers.
  • event_source_factory (EventSourceFactory[E]) — The factory used to construct event sources from the subscribers’ requested identifiers.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution. Defaults to a RetryErrorHandler.
  • logger (FilteringBoundLogger) — The logger to use.
  • distribution_interval (timedelta) — The time to wait between distribution rounds.
property status: ProcessStatus
async def register(subscriber: EventSubscriber[E]) -> None
async def execute() -> None

Run the broker until cancelled, distributing event sources.

Errors are handled by the configured error handler.

class SingletonEventBrokerSettings(
    distribution_interval: timedelta = timedelta(seconds=60),
)

Settings for a singleton event broker.

Attributes:

  • distribution_interval (timedelta) — The time to wait between rounds of passing event sources to subscribers.
distribution_interval: timedelta = timedelta(seconds=60)
SingletonEventBrokerTypeType = type[_SingletonEventBrokerType]
DistributedEventBrokerTypeType = type[_DistributedEventBrokerType]
class TypeMappingErrorHandler[T](
    type_mappings: TypeMappings[T] = MappingProxyType({}),
    default_error_handler: ErrorHandler[T] = RaiseErrorHandler(),
)

Bases: ErrorHandler[T]

Error handler that delegates based on the type of the exception.

The most specific mapped type in the exception’s class hierarchy determines the handler used. Its callback is invoked before handling. Unmapped exceptions are passed to the default error handler.

Parameters:

  • type_mappings (TypeMappings[T]) — Error handling definitions keyed by exception type, typically built with error_handler_type_mappings.
  • default_error_handler (ErrorHandler[T]) — The handler for unmapped exceptions. Defaults to raising them.
type_mappings
default_error_handler
def handle(exception: BaseException) -> ErrorHandlerDecision[T]
class WaitRetryStrategyDecision(duration: timedelta)

Bases: RetryStrategyDecision

Decision to wait before retrying.

Attributes:

  • duration (timedelta) — The time to wait before retrying.
duration: timedelta
def determine_multi_process_status(*statuses: ProcessStatus) -> ProcessStatus

Combine the statuses of several processes into a single status.

UNKNOWN statuses are ignored. If all remaining statuses are the same, that status is returned. Otherwise, in order of precedence: any ERRORED gives ERRORED; any STOPPING or STOPPED gives STOPPING; any STARTING gives STARTING; a mix of only RUNNING and WAITING gives RUNNING; and a mix including INITIALISED gives STARTING.

Parameters:

Returns:

  • ProcessStatus — the combined status, or UNKNOWN if no known statuses are given.
def exit_fatally_type_mapping(
    types: Sequence[type[BaseException]],
    exit_code_factory: Callable[[BaseException], int] = default_exit_code_factory,
    callback: Callable[[BaseException], None] | None = None,
) -> ExitFatallyTypeMappingDict

Build an exit fatally type mapping for error_handler_type_mappings.

Parameters:

  • types (Sequence[type[BaseException]]) — The exception types to exit on.
  • exit_code_factory (Callable[[BaseException], int]) — Returns the exit code for an exception. Defaults to always returning 1.
  • callback (Callable[[BaseException], None] | None) — Called with the exception before exiting.

Returns:

  • ExitFatallyTypeMappingDict — the type mapping.
def raise_exception_type_mapping(
    types: Sequence[type[BaseException]],
    exception_factory: Callable[[BaseException], BaseException],
    callback: Callable[[BaseException], None] | None = None,
) -> RaiseExceptionTypeMappingDict

Build a raise exception type mapping.

For use with error_handler_type_mappings.

Parameters:

  • types (Sequence[type[BaseException]]) — The exception types to raise on.
  • exception_factory (Callable[[BaseException], BaseException]) — Returns the exception to raise for a handled exception.
  • callback (Callable[[BaseException], None] | None) — Called with the exception before raising.

Returns:

  • RaiseExceptionTypeMappingDict — the type mapping.
def continue_execution_type_mapping[T](
    types: Sequence[type[BaseException]],
    value_factory: Callable[[BaseException], T],
    callback: Callable[[BaseException], None] | None = None,
) -> ContinueExecutionTypeMappingDict[T]

Build a continue execution type mapping.

For use with error_handler_type_mappings.

Parameters:

  • types (Sequence[type[BaseException]]) — The exception types to continue on.
  • value_factory (Callable[[BaseException], T]) — Returns the value to continue with for an exception.
  • callback (Callable[[BaseException], None] | None) — Called with the exception before continuing.

Returns:

  • ContinueExecutionTypeMappingDict[T] — the type mapping.
def retry_execution_type_mapping(
    types: Sequence[type[BaseException]],
    callback: Callable[[BaseException], None] | None = None,
    retry_strategy: RetryStrategy | None = None,
) -> RetryExecutionTypeMappingDict

Build a retry execution type mapping.

For use with error_handler_type_mappings.

Parameters:

  • types (Sequence[type[BaseException]]) — The exception types to retry on.
  • callback (Callable[[BaseException], None] | None) — Called with the exception before retrying.
  • retry_strategy (RetryStrategy | None) — Decides how to retry for each exception. If None, retries immediately.

Returns:

  • RetryExecutionTypeMappingDict — the type mapping.
def error_handler_type_mappings[T](
    exit_fatally: ExitFatallyTypeMappingValue = None,
    raise_exception: RaiseExceptionTypeMappingValue = None,
    continue_execution: ContinueExecutionTypeMappingValue[T] = None,
    retry_execution: RetryExecutionTypeMappingValue = None,
) -> TypeMappings[T]

Build type mappings for a TypeMappingErrorHandler.

Each argument is either a sequence of exception types, using the default behaviour for that kind of handling, or a mapping produced by the corresponding *_type_mapping function, except continue_execution, which must be a mapping. If an exception type appears under more than one argument, retry_execution takes precedence, then continue_execution, raise_exception and finally exit_fatally.

Parameters:

  • exit_fatally (ExitFatallyTypeMappingValue) — Exception types that should exit the process.
  • raise_exception (RaiseExceptionTypeMappingValue) — Exception types that should be raised.
  • continue_execution (ContinueExecutionTypeMappingValue[T]) — Exception types that should continue with a value, from continue_execution_type_mapping.
  • retry_execution (RetryExecutionTypeMappingValue) — Exception types that should be retried.

Returns:

  • TypeMappings[T] — the mapping from exception type to error handling definition.
@overload
def make_event_broker(
    node_id: str,
    broker_type: DistributedEventBrokerTypeType,
    storage_type: InMemoryEventBrokerStorageTypeType,
    **kwargs: Unpack[InMemoryDistributedBrokerParams],
) -> EventBroker[StoredEvent]
@overload
def make_event_broker(
    node_id: str,
    broker_type: DistributedEventBrokerTypeType,
    storage_type: PostgresEventBrokerStorageTypeType,
    **kwargs: Unpack[PostgresDistributedBrokerParams],
) -> EventBroker[StoredEvent]
@overload
def make_event_broker(
    node_id: str,
    broker_type: SingletonEventBrokerTypeType,
    storage_type: InMemoryEventBrokerStorageTypeType,
    **kwargs: Unpack[InMemorySingletonBrokerParams],
) -> EventBroker[StoredEvent]
@overload
def make_event_broker(
    node_id: str,
    broker_type: SingletonEventBrokerTypeType,
    storage_type: PostgresEventBrokerStorageTypeType,
    **kwargs: Unpack[PostgresSingletonBrokerParams],
) -> EventBroker[StoredEvent]

Construct an event broker of the given type and storage.

The keyword arguments required depend on the combination of broker and storage type:

  • in-memory storage requires settings and adapter;
  • Postgres storage requires settings, connection_settings and connection_pool, and optionally accepts adapter, defaulting to a PostgresEventStorageAdapter using connection_pool.

All combinations optionally accept an error_handler, defaulting to a RetryErrorHandler that retries immediately and indefinitely.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • broker_type (SingletonEventBrokerTypeType | DistributedEventBrokerTypeType) — The broker strategy, one of EventBrokerType.Singleton or EventBrokerType.Distributed.
  • storage_type (InMemoryEventBrokerStorageTypeType | PostgresEventBrokerStorageTypeType) — The storage for broker state, one of EventBrokerStorageType.InMemory or EventBrokerStorageType.Postgres.
  • **kwargs (Unpack[CombinedBrokerParams]) — The settings and dependencies for the chosen combination, where settings must be a SingletonEventBrokerSettings or DistributedEventBrokerSettings to match broker_type.

Returns:

Raises:

  • ValueError — If the broker or storage type is not recognised.
def make_event_store_subscriber[E: StoredEvent](
    *,
    subscriber_group: str,
    subscriber_id: str | None = None,
    subscription_request: EventSourceIdentifier,
    subscriber_state_category: EventCategory,
    subscriber_state_persistence_interval: EventCount = EventCount(100),
    event_processor: EventProcessor[E],
    logger: FilteringBoundLogger = default_logger,
    save_state_after_consumption: bool = True,
) -> EventSubscriptionConsumer[E]

Create a subscriber that consumes stored events from the event store.

A convenience wrapper around make_subscriber that tracks progress by the sequence number of the last processed event.

Parameters:

  • subscriber_group (str) — The group the subscriber belongs to. Subscribers in the same group share the work of consuming the requested sources.
  • subscriber_id (str | None) — The unique identifier of the subscriber. Defaults to a random UUID.
  • subscription_request (EventSourceIdentifier) — The identifier of the event source to subscribe to.
  • subscriber_state_category (EventCategory) — The category in which to store subscriber state.
  • subscriber_state_persistence_interval (EventCount) — The number of processed events after which subscriber state is saved.
  • event_processor (EventProcessor[E]) — The processor to pass events to.
  • logger (FilteringBoundLogger) — The logger to use.
  • save_state_after_consumption (bool) — Whether to save subscriber state after all available events have been consumed.

Returns:

def make_in_memory_event_broker(
    node_id: str,
    settings: DistributedEventBrokerSettings,
    adapter: EventStorageAdapter,
) -> EventBroker[StoredEvent]

Construct an in-memory distributed event broker.

Deprecated: use make_event_broker with EventBrokerType.Distributed and EventBrokerStorageType.InMemory instead.

Parameters:

Returns:

def make_postgres_event_broker(
    node_id: str,
    connection_settings: ConnectionSettings,
    connection_pool: AsyncConnectionPool[AsyncConnection],
    settings: DistributedEventBrokerSettings,
) -> EventBroker[StoredEvent]

Construct a Postgres-backed distributed event broker.

Deprecated: use make_event_broker with EventBrokerType.Distributed and EventBrokerStorageType.Postgres instead.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • connection_settings (ConnectionSettings) — The settings used to connect for locking.
  • connection_pool (AsyncConnectionPool[AsyncConnection]) — The pool used for broker state and event storage.
  • settings (DistributedEventBrokerSettings) — The settings for the distributed broker.

Returns:

def make_subscriber[E: Event](
    *,
    subscriber_group: str,
    subscriber_id: str | None = None,
    subscription_request: EventSourceIdentifier,
    subscriber_state_category: EventCategory,
    subscriber_state_persistence_interval: EventCount = EventCount(100),
    subscriber_state_converter: EventConsumerStateConverter[E],
    event_processor: EventProcessor[E],
    logger: FilteringBoundLogger = default_logger,
    save_state_after_consumption: bool = True,
) -> EventSubscriptionConsumer[E]

Create a subscriber that consumes events from the sources it is given.

The returned subscriber creates an EventSourceConsumer for each source it accepts, all sharing a single EventConsumerStateStore.

Parameters:

  • subscriber_group (str) — The group the subscriber belongs to. Subscribers in the same group share the work of consuming the requested sources.
  • subscriber_id (str | None) — The unique identifier of the subscriber. Defaults to a random UUID.
  • subscription_request (EventSourceIdentifier) — The identifier of the event source to subscribe to.
  • subscriber_state_category (EventCategory) — The category in which to store subscriber state.
  • subscriber_state_persistence_interval (EventCount) — The number of processed events after which subscriber state is saved.
  • subscriber_state_converter (EventConsumerStateConverter[E]) — The converter between events and subscriber state.
  • event_processor (EventProcessor[E]) — The processor to pass events to.
  • logger (FilteringBoundLogger) — The logger to use.
  • save_state_after_consumption (bool) — Whether to save subscriber state after all available events have been consumed.

Returns:

class RetryStrategy[T]

Bases: ABC

Base class for strategies deciding how to retry after an exception.

Used by RetryErrorHandler.

def calculate(exception: Exception) -> RetryStrategyDecision

Decide how to retry after the given exception.

Parameters:

  • exception (Exception) — The exception that caused the failure.

Returns:

class ConstantRetryStrategy[T](time: timedelta)

Bases: RetryStrategy[T]

Retry strategy that always waits the same time before retrying.

Attributes:

  • time (timedelta) — The time to wait before each retry.
time: timedelta
def calculate(exception: Exception) -> RetryStrategyDecision
class IncludeExceptionsRetryStrategy[T](
    delegate: RetryStrategy,
    include_list: Sequence[type[Exception]],
)

Bases: RetryStrategy[T]

Applies a delegate strategy only to the listed exception types.

Exceptions not matching any of the listed types are retried immediately, rather than not retried.

Parameters:

  • delegate (RetryStrategy) — The strategy to apply to matching exceptions.
  • include_list (Sequence[type[Exception]]) — The exception types to apply the delegate to, including their subclasses.
def calculate(exception: Exception) -> RetryStrategyDecision
class ExcludeExceptionsRetryStrategy[T](
    delegate: RetryStrategy,
    exclude_list: Sequence[type[Exception]],
)

Bases: RetryStrategy[T]

Applies a delegate strategy to all but the listed exception types.

Exceptions matching any of the listed types are retried immediately, rather than not retried.

Parameters:

  • delegate (RetryStrategy) — The strategy to apply to non-matching exceptions.
  • exclude_list (Sequence[type[Exception]]) — The exception types not to apply the delegate to, including their subclasses.
def calculate(exception: Exception) -> RetryStrategyDecision
def make_in_memory_distributed_event_broker(
    node_id: str,
    settings: DistributedEventBrokerSettings,
    adapter: EventStorageAdapter,
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
) -> EventBroker[StoredEvent]

Construct a distributed event broker with in-memory state.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • settings (DistributedEventBrokerSettings) — The settings for the broker.
  • adapter (EventStorageAdapter) — The storage adapter from which event sources are read.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution.

Returns:

def make_postgres_distributed_event_broker(
    node_id: str,
    connection_settings: ConnectionSettings,
    connection_pool: AsyncConnectionPool[AsyncConnection],
    settings: DistributedEventBrokerSettings,
    adapter: EventStorageAdapter | None = None,
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
) -> EventBroker[StoredEvent]

Construct a distributed event broker with Postgres-backed state.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • connection_settings (ConnectionSettings) — The settings used to connect for leadership locking.
  • connection_pool (AsyncConnectionPool[AsyncConnection]) — The pool used for subscriber and subscription state, and for the default adapter.
  • settings (DistributedEventBrokerSettings) — The settings for the broker.
  • adapter (EventStorageAdapter | None) — The storage adapter from which event sources are read. Defaults to a PostgresEventStorageAdapter on connection_pool.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution.

Returns:

def make_in_memory_singleton_event_broker(
    node_id: str,
    settings: SingletonEventBrokerSettings,
    adapter: EventStorageAdapter,
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
) -> EventBroker[StoredEvent]

Construct a singleton event broker reading from the given adapter.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • settings (SingletonEventBrokerSettings) — The settings for the broker.
  • adapter (EventStorageAdapter) — The storage adapter from which event sources are read.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution.

Returns:

def make_postgres_singleton_event_broker(
    node_id: str,
    connection_settings: ConnectionSettings,
    connection_pool: AsyncConnectionPool[AsyncConnection],
    settings: SingletonEventBrokerSettings,
    adapter: EventStorageAdapter | None = None,
    error_handler: ErrorHandler[NoneType] = RetryErrorHandler(),
) -> EventBroker[StoredEvent]

Construct a singleton event broker backed by Postgres.

Parameters:

  • node_id (str) — The ID of the node on which the broker runs.
  • connection_settings (ConnectionSettings) — The Postgres connection settings. Currently unused by the singleton broker.
  • connection_pool (AsyncConnectionPool[AsyncConnection]) — The pool used to construct the default adapter.
  • settings (SingletonEventBrokerSettings) — The settings for the broker.
  • adapter (EventStorageAdapter | None) — The storage adapter from which event sources are read. Defaults to a PostgresEventStorageAdapter on connection_pool.
  • error_handler (ErrorHandler[NoneType]) — The handler for errors raised during execution.

Returns:

logicblocks.event.processing.broker.strategies.distributed

Section titled “logicblocks.event.processing.broker.strategies.distributed”
COORDINATOR_LOCK_NAME = class_fullname(DefaultEventSubscriptionCoordinator)
class DefaultEventSubscriberManager[E: Event](
    node_id: str,
    subscriber_store: EventSubscriberStore[E],
    subscriber_state_store: EventSubscriberStateStore,
    logger: FilteringBoundLogger = default_logger,
    heartbeat_interval: timedelta = timedelta(seconds=10),
    purge_interval: timedelta = timedelta(minutes=1),
    subscriber_max_age: timedelta = timedelta(minutes=10),
)

Bases: EventSubscriberManager[E]

Registers subscribers, sends heartbeats and purges stale subscribers.

Only subscribers present when start is called are registered in shared state; heartbeats are sent only for healthy subscribers.

Parameters:

  • node_id (str) — The ID of the node on which the manager runs.
  • subscriber_store (EventSubscriberStore[E]) — The store of this node’s subscribers.
  • subscriber_state_store (EventSubscriberStateStore) — The shared store of subscriber state.
  • logger (FilteringBoundLogger) — The logger to use.
  • heartbeat_interval (timedelta) — The time to wait between heartbeats.
  • purge_interval (timedelta) — The time to wait between purges.
  • subscriber_max_age (timedelta) — How long since last seen before any subscriber, on any node, is purged.
async def add(subscriber: EventSubscriber[E]) -> Self
async def start()
async def maintain()

Run heartbeats and purges concurrently until cancelled.

async def stop()
async def register()

Add each subscriber to the subscriber state store.

async def unregister()

Remove each subscriber from the subscriber state store.

async def heartbeat()

Periodically record healthy subscribers as seen, until cancelled.

Unhealthy subscribers are logged and not recorded.

async def purge()

Periodically purge stale subscribers, until cancelled.

class DefaultEventSubscriptionCoordinator(
    node_id: str,
    lock_manager: LockManager,
    subscriber_state_store: EventSubscriberStateStore,
    subscription_state_store: EventSubscriptionStateStore,
    logger: FilteringBoundLogger = default_logger,
    subscriber_max_time_since_last_seen: timedelta = timedelta(seconds=60),
    distribution_interval: timedelta = timedelta(seconds=20),
    leadership_max_duration: timedelta = timedelta(minutes=15),
    leadership_attempt_interval: timedelta = timedelta(seconds=5),
)

Bases: EventSubscriptionCoordinator

Coordinates allocation as the leader elected via a shared lock.

Nodes compete for the lock named by LOCK_NAME. The holder distributes event sources at each distribution interval until its leadership maximum duration elapses, then releases the lock and competes again.

Parameters:

  • node_id (str) — The ID of the node on which the coordinator runs.
  • lock_manager (LockManager) — The lock manager used for leader election.
  • subscriber_state_store (EventSubscriberStateStore) — The shared store of subscriber state.
  • subscription_state_store (EventSubscriptionStateStore) — The shared store of subscription state.
  • logger (FilteringBoundLogger) — The logger to use.
  • subscriber_max_time_since_last_seen (timedelta) — How long since last seen before a subscriber is no longer allocated event sources.
  • distribution_interval (timedelta) — The time to wait between distributions while leader.
  • leadership_max_duration (timedelta) — The maximum time to hold leadership.
  • leadership_attempt_interval (timedelta) — The time to wait between attempts to acquire leadership.
property status: ProcessStatus
async def coordinate() -> None
async def distribute() -> None

Recalculate and store subscriptions for all live subscribers.

Subscriptions belonging to subscribers not seen recently are removed. Within each subscriber group, the target event sources are the longest set of subscription requests in the group; subscribers requesting anything else lose their subscription. Event sources not yet allocated to an eligible subscriber are spread round-robin across the eligible subscribers, and sources no longer requested are removed. All changes are applied to the subscription state store in a single call.

class DefaultEventSubscriptionObserver[E: Event](
    node_id: str,
    subscriber_store: EventSubscriberStore[E],
    subscription_state_store: EventSubscriptionStateStore,
    event_source_factory: EventSourceFactory[E],
    subscription_difference: EventSubscriptionDifference = EventSubscriptionDifference(),
    logger: FilteringBoundLogger = default_logger,
    synchronisation_interval: timedelta = timedelta(seconds=20),
)

Bases: EventSubscriptionObserver[E]

Periodically synchronises stored subscriptions to local subscribers.

Parameters:

  • node_id (str) — The ID of the node on which the observer runs.
  • subscriber_store (EventSubscriberStore[E]) — The store of this node’s subscribers.
  • subscription_state_store (EventSubscriptionStateStore) — The shared store of subscription state.
  • event_source_factory (EventSourceFactory[E]) — The factory used to construct event sources from allocated identifiers.
  • subscription_difference (EventSubscriptionDifference) — Calculates changes between subscription snapshots.
  • logger (FilteringBoundLogger) — The logger to use.
  • synchronisation_interval (timedelta) — The time to wait between synchronisations.
property status: ProcessStatus
async def observe()
async def synchronise()

Apply changes in subscriptions since the last synchronisation.

Revoked event sources are withdrawn before newly allocated ones are accepted. Changes for subscribers not on this node are ignored.

class EventSubscriberManager[E: Event]

Bases: ABC

Manages the registration of this node’s subscribers in shared state.

async def add(subscriber: EventSubscriber[E]) -> Self

Add a subscriber to be managed.

Returns:

  • Self — this manager, for chaining.
async def start() -> None

Register the managed subscribers in shared state.

async def maintain() -> None

Keep the subscribers’ shared state current until cancelled.

async def stop() -> None

Unregister the managed subscribers from shared state.

class EventSubscriberState(
    group: str,
    id: str,
    node_id: str,
    subscription_requests: Sequence[EventSourceIdentifier],
    last_seen: datetime,
)

The shared record of a subscriber, as seen by the coordinator.

Attributes:

  • group (str) — The group the subscriber belongs to.
  • id (str) — The ID of the subscriber within its group.
  • node_id (str) — The ID of the node on which the subscriber runs.
  • subscription_requests (Sequence[EventSourceIdentifier]) — The event sources the subscriber requests.
  • last_seen (datetime) — When the subscriber was last registered or heartbeated.
group: str
id: str
node_id: str
subscription_requests: Sequence[EventSourceIdentifier]
last_seen: datetime
property key: EventSubscriberKey
property subscription_key: EventSubscriptionKey
class EventSubscriberStateStore

Bases: ABC

Shared store of subscriber state used for distributed coordination.

async def add(subscriber: EventSubscriber[Any]) -> None

Record a subscriber as present on this node and seen now.

If the subscriber is already present, it is refreshed instead.

async def remove(subscriber: EventSubscriber[Any]) -> None

Remove a subscriber’s state, if present.

async def list(
    subscriber_group: str | None = None,
    max_time_since_last_seen: timedelta | None = None,
) -> Sequence[EventSubscriberState]

List subscriber state, optionally filtered.

Parameters:

  • subscriber_group (str | None) — If given, only include subscribers in this group.
  • max_time_since_last_seen (timedelta | None) — If given, only include subscribers seen more recently than this.

Returns:

async def heartbeat(subscriber: EventSubscriber[Any]) -> None

Record an existing subscriber as seen now.

Subscribers not already in the store are ignored.

async def purge(
    max_time_since_last_seen: timedelta = timedelta(minutes=5),
) -> None

Remove all subscribers not seen within the given time.

class EventSubscriptionChange(
    group: str,
    id: str,
    event_source: EventSourceIdentifier,
)

The allocation or revocation of one event source for a subscriber.

group: str
id: str
event_source: EventSourceIdentifier
property key: EventSubscriptionKey
property subscriber_key: EventSubscriberKey
class EventSubscriptionChangeset(
    allocations: Set[EventSubscriptionChange],
    revocations: Set[EventSubscriptionChange],
)

The event sources allocated and revoked between two snapshots.

allocations: Set[EventSubscriptionChange]
revocations: Set[EventSubscriptionChange]
class EventSubscriptionCoordinator

Bases: Process, ABC

Allocates requested event sources to subscribers across nodes.

async def coordinate() -> None

Coordinate event source allocation until cancelled.

class EventSubscriptionDifference

Calculates per event source changes between subscription snapshots.

@staticmethod
def diff(
    existing: Sequence[EventSubscriptionState],
    updated: Sequence[EventSubscriptionState],
) -> EventSubscriptionChangeset

Calculate the event sources allocated and revoked.

Parameters:

Returns:

  • EventSubscriptionChangeset — the changeset, with an allocation for each event source present only in updated and a revocation for each event source present only in existing, per subscription.
class EventSubscriptionKey(group: str, id: str)

The unique identity of a subscription, matching its subscriber’s.

group: str
id: str
class EventSubscriptionObserver[E: Event]

Bases: Process, ABC

Applies stored subscriptions to the subscribers on this node.

async def observe() -> None

Synchronise subscriptions to subscribers until cancelled.

class EventSubscriptionState(
    group: str,
    id: str,
    node_id: str,
    event_sources: Sequence[EventSourceIdentifier],
)

The event sources allocated to a subscriber.

Attributes:

  • group (str) — The group of the subscriber.
  • id (str) — The ID of the subscriber within its group.
  • node_id (str) — The ID of the node recorded against the subscription.
  • event_sources (Sequence[EventSourceIdentifier]) — The event sources allocated to the subscriber.
group: str
id: str
node_id: str
event_sources: Sequence[EventSourceIdentifier]
property key: EventSubscriptionKey
property subscriber_key: EventSubscriberKey
class EventSubscriptionStateChange(
    type: EventSubscriptionStateChangeType,
    subscription: EventSubscriptionState,
)

A change to apply to the subscription state store.

type: EventSubscriptionStateChangeType
subscription: EventSubscriptionState
class EventSubscriptionStateChangeType

Bases: StrEnum

The kind of change to make to a stored subscription.

ADD = 'add'
REMOVE = 'remove'
REPLACE = 'replace'
class EventSubscriptionStateStore

Bases: ABC

Shared store of subscriptions used for distributed coordination.

async def list() -> Sequence[EventSubscriptionState]

Return all subscriptions.

async def get(key: EventSubscriptionKey) -> EventSubscriptionState | None

Return the subscription with the given key, or None if absent.

async def add(subscription: EventSubscriptionState) -> None

Add a new subscription.

Raises:

  • ValueError — If a subscription with the same key already exists.
async def remove(subscription: EventSubscriptionState) -> None

Remove an existing subscription.

Raises:

  • ValueError — If no subscription with the same key exists.
async def replace(subscription: EventSubscriptionState) -> None

Replace the event sources of an existing subscription.

Raises:

  • ValueError — If no subscription with the same key exists.
async def apply(changes: Sequence[EventSubscriptionStateChange]) -> None

Apply a batch of additions, removals and replacements.

Parameters:

Raises:

  • ValueError — If more than one change targets the same key, or if any individual change is invalid.
class InMemoryEventSubscriberStateStore(
    node_id: str,
    clock: Clock = SystemClock(),
)

Bases: EventSubscriberStateStore

An event subscriber state store holding state in memory.

Parameters:

  • node_id (str) — The ID of the node recorded against added subscribers.
  • clock (Clock) — The clock used to timestamp subscribers.
node_id
clock
subscribers: list[EventSubscriberState]
async def add(subscriber: EventSubscriber[Any]) -> None
async def remove(subscriber: EventSubscriber[Any]) -> None
async def list(
    subscriber_group: str | None = None,
    max_time_since_last_seen: timedelta | None = None,
) -> Sequence[EventSubscriberState]
async def heartbeat(subscriber: EventSubscriber[Any]) -> None
async def purge(
    max_time_since_last_seen: timedelta = timedelta(minutes=5),
) -> None
class InMemoryEventSubscriptionStateStore(node_id: str)

Bases: EventSubscriptionStateStore

An event subscription state store holding state in memory.

Parameters:

  • node_id (str) — The node ID recorded against stored subscriptions, in place of the node ID on the given subscription.
node_id
async def list() -> Sequence[EventSubscriptionState]
async def get(key: EventSubscriptionKey) -> EventSubscriptionState | None
async def add(subscription: EventSubscriptionState) -> None
async def remove(subscription: EventSubscriptionState) -> None
async def replace(subscription: EventSubscriptionState) -> None
async def apply(changes: Sequence[EventSubscriptionStateChange]) -> None
class PostgresEventSubscriberStateStore(
    *,
    node_id: str,
    connection_source: postgres.ConnectionSource,
    clock: Clock = SystemClock(),
    table_settings: postgres.TableSettings = postgres.TableSettings(table_name='subscribers'),
    query_converter: postgres.QueryConverter | None = None,
)

Bases: EventSubscriberStateStore

An event subscriber state store persisting state in Postgres.

Parameters:

  • node_id (str) — The ID of the node recorded against added subscribers.
  • connection_source (postgres.ConnectionSource) — The connection pool, or connection settings from which to create one.
  • clock (Clock) — The clock used to timestamp subscribers.
  • table_settings (postgres.TableSettings) — The settings of the table holding subscriber state.
  • query_converter (postgres.QueryConverter | None) — The converter used to build list queries. Defaults to one with the default clause and query converters.
connection_pool
node_id
clock
table_settings
query_converter
async def add(subscriber: EventSubscriber[Any]) -> None
async def remove(subscriber: EventSubscriber[Any]) -> None
async def list(
    subscriber_group: str | None = None,
    max_time_since_last_seen: timedelta | None = None,
) -> Sequence[EventSubscriberState]
async def heartbeat(subscriber: EventSubscriber[Any]) -> None
async def purge(
    max_time_since_last_seen: timedelta = timedelta(minutes=5),
) -> None
class PostgresEventSubscriptionStateStore(
    *,
    node_id: str,
    connection_source: postgres.ConnectionSource,
    table_settings: postgres.TableSettings = postgres.TableSettings(table_name='subscriptions'),
    query_converter: postgres.QueryConverter | None = None,
)

Bases: EventSubscriptionStateStore

An event subscription state store persisting state in Postgres.

get is not supported and raises NotImplementedError.

Parameters:

  • node_id (str) — The node ID recorded against stored subscriptions, in place of the node ID on the given subscription.
  • connection_source (postgres.ConnectionSource) — The connection pool, or connection settings from which to create one.
  • table_settings (postgres.TableSettings) — The settings of the table holding subscriptions.
  • query_converter (postgres.QueryConverter | None) — The converter used to build list queries. Defaults to one with the default clause and query converters.
connection_pool
node_id
table_settings
query_converter
async def list() -> Sequence[EventSubscriptionState]
async def get(key: EventSubscriptionKey) -> EventSubscriptionState | None
async def add(subscription: EventSubscriptionState) -> None
async def remove(subscription: EventSubscriptionState) -> None
async def replace(subscription: EventSubscriptionState) -> None
async def apply(changes: Sequence[EventSubscriptionStateChange]) -> None

Apply a batch of changes using a single connection.

Raises:

  • ValueError — If more than one change targets the same key, or if any individual change is invalid.

logicblocks.event.processing.broker.subscribers

Section titled “logicblocks.event.processing.broker.subscribers”
class EventSubscriberStore[E: Event]

Bases: ABC

Holds the subscriber instances registered with a broker on a node.

async def add(subscriber: EventSubscriber[E]) -> None

Add a subscriber, replacing any existing one with the same key.

async def remove(subscriber: EventSubscriber[E]) -> None

Remove a subscriber, if present.

async def get(key: EventSubscriberKey) -> EventSubscriber[E] | None

Return the subscriber with the given key, or None if absent.

async def list() -> Sequence[EventSubscriber[E]]

Return all subscribers in the store.

class InMemoryEventSubscriberStore[E: Event]

Bases: EventSubscriberStore[E]

An event subscriber store holding subscribers in memory.

subscribers: dict[EventSubscriberKey, EventSubscriber[E]]
async def add(subscriber: EventSubscriber[E]) -> None
async def remove(subscriber: EventSubscriber[E]) -> None
async def get(key: EventSubscriberKey) -> EventSubscriber[E] | None
async def list() -> Sequence[EventSubscriber[E]]
class AutoCommitEventIteratorProcessor[E: Event]

Bases: ABC

Base class for processors that consume an iterator of events.

Each event is acknowledged, and the consumer state saved if needed, when the processor requests the next event from the iterator.

async def process(events: EventIterator[E]) -> None

Process events from the provided iterator.

Parameters:

  • events (EventIterator[E]) — An asynchronous iterator over the events to process.
type EventIterator[E: Event] = AsyncIterator[E]
class EventProcessorManager[E: Event]

Bases: ABC

Allows a processor to control acknowledgement and state persistence.

Passed to a ManagedEventIteratorProcessor so that it can decide when events count as processed and when consumer state is saved.

def acknowledge(events: E | Sequence[E]) -> None

Record one or more events as processed.

Parameters:

  • events (E | Sequence[E]) — The event, or sequence of events, to acknowledge.
async def commit(*, force: bool = False) -> None

Persist the consumer state for acknowledged events.

Parameters:

  • force (bool) — If True, save the state regardless of the persistence interval. Otherwise, save only if the persistence interval has been reached.
class ManagedEventIteratorProcessor[E: Event]

Bases: ABC

Base class for processors that manage acknowledgement themselves.

Events are not acknowledged automatically; the processor must use the provided manager to acknowledge events and commit state.

async def process(
    events: EventIterator[E],
    manager: EventProcessorManager[E],
) -> None

Process events from the provided iterator.

Parameters:

  • events (EventIterator[E]) — An asynchronous iterator over the events to process.
  • manager (EventProcessorManager[E]) — The manager used to acknowledge events and commit consumer state.
type SupportedProcessors[E: Event] = EventProcessor[E] | AutoCommitEventIteratorProcessor[E] | ManagedEventIteratorProcessor[E]
class StoredEventEventConsumerStateConverter

Bases: EventConsumerStateConverter[StoredEvent]

Tracks consumption of stored events by their sequence number.

State takes the form {"last_sequence_number": <int | None>}.

def event_to_state(event: StoredEvent) -> JsonObject
def initial_state() -> JsonObject
def state_to_query_constraint(
    state: JsonObject,
) -> constraints.QueryConstraint | None
type CallableServiceCallable[T] = Callable[[], Awaitable[T]]