rowbuffer

package

v0.1.7 Latest Latest Go to latest Published: Feb 26, 2026 License: GPL-3.0 Imports: 6 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/ethpandaops/execution-processor

Links

Open Source Insights

Documentation ¶

Overview ¶

Package rowbuffer provides thread-safe row batching for ClickHouse inserts. It pools rows in memory across concurrent tasks and flushes when hitting a row limit or timer interval.

Index ¶

type Buffer
- func New[R any](cfg Config, flushFn FlushFunc[R], log logrus.FieldLogger) *Buffer[R]
type Config
type FlushFunc

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Buffer ¶

type Buffer[R any] struct {
	// contains filtered or unexported fields
}

Buffer provides thread-safe row batching for ClickHouse inserts.

func New ¶

func New[R any](cfg Config, flushFn FlushFunc[R], log logrus.FieldLogger) *Buffer[R]

New creates a new Buffer with the given configuration and flush function.

func (*Buffer[R]) Len ¶

func (b *Buffer[R]) Len() int

Len returns the current number of buffered rows.

func (*Buffer[R]) Start ¶

func (b *Buffer[R]) Start(ctx context.Context) error

Start starts the flush timer goroutine.

func (*Buffer[R]) Stop ¶

func (b *Buffer[R]) Stop(ctx context.Context) error

Stop stops the buffer, flushing any remaining rows.

func (*Buffer[R]) Submit ¶

func (b *Buffer[R]) Submit(ctx context.Context, rows []R) error

Submit adds rows to the buffer and blocks until they are successfully flushed. Returns an error if the flush fails or the context is cancelled.

func (*Buffer[R]) WaiterCount ¶

func (b *Buffer[R]) WaiterCount() int

WaiterCount returns the current number of waiting tasks.

type Config ¶

type Config struct {
	MaxRows       int           // Flush threshold (default: 100000)
	FlushInterval time.Duration // Max wait before flush (default: 1s)
	Network       string        // For metrics
	Processor     string        // For metrics
	Table         string        // For metrics
}

Config holds configuration for the row buffer.

type FlushFunc ¶

type FlushFunc[R any] func(ctx context.Context, rows []R) error

FlushFunc is the function called to flush rows to the database.

Source Files ¶

View all Source files

buffer.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL