partitionmap

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2025 License: GPL-3.0 Imports: 8 Imported by: 0

README

PartitionMap

golang GoDoc Go Report Issues Size Tag View examples License

Purpose

This is a Go package that implements a partitioned data structure for storing key/value pairs. Key features include:

  • Using a partitioning strategy with CRC32 hashing to distribute keys across multiple partitions,
  • thread-safe implementation with mutex locks for concurrent data access,
  • generic implementation supporting any key and value types,
  • designed for efficient concurrent access by reducing lock contention,
  • implements standard map operations (Get, Put, Delete, etc.).

The package is designed to provide better performance than a standard map when used in highly concurrent environments by distributing the data across multiple partitions.

Installation

You can use Go to install this package for you:

go get -u github.com/mwat56/partitionmap

Usage

Basic Usage
package main

import (
	"fmt"
	"github.com/mwat56/partitionmap"
)

func main() {
	// Create a new partition map with string keys and string values
	pm := partitionmap.New[string, string]()

	// Store key-value pairs
	pm.Put("user1", "John Doe")
	pm.Put("user2", "Jane Smith")

	// Retrieve values
	value, exists := pm.Get("user1")
	if exists {
		fmt.Printf("Found user1: %s\n", value)
	}

	// Delete a key
	pm.Delete("user2")

	// Check map size
	fmt.Printf("Map contains %d entries\n", pm.Len())

	// Iterate over all entries
	pm.ForEach(func(aKey string, aValue string) {
		fmt.Printf("%s: %s\n", aKey, aValue)
	})

	// Get all keys
	keys := pm.Keys()
	fmt.Printf("Keys: %v\n", keys)

	// Clear all entries
	pm.Clear()
} // main()
Key Features

  1. Generic Implementation: Works with any key and value types using Go generics.

     intMap := partitionmap.New[string, int]()
 structMap := partitionmap.New[int, MyStruct]()

  2. Thread-Safety: All operations are thread-safe, making it suitable for concurrent access.

     // Can be safely used from multiple goroutines
 go func() { pm.Put("key1", "value1") }()
 go func() { pm.Get("key2") }()

  3. Method Chaining: Most methods return the map itself for chaining operations.

     pm.Put("key1", "value1").Put("key2", "value2").Delete("key3")

  4. Efficient Partitioning: Uses CRC32 hashing to distribute keys across 64 partitions, reducing lock contention.

  5. Lazy Partition Creation: Partitions are created lazily when needed, saving memory in a sparse map.

Performance Considerations
  • The map uses partitioning to reduce lock contention in concurrent scenarios.
  • Partitions are created lazily when needed.
  • For high-concurrency applications, this implementation can offer better performance than a standard map with a global lock.

The package is ideal for scenarios where you need a thread-safe map with good performance under concurrent access patterns.

Libraries

The following external libraries were used building partitionmap:

  • No external libraries are used by this module.

Licence

    Copyright © 2024, 2025  M.Watermann, 10247 Berlin, Germany
                    All rights reserved
                EMail : <[email protected]>

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

You should have received a copy of the GNU General Public License along with this program. If not, see the GNU General Public License for details.

GFDL

Documentation

Overview

Package partitionmap implements a partitions map to store key/value pairs. 

Copyright © 2024. 2025  M.Watermann, 10247 Berlin, Germany
                All rights reserved
            EMail : <[email protected]>

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

You should have received a copy of the GNU General Public License along with this program. If not, see the [GNU General Public License](http://www.gnu.org/licenses/gpl.html) for details.

Copyright © 2024, 2025 M.Watermann, 10247 Berlin, Germany 

    All rights reserved
EMail : <[email protected]>

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type TMetrics added in v0.4.0 

type TMetrics struct {
	Parts    int
	Keys     int
	Avg      int
	PartKeys map[int]int
}

`TMetrics` provides statistics about the partition usage.

`Parts` is the number of partitions that are actually in use. `Keys` is the total number of keys across all partitions. `Avg` is the average number of keys per partition. `PartKeys` is a map where the key is the partition index and the value is the number of keys in that partition.

type TPartitionMap 

type TPartitionMap[K cmp.Ordered, V any] struct {
	sync.RWMutex // protect the list of partitions
	// contains filtered or unexported fields
}

`TPartitionMap` is a slice of partitions holding the key/value pairs.

func New added in v0.2.0 

func New[K cmp.Ordered, V any]() *TPartitionMap[K, V]

`New()` creates and initialises a new partitioned map instance.

This is a constructor function that returns a pointer to a new `TPartitionMap` instance with the specified key and value types.

The returned partitioned map is initialised with the predefined number of partitions (128), but the actual partition instances are created lazily when needed.

Example usage: 

pm := New[string, string]()
pm.Put("key1", "value1")
value, exists := pm.Get("key1")

Returns:

  • `*TPartitionMap[K, V]`: A pointer to a newly created partitioned map.

func (*TPartitionMap[K, V]) Clear added in v0.1.2 

func (pm *TPartitionMap[K, V]) Clear() *TPartitionMap[K, V]

`Clear()` removes all key/value pairs from the partitioned map.

Returns:

  • `*TPartitionMap[K, V]`: The partitioned map itself, allowing method chaining.

func (*TPartitionMap[K, V]) Delete 

func (pm *TPartitionMap[K, V]) Delete(aKey K) *TPartitionMap[K, V]

`Delete()` removes a key/value pair from the partitioned map.

Parameters:

  • `aKey`: The key of the key/value pair to be deleted.

Returns:

  • `*TPartitionMap[K, V]`: The partitioned map itself, allowing method chaining.

func (*TPartitionMap[K, V]) ForEach added in v0.1.2 

func (pm *TPartitionMap[K, V]) ForEach(aFunc func(aKey K, aValue V)) *TPartitionMap[K, V]

`ForEach()` executes the provided function for each key/value pair in the partitioned map.

Parameters:

  • `aFunc`: The function to execute for each key/value pair.

Returns:

  • `*TPartitionMap[K, V]`: The partitioned map itself, allowing method chaining.

func (*TPartitionMap[K, V]) Get 

func (pm *TPartitionMap[K, V]) Get(aKey K) (V, bool)

`Get()` retrieves a key/value pair from the partitioned map.

If the partitioned map contains a key/value pair with the specified key, the function returns the associated value and a boolean value indicating whether the key was found. If the key is not found, the function returns the zero value of type V and a boolean value of false.

Parameters:

  • `aKey`: The key of the key/value pair to be retrieved.

Returns:

  • `V`: The value associated with the key (if found).
  • `bool`: Indicating for whether the key was found.

func (*TPartitionMap[K, V]) GetOrDefault added in v0.3.0 

func (pm *TPartitionMap[K, V]) GetOrDefault(aKey K, aDefault V) V

`GetOrDefault()` retrieves a value for the given key, or returns the given default value if the key doesn't exist in the partitioned map.

Parameters:

  • `aKey`: The key to look up.
  • `aDefault`: The default value to return if the key is not found.

Returns:

  • `V`: The value associated with `aKey`, or `aDefault` if not found.

func (*TPartitionMap[K, V]) Keys 

func (pm *TPartitionMap[K, V]) Keys() []K

`Keys()` returns a slice of all keys in the partitioned map.

The partitioned map is divided into multiple partitions, each holding a subset of the keys. This method retrieves the keys from all partitions and returns them in a sorted slice.

The returned slice is a copy of the keys from all partitions, sorted in ascending order.

Returns:

  • `[]K`: A slice of all the keys in the current partitioned map.

func (*TPartitionMap[K, V]) Len 

func (pm *TPartitionMap[K, V]) Len() (rLen int)

`Len()` returns the total number of key/value pairs in the partitioned map.

Returns:

  • `rLen`: The number of all key/value pairs in the partitioned map.

func (*TPartitionMap[K, V]) PartitionStats added in v0.4.0 

func (pm *TPartitionMap[K, V]) PartitionStats() *TMetrics

`PartitionStats()` returns statistics about the partition usage.

This method returns information about how many partitions are actually in use, how many keys are distributed across the partitions, and how many key/value pairs are stored in each partition. This can be useful for monitoring and optimising the distribution of keys across partitions.

Returns:

  • `*TMetrics`: A pointer to a `TMetrics` instance containing the statistics.

func (*TPartitionMap[K, V]) Put 

func (pm *TPartitionMap[K, V]) Put(aKey K, aValue V) *TPartitionMap[K, V]

`Put()` stores a key/value pair into the partitioned map. If the key already exists, it will be updated.

Parameters:

  • `aKey`: The key to be put into the partitioned map.
  • `aValue`: The value associated with the key.

Returns:

  • `*TPartitionMap[K, V]`: The partitioned map itself, allowing method chaining.

func (*TPartitionMap[K, V]) String 

func (pm *TPartitionMap[K, V]) String() string

`String()` returns a string representation of the `TPartitionMap`. It iterates over all existing partitions and concatenates their string representations.

The keys in returned string are sorted in ascending order.

Returns:

  • `string`: A string representation of the partitioned map.

func (*TPartitionMap[K, V]) Values added in v0.3.0 

func (pm *TPartitionMap[K, V]) Values() []V

`Values()` returns a slice of all values in the partitioned map.

The partitioned map is divided into multiple partitions, each holding a subset of the values. This method retrieves the values from all partitions and returns them in a slice.

The order of values in the returned slice corresponds to the order of keys returned by the `Keys()` method.

Returns:

  • `[]V`: A slice of all the values in the current partitioned map.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.