micro/sync
Jake Sanders 1b4e881d74
Rewrite the store interface (#1335)
* WIP store rewrite

* Fix memory store tests

* Store hard expiry times rather than duration!

* Clarify memory test

* Add limit to store interface

* Implement suffix option

* Don't return nils from noop store

* Fix syncmap

* Start fixing store service

* wip service and cache

* Use _ for special characters in cockroachdb namespace

* Improve cockroach namespace comment

* Use service name as default store namespace

* Fixes

* Implement Store Scope

* Start fixing etcd

* implement read and write with expiry and prefix

* Fix etcd tests

* Fix cockroach store

* Fix cloudflare interface

* Fix certmagic / cloudflare store

* comment lint

* cache isn't implemented yet

* Only prepare DB staements once

Co-authored-by: Ben Toogood <ben@micro.mu>
Co-authored-by: ben-toogood <bentoogood@gmail.com>
2020-03-12 13:41:30 +00:00
..
event Add sync => go-sync 2019-05-31 00:43:23 +01:00
leader fix import paths for v2 release 2020-01-30 14:44:40 +03:00
lock fix import paths for v2 release 2020-01-30 14:44:40 +03:00
task fix import paths for v2 release 2020-01-30 14:44:40 +03:00
time fix import paths for v2 release 2020-01-30 14:44:40 +03:00
cron.go minimize allocations in logger and tunnel code (#1323) 2020-03-11 17:55:39 +00:00
map.go Rewrite the store interface (#1335) 2020-03-12 13:41:30 +00:00
options.go fix import paths for v2 release 2020-01-30 14:44:40 +03:00
README.md Move data to top level 2019-06-11 17:20:52 +01:00
sync.go fix import paths for v2 release 2020-01-30 14:44:40 +03:00

Sync

Sync is a synchronization library for distributed systems.

Overview

Distributed systems by their very nature are decoupled and independent. In most cases they must honour 2 out of 3 letters of the CAP theorem e.g Availability and Partitional tolerance but sacrificing consistency. In the case of microservices we often offload this concern to an external database or eventing system. Go Sync provides a framework for synchronization which can be used in the application by the developer.

Getting Started

  • Leader - leadership election for group coordination
  • Lock - distributed locking for exclusive resource access
  • Task - distributed job execution
  • Time - provides synchronized time

Lock

The Lock interface provides distributed locking. Multiple instances attempting to lock the same id will block until available.

import "github.com/micro/go-micro/sync/lock/consul"

lock := consul.NewLock()

// acquire lock
err := lock.Acquire("id")
// handle err

// release lock
err = lock.Release("id")
// handle err

Leader

Leader provides leadership election. Useful where one node needs to coordinate some action.

import (
	"github.com/micro/go-micro/sync/leader"
	"github.com/micro/go-micro/sync/leader/consul"
)

l := consul.NewLeader(
	leader.Group("name"),
)

// elect leader
e, err := l.Elect("id")
// handle err


// operate while leader
revoked := e.Revoked()

for {
	select {
	case <-revoked:
		// re-elect
		e.Elect("id")
	default:
		// leader operation
	}
}

// resign leadership
e.Resign() 

Task

Task provides distributed job execution. It's a simple way to distribute work across a coordinated pool of workers.

import (
	"github.com/micro/go-micro/sync/task"
	"github.com/micro/go-micro/sync/task/local"
)

t := local.NewTask(
	task.WithPool(10),
)

err := t.Run(task.Command{
	Name: "atask",
	Func: func() error {
		// exec some work
		return nil
	},
})

if err != nil {
	// do something
}

Time

Time provides synchronized time. Local machines may have clock skew and time cannot be guaranteed to be the same everywhere. Synchronized Time allows you to decide how time is defined for your applications.

import (
	"github.com/micro/go-micro/sync/time/ntp"
)


t := ntp.NewTime()
time, err := t.Now()

TODO

  • Event package - strongly consistent event stream e.g kafka