micro/README.md

386 lines
12 KiB
Markdown
Raw Normal View History

2016-06-20 04:54:06 +03:00
# Go Micro [![License](https://img.shields.io/:license-apache-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![GoDoc](https://godoc.org/github.com/micro/go-micro?status.svg)](https://godoc.org/github.com/micro/go-micro) [![Travis CI](https://api.travis-ci.org/micro/go-micro.svg?branch=master)](https://travis-ci.org/micro/go-micro) [![Go Report Card](https://goreportcard.com/badge/micro/go-micro)](https://goreportcard.com/report/github.com/micro/go-micro)
2015-01-14 02:31:27 +03:00
2016-12-25 23:57:27 +03:00
Go Micro is a pluggable RPC framework for **microservices**. It is part of the [Micro](https://github.com/micro/micro) toolkit.
2015-11-29 23:46:31 +03:00
2016-06-20 04:52:38 +03:00
The **Micro** philosophy is sane defaults with a pluggable architecture. We provide defaults to get you started quickly but everything can be easily swapped out. It comes with built in support for {json,proto}-rpc encoding, consul or multicast dns for service discovery, http for communication and random hashed client side load balancing.
Everything in go-micro is **pluggable**. You can find and contribute to plugins at [github.com/micro/go-plugins](https://github.com/micro/go-plugins).
2015-01-14 11:38:39 +03:00
2017-07-16 16:58:31 +03:00
Follow us on [Twitter](https://twitter.com/microhq) or join the [Slack](http://slack.micro.mu/) community.
2016-12-25 23:53:08 +03:00
2016-12-07 19:54:19 +03:00
## Features
2016-07-03 04:49:13 +03:00
2017-06-15 21:57:27 +03:00
Go Micro abstracts away the details of distributed systems. Here are the main features.
2016-07-03 04:49:13 +03:00
2017-10-03 13:14:39 +03:00
- **Service Discovery** - Automatic registration and name resolution with service discovery
- **Load Balancing** - Smart client side load balancing of services built on discovery
- **Synchronous Comms** - RPC based communication with support for bidirectional streaming
- **Asynchronous Comms** - PubSub interface built in for event driven architectures
- **Message Encoding** - Dynamic encoding based on content-type with protobuf and json out of the box
- **Service Interface** - All features are packaged in a simple high level interface for developing microservices
2016-07-03 04:49:13 +03:00
2017-06-01 14:26:57 +03:00
Go Micro supports both the Service and Function programming models. Read on to learn more.
2017-02-10 15:36:42 +03:00
## Docs
For more detailed information on the architecture, installation and use of go-micro checkout the [docs](https://micro.mu/docs).
2016-07-03 04:49:13 +03:00
## Learn By Example
2017-10-03 13:03:46 +03:00
An example service can be found in [**examples/service**](https://github.com/micro/examples/tree/master/service) and function in [**examples/function**](https://github.com/micro/examples/tree/master/function).
2015-01-14 11:38:39 +03:00
2017-10-03 13:03:46 +03:00
The [**examples**](https://github.com/micro/examples) directory contains examples for using things such as middleware/wrappers, selector filters, pub/sub, grpc, plugins and much more. For the complete greeter example look at [**examples/greeter**](https://github.com/micro/examples/tree/master/greeter). Other examples can be found throughout the GitHub repository.
Watch the [Golang UK Conf 2016](https://www.youtube.com/watch?v=xspaDovwk34) video for a high level overview.
2016-04-13 13:30:17 +03:00
2015-01-14 13:50:43 +03:00
## Getting Started
2016-02-06 16:32:01 +03:00
This is a quick getting started guide with the greeter service example.
2017-10-03 13:03:46 +03:00
### Prereq: Service Discovery
Service discovery is required to resolve services to their addresses.
2016-02-06 16:32:01 +03:00
2017-10-03 13:03:46 +03:00
The default discovery plugin is consul. Discovery is however pluggable so you can use
etcd, kubernetes, zookeeper, etc. Plugins are in [micro/go-plugins](https://github.com/micro/go-plugins).
2016-02-06 16:32:01 +03:00
2017-01-22 17:12:53 +03:00
### Multicast DNS
2016-06-19 01:19:52 +03:00
2017-10-03 13:05:54 +03:00
[Multicast DNS](https://en.wikipedia.org/wiki/Multicast_DNS) is a built in service discovery plugin for a zero dependency configuration.
2016-02-06 16:32:01 +03:00
2017-10-03 13:03:46 +03:00
Pass `--registry=mdns` to any command or the enviroment variable MICRO_REGISTRY=mdns
2016-11-19 17:31:07 +03:00
```
2017-10-03 13:03:46 +03:00
go run main.go --registry=mdns
2016-11-19 17:31:07 +03:00
```
2017-01-22 17:12:53 +03:00
### Consul
2016-11-19 17:31:07 +03:00
2017-10-03 13:05:54 +03:00
Here's a quick start for the default service discovery system consul.
2016-11-19 17:31:07 +03:00
2017-10-03 13:19:03 +03:00
On Mac OS
2016-11-19 17:31:07 +03:00
```
brew install consul
2017-01-22 17:12:53 +03:00
consul agent -dev
2016-11-19 17:31:07 +03:00
```
2017-01-22 17:12:53 +03:00
[Further installation instructions](https://www.consul.io/intro/getting-started/install.html)
2015-01-14 13:50:43 +03:00
### Run Service
2017-01-22 17:16:16 +03:00
2015-01-14 11:38:39 +03:00
```
2017-10-03 13:03:46 +03:00
go get github.com/micro/examples/service && service
```
Output
```
2016-03-14 14:01:10 +03:00
2016/03/14 10:59:14 Listening on [::]:50137
2016/03/14 10:59:14 Broker Listening on [::]:50138
2016/03/14 10:59:14 Registering node: greeter-ca62b017-e9d3-11e5-9bbb-68a86d0d36b6
2015-01-14 11:38:39 +03:00
```
2017-01-22 17:16:16 +03:00
### Call Service
2015-01-14 11:38:39 +03:00
```
2017-10-03 13:03:46 +03:00
service --run_client
```
Output
```
2016-01-02 03:38:57 +03:00
Hello John
2015-01-14 11:38:39 +03:00
```
2015-01-14 13:50:43 +03:00
## Writing a service
2017-10-03 13:03:46 +03:00
This is a simple greeter RPC service example
Find this example at [examples/service](https://github.com/micro/examples/tree/master/service).
2016-12-07 19:54:19 +03:00
### Create service proto
2016-03-12 02:58:55 +03:00
2017-10-03 13:03:46 +03:00
One of the key requirements of microservices is strongly defined interfaces. Micro uses protobuf to achieve this.
2016-03-12 02:58:55 +03:00
2017-10-03 13:03:46 +03:00
Here we define the Greeter handler with the method Hello. It takes a HelloRequest and HelloResponse both with one string arguments.
2015-01-14 13:50:43 +03:00
2016-01-04 02:36:14 +03:00
```proto
2015-05-25 20:16:42 +03:00
syntax = "proto3";
2015-01-14 13:50:43 +03:00
2016-01-02 03:38:57 +03:00
service Greeter {
rpc Hello(HelloRequest) returns (HelloResponse) {}
2015-01-14 13:50:43 +03:00
}
2016-01-02 03:38:57 +03:00
message HelloRequest {
string name = 1;
2015-01-14 13:50:43 +03:00
}
2016-01-02 03:38:57 +03:00
message HelloResponse {
string greeting = 2;
}
```
2016-12-07 19:54:19 +03:00
### Install protobuf
2016-01-02 03:38:57 +03:00
2017-10-03 13:03:46 +03:00
Install [protobuf](https://developers.google.com/protocol-buffers/)
Now install the micro fork of protoc-gen-go. The protobuf compiler for Go.
2016-01-02 03:38:57 +03:00
```shell
go get github.com/micro/protobuf/{proto,protoc-gen-go}
2015-01-14 13:50:43 +03:00
```
2017-10-03 13:03:46 +03:00
### Generate the proto
2016-03-12 03:04:33 +03:00
2017-10-03 13:03:46 +03:00
After writing the proto definition we must compile it using protoc with the micro plugin.
2016-03-12 03:04:33 +03:00
2016-11-20 18:02:05 +03:00
```shell
protoc -I$GOPATH/src --go_out=plugins=micro:$GOPATH/src \
2017-01-05 18:09:27 +03:00
$GOPATH/src/github.com/micro/examples/service/proto/greeter.proto
2016-03-12 03:04:33 +03:00
```
2015-01-14 13:50:43 +03:00
2017-10-03 13:03:46 +03:00
### Write the service
Below is the code for the greeter service.
2016-03-12 03:04:33 +03:00
2017-10-03 13:03:46 +03:00
It does the following:
2016-03-12 03:04:33 +03:00
2017-10-03 13:03:46 +03:00
1. Implements the interface defined for the Greeter handler
2. Initialises a micro.Service
3. Registers the Greeter handler
4. Runs the service
2015-01-14 13:50:43 +03:00
2015-02-02 21:53:16 +03:00
```go
2016-01-02 03:38:57 +03:00
package main
2015-01-14 13:50:43 +03:00
import (
2016-01-02 03:38:57 +03:00
"fmt"
2015-01-14 13:50:43 +03:00
2016-01-02 03:38:57 +03:00
micro "github.com/micro/go-micro"
2017-01-05 18:09:27 +03:00
proto "github.com/micro/examples/service/proto"
2015-11-08 14:12:09 +03:00
"golang.org/x/net/context"
2015-01-14 13:50:43 +03:00
)
2016-01-02 03:38:57 +03:00
type Greeter struct{}
2015-01-14 13:50:43 +03:00
2016-01-02 03:38:57 +03:00
func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error {
rsp.Greeting = "Hello " + req.Name
2015-11-08 14:12:09 +03:00
return nil
2015-01-14 13:50:43 +03:00
}
2016-01-02 03:38:57 +03:00
func main() {
// Create a new service. Optionally include some options here.
service := micro.NewService(
micro.Name("greeter"),
micro.Version("latest"),
)
2017-07-14 10:24:46 +03:00
// Init will parse the command line flags.
2016-01-02 03:38:57 +03:00
service.Init()
// Register handler
proto.RegisterGreeterHandler(service.Server(), new(Greeter))
// Run the server
if err := service.Run(); err != nil {
fmt.Println(err)
}
}
2015-01-14 13:50:43 +03:00
```
2016-01-02 03:38:57 +03:00
### Run service
```
2016-03-14 14:01:10 +03:00
go run examples/service/main.go
2017-10-03 13:03:46 +03:00
```
Output
```
2016-03-14 14:01:10 +03:00
2016/03/14 10:59:14 Listening on [::]:50137
2016/03/14 10:59:14 Broker Listening on [::]:50138
2016/03/14 10:59:14 Registering node: greeter-ca62b017-e9d3-11e5-9bbb-68a86d0d36b6
2016-01-02 03:38:57 +03:00
```
### Define a client
2017-10-03 13:03:46 +03:00
Below is the client code to query the greeter service.
2016-03-12 03:04:33 +03:00
2017-10-03 13:03:46 +03:00
The generated proto includes a greeter client to reduce boilerplate code.
2015-01-14 13:50:43 +03:00
2015-02-02 21:53:16 +03:00
```go
2015-01-14 13:50:43 +03:00
package main
import (
2016-01-02 03:38:57 +03:00
"fmt"
micro "github.com/micro/go-micro"
2017-01-05 18:09:27 +03:00
proto "github.com/micro/examples/service/proto"
2016-01-02 03:38:57 +03:00
"golang.org/x/net/context"
2015-01-14 13:50:43 +03:00
)
2015-05-25 20:16:42 +03:00
2016-01-02 03:38:57 +03:00
func main() {
// Create a new service. Optionally include some options here.
service := micro.NewService(micro.Name("greeter.client"))
2015-01-14 13:50:43 +03:00
2016-01-02 03:38:57 +03:00
// Create new greeter client
greeter := proto.NewGreeterClient("greeter", service.Client())
2015-01-14 13:50:43 +03:00
2016-01-02 03:38:57 +03:00
// Call the greeter
rsp, err := greeter.Hello(context.TODO(), &proto.HelloRequest{Name: "John"})
if err != nil {
fmt.Println(err)
2015-01-14 13:50:43 +03:00
}
2016-01-02 03:38:57 +03:00
// Print response
fmt.Println(rsp.Greeting)
2015-01-14 13:50:43 +03:00
}
```
2016-01-02 03:38:57 +03:00
### Run the client
```shell
go run client.go
2017-10-03 13:03:46 +03:00
```
Output
```
2016-01-02 03:38:57 +03:00
Hello John
2015-01-14 13:50:43 +03:00
```
2016-03-12 03:07:14 +03:00
2017-06-01 14:26:57 +03:00
## Writing a Function
2017-10-03 13:03:46 +03:00
Go Micro includes the Function programming model.
A Function is a one time executing Service which exits after completing a request.
2017-06-01 14:26:57 +03:00
### Defining a Function
```go
package main
import (
proto "github.com/micro/examples/function/proto"
"github.com/micro/go-micro"
"golang.org/x/net/context"
)
type Greeter struct{}
func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error {
rsp.Greeting = "Hello " + req.Name
return nil
}
func main() {
// create a new function
fnc := micro.NewFunction(
micro.Name("go.micro.fnc.greeter"),
)
// init the command line
fnc.Init()
// register a handler
fnc.Handle(new(Greeter))
// run the function
fnc.Run()
}
```
It's that simple.
2017-10-03 13:03:46 +03:00
## How it works
2016-10-04 19:33:58 +03:00
<p align="center">
<img src="go-micro.png" />
</p>
2017-10-03 13:03:46 +03:00
Go Micro is a framework that addresses the fundamental requirements for writing microservices.
2016-10-04 19:33:58 +03:00
Let's dig into the core components.
### Registry
The registry provides a service discovery mechanism to resolve names to addresses. It can be backed by consul, etcd, zookeeper, dns, gossip, etc.
Services should register using the registry on startup and deregister on shutdown. Services can optionally provide an expiry TTL and reregister
on an interval to ensure liveness and that the service is cleaned up if it dies.
### Selector
The selector is a load balancing abstraction which builds on the registry. It allows services to be "filtered" using filter functions and "selected"
using a choice of algorithms such as random, roundrobin, leastconn, etc. The selector is leveraged by the Client when making requests. The client
will use the selector rather than the registry as it provides that built in mechanism of load balancing.
### Transport
The transport is the interface for synchronous request/response communication between services. It's akin to the golang net package but provides
a higher level abstraction which allows us to switch out communication mechanisms e.g http, rabbitmq, websockets, NATS. The transport also
supports bidirectional streaming. This is powerful for client side push to the server.
### Broker
The broker provides an interface to a message broker for asynchronous pub/sub communication. This is one of the fundamental requirements of an event
driven architecture and microservices. By default we use an inbox style point to point HTTP system to minimise the number of dependencies required
to get started. However there are many message broker implementations available in go-plugins e.g RabbitMQ, NATS, NSQ, Google Cloud Pub Sub.
### Codec
The codec is used for encoding and decoding messages before transporting them across the wire. This could be json, protobuf, bson, msgpack, etc.
Where this differs from most other codecs is that we actually support the RPC format here as well. So we have JSON-RPC, PROTO-RPC, BSON-RPC, etc.
It separates encoding from the client/server and provides a powerful method for integrating other systems such as gRPC, Vanadium, etc.
### Server
The server is the building block for writing a service. Here you can name your service, register request handlers, add middeware, etc. The service
builds on the above packages to provide a unified interface for serving requests. The built in server is an RPC system. In the future there maybe
other implementations. The server also allows you to define multiple codecs to serve different encoded messages.
### Client
The client provides an interface to make requests to services. Again like the server, it builds on the other packages to provide a unified interface
for finding services by name using the registry, load balancing using the selector, making synchronous requests with the transport and asynchronous
messaging using the broker.
The above components are combined at the top-level of micro as a **Service**.
2016-12-07 19:54:19 +03:00
## Plugins
2016-10-04 19:34:45 +03:00
By default go-micro only provides a few implementation of each interface at the core but it's completely pluggable. There's already dozens of plugins which are available at [github.com/micro/go-plugins](https://github.com/micro/go-plugins). Contributions are welcome!
2017-01-04 17:54:31 +03:00
### Build with plugins
If you want to integrate plugins simply link them in a separate file and rebuild
Create a plugins.go file
2017-02-15 18:17:10 +03:00
2017-01-04 17:54:31 +03:00
```go
import (
// etcd v3 registry
_ "github.com/micro/go-plugins/registry/etcdv3"
// nats transport
_ "github.com/micro/go-plugins/transport/nats"
// kafka broker
_ "github.com/micro/go-plugins/broker/kafka"
2017-01-04 19:43:43 +03:00
)
2017-01-04 17:54:31 +03:00
```
Build binary
2017-02-15 18:17:10 +03:00
2017-01-04 17:54:31 +03:00
```shell
// For local use
go build -i -o service ./main.go ./plugins.go
```
Flag usage of plugins
```shell
service --registry=etcdv3 --transport=nats --broker=kafka
```
2017-02-07 21:33:50 +03:00
## Other Languages
2017-02-15 17:31:20 +03:00
Check out [ja-micro](https://github.com/Sixt/ja-micro) to write services in Java
2017-02-07 21:33:50 +03:00
2016-04-26 14:18:18 +03:00
## Sponsors
2016-12-25 23:53:08 +03:00
Open source development of Micro is sponsored by Sixt
2016-04-26 14:18:18 +03:00
2017-02-11 17:26:47 +03:00
<a href="https://micro.mu/blog/2016/04/25/announcing-sixt-sponsorship.html"><img src="https://micro.mu/sixt_logo.png" width=150px height="auto" /></a>