unistack-org/cloudinit
4
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #330 from crawford/etcd

Browse Source 
config: add support for etcd2
This commit is contained in:
Alex Crawford 2015-04-07 14:40:47 -07:00
commit 66ec7d805c
8 changed files with 163 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

66
Documentation/cloud-config.md
View File

@ -39,7 +39,7 @@ CoreOS tries to conform to each platform's native method to provide user data. E
### coreos ### coreos
#### etcd #### etcd (deprecated. see etcd2)
The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file. The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document... If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
@ -49,15 +49,15 @@ If the platform environment supports the templating feature of coreos-cloudinit
coreos: coreos:
etcd: etcd:
name: node001 name: node001
# generate a new token for each unique cluster from https://discovery.etcd.io/new # generate a new token for each unique cluster from https://discovery.etcd.io/new
discovery: https://discovery.etcd.io/<token> discovery: https://discovery.etcd.io/<token>
# multi-region and multi-cloud deployments need to use $public_ipv4 # multi-region and multi-cloud deployments need to use $public_ipv4
addr: $public_ipv4:4001 addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001 peer-addr: $private_ipv4:7001
``` ```
...will generate a systemd unit drop-in like this: ...will generate a systemd unit drop-in for etcd.service with the following contents:
```yaml ```yaml
[Service] [Service]
@ -71,11 +71,49 @@ For more information about the available configuration parameters, see the [etcd
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._ _Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md [etcd-config]: https://github.com/coreos/etcd/blob/9fa3bea5a22265151f0d5063ce38a79c5b5d0271/Documentation/configuration.md
#### etcd2
The `coreos.etcd2.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
```yaml
#cloud-config
coreos:
etcd2:
# generate a new token for each unique cluster from https://discovery.etcd.io/new
discovery: https://discovery.etcd.io/<token>
# multi-region and multi-cloud deployments need to use $public_ipv4
advertise-client-urls: http://$public_ipv4:2379
initial-advertise-peer-urls: http://$private_ipv4:2380
# listen on both the official ports and the legacy ports
# legacy ports can be omitted if your application doesn't depend on them
listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
```
...will generate a systemd unit drop-in for etcd2.service with the following contents:
```yaml
[Service]
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
Environment="ETCD_ADVERTISE_CLIENT_URLS=http://203.0.113.29:2379"
Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=http://192.0.2.13:2380"
Environment="ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001"
Environment="ETCD_LISTEN_PEERS_URLS=http://192.0.2.13:2380,http://192.0.2.13:7001"
```
For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/86e616c6e974828fc9119c1eb0f6439577a9ce0b/Documentation/configuration.md
#### fleet #### fleet
The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document... The `coreos.fleet.*` parameters work very similarly to `coreos.etcd2.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
```yaml ```yaml
#cloud-config #cloud-config
@ -100,7 +138,7 @@ For more information on fleet configuration, see the [fleet documentation][fleet
#### flannel #### flannel
The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd.*` The `coreos.flannel.*` parameters also work very similarly to `coreos.etcd2.*`
and `coreos.fleet.*`. They can be used to set environment variables for and `coreos.fleet.*`. They can be used to set environment variables for
flanneld. For example, the following cloud-config... flanneld. For example, the following cloud-config...
@ -125,7 +163,7 @@ List of flannel configuration parameters:
- **etcd_cafile**: Path to CA file used for TLS communication with etcd - **etcd_cafile**: Path to CA file used for TLS communication with etcd
- **etcd_certfile**: Path to certificate file used for TLS communication with etcd - **etcd_certfile**: Path to certificate file used for TLS communication with etcd
- **etcd_keyfile**: Path to private key file used for TLS communication with etcd - **etcd_keyfile**: Path to private key file used for TLS communication with etcd
- **etcd_prefix**: Etcd prefix path to be used for flannel keys - **etcd_prefix**: etcd prefix path to be used for flannel keys
- **ip_masq**: Install IP masquerade rules for traffic outside of flannel subnet - **ip_masq**: Install IP masquerade rules for traffic outside of flannel subnet
- **subnet_file**: Path to flannel subnet file to write out - **subnet_file**: Path to flannel subnet file to write out
- **interface**: Interface (name or IP) that should be used for inter-host communication - **interface**: Interface (name or IP) that should be used for inter-host communication
@ -242,14 +280,14 @@ coreos:
Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"' Environment=DOCKER_OPTS='--insecure-registry="10.0.1.0/24"'
``` ```
Start the built-in `etcd` and `fleet` services: Start the built-in `etcd2` and `fleet` services:
```yaml ```yaml
#cloud-config #cloud-config
coreos: coreos:
units: units:
- name: etcd.service - name: etcd2.service
command: start command: start
- name: fleet.service - name: fleet.service
command: start command: start

1
config/config.go
View File

@ -37,6 +37,7 @@ type CloudConfig struct {
type CoreOS struct { type CoreOS struct {
Etcd Etcd `yaml:"etcd"` Etcd Etcd `yaml:"etcd"`
Etcd2 Etcd2 `yaml:"etcd2"`
Flannel Flannel `yaml:"flannel"` Flannel Flannel `yaml:"flannel"`
Fleet Fleet `yaml:"fleet"` Fleet Fleet `yaml:"fleet"`
Locksmith Locksmith `yaml:"locksmith"` Locksmith Locksmith `yaml:"locksmith"`

32
config/etcd.go
View File

@ -16,7 +16,7 @@ package config
type Etcd struct { type Etcd struct {
Addr string `yaml:"addr" env:"ETCD_ADDR"` Addr string `yaml:"addr" env:"ETCD_ADDR"`
AdvertiseClientURLs string `yaml:"advertise_client_urls" env:"ETCD_ADVERTISE_CLIENT_URLS"` AdvertiseClientURLs string `yaml:"advertise_client_urls" env:"ETCD_ADVERTISE_CLIENT_URLS" deprecated:"etcd2 options no longer work for etcd"`
BindAddr string `yaml:"bind_addr" env:"ETCD_BIND_ADDR"` BindAddr string `yaml:"bind_addr" env:"ETCD_BIND_ADDR"`
CAFile string `yaml:"ca_file" env:"ETCD_CA_FILE"` CAFile string `yaml:"ca_file" env:"ETCD_CA_FILE"`
CertFile string `yaml:"cert_file" env:"ETCD_CERT_FILE"` CertFile string `yaml:"cert_file" env:"ETCD_CERT_FILE"`
@ -26,26 +26,26 @@ type Etcd struct {
CorsOrigins string `yaml:"cors" env:"ETCD_CORS"` CorsOrigins string `yaml:"cors" env:"ETCD_CORS"`
DataDir string `yaml:"data_dir" env:"ETCD_DATA_DIR"` DataDir string `yaml:"data_dir" env:"ETCD_DATA_DIR"`
Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"` Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"`
DiscoveryFallback string `yaml:"discovery_fallback" env:"ETCD_DISCOVERY_FALLBACK"` DiscoveryFallback string `yaml:"discovery_fallback" env:"ETCD_DISCOVERY_FALLBACK" deprecated:"etcd2 options no longer work for etcd"`
DiscoverySRV string `yaml:"discovery_srv" env:"ETCD_DISCOVERY_SRV"` DiscoverySRV string `yaml:"discovery_srv" env:"ETCD_DISCOVERY_SRV" deprecated:"etcd2 options no longer work for etcd"`
DiscoveryProxy string `yaml:"discovery_proxy" env:"ETCD_DISCOVERY_PROXY"` DiscoveryProxy string `yaml:"discovery_proxy" env:"ETCD_DISCOVERY_PROXY" deprecated:"etcd2 options no longer work for etcd"`
ElectionTimeout int `yaml:"election_timeout" env:"ETCD_ELECTION_TIMEOUT"` ElectionTimeout int `yaml:"election_timeout" env:"ETCD_ELECTION_TIMEOUT" deprecated:"etcd2 options no longer work for etcd"`
ForceNewCluster bool `yaml:"force_new_cluster" env:"ETCD_FORCE_NEW_CLUSTER"` ForceNewCluster bool `yaml:"force_new_cluster" env:"ETCD_FORCE_NEW_CLUSTER" deprecated:"etcd2 options no longer work for etcd"`
GraphiteHost string `yaml:"graphite_host" env:"ETCD_GRAPHITE_HOST"` GraphiteHost string `yaml:"graphite_host" env:"ETCD_GRAPHITE_HOST"`
HeartbeatInterval int `yaml:"heartbeat_interval" env:"ETCD_HEARTBEAT_INTERVAL"` HeartbeatInterval int `yaml:"heartbeat_interval" env:"ETCD_HEARTBEAT_INTERVAL" deprecated:"etcd2 options no longer work for etcd"`
HTTPReadTimeout float64 `yaml:"http_read_timeout" env:"ETCD_HTTP_READ_TIMEOUT"` HTTPReadTimeout float64 `yaml:"http_read_timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
HTTPWriteTimeout float64 `yaml:"http_write_timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"` HTTPWriteTimeout float64 `yaml:"http_write_timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
InitialAdvertisePeerURLs string `yaml:"initial_advertise_peer_urls" env:"ETCD_INITIAL_ADVERTISE_PEER_URLS"` InitialAdvertisePeerURLs string `yaml:"initial_advertise_peer_urls" env:"ETCD_INITIAL_ADVERTISE_PEER_URLS" deprecated:"etcd2 options no longer work for etcd"`
InitialCluster string `yaml:"initial_cluster" env:"ETCD_INITIAL_CLUSTER"` InitialCluster string `yaml:"initial_cluster" env:"ETCD_INITIAL_CLUSTER" deprecated:"etcd2 options no longer work for etcd"`
InitialClusterState string `yaml:"initial_cluster_state" env:"ETCD_INITIAL_CLUSTER_STATE"` InitialClusterState string `yaml:"initial_cluster_state" env:"ETCD_INITIAL_CLUSTER_STATE" deprecated:"etcd2 options no longer work for etcd"`
InitialClusterToken string `yaml:"initial_cluster_token" env:"ETCD_INITIAL_CLUSTER_TOKEN"` InitialClusterToken string `yaml:"initial_cluster_token" env:"ETCD_INITIAL_CLUSTER_TOKEN" deprecated:"etcd2 options no longer work for etcd"`
KeyFile string `yaml:"key_file" env:"ETCD_KEY_FILE"` KeyFile string `yaml:"key_file" env:"ETCD_KEY_FILE"`
ListenClientURLs string `yaml:"listen_client_urls" env:"ETCD_LISTEN_CLIENT_URLS"` ListenClientURLs string `yaml:"listen_client_urls" env:"ETCD_LISTEN_CLIENT_URLS" deprecated:"etcd2 options no longer work for etcd"`
ListenPeerURLs string `yaml:"listen_peer_urls" env:"ETCD_LISTEN_PEER_URLS"` ListenPeerURLs string `yaml:"listen_peer_urls" env:"ETCD_LISTEN_PEER_URLS" deprecated:"etcd2 options no longer work for etcd"`
MaxResultBuffer int `yaml:"max_result_buffer" env:"ETCD_MAX_RESULT_BUFFER"` MaxResultBuffer int `yaml:"max_result_buffer" env:"ETCD_MAX_RESULT_BUFFER"`
MaxRetryAttempts int `yaml:"max_retry_attempts" env:"ETCD_MAX_RETRY_ATTEMPTS"` MaxRetryAttempts int `yaml:"max_retry_attempts" env:"ETCD_MAX_RETRY_ATTEMPTS"`
MaxSnapshots int `yaml:"max_snapshots" env:"ETCD_MAX_SNAPSHOTS"` MaxSnapshots int `yaml:"max_snapshots" env:"ETCD_MAX_SNAPSHOTS" deprecated:"etcd2 options no longer work for etcd"`
MaxWALs int `yaml:"max_wals" env:"ETCD_MAX_WALS"` MaxWALs int `yaml:"max_wals" env:"ETCD_MAX_WALS" deprecated:"etcd2 options no longer work for etcd"`
Name string `yaml:"name" env:"ETCD_NAME"` Name string `yaml:"name" env:"ETCD_NAME"`
PeerAddr string `yaml:"peer_addr" env:"ETCD_PEER_ADDR"` PeerAddr string `yaml:"peer_addr" env:"ETCD_PEER_ADDR"`
PeerBindAddr string `yaml:"peer_bind_addr" env:"ETCD_PEER_BIND_ADDR"` PeerBindAddr string `yaml:"peer_bind_addr" env:"ETCD_PEER_BIND_ADDR"`
@ -56,7 +56,7 @@ type Etcd struct {
PeerKeyFile string `yaml:"peer_key_file" env:"ETCD_PEER_KEY_FILE"` PeerKeyFile string `yaml:"peer_key_file" env:"ETCD_PEER_KEY_FILE"`
Peers string `yaml:"peers" env:"ETCD_PEERS"` Peers string `yaml:"peers" env:"ETCD_PEERS"`
PeersFile string `yaml:"peers_file" env:"ETCD_PEERS_FILE"` PeersFile string `yaml:"peers_file" env:"ETCD_PEERS_FILE"`
Proxy string `yaml:"proxy" env:"ETCD_PROXY"` Proxy string `yaml:"proxy" env:"ETCD_PROXY" deprecated:"etcd2 options no longer work for etcd"`
RetryInterval float64 `yaml:"retry_interval" env:"ETCD_RETRY_INTERVAL"` RetryInterval float64 `yaml:"retry_interval" env:"ETCD_RETRY_INTERVAL"`
Snapshot bool `yaml:"snapshot" env:"ETCD_SNAPSHOT"` Snapshot bool `yaml:"snapshot" env:"ETCD_SNAPSHOT"`
SnapshotCount int `yaml:"snapshot_count" env:"ETCD_SNAPSHOTCOUNT"` SnapshotCount int `yaml:"snapshot_count" env:"ETCD_SNAPSHOTCOUNT"`

44
config/etcd2.go Normal file
View File

@ -0,0 +1,44 @@
// Copyright 2015 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package config
type Etcd2 struct {
AdvertiseClientURLs string `yaml:"advertise_client_urls" env:"ETCD_ADVERTISE_CLIENT_URLS"`
CAFile string `yaml:"ca_file" env:"ETCD_CA_FILE"`
CertFile string `yaml:"cert_file" env:"ETCD_CERT_FILE"`
CorsOrigins string `yaml:"cors" env:"ETCD_CORS"`
DataDir string `yaml:"data_dir" env:"ETCD_DATA_DIR"`
Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"`
DiscoveryFallback string `yaml:"discovery_fallback" env:"ETCD_DISCOVERY_FALLBACK"`
DiscoverySRV string `yaml:"discovery_srv" env:"ETCD_DISCOVERY_SRV"`
DiscoveryProxy string `yaml:"discovery_proxy" env:"ETCD_DISCOVERY_PROXY"`
ElectionTimeout int `yaml:"election_timeout" env:"ETCD_ELECTION_TIMEOUT"`
HeartbeatInterval int `yaml:"heartbeat_interval" env:"ETCD_HEARTBEAT_INTERVAL"`
InitialAdvertisePeerURLs string `yaml:"initial_advertise_peer_urls" env:"ETCD_INITIAL_ADVERTISE_PEER_URLS"`
InitialCluster string `yaml:"initial_cluster" env:"ETCD_INITIAL_CLUSTER"`
InitialClusterState string `yaml:"initial_cluster_state" env:"ETCD_INITIAL_CLUSTER_STATE"`
InitialClusterToken string `yaml:"initial_cluster_token" env:"ETCD_INITIAL_CLUSTER_TOKEN"`
KeyFile string `yaml:"key_file" env:"ETCD_KEY_FILE"`
ListenClientURLs string `yaml:"listen_client_urls" env:"ETCD_LISTEN_CLIENT_URLS"`
ListenPeerURLs string `yaml:"listen_peer_urls" env:"ETCD_LISTEN_PEER_URLS"`
MaxSnapshots int `yaml:"max_snapshots" env:"ETCD_MAX_SNAPSHOTS"`
MaxWALs int `yaml:"max_wals" env:"ETCD_MAX_WALS"`
Name string `yaml:"name" env:"ETCD_NAME"`
PeerCAFile string `yaml:"peer_ca_file" env:"ETCD_PEER_CA_FILE"`
PeerCertFile string `yaml:"peer_cert_file" env:"ETCD_PEER_CERT_FILE"`
PeerKeyFile string `yaml:"peer_key_file" env:"ETCD_PEER_KEY_FILE"`
Proxy string `yaml:"proxy" env:"ETCD_PROXY"`
SnapshotCount int `yaml:"snapshot_count" env:"ETCD_SNAPSHOTCOUNT"`
}

3
config/validate/rules.go
View File

@ -82,6 +82,9 @@ func checkNodeStructure(n, g node, r *Report) {
case reflect.Struct: case reflect.Struct:
for _, cn := range n.children { for _, cn := range n.children {
if cg := g.Child(cn.name); cg.IsValid() { if cg := g.Child(cn.name); cg.IsValid() {
if msg := cg.field.Tag.Get("deprecated"); msg != "" {
r.Warning(cn.line, fmt.Sprintf("deprecated key %q (%s)", cn.name, msg))
}
checkNodeStructure(cn, cg, r) checkNodeStructure(cn, cg, r)
} else { } else {
r.Warning(cn.line, fmt.Sprintf("unrecognized key %q", cn.name)) r.Warning(cn.line, fmt.Sprintf("unrecognized key %q", cn.name))

9
config/validate/rules_test.go
View File

@ -119,6 +119,15 @@ func TestCheckStructure(t *testing.T) {
config: "coreos:\n etcd:\n discovery: good", config: "coreos:\n etcd:\n discovery: good",
}, },
// Test for deprecated keys
{
config: "coreos:\n etcd:\n addr: hi",
},
{
config: "coreos:\n etcd:\n proxy: hi",
entries: []Entry{{entryWarning, "deprecated key \"proxy\" (etcd2 options no longer work for etcd)", 3}},
},
// Test for error on list of nodes // Test for error on list of nodes
{ {
config: "coreos:\n units:\n - hello\n - goodbye", config: "coreos:\n units:\n - hello\n - goodbye",

1
initialize/config.go
View File

@ -135,6 +135,7 @@ func Apply(cfg config.CloudConfig, ifaces []network.InterfaceGenerator, env *Env
for _, ccu := range []CloudConfigUnit{ for _, ccu := range []CloudConfigUnit{
system.Etcd{Etcd: cfg.CoreOS.Etcd}, system.Etcd{Etcd: cfg.CoreOS.Etcd},
system.Etcd2{Etcd2: cfg.CoreOS.Etcd2},
system.Fleet{Fleet: cfg.CoreOS.Fleet}, system.Fleet{Fleet: cfg.CoreOS.Fleet},
system.Locksmith{Locksmith: cfg.CoreOS.Locksmith}, system.Locksmith{Locksmith: cfg.CoreOS.Locksmith},
system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig}, system.Update{Update: cfg.CoreOS.Update, ReadConfig: system.DefaultReadConfig},

37
system/etcd2.go Normal file
View File

@ -0,0 +1,37 @@
// Copyright 2015 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package system
import (
"github.com/coreos/coreos-cloudinit/config"
)
// Etcd2 is a top-level structure which embeds its underlying configuration,
// config.Etcd2, and provides the system-specific Unit().
type Etcd2 struct {
config.Etcd2
}
// Units creates a Unit file drop-in for etcd, using any configured options.
func (ee Etcd2) Units() []Unit {
return []Unit{{config.Unit{
Name: "etcd2.service",
Runtime: true,
DropIns: []config.UnitDropIn{{
Name: "20-cloudinit.conf",
Content: serviceContents(ee.Etcd2),
}},
}}}
}