Compare commits

..

322 Commits

Author SHA1 Message Date
Alex Crawford
0785840fe3 coreos-cloudinit: bump to 0.10.4 2014-09-24 19:25:34 -07:00
Alex Crawford
c10bfc2f56 Merge pull request #240 from epankala/euca4_compat_fix
AWS: Eucalyptus 4.x compatibility fix
2014-09-24 10:55:39 -07:00
Janne Paenkaelae
2f954dcdc2 AWS: Eucalyptus 4.x compatibility fix
For Eucalyptus 4.0.1 requesting metadata seem to work differently as with EC2.

In Euca:
> curl http://169.254.169.254/2009-04-04
<?xml version="1.0"?><Response><Errors><Error><Code>404 Not Found</Code><Message>unknown</Message></Error></Errors><RequestID>unknown</RequestID></Response>core@localhost ~ $

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

In AWS EC2
> curl http://169.254.169.254/2009-04-04
"" (zero bytes)

> curl http://169.254.169.254/2009-04-04/
dynamic
meta-data
user-data

As the isAvailable() function in metadata.go tests only for errorcode
it fails in Euca.
2014-09-24 20:33:29 +03:00
Alex Crawford
cdfc94f4e9 Merge pull request #234 from crawford/validate
config: explicitly specify fields and seperate config and application
2014-09-24 07:42:09 -07:00
Alex Crawford
18e2f98414 cloudconfig: refactor config
- Move CloudConfig into config package
- Add YAML tags to CloudConfig
2014-09-23 17:59:32 -07:00
Alex Crawford
4b472795c4 user: move User into config package
- Add YAML tags for the fields
2014-09-23 17:59:19 -07:00
Alex Crawford
85b8d804c8 file: refactor config
- Seperate the config from Permissions()
- Add YAML tags for the fields
2014-09-23 17:59:16 -07:00
Alex Crawford
1fbbaaec19 unit: refactor config
- Seperate the config from Destination()
- Add YAML tags for the fields
2014-09-23 17:58:32 -07:00
Alex Crawford
667dbd8fb7 update: refactor config
- Explicitly specify all of the valid options for Update
- Seperate the config from File() and Units()
- Add YAML tags for the fields
2014-09-23 17:57:43 -07:00
Alex Crawford
6730cb7227 oem: refactor the config
- Seperate the config from File()
- Add YAML tags for the fields
2014-09-23 16:08:23 -07:00
Alex Crawford
9454522033 fleet: refactor config
- Explicitly specify all of the valid options for fleet
- Seperate the config from Units()
- Add YAML tags for the fields
2014-09-23 16:07:53 -07:00
Alex Crawford
c255739a93 etcd: refactor config
- Explicitly specify all of the valid options for etcd
- Remove the default name generation (ETCD_NAME is set by its unit file now)
- Seperate the etcd config from Units()
- Remove support for DISCOVERY_URL
- Add YAML tags for the fields
2014-09-23 16:07:13 -07:00
Alex Crawford
2051cd3e1c Merge pull request #238 from crawford/docs
docs: fix documentation of coreos.units.command
2014-09-23 11:33:44 -07:00
Alex Crawford
b52cb3fea3 docs: fix documentation of coreos.units.command 2014-09-23 11:32:15 -07:00
Alex Crawford
da5f85b3fb coreos-cloudinit: bump to 0.10.3+git 2014-09-17 12:19:27 -07:00
Alex Crawford
9999178538 coreos-cloudinit: bump to 0.10.3 2014-09-17 12:19:13 -07:00
Alex Crawford
8f766e4666 Merge pull request #235 from crawford/routes
network: add support for CIDR addresses Debian routes
2014-09-17 12:18:16 -07:00
Alex Crawford
2d28d16c92 network: add support for CIDR addresses Debian routes
OnMetal is changing their template from:
`route add -net 1.2.3.0 netmask 255.255.255.0 gw 10.1.2.1 || true`
to:
`route add -net 1.2.3.0/24 gw 10.1.2.1 || true`
2014-09-16 17:36:34 -07:00
Alex Crawford
e9cd09dd7b coreos-cloudinit: bump to 0.10.2+git 2014-09-14 08:19:57 -07:00
Alex Crawford
8370b30aa2 coreos-cloudinit: bump to 0.10.2 2014-09-14 08:19:33 -07:00
Alex Crawford
3e015cc3a1 Merge pull request #233 from crawford/configdrive
configdrive: don't fail if no network config was provided
2014-09-14 08:18:14 -07:00
Alex Crawford
a0fe6d0884 configdrive: return an empty network config when filename is empty
Additionally, don't bother checking for a network config if it isn't going to
be processed.
2014-09-13 21:51:51 -07:00
Alex Crawford
585ce5fcd9 Revert "metadata: don't fail if no network config was provided"
This reverts commit c1f373e648.
2014-09-13 21:01:42 -07:00
Alex Crawford
72445796ca coreos-cloudinit: bump to 0.10.1+git 2014-09-12 16:48:15 -07:00
Alex Crawford
7342d91a85 coreos-cloudinit: bump to 0.10.1 2014-09-12 16:47:58 -07:00
Alex Crawford
db1bc51c98 Merge pull request #231 from crawford/netconf
metadata: don't fail if no network config was provided
2014-09-12 16:35:24 -07:00
Alex Crawford
c1f373e648 metadata: don't fail if no network config was provided 2014-09-12 16:29:27 -07:00
Alex Crawford
db49a16002 coreos-cloudinit: bump to 0.10.0+git 2014-09-11 17:37:05 -07:00
Alex Crawford
a4a6c281d9 coreos-cloudinit: bump to 0.10.0 2014-09-11 17:36:38 -07:00
Alex Crawford
17f8733121 Merge pull request #228 from crawford/sub
env: add support for escaping environment substitutions
2014-09-11 15:34:03 -07:00
Alex Crawford
7dec922618 env: add support for escaping environment substitutions 2014-09-11 15:30:33 -07:00
Alex Crawford
54d3ae27af Merge pull request #226 from crawford/oem
flags: add oem flag
2014-09-11 13:25:21 -07:00
Alex Crawford
ee2416af64 flags: move the flags into their own namespace 2014-09-11 12:00:17 -07:00
Alex Crawford
cda037f9a5 flags: add oem flag
The oem flag will allow each of the OEMs to specify one flag only, acting as a
shortcut to their specific configuration. This will allow us to update which
options each OEM uses when running cloudinit.
2014-09-11 12:00:17 -07:00
Alex Crawford
549806cf64 Merge pull request #227 from crawford/ipv6
metadata: add support for IPv6 variable substitution
2014-09-11 10:45:33 -07:00
Alex Crawford
56815a6756 metadata: add support for IPv6 variable substitution 2014-09-11 10:43:02 -07:00
Alex Crawford
24a6f7c49c Merge pull request #225 from crawford/exit
userdata: change handling of bad userdata
2014-09-10 19:16:12 -07:00
Alex Crawford
98484be434 userdata: change handling of bad userdata
Don't fail after encountering bad userdata. Continue processing the metadata
and then exit. This will allow people with bad userdata to actually log in and
see the error.
2014-09-10 17:50:23 -07:00
Jonathan Boulle
9024659296 Merge pull request #217 from ecnahc515/patch-1
Fix broken link to fleet config
2014-09-09 15:19:18 -07:00
Chance Zibolski
fc6940f7ba Documentation: More specific link to fleet config.
Add an anchor tag to the url to take person directly to config section.
2014-09-09 15:15:55 -07:00
Brian Waldon
f2fd95699b Merge pull request #224 from bcwaldon/typo
docs: fix a typo
2014-09-09 12:36:42 -07:00
bdevloed
65db96cc7c docs: fix a typo 2014-09-09 12:31:54 -07:00
Alex Crawford
c17b93b5c0 Merge pull request #223 from crawford/yaml
third_party: sync third_party/gopkg.in/yaml.v1
2014-09-08 19:28:59 -07:00
Alex Crawford
d352f8ce6a Merge pull request #222 from crawford/contribute
docs: Update maintainers and contribution guide
2014-09-08 15:54:30 -07:00
Alex Crawford
78aa2c56ec yaml: replace goyaml with yaml 2014-09-08 13:25:27 -07:00
Alex Crawford
c5b3788282 third_party: sync third_party/gopkg.in/yaml.v1
Update launchpad.net/goyaml to gopkg.in/yaml.v1
2014-09-08 13:23:50 -07:00
Alex Crawford
5e98970bb5 docs: Update maintainers and contribution guide 2014-09-08 12:55:17 -07:00
Alex Crawford
cbdd446c55 Merge pull request #220 from crawford/docs
docs: Update list of platforms supporting variable substitutions
2014-09-05 11:51:45 -07:00
Alex Crawford
316cadcf44 docs: Update list of platforms supporting variable substitutions 2014-09-04 12:57:19 -07:00
Alex Crawford
5a939be21b coreos-cloudinit: bump to 0.9.6+git 2014-09-02 17:49:09 -07:00
Alex Crawford
8d76c64386 coreos-cloudinit: bump to 0.9.6 2014-09-02 17:48:45 -07:00
Alex Crawford
1b854eb51e Merge pull request #218 from crawford/units
units: Ensure that the units are executed in order
2014-09-02 17:40:37 -07:00
Alex Crawford
9fcf338bf3 units: Ensure that the units are executed in order 2014-09-02 17:15:32 -07:00
Alex Crawford
fda72bdb5c coreos-cloudinit: bump to 0.9.5+git 2014-09-02 10:10:59 -07:00
Alex Crawford
685a38c6c8 coreos-cloudinit: bump to 0.9.5 2014-09-02 10:10:41 -07:00
Alex Crawford
9d15f2cfaf Merge pull request #213 from crawford/digitalocean
digitalocean: Add support for DigitalOcean
2014-09-01 16:55:12 -07:00
Alex Crawford
2134fce791 digitalocean: Add tests for network unit generation 2014-09-01 16:53:15 -07:00
Alex Crawford
3abd6b2225 digitalocean: Add DigitalOcean metadata service
Move debian-related processing into its own file.
2014-09-01 16:53:15 -07:00
Alex Crawford
2a8e6c9566 network: Fall back to MAC address if there is no name 2014-09-01 09:29:45 -07:00
Alex Crawford
abe43537da metadata: Merge the network config 2014-09-01 09:29:45 -07:00
Jonathan Boulle
3a550af651 Merge pull request #216 from robszumski/patch-2
docs: fix broken link to fleet docs
2014-08-29 11:22:13 -07:00
Rob Szumski
61c3a0eb2d docs: fix broken link to fleet docs 2014-08-29 11:17:05 -07:00
Brian Waldon
480176bc11 Merge pull request #214 from bcwaldon/clarify-write-files
doc: clarify docs around write_files
2014-08-28 20:24:11 -07:00
Brian Waldon
01b18eb551 squash: fix spacing 2014-08-28 13:48:58 -07:00
Brian Waldon
970ef435b6 doc: clarify docs around write_files 2014-08-28 13:33:59 -07:00
Alex Crawford
e8d0021140 Merge pull request #212 from crawford/metadata
refactor: Refactor metadata and datasources to be more testable
2014-08-26 18:45:10 -07:00
Alex Crawford
e9ec78ac6f test: Refactor interface tests a little 2014-08-26 13:18:46 -07:00
Alex Crawford
4a2e417781 network: Add support for multiple addresses and HW addresses
Logical Interfaces can be assigned a hardware address allowing them
to match on MAC address. The static config method also needs to
support specifying multiple addresses.
2014-08-26 13:07:38 -07:00
Alex Crawford
604ef7ecb4 datasource: Add FetchNetworkConfig
FetchNetworkConfig is currently only used by ConfigDrive to read the
network config file from the disk.
2014-08-26 13:04:43 -07:00
Alex Crawford
c39dd5cc67 networkd: Fix bug causing bonding to always be loaded 2014-08-26 13:04:21 -07:00
Alex Crawford
a923161f4a metadata: Refactor common parts out of ec2 2014-08-26 12:02:56 -07:00
Alex Crawford
e59e2f6cd5 Merge pull request #210 from crawford/test
test: Add gofmt to test
2014-08-25 17:04:04 -07:00
Alex Crawford
e90fe3eba8 test: Add gofmt to test 2014-08-25 12:48:52 -07:00
Alex Crawford
fb0187b197 gofmt: sort 2014-08-25 12:35:40 -07:00
Michael Marineau
6babe74716 Merge pull request #209 from marineam/go13
travis: enable testing under go 1.3
2014-08-25 12:26:23 -07:00
Michael Marineau
b1e88284ca travis: enable testing under go 1.3 2014-08-25 12:21:07 -07:00
Alex Crawford
18a65f7dac Merge pull request #208 from crawford/go
test: Fix tests for Go 1.3
2014-08-25 12:19:52 -07:00
Alex Crawford
0c212c72c9 test: Fix tests for Go 1.3 2014-08-25 12:01:27 -07:00
Alex Crawford
6a800d8cc0 coreos-cloudinit: bump to 0.9.4+git 2014-08-24 18:41:20 -07:00
Alex Crawford
5e112147bb coreos-cloudinit: bump to 0.9.4 2014-08-24 18:40:53 -07:00
Alex Crawford
7e78b1563f Merge pull request #206 from crawford/tests
test: Enable tests for CloudSigma datasource
2014-08-24 18:36:38 -07:00
Alex Crawford
ecbe81f103 test: Enable tests for CloudSigma datasource 2014-08-24 17:08:49 -07:00
Alex Crawford
45c20c1dd3 Merge pull request #196 from Vladimiroff/cloudsigma
cloudsigma: Add support for CloudSigma datasource
2014-08-15 15:21:33 -07:00
Alex Crawford
8ce925a060 coreos-cloudinit: bump to 0.9.3+git 2014-08-15 10:47:28 -07:00
Alex Crawford
eadb6ef42c coreos-cloudinit: bump to 0.9.3 2014-08-15 10:46:46 -07:00
Alex Crawford
7518f0ec93 Merge pull request #204 from crawford/configdrive
configdrive: Remove broken support for ec2 metadata
2014-08-15 10:43:26 -07:00
Alex Crawford
f0b9eaf2fe configdrive: Remove broken support for ec2 metadata
As it turns out, certain metadata is only present in the ec2 flavor
of metadata (e.g. public_ipv4) and other data is only present in
the openstack flavor (e.g. network_config). For now, just read the
openstack metadata.
2014-08-15 10:35:21 -07:00
Kiril Vladimirov
7320a2cbf2 feat(datasource/metadata): Add datasource for CloudSigma 2014-08-15 12:08:55 +03:00
Kiril Vladimirov
57950b3ed9 add(goserial): import github.com/tarm/goserial 2014-08-15 12:08:34 +03:00
Kiril Vladimirov
85c6a2a16a add(cepgo): import github.com/cloudsigma/cepgo 2014-08-15 12:07:58 +03:00
Jonathan Boulle
24b44e86a6 coreos-cloudinit: bump to 0.9.2+git 2014-08-12 11:38:51 -07:00
Jonathan Boulle
2f52ad4ef8 coreos-cloudinit: bump to 0.9.2 2014-08-12 11:38:12 -07:00
Jonathan Boulle
735d6c6161 Merge pull request #202 from jonboulle/env
environment: write new keys in consistent order
2014-08-11 22:40:42 -07:00
Alex Crawford
1cf275bad6 Merge pull request #201 from crawford/configdrive
configdrive: fix root path
2014-08-11 20:11:17 -07:00
Jonathan Boulle
f1c97cb4d5 environment: write new keys in consistent order 2014-08-11 18:24:58 -07:00
Alex Crawford
d143904aa9 configdrive: fix root path 2014-08-11 17:57:10 -07:00
Jonathan Boulle
c428ce2cc5 Merge pull request #200 from jonboulle/fu
initialize: use correct heuristic to check if etcdenvironment is set
2014-08-11 17:44:44 -07:00
Jonathan Boulle
dfb5b4fc3a initialize: use correct heuristic to check if etcdenvironment is set
In some circumstances (e.g. nova-agent-watcher) cloudconfig files will
be created where the EtcdEnvironment is an empty map, and hence != nil.
If this is the case we should not do anything at all (because the user
hasn't explicitly asked us to configure etcd). This change standardises
behaviour with the check that we already do for FleetEnvironment.
2014-08-11 16:01:08 -07:00
Alex Crawford
97d5538533 Merge pull request #197 from crawford/ec2
datasource: Fix ec2 URLs
2014-08-06 22:45:03 -07:00
Alex Crawford
6b8f82b5d3 datasource: Fix ec2 URLs
_ vs -
2014-08-06 21:31:43 -07:00
Alex Crawford
facde6609f Merge pull request #194 from crawford/metadata
datasource: Refactoring datasources
2014-08-06 15:55:13 -07:00
Alex Crawford
d68ae84b37 metadata: Refactor metadata service into ec2 metadata
Added more testing.
2014-08-05 17:19:43 -07:00
Alex Crawford
54aa39543b timeouts: Use After() instead of Tick() 2014-08-04 15:10:14 -07:00
Alex Crawford
8566a2c118 datasource: Move datasources into their own packages. 2014-08-04 15:10:07 -07:00
Alex Crawford
49ac083af5 coreos-cloudinit: bump to 0.9.1+git 2014-08-04 14:14:24 -07:00
Alex Crawford
5d65ca230a coreos-cloudinit: bump to 0.9.1 2014-08-04 14:13:51 -07:00
Alex Crawford
38b3e1213a Merge pull request #188 from crawford/configdrive
configdrive: Use the EC2 metadata over OpenStack
2014-08-04 11:12:06 -07:00
Alex Crawford
4eedca26e9 configdrive: Use the EC2 metadata over OpenStack
Standardize on specific EC2 and OpenStack versions and add tests.
2014-08-04 10:18:29 -07:00
Brian Waldon
f2b342c8be doc: escape user.home example 2014-08-01 13:20:44 -07:00
Michael Marineau
c19d8f6b61 Merge pull request #193 from benjic/cloudconfig_variables
docs(quick-start): Clarified use of fields in cloud config
2014-07-24 11:02:03 -07:00
Benjamin Campbell
7913f74351 docs(quick-start) Enumerated supported platforms
Following suggestion a list of platforms that *do* support cloud config variables. In addition minor mark up formatting is added.
2014-07-24 11:54:31 -06:00
Benjamin Campbell
5593408be8 docs(quick-start): Clarified use of fields in cloud config
Updated the language to illustrate that fields in a cloud config is not
supported in all environments. This is expressed explicitly in PXE and
install to disk pages. The quick start lacked this information and is
inconsistent.
2014-07-24 11:27:35 -06:00
Alex Crawford
7fc67c2acf Merge pull request #191 from crawford/panic
config: Verify that type assertions are valid
2014-07-22 11:51:39 -07:00
Alex Crawford
b093094292 config: Verify that type assertions are valid 2014-07-22 11:39:20 -07:00
Michael Marineau
9a80fd714a Merge pull request #181 from robszumski/docs-startup
fix(docs): clarity around boot behavior and unit usage
2014-07-21 22:12:19 -07:00
Rob Szumski
fef5473881 fix(docs): clarity around boot behavior and unit usage 2014-07-21 21:41:00 -07:00
Alex Crawford
bf5a2b208f coreos-cloudinit: bump to 0.9.0+git 2014-07-21 19:17:14 -07:00
Alex Crawford
364507fb75 coreos-cloudinit: bump to 0.9.0 2014-07-21 19:16:11 -07:00
Alex Crawford
08d4842502 Merge pull request #190 from crawford/logs
Logs
2014-07-21 12:22:41 -07:00
Alex Crawford
21e32e44f8 system: Add more logging for networkd 2014-07-21 11:25:22 -07:00
Alex Crawford
7a06dee16f system: Cleanup redundant code 2014-07-21 11:24:42 -07:00
Alex Crawford
ff9cf5743d Merge pull request #187 from crawford/order
networkd: Reverse lexicographic order of generated unit files
2014-07-18 13:23:58 -07:00
Alex Crawford
1b10a3a187 networkd: Reverse lexicographic order of generated unit files 2014-07-17 20:47:37 -07:00
Michael Marineau
10838e001d Merge pull request #186 from robszumski/add-highlighting
feat(docs): add syntax highlighting
2014-07-15 15:26:33 -07:00
Rob Szumski
96370ac5b9 feat(docs): add syntax highlighting 2014-07-14 16:16:14 -07:00
Michael Marineau
0b82cd074d Merge pull request #180 from marineam/systemd_testing
chore(*): split out unit processing from config.Apply
2014-07-11 20:09:08 -07:00
Alex Crawford
a974e85103 Merge pull request #174 from crawford/teeth
networkd: Fix issues with bonding and VLANs
2014-07-11 15:48:02 -07:00
Michael Marineau
f0450662b0 Merge pull request #183 from marineam/fix
tests: fix error messages, use Fatalf
2014-07-11 15:40:54 -07:00
Michael Marineau
03e29d1291 tests: fix error messages, use Fatalf 2014-07-11 15:38:04 -07:00
Michael Marineau
98ae5d88aa coreos-cloudinit: bump to 0.8.9+git 2014-07-11 14:40:57 -07:00
Michael Marineau
bf5d3539c9 coreos-cloudinit: bump to 0.8.9 2014-07-11 14:40:34 -07:00
Michael Marineau
5e4cbcd909 Merge pull request #182 from marineam/fix
env_file: fix broken test cases
2014-07-11 14:38:56 -07:00
Michael Marineau
a270c4c737 env_file: fix broken test cases
TestWriteEnvFileDos2Unix had a copy/paste bug, it shouldn't have
asserted that mtime doesn't change because the file is actually being
modified in this test. This didn't come up earlier because the actual
comparison wasn't using Time.Equal as it should have.

Instead switch to comparing inode numbers which is the actual thing I
wanted to test for in the first place, just accessing them is much more
awkard. Now all tests where it is relevant check the inode in addition
to the contents.
2014-07-11 13:35:10 -07:00
Michael Marineau
f356a8a690 coreos-cloudinit: bump to 0.8.8+git 2014-07-11 11:13:01 -07:00
Michael Marineau
b1a897d75c coreos-cloudinit: bump to 0.8.8 2014-07-11 11:12:15 -07:00
Jonathan Boulle
be51f4eba0 chore(*): split out unit processing from config.Apply 2014-07-11 10:44:19 -07:00
Michael Marineau
a55e2cd49b Merge pull request #178 from marineam/env
Write /etc/environment
2014-07-11 10:39:33 -07:00
Michael Marineau
983501e43b environment: add support for updating /etc/environment with IP values
To maintain the behavior of the coreos-setup-environment that has
started to move into cloudinit we need to write out /etc/environment
with the public and private addresses, if known. The file is updated so
that other contents are not replaced. This behavior is disabled entirely
if /etc/environment was written by a write_files entry.
2014-07-11 10:34:44 -07:00
Alex Crawford
e3037f18a6 networkd: Restart networkd twice to work around race
https://bugs.freedesktop.org/show_bug.cgi?id=76077
2014-07-10 23:40:42 -07:00
Alex Crawford
fe388a3ab6 networkd: Create config directory before writing config 2014-07-10 23:40:42 -07:00
Alex Crawford
c820f2b1cf bonding: Add support for probing the bonding module with parameters
Until support for bonding params is added to networkd, this will be
neccessary in order to use bonding parameters (i.e. miimon, mode).
This also makes it such that the 8012q module will only be loaded if
the network config makes use of VLANs.
2014-07-10 23:40:42 -07:00
Michael Marineau
81824be3bf system: new file writer for updating env-style files
This can be used to safely update config files cloudinit does not have
exclusive control over. For example update.conf or /etc/environment.
2014-07-10 15:53:32 -07:00
Michael Marineau
98c26440be Merge pull request #176 from jayofdoom/master
Document need for #cloud-config in cloud-config.yml
2014-07-09 16:41:00 -07:00
Jay Faulkner
3b5fcc393b Document need for #cloud-config in cloud-config.yml
- cloud-config.yml does not work if it's missing the #cloud-config
  directive at the top. This is undocumented, except in the examples.
2014-07-09 16:36:11 -07:00
Alex Crawford
9528077340 coreos-cloudinit: bump to 0.8.7+git 2014-07-02 15:20:45 -07:00
Alex Crawford
4355a05d55 coreos-cloudinit: bump to 0.8.7 2014-07-02 15:20:26 -07:00
Alex Crawford
52c44923dd Merge pull request #173 from crawford/metadata
metadata-service: remove check for OpenStack meta_data.json
2014-07-02 15:19:37 -07:00
Alex Crawford
47748ef4b6 metadata-service: remove check for OpenStack meta_data.json
The meta_data.json blob under OpenStack doesn't actually contain all
of the metadata... Fall back to explicitly requesting each attribute.
2014-07-02 14:38:23 -07:00
Alex Crawford
8eca10200e coreos-cloudinit: bump to 0.8.6+git 2014-07-01 16:17:00 -07:00
Alex Crawford
43be8c8996 coreos-cloudinit: bump to 0.8.6 2014-07-01 16:16:41 -07:00
Alex Crawford
19b4b1160e Merge pull request #171 from crawford/err
metadata-service: Handle no user-data
2014-07-01 16:15:32 -07:00
Alex Crawford
ce6fccfb3c metadata-service: Handle no user-data 2014-07-01 16:10:18 -07:00
Alex Crawford
7d89aefb82 coreos-cloudinit: bump to 0.8.5+git 2014-07-01 15:45:49 -07:00
Alex Crawford
2369e2a920 coreos-cloudinit: bump to 0.8.5 2014-07-01 15:45:23 -07:00
Alex Crawford
6d808048d3 Merge pull request #170 from crawford/metadata
metadata: Fetch the public and private IP addresses
2014-07-01 15:44:14 -07:00
Alex Crawford
276f0b5d99 metadata: Fetch the public and private IP addresses 2014-07-01 14:43:19 -07:00
Jonathan Boulle
92bd5ca5d4 coreos-cloudinit: bump to 0.8.4+git 2014-07-01 12:16:09 -07:00
Jonathan Boulle
5b5ffea126 coreos-cloudinit: bump to 0.8.4 2014-07-01 12:15:48 -07:00
Jonathan Boulle
18068e9375 Merge pull request #169 from jonboulle/pebkac
coreos-cloudinit: apply environment to userdata string
2014-07-01 12:15:06 -07:00
Jonathan Boulle
1b3cabb035 coreos-cloudinit: apply environment to userdata string 2014-07-01 12:08:42 -07:00
Jonathan Boulle
1be2bec1c2 coreos-cloudinit: bump to 0.8.3+git 2014-06-30 22:12:13 -07:00
Jonathan Boulle
f3bd5f543e coreos-cloudinit: bump to 0.8.3 2014-06-30 22:11:15 -07:00
Jonathan Boulle
660feb59b9 Merge pull request #168 from jonboulle/foo
fix ordering error in mergeCloudConfig
2014-06-30 22:08:47 -07:00
Jonathan Boulle
9673dbe12b coreos-cloudinit: fix ordering error in merge invocation 2014-06-30 22:07:05 -07:00
Alex Crawford
2be435dd83 coreos-cloudinit: bump to 0.8.2+git 2014-06-30 18:11:14 -07:00
Alex Crawford
2d91369596 coreos-cloudinit: bump to 0.8.2 2014-06-30 18:10:20 -07:00
Alex Crawford
d8d3928978 Merge pull request #167 from crawford/sshkeys
metadata-service: fix ssh key retrieval and application
2014-06-30 18:08:04 -07:00
Alex Crawford
7fcc540154 metadata-service: fix ssh key retrieval and application
The metadata service wasn't properly fetching the ssh keys from metadata.
Drop the key traversal in favor of explict key urls.
2014-06-30 17:45:08 -07:00
Jonathan Boulle
cb7fbd4668 Merge pull request #166 from jonboulle/merge
cloudinit: merge cloudconfig info from user-data and meta-data
2014-06-30 11:27:47 -07:00
Jonathan Boulle
d4e048a1f4 ParseUserData: return nil on empty input string 2014-06-30 11:27:33 -07:00
Jonathan Boulle
231c0fa20b initialize: add tests for ParseMetadata 2014-06-27 23:53:06 -07:00
Jonathan Boulle
1aabacc769 cloudinit: merge cloudconfig info from user-data and meta-data
This attempts to retrieve cloudconfigs from two sources: the meta-data
service, and the user-data service. If only one cloudconfig is found,
that is applied to the system. If both services return a cloudconfig,
the two are merged into a single cloudconfig which is then applied to
the system.

Only a subset of parameters are merged (because the meta-data service
currently only partially populates a cloudconfig). In the event of any
conflicts, parameters in the user-data cloudconfig take precedence over
those in the meta-data cloudconfig.
2014-06-27 23:48:48 -07:00
Alex Crawford
6a2927d701 coreos-cloudinit: bump to 0.8.1+git 2014-06-27 15:00:05 -07:00
Alex Crawford
126188510b coreos-cloudinit: bump to 0.8.1 2014-06-27 14:59:56 -07:00
Alex Crawford
4627ccb444 Merge pull request #165 from crawford/units
units: update dependencies
2014-06-27 14:55:48 -07:00
Alex Crawford
aff372111a units: update dependencies 2014-06-27 14:29:59 -07:00
Alex Crawford
c7081b9918 coreos-cloudinit: bump to 0.8.0+git 2014-06-27 11:33:56 -07:00
Alex Crawford
9ba3b18b59 coreos-cloudinit: bump to 0.8.0 2014-06-27 11:32:52 -07:00
Alex Crawford
099de62e9a Merge pull request #164 from crawford/datasources
datasources: Add support for specifying multiple datasources
2014-06-27 00:25:09 -07:00
Alex Crawford
c089216cb5 datasources: Add support for specifying multiple datasources
If multiple sources are specified, the first available source is used.
2014-06-26 22:32:39 -07:00
Alex Crawford
68dc902ed1 HttpClient: Refactor timeout into two seperate functions 2014-06-26 15:16:22 -07:00
Jonathan Boulle
ad66b1c92f Merge pull request #163 from jonboulle/net
coreos-cloudinit: restrict convert-netconf to configdrive
2014-06-25 14:35:27 -07:00
Jonathan Boulle
fbdece2762 coreos-cloudinit: restrict convert-netconf to configdrive 2014-06-25 14:28:11 -07:00
Jonathan Boulle
f85eafb7ca Merge pull request #162 from jonboulle/fffffffffffffffffff
initialize/env: handle nil substitution maps properly
2014-06-25 12:10:00 -07:00
Jonathan Boulle
f0dba2294e initialize/env: handle nil substitution maps properly 2014-06-25 12:07:48 -07:00
Alex Crawford
bda3948382 Merge pull request #159 from crawford/metadata
metadataService: Check both ec2 and openstack urls more explicitly
2014-06-25 11:26:19 -07:00
Alex Crawford
fae81c78f3 metadataService: Check both ec2 and openstack urls more explicitly
Remove the root url parameter for -from-metadata-service since this
is a guarenteed value. Additionally, check for both ec2 and openstack
urls for the metadata and userdata. Fix a bug with the -from-url
option and a panic on an empty response.
2014-06-25 11:19:11 -07:00
Alex Crawford
a5dec7d7bd cloudconfig: Process metadata before userdata
This gives the options in userdata a higher precedence over metadata.
2014-06-25 10:35:26 -07:00
Jonathan Boulle
e1222c9885 Merge pull request #161 from jonboulle/doc
metadata: add links to metadata source information
2014-06-25 08:55:26 -07:00
Jonathan Boulle
ded3bcf122 metadata: add links to metadata source information 2014-06-24 19:26:07 -07:00
Jonathan Boulle
80d00cde94 Merge pull request #158 from jonboulle/nettttt
cloudinit: retrieve IPv4 addresses from metadata
2014-06-24 19:11:15 -07:00
Jonathan Boulle
2805d70ece initialize/env: add notes about tests 2014-06-24 18:52:08 -07:00
Jonathan Boulle
439b7e8b98 initialize/env: fall back to COREOS_*_IPV4 env variables 2014-06-24 18:49:49 -07:00
Jonathan Boulle
ba1c1e97d0 cloudinit: retrieve IPv4 addresses from metadata
This uses the new MetadataService implementation to retrieve values for
$private_ipv4 and $public_ipv4 substitutions, instead of using
environment variables.
2014-06-24 17:46:06 -07:00
Alex Crawford
8a50fd8595 Merge pull request #154 from crawford/metadata
metadata-service: Add new datasource to download from metadata service
2014-06-24 15:18:27 -07:00
Alex Crawford
465bcce72c metadata_service: Add tests for constructing metadata 2014-06-24 15:08:03 -07:00
Alex Crawford
361edeebc6 metadata-service: Add metadata-service datasource
Move the old metadata-service datasource to url datasource. This new datasource
checks for the existance of meta-data.json and if it doesn't exist, walks the
meta-data directory to build a metadata blob.
2014-06-24 15:08:03 -07:00
Jonathan Boulle
29a7b0e34f Merge pull request #155 from jonboulle/etcd
etcdenvironment: order map keys consistently
2014-06-24 15:02:04 -07:00
Alex Crawford
8496ffb53a HttpClient: Wrap errors with error classes 2014-06-24 15:01:31 -07:00
Jonathan Boulle
2c717a6cd1 Merge pull request #157 from jonboulle/core
coreos-cloudinit: clean up flag handling
2014-06-24 14:57:18 -07:00
Jonathan Boulle
13a91c9181 coreos-cloudinit: clean up flag handling 2014-06-24 14:19:55 -07:00
Jonathan Boulle
338e1b64ab etcdenvironment: order map keys consistently 2014-06-23 15:13:11 -07:00
Alex Crawford
8eb0636034 Merge pull request #149 from crawford/network
feat(network): Add support for blind interfaces, support for hwaddress, and bug fixes
2014-06-20 17:53:37 -07:00
Alex Crawford
f7c25a1b83 doc(debian-interfaces): Add basic docs for convert-netconf 2014-06-20 17:51:57 -07:00
Alex Crawford
d6a0d0908c fix(network): Generate prefixes to ensure proper lexicographical ordering
In order for networkd to properly configure the network interfaces, the configs must be
prefixed to ensure that they load in the correct order (parent interfaces have a lower
prefix than their children).
2014-06-20 17:51:57 -07:00
Alex Crawford
5c89afc18a Merge pull request #152 from crawford/metadata
feat(meta_data): Add partial support for meta_data.json
2014-06-20 17:47:08 -07:00
Michael Marineau
376cc4bcac chore(coreos-cloudinit): bump to 0.7.7+git 2014-06-18 15:01:13 -07:00
Michael Marineau
d0a6d6f92f chore(coreos-cloudinit): bump to 0.7.7 2014-06-18 14:55:38 -07:00
Michael Marineau
2be1e52f32 Merge pull request #151 from marineam/mount
fix(configdrive): Use mount units, give virtfs a new mount point.
2014-06-18 13:51:11 -07:00
Michael Marineau
784a71e2bf fix(configdrive): Use mount units, give virtfs a new mount point.
Currently systemd cannot track dependencies on configdrive very well
because it is mounted via a service instead of a mount unit. Also since
the interaction between path and mount units can lead to unexpected
behavior if something goes wrong the cloudinit service is now triggered
explicitly by the mount again. The configdrive path unit remains only as
a fall back for containers where the mount unit doesn't kick in. Better
to have two mechanisms that trigger the cloudinit service than none. :)

Since mounting a virtfs based configdrive requires different mount
options and two different mount units cannot refer to the same path the
virtfs version now mounts to /media/configvirtfs.

There are also two new kernel options:
- `coreos.configdrive=1`: enable config drive on physical hardware.
- `coreos.configdrive=0`: disable config drive on virtual machines.
2014-06-18 13:01:19 -07:00
Alex Crawford
e6cf83a2e5 refactor(netconf): Move netconf processing and handle metadata 2014-06-18 12:43:41 -07:00
Alex Crawford
840c208b60 feat(metadata): Distinguish between userdata and metadata for datasources 2014-06-18 12:34:31 -07:00
Alex Crawford
29ed6b38bd refactor(env): Add the config root and netconf type to datasource and env 2014-06-18 12:27:15 -07:00
Alex Crawford
259c7e1fe2 fix(sshKeyName): Use the SSH key name provided 2014-06-18 11:47:17 -07:00
Alex Crawford
033c8d352f feat(network): Add support for hwaddress
Currently only supports the ether mode of hwaddress. No immediate plans
to support ax25, ARCnet, or netrom.
2014-06-14 21:30:14 -07:00
Alex Crawford
16d7e8af48 fix(network): Take down all interfaces properly
The map of interfaces wasn't being populated correctly. Also, clean up some prints.
2014-06-13 20:53:59 -07:00
Alex Crawford
159f4a2c7c feat(network): Add support for blind interfaces
It is valid for an interface to reference another, otherwise undeclared,
interface (i.e. a bond enslaves eth0 without eth0 having its own iface stanza).
In order to associate the two interfaces, the undeclared interface needs to be
implicitly created so that it can be referenced by the other. This adds the
capability to forward-declare interfaces in addition to cleaning up the
process a little bit.
2014-06-11 22:07:33 -07:00
Michael Marineau
160668284c chore(coreos-cloudinit): bump to 0.7.6+git 2014-06-07 16:04:33 -04:00
Michael Marineau
41b9dfcb1c chore(coreos-cloudinit): bump to 0.7.6 2014-06-07 16:01:31 -04:00
Michael Marineau
ef4c3483b6 Merge pull request #146 from marineam/fix
fix(update): Fix restart of update-engine
2014-06-07 13:04:49 -07:00
Michael Marineau
4bdf633075 fix(update): Fix restart of update-engine
The name was missing .service.
2014-06-07 12:08:22 -07:00
Brian Waldon
c9fc718e18 Merge pull request #145 from bcwaldon/drop-group-req
Relax requirements of update group value
2014-06-06 11:43:22 -07:00
Brian Waldon
4461b3d33d fix(update): Relax requirements of update group value 2014-06-06 11:29:09 -07:00
Jonathan Boulle
c6a1412f6b chore(coreos-cloudinit): bump to 0.7.5+git 2014-06-06 11:14:39 -07:00
Jonathan Boulle
d0cbbd2007 chore(coreos-cloudinit): bump to 0.7.5 2014-06-06 11:10:48 -07:00
Jonathan Boulle
7b5e542eb4 Merge pull request #132 from jonboulle/locksmith
reboot-strategy=off breaks subsequent reboot strategies
2014-06-06 11:08:06 -07:00
Jonathan Boulle
376d82ba63 doc(*): add note about runtime locksmithd unit file 2014-06-06 10:55:42 -07:00
Jonathan Boulle
a6aa9f82b8 fix(systemd): unmask runtime units when mask=False 2014-06-06 10:55:42 -07:00
Jonathan Boulle
00ee047753 fix(locksmith): use a runtime unit for locksmith 2014-06-06 10:55:42 -07:00
Jonathan Boulle
f127406d01 Merge pull request #140 from jonboulle/atomic
fix(system): write all files atomically
2014-06-06 10:37:09 -07:00
Jonathan Boulle
0ddc08d55a fix(system): write all files atomically 2014-06-06 10:36:36 -07:00
Jonathan Boulle
56f455f890 Merge pull request #141 from jonboulle/141
cloudinit doesn't restart update-engine.service
2014-06-06 10:25:24 -07:00
Jonathan Boulle
dd861b9f88 fix(initialize): ensure update-engine is restarted after group/server
changes
2014-06-05 16:12:40 -07:00
Alex Crawford
f7d01da267 Merge pull request #138 from spkane/github-ent-key-docs
Add a valid URL example for Github Enterprise token based API auth
2014-06-04 16:15:04 -07:00
Sean P. Kane
fc8f30bf08 Add a valid URL example for Github Enterprise token based API auth 2014-06-04 16:03:02 -07:00
Brandon Philips
075c0557e7 Merge pull request #137 from robszumski/patch-1
fix(docs): remove unneeded install section
2014-06-04 14:22:55 -07:00
Rob Szumski
d25e13a2c6 fix(docs): remove unneeded install section 2014-06-04 13:57:18 -07:00
Alex Crawford
cf1ffad533 chore(coreos-cloudinit): bump to 0.7.4+git 2014-06-03 14:14:47 -07:00
Alex Crawford
82706b1d5f chore(coreos-cloudinit): bump to 0.7.4 2014-06-03 14:13:56 -07:00
Alex Crawford
38c8fda0d1 Merge pull request #124 from crawford/networkd
feat(networkd): Adding support for debian-interface-to-networkd conversion
2014-06-03 13:55:06 -07:00
Alex Crawford
69240a7e39 feat(systemd): Update the systemd unit files to use configdrive
This makes it so that /media/configdrive can be used for user-data
and network configs.
2014-06-02 18:43:22 -07:00
Brian Waldon
c4f1996843 fix(doc): Correct spacing in cloud-config.md 2014-06-02 16:49:44 -07:00
Alex Crawford
48df1be793 feat(convertNetconf): Add support for network config conversion
Adding the flag -convertNetconf which is used to specify the config
format to convert from (right now, only 'debian' is supported).
Once the network configs are generated, they are written to
systemd's runtime network directory and the network is restarted.
2014-06-02 15:31:30 -07:00
Alex Crawford
79a40a38d8 add(netlink): import dotcloud/docker/pkg/netlink 2014-06-02 15:31:30 -07:00
Alex Crawford
856061b445 test(interfaces): Add tests for network conversion
These tests should be an exhaustive set of tests for the parsing
of Debian interface files and generation of equivilent networkd
config files.
2014-06-02 15:31:27 -07:00
Alex Crawford
38321fedce feat(interfaces): Add support for interfaces file
This adds the ability for cloudinit to parse a debian interfaces
file and generate the coresponding networkd configs.
2014-06-02 15:30:37 -07:00
Alex Crawford
f8a823cf7e refactor(userdata): Move userdata processing into a function 2014-06-02 14:59:01 -07:00
Alex Crawford
a4035cffea feat(config-drive): Add support for reading user-data from config-drive
The -config-drive flag tells cloudinit to read the user-data from
within the config-drive (./openstack/latest/user-data).
2014-06-02 14:58:57 -07:00
Brian Waldon
5c8fb7f465 fix(doc): Add newlines for proper formatting 2014-06-02 11:42:43 -07:00
Alex Crawford
7a02bf54ed Merge pull request #130 from crawford/docs
fix(docs): Fix minor typo describing runtime field for units
2014-05-30 11:52:30 -07:00
Alex Crawford
388dd67388 fix(docs): Fix minor typo describing runtime field for units 2014-05-30 11:45:44 -07:00
Jonathan Boulle
ded6d94180 chore(coreos-cloudinit): bump to 0.7.3+git 2014-05-29 14:55:34 -07:00
Jonathan Boulle
a9a910b5c4 chore(coreos-cloudinit): bump to 0.7.3 2014-05-29 14:52:58 -07:00
Jonathan Boulle
8e94b4140a Merge pull request #122 from jonboulle/122
ec2-cloudinit service fails after reboot with "reboot-strategy: off"
2014-05-29 14:25:58 -07:00
Jonathan Boulle
cd322863e9 Merge pull request #129 from jonboulle/exp
fix(pkg): simplify exponential backoff to avoid overflows
2014-05-29 14:02:47 -07:00
Jonathan Boulle
786e4bef65 fix(systemd): remove any existing unit when calling mask 2014-05-29 13:59:55 -07:00
Jonathan Boulle
269a658d4b fix(pkg): simplify exponential backoff to avoid overflows 2014-05-29 11:11:18 -07:00
Michael Marineau
e317c7eb9a chore(coreos-cloudinit): bump to 0.7.2+git 2014-05-27 14:02:11 -07:00
Michael Marineau
974de943e0 chore(coreos-cloudinit): bump to 0.7.2 2014-05-27 13:37:58 -07:00
Jonathan Boulle
db3f008543 Merge pull request #127 from jonboulle/127
"Enable" option does not support units in /usr/lib64/systemd
2014-05-26 15:24:30 -07:00
Jonathan Boulle
b04509ae54 fix(systemd): EnableUnitFile unit name rather than absolute destination 2014-05-26 15:16:24 -07:00
Jonathan Boulle
6c07e8784f Merge pull request #125 from jonboulle/no_locksmith_enable
Dies trying to enable non-existent /etc/systemd/system/locksmithd.service
2014-05-26 13:11:47 -07:00
Jonathan Boulle
60ab4222de fix(update): locksmith service does not need disabling/enabling 2014-05-26 12:33:23 -07:00
Brandon Philips
1a295f65c7 Merge pull request #123 from c4milo/shared-http-client
feat(util/http_client): Adds generic HTTP client
2014-05-22 14:37:32 -07:00
Camilo Aguilar
cec0926c5c fix(pkg/http_client): Printf is smarter than you think
Printf determines what the duration unit is
and prints it accordingly.
2014-05-22 14:53:54 -04:00
Camilo Aguilar
8ca3c2ed1f style(httpbackoff -> pkg): Adjusts package name to follow convention 2014-05-22 14:37:19 -04:00
Camilo Aguilar
2cedebb4eb style(util->httpbackoff): Changes package as per @philips suggestion 2014-05-21 21:12:16 -04:00
Camilo Aguilar
3e00a37ef5 feat(util/http_client): Adds generic HTTP client
Supports retries with exponential backoff as well as connection
timeouts and the ability to skip SSL/TLS verification.

This commit also refactors datasource and initialize packages
in order to use the new HTTP client.
2014-05-21 13:31:50 -04:00
Jonathan Boulle
59d1eba423 Merge pull request #111 from namsral/patch-1
Trim newlines from the cloud-config-url option
2014-05-21 10:18:24 -07:00
Jonathan Boulle
af69149260 Merge pull request #120 from brianredbeard/pr20-fix
fix(docs) Clear description of update server changes
2014-05-21 10:01:25 -07:00
Brandon Philips
5fa2ad8dfd Merge pull request #121 from iamveen/master
removed tricky space from cloud-config header
2014-05-21 05:33:05 -07:00
Lars Wiegman
513a1eb602 Trim newlines from the cloud-config-url kernel parameter and added a test
- In the Fetch function trim whitespace from /proc/cmdline
- New test for Fetch function
- Added Location field to the procCmdline struct for testing
2014-05-21 11:09:39 +02:00
Gavin Dunne
5189e1594e removed tricky space from cloud-config header 2014-05-21 01:22:09 -07:00
Brian 'Redbeard' Harrington
8b5bc47429 fix(doc) more sensible ordering
It makes a bit more sense to specify the scope of the section
before getting into details about how it's done.
2014-05-20 23:29:56 -07:00
Brian 'Redbeard' Harrington
a64fcd2893 fix(docs) Clear description of update server changes TBD
Pulling in @philips' changes from coreos/coreos-cloudinit#6 after
trashing PR coreos/coreos-cloudinit#20.  Cleanup of that PR was
beyond my git-fu.

cc @jonboulle
2014-05-20 22:53:29 -07:00
Brandon Philips
5b1145c044 Merge pull request #118 from c4milo/log-timestamp-fix
chore(logging): Removes duplicated timestamp during booting
2014-05-17 16:31:07 -07:00
Michael Marineau
a49877b99f chore(coreos-cloudinit): bump to 0.7.1+git 2014-05-16 21:23:34 -07:00
Michael Marineau
24f181f7a3 chore(coreos-cloudinit): bump to 0.7.1 2014-05-16 21:21:47 -07:00
Michael Marineau
61e70fcce8 Merge pull request #119 from marineam/container
container and panic fixes
2014-05-16 21:19:43 -07:00
Michael Marineau
ea6262f0ae fix(etcd): fix runtime panic when etcd section is missing.
The etcd code tries to assign ee["name"] even when the map was never
defined and assigning to an uninitialized map causes a panic.
2014-05-16 20:38:49 -07:00
Michael Marineau
f83ce07416 feat(units): Add generic cloudinit path unit
Switch to triggering common user configs via a path unit. This is
particularly useful for config drive so that a config drive can be
mounted by something other than the udev triggered services, a bind
mount when running in a container for example.
2014-05-16 20:38:49 -07:00
Brandon Philips
140682350d chore(coreos-cloudinit): bump to 0.7.0+git 2014-05-16 18:22:22 -07:00
Brandon Philips
289ada4668 chore(coreos-cloudinit): bump to 0.7.0 2014-05-16 18:22:22 -07:00
Camilo Aguilar
5d58c6c1c1 chore(logging): Removes duplicated timestamp during booting 2014-05-16 17:35:31 -04:00
Jonathan Boulle
d95df78c6d Merge pull request #117 from c4milo/travis-support
chore(travis): Adds travis yaml file as well as badge in README
2014-05-16 14:11:37 -07:00
Camilo Aguilar
ac4c969454 chore(travis): Adds travis yaml file and badge in README 2014-05-16 17:09:59 -04:00
Jonathan Boulle
04fcd3935f Merge pull request #114 from c4milo/fetch-url-refactor
refactor(datastore/fetch): Makes more failure proof fetching user-data files.
2014-05-16 14:03:54 -07:00
Camilo Aguilar
36efcc9d69 test(datastore/fetch): Makes sure err is not nil 2014-05-16 16:57:58 -04:00
Jonathan Boulle
f7ecc2461c Merge pull request #109 from jonboulle/fleet
fix(docs): add documentation for fleet section
2014-05-16 13:38:12 -07:00
Jonathan Boulle
8df9ee3ca2 Merge pull request #115 from burke/master
Response body must not be closed if request error'd.
2014-05-16 13:20:27 -07:00
Burke Libbey
321ceaa0da Response body must not be closed if request error'd. 2014-05-16 15:42:11 -04:00
Jonathan Boulle
05daad692e fix(docs): add documentation for fleet section 2014-05-16 12:10:21 -07:00
Camilo Aguilar
4b6fc63e8c fix(datastore/fetch): off-by-one oversight 2014-05-16 12:36:05 -04:00
Camilo Aguilar
fcccfb085f style(datastore/fetch): Adjusts comments formatting 2014-05-16 12:35:39 -04:00
Camilo Aguilar
ebf134f181 refactor(datastore/fetch): Makes more failure proof fetching user-data files
- Adds URL validations
- Adds timeout support for http client
- Limits the amount of retries to not spin forever
- Fails faster if response status code is 4xx
- Does a little bit more of logging
- Adds more tests
2014-05-16 12:35:06 -04:00
Jonathan Boulle
51d77516a5 Merge pull request #90 from jonboulle/90
Warn or error on unrecognized keys in cloud-config.yml
2014-05-15 18:53:48 -07:00
Jonathan Boulle
98f5ead730 fix(*): catch more unknown keys in user and file sections 2014-05-15 18:53:17 -07:00
Jonathan Boulle
81fe0dc9e0 fix(initialize): also check for unknown coreos keys 2014-05-15 18:53:17 -07:00
Jonathan Boulle
e852be65f7 feat(*): warn on encountering unrecognized keys in cloud-config 2014-05-15 18:53:17 -07:00
Brandon Philips
0a16532d4b Merge pull request #113 from c4milo/exponential_backoff
Exponential backoff with sleep capping
2014-05-15 10:16:42 -07:00
Camilo Aguilar
ff70a60fbc Adds sleep cap to exponential backoff so it does not go too high 2014-05-15 13:04:37 -04:00
Kelsey Hightower
31f61d7531 Use exponential backoff when fetching user-data from an URL.
The user-cloudinit-proc-cmdline systemd unit is responsible for
fetching user-data from various sources during the cloud-init
process. When fetching user-data from an URL datasource we face
a race condition since the network may not be available, which
can cause the job to fail and no further attempts to fetch the
user-data are made.

Eliminate the race condition when fetching user-data from an URL
datasource. Retry the fetch using an exponential backoff until
the user-data is retrieved.

Fixes issue 105.
2014-05-14 23:15:49 -07:00
Jonathan Boulle
b505e6241c Merge pull request #103 from jonboulle/20
feat(*): add more configuration options for update.conf
2014-05-14 13:14:35 -07:00
Jonathan Boulle
e413a97741 feat(update): add more configuration options for update.conf 2014-05-14 13:13:19 -07:00
Jonathan Boulle
41cbec8729 Merge pull request #101 from jonboulle/fleet
feat(*): add basic fleet configuration to cloud-config
2014-05-14 12:28:52 -07:00
Jonathan Boulle
919298e545 feat(fleet): add basic fleet configuration to cloud-config 2014-05-14 12:28:20 -07:00
Jonathan Boulle
ae424b5637 Merge pull request #106 from jonboulle/locksmith_to_update
refactor(init): rename locksmith to update
2014-05-14 11:50:09 -07:00
Jonathan Boulle
e93911344d refactor(init): rename locksmith to update 2014-05-14 11:40:39 -07:00
Jonathan Boulle
32c52d8729 Merge pull request #100 from jonboulle/rework
refactor(*): rework cloudconfig for better extensibility and consistency
2014-05-14 11:39:53 -07:00
Jonathan Boulle
cdee32d245 refactor(systemd): don't allow users to set DropIn=true yet 2014-05-14 11:34:13 -07:00
Jonathan Boulle
31cfad91e3 refactor(*): rework cloudconfig for better extensibility and consistency
This change creates a few simple interfaces for coreos-specific
configuration options and moves things to them wherever possible; so if
an option needs to write a file, or create a unit, it is acted on
exactly the same way as every other file/unit that needs to be written
during the cloud configuration process.
2014-05-14 11:34:07 -07:00
Brian Waldon
e814b37839 Merge pull request #107 from bcwaldon/locksmith-no-etc
fix(coreos-cloudinit): Ensure /etc/coreos exists before writing to it
2014-05-14 10:49:23 -07:00
Brian Waldon
cb4d9e81a4 fix(coreos-cloudinit): Ensure /etc/coreos exists before writing to it 2014-05-14 10:47:18 -07:00
Jonathan Boulle
b87a4628e6 Merge pull request #99 from jonboulle/simple
chore(cloudinit): remove superfluous check
2014-05-12 10:51:51 -07:00
Jonathan Boulle
b22fdd5ac9 Merge pull request #104 from jonboulle/tests
feat(tests): add coverage script
2014-05-12 10:51:38 -07:00
Jonathan Boulle
6939fc2ddc feat(tests): add cover script 2014-05-10 01:42:57 -07:00
Jonathan Boulle
e3117269cb chore(cloudinit): remove superfluous check 2014-05-09 20:32:51 -07:00
Brandon Philips
3bb3a683a4 Merge pull request #98 from philips/remove-oem-from-default
chore(Documentation): move OEM into its own doc
2014-05-08 09:41:42 -07:00
Brandon Philips
e1033c979e chore(Documentation): move OEM into its own doc
People are customizing the OEM needlessly. Just move it into its own
doc.
2014-05-08 09:32:21 -07:00
Jonathan Boulle
9a4d24826f Merge pull request #80 from jonboulle/master
users[i].primary-group option seems invalid
2014-05-07 21:12:45 -07:00
Jonathan Boulle
7bed1307e1 fix(user): user correct primary group flag for useradd 2014-05-07 14:06:51 -07:00
Brandon Philips
47b536532d chore(coreos-cloudinit): version +git 2014-05-06 21:09:40 -07:00
143 changed files with 10552 additions and 1801 deletions

3
.gitignore vendored
View File

@@ -1,3 +1,4 @@
*.swp
bin/
pkg/
coverage/
gopath/

10
.travis.yml Normal file
View File

@@ -0,0 +1,10 @@
language: go
go:
- 1.3
- 1.2
install:
- go get code.google.com/p/go.tools/cmd/cover
script:
- ./test

View File

@@ -39,22 +39,25 @@ Thanks for your contributions!
### Format of the Commit Message
We follow a rough convention for commit messages borrowed from AngularJS. This
is an example of a commit:
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
feat(scripts/test-cluster): add a cluster test command
environment: write new keys in consistent order
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Go 1.3 randomizes the ordering of keys when iterating over a map.
Sort the keys to make this ordering consistent.
Fixes #38
```
The format can be described more formally as follows:
```
<type>(<scope>): <subject>
<subsystem>: <what changed>
<BLANK LINE>
<body>
<why this change was made>
<BLANK LINE>
<footer>
```
@@ -63,25 +66,3 @@ The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
#### Subject Line
The subject line contains a succinct description of the change.
#### Allowed `<type>`s
- *feat* (feature)
- *fix* (bug fix)
- *docs* (documentation)
- *style* (formatting, missing semi colons, …)
- *refactor*
- *test* (when adding missing tests)
- *chore* (maintain)
#### Allowed `<scope>`s
Scopes can anything specifying the place of the commit change in the code base -
for example, "api", "store", etc.
For more details on the commit format, see the [AngularJS commit style
guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).

View File

@@ -0,0 +1,37 @@
## OEM configuration
The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine. Customizing this section is only needed when generating a new OEM of CoreOS from the SDK. The fields include:
- **id**: Lowercase string identifying the OEM
- **name**: Human-friendly string representing the OEM
- **version-id**: Lowercase string identifying the version of the OEM
- **home-url**: Link to the homepage of the provider or OEM
- **bug-report-url**: Link to a place to file bug reports about this OEM
coreos-cloudinit renders these fields to `/etc/oem-release`.
If no **id** field is provided, coreos-cloudinit will ignore this section.
For example, the following cloud-config document...
```yaml
#cloud-config
coreos:
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
```
...would be rendered to the following `/etc/oem-release`:
```yaml
ID=rackspace
NAME="Rackspace Cloud Servers"
VERSION_ID=168.0.0
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
```
[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html

View File

@@ -1,10 +1,10 @@
# Using Cloud-Config
CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime.
CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime. Your cloud-config is processed during each boot.
## Configuration File
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file, which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
@@ -16,7 +16,7 @@ We've designed our implementation to allow the same cloud-config file to work ac
The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
A cloud-config file should contain an associative array which has zero or more of the following keys:
A cloud-config file should contain `#cloud-config`, followed by an associative array which has zero or more of the following keys:
- `coreos`
- `ssh_authorized_keys`
@@ -40,9 +40,9 @@ CoreOS tries to conform to each platform's native method to provide user data. E
#### etcd
The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
If the platform environment supports the templating feature of coreos-cloudinit it is possible to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
```
```yaml
#cloud-config
coreos:
@@ -57,7 +57,7 @@ coreos:
...will generate a systemd unit drop-in like this:
```
```yaml
[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
@@ -68,72 +68,73 @@ Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
_Note: The `$private_ipv4` and `$public_ipv4` substitution variables referenced in other documents are only supported on Amazon EC2, Google Compute Engine, OpenStack, Rackspace, DigitalOcean, and Vagrant._
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
#### fleet
The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
```yaml
#cloud-config
coreos:
fleet:
public-ip: $public_ipv4
metadata: region=us-west
```
...will generate a systemd unit drop-in like this:
```yaml
[Service]
Environment="FLEET_PUBLIC_IP=203.0.113.29"
Environment="FLEET_METADATA=region=us-west"
```
For more information on fleet configuration, see the [fleet documentation][fleet-config].
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#configuration
#### update
The `coreos.update.*` parameters manipulate settings related to how CoreOS instances are updated.
These fields will be written out to and replace `/etc/coreos/update.conf`. If only one of the parameters is given it will only overwrite the given field.
The `reboot-strategy` parameter also affects the behaviour of [locksmith](https://github.com/coreos/locksmith).
- **reboot-strategy**: One of "reboot", "etcd-lock", "best-effort" or "off" for controlling when reboots are issued after an update is performed.
- _reboot_: Reboot immediately after an update is applied.
- _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
- _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
- _off_ - Disable rebooting after updates are applied (not recommended).
- **server**: is the omaha endpoint URL which will be queried for updates.
- **group**: signifies the channel which should be used for automatic updates. This value defaults to the version of the image initially downloaded. (one of "master", "alpha", "beta", "stable")
```
*Note: cloudinit will only manipulate the locksmith unit file in the systemd runtime directory (`/run/systemd/system/locksmithd.service`). If any manual modifications are made to an overriding unit configuration file (e.g. `/etc/systemd/system/locksmithd.service`), cloudinit will no longer be able to control the locksmith service unit.*
##### Example
```yaml
#cloud-config
coreos:
update:
reboot-strategy: etcd-lock
```
#### oem
The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine:
- **id**: Lowercase string identifying the OEM
- **name**: Human-friendly string representing the OEM
- **version-id**: Lowercase string identifying the version of the OEM
- **home-url**: Link to the homepage of the provider or OEM
- **bug-report-url**: Link to a place to file bug reports about this OEM
coreos-cloudinit renders these fields to `/etc/oem-release`.
If no **id** field is provided, coreos-cloudinit will ignore this section.
For example, the following cloud-config document...
```
#cloud-config
coreos:
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
```
...would be rendered to the following `/etc/oem-release`:
```
ID=rackspace
NAME="Rackspace Cloud Servers"
VERSION_ID=168.0.0
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
```
[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html
#### units
The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields:
The `coreos.units.*` parameters define a list of arbitrary systemd units to start after booting. This feature is intended to help you start essential services required to mount storage and configure networking in order to join the CoreOS cluster. It is not intended to be a Chef/Puppet replacement.
Each item is an object with the following fields:
- **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` argument to `systemd enable`. Default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemctl enable`. The default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. The default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. The default behavior is to not execute any commands.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. The default value is false.
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
@@ -141,7 +142,7 @@ The `coreos.units.*` parameters define a list of arbitrary systemd units to star
Write a unit to disk, automatically starting it.
```
```yaml
#cloud-config
coreos:
@@ -158,15 +159,12 @@ coreos:
Restart=always
ExecStart=/usr/bin/docker start -a redis_server
ExecStop=/usr/bin/docker stop -t 2 redis_server
[Install]
WantedBy=local.target
```
Start the builtin `etcd` and `fleet` services:
Start the built-in `etcd` and `fleet` services:
```
# cloud-config
```yaml
#cloud-config
coreos:
units:
@@ -183,7 +181,7 @@ The `ssh_authorized_keys` parameter adds public SSH keys which will be authorize
The keys will be named "coreos-cloudinit" by default.
Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`.
```
```yaml
#cloud-config
ssh_authorized_keys:
@@ -195,7 +193,7 @@ ssh_authorized_keys:
The `hostname` parameter defines the system's hostname.
This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
```
```yaml
#cloud-config
hostname: coreos1
@@ -209,7 +207,7 @@ All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the use
- **name**: Required. Login name of user
- **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name>
- **homedir**: User's home directory. Defaults to /home/\<name\>
- **no-create-home**: Boolean. Skip home directory creation.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups
@@ -228,7 +226,7 @@ The following fields are not yet implemented:
- **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad.
```
```yaml
#cloud-config
users:
@@ -267,7 +265,7 @@ Using a higher number of rounds will help create more secure passwords, but give
Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
```
```yaml
#cloud-config
users:
@@ -280,17 +278,17 @@ users:
We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
```
```yaml
#cloud-config
users:
- name: elroy
coreos-ssh-import-url: https://token:<OAUTH-TOKEN>@github-enterprise.example.com/users/elroy/keys
coreos-ssh-import-url: https://github-enterprise.example.com/api/v3/users/elroy/keys?access_token=<TOKEN>
```
You can also specify any URL whose response matches the JSON format for public keys:
```
```yaml
#cloud-config
users:
@@ -300,7 +298,8 @@ users:
### write_files
The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys:
The `write_files` directive defines a set of files to create on the local filesystem.
Each item in the list may have the following keys:
- **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path`
@@ -310,14 +309,19 @@ The `write-file` parameter defines a list of files to create on the local filesy
Explicitly not implemented is the **encoding** attribute.
The **content** field must represent exactly what should be written to disk.
```
```yaml
#cloud-config
write_files:
- path: /etc/fleet/fleet.conf
- path: /etc/resolv.conf
permissions: 0644
owner: root
content: |
verbosity=1
metadata="region=us-west,type=ssd"
nameserver 8.8.8.8
- path: /etc/motd
permissions: 0644
owner: root
content: |
Good news, everyone!
```
### manage_etc_hosts
@@ -327,7 +331,7 @@ Currently, the only supported value is "localhost" which will cause your system'
to resolve to "127.0.0.1". This is helpful when the host does not have DNS
infrastructure in place to resolve its own hostname, for example, when using Vagrant.
```
```yaml
#cloud-config
manage_etc_hosts: localhost

View File

@@ -14,17 +14,21 @@ The image should be a single FAT or ISO9660 file system with the label
For example, to wrap up a config named `user_data` in a config drive image:
mkdir -p /tmp/new-drive/openstack/latest
cp user_data /tmp/new-drive/openstack/latest/user_data
mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
rm -r /tmp/new-drive
```sh
mkdir -p /tmp/new-drive/openstack/latest
cp user_data /tmp/new-drive/openstack/latest/user_data
mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
rm -r /tmp/new-drive
```
## QEMU virtfs
One exception to the above, when using QEMU it is possible to skip creating an
image and use a plain directory containing the same contents:
qemu-system-x86_64 \
```sh
qemu-system-x86_64 \
-fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \
-device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \
[usual qemu options here...]
```

View File

@@ -0,0 +1,27 @@
#Debian Interfaces#
**WARNING**: This option is EXPERIMENTAL and may change or be removed at any
point.
There is basic support for converting from a Debian network configuration to
networkd unit files. The -convert-netconf=debian option is used to activate
this feature.
#convert-netconf#
Default: ""
Read the network config provided in cloud-drive and translate it from the
specified format into networkd unit files (requires the -from-configdrive
flag). Currently only supports "debian" which provides support for a small
subset of the [Debian network configuration]
(https://wiki.debian.org/NetworkConfiguration). These options include:
- interface config methods
- static
- address/netmask
- gateway
- hwaddress
- dns-nameservers
- dhcp
- hwaddress
- manual
- loopback
- vlan_raw_device
- bond-slaves

3
MAINTAINERS Normal file
View File

@@ -0,0 +1,3 @@
Alex Crawford <alex.crawford@coreos.com> (@crawford)
Jonathan Boulle <jonathan.boulle@coreos.com> (@jonboulle)
Brian Waldon <brian.waldon@coreos.com> (@bcwaldon)

View File

@@ -1,4 +1,4 @@
# coreos-cloudinit
# coreos-cloudinit [![Build Status](https://travis-ci.org/coreos/coreos-cloudinit.png?branch=master)](https://travis-ci.org/coreos/coreos-cloudinit)
coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.

14
build
View File

@@ -1,6 +1,14 @@
#!/bin/bash -e
export GOBIN=${PWD}/bin
export GOPATH=${PWD}
ORG_PATH="github.com/coreos"
REPO_PATH="${ORG_PATH}/coreos-cloudinit"
go build -o bin/coreos-cloudinit github.com/coreos/coreos-cloudinit
if [ ! -h gopath/src/${REPO_PATH} ]; then
mkdir -p gopath/src/${ORG_PATH}
ln -s ../../../.. gopath/src/${REPO_PATH} || exit 255
fi
export GOBIN=${PWD}/bin
export GOPATH=${PWD}/gopath
go build -o bin/coreos-cloudinit ${REPO_PATH}

198
config/config.go Normal file
View File

@@ -0,0 +1,198 @@
package config
import (
"fmt"
"log"
"reflect"
"strings"
"github.com/coreos/coreos-cloudinit/third_party/gopkg.in/yaml.v1"
)
// CloudConfig encapsulates the entire cloud-config configuration file and maps
// directly to YAML. Fields that cannot be set in the cloud-config (fields
// used for internal use) have the YAML tag '-' so that they aren't marshalled.
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd Etcd `yaml:"etcd"`
Fleet Fleet `yaml:"fleet"`
OEM OEM `yaml:"oem"`
Update Update `yaml:"update"`
Units []Unit `yaml:"units"`
} `yaml:"coreos"`
WriteFiles []File `yaml:"write_files"`
Hostname string `yaml:"hostname"`
Users []User `yaml:"users"`
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
NetworkConfigPath string `yaml:"-"`
NetworkConfig string `yaml:"-"`
}
// NewCloudConfig instantiates a new CloudConfig from the given contents (a
// string of YAML), returning any error encountered. It will ignore unknown
// fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := yaml.Unmarshal([]byte(contents), &cfg)
if err != nil {
return &cfg, err
}
warnOnUnrecognizedKeys(contents, log.Printf)
return &cfg, nil
}
func (cc CloudConfig) String() string {
bytes, err := yaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
// IsZero returns whether or not the parameter is the zero value for its type.
// If the parameter is a struct, only the exported fields are considered.
func IsZero(c interface{}) bool {
return isZero(reflect.ValueOf(c))
}
// AssertValid checks the fields in the structure and makes sure that they
// contain valid values as specified by the 'valid' flag. Empty fields are
// implicitly valid.
func AssertValid(c interface{}) error {
ct := reflect.TypeOf(c)
cv := reflect.ValueOf(c)
for i := 0; i < ct.NumField(); i++ {
ft := ct.Field(i)
if !isFieldExported(ft) {
continue
}
valid := ft.Tag.Get("valid")
val := cv.Field(i)
if !isValid(val, valid) {
return fmt.Errorf("invalid value \"%v\" for option %q (valid options: %q)", val.Interface(), ft.Name, valid)
}
}
return nil
}
func isZero(v reflect.Value) bool {
switch v.Kind() {
case reflect.Struct:
vt := v.Type()
for i := 0; i < v.NumField(); i++ {
if isFieldExported(vt.Field(i)) && !isZero(v.Field(i)) {
return false
}
}
return true
default:
return v.Interface() == reflect.Zero(v.Type()).Interface()
}
}
func isFieldExported(f reflect.StructField) bool {
return f.PkgPath == ""
}
func isValid(v reflect.Value, valid string) bool {
if valid == "" || isZero(v) {
return true
}
vs := fmt.Sprintf("%v", v.Interface())
for _, valid := range strings.Split(valid, ",") {
if vs == valid {
return true
}
}
return false
}
type warner func(format string, v ...interface{})
// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
func warnOnUnrecognizedKeys(contents string, warn warner) {
// Generate a map of all understood cloud config options
var cc map[string]interface{}
b, _ := yaml.Marshal(&CloudConfig{})
yaml.Unmarshal(b, &cc)
// Now unmarshal the entire provided contents
var c map[string]interface{}
yaml.Unmarshal([]byte(contents), &c)
// Check that every key in the contents exists in the cloud config
for k, _ := range c {
if _, ok := cc[k]; !ok {
warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
}
}
// Check for unrecognized coreos options, if any are set
if coreos, ok := c["coreos"]; ok {
if set, ok := coreos.(map[interface{}]interface{}); ok {
known := cc["coreos"].(map[interface{}]interface{})
for k, _ := range set {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", k)
}
}
}
}
// Check for any badly-specified users, if any are set
if users, ok := c["users"]; ok {
var known map[string]interface{}
b, _ := yaml.Marshal(&User{})
yaml.Unmarshal(b, &known)
if set, ok := users.([]interface{}); ok {
for _, u := range set {
if user, ok := u.(map[interface{}]interface{}); ok {
for k, _ := range user {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", k)
}
}
}
}
}
}
// Check for any badly-specified files, if any are set
if files, ok := c["write_files"]; ok {
var known map[string]interface{}
b, _ := yaml.Marshal(&File{})
yaml.Unmarshal(b, &known)
if set, ok := files.([]interface{}); ok {
for _, f := range set {
if file, ok := f.(map[interface{}]interface{}); ok {
for k, _ := range file {
if key, ok := k.(string); ok {
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
}
} else {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", k)
}
}
}
}
}
}
}

473
config/config_test.go Normal file
View File

@@ -0,0 +1,473 @@
package config
import (
"errors"
"fmt"
"reflect"
"strings"
"testing"
)
func TestIsZero(t *testing.T) {
for _, tt := range []struct {
c interface{}
empty bool
}{
{struct{}{}, true},
{struct{ a, b string }{}, true},
{struct{ A, b string }{}, true},
{struct{ A, B string }{}, true},
{struct{ A string }{A: "hello"}, false},
{struct{ A int }{}, true},
{struct{ A int }{A: 1}, false},
} {
if empty := IsZero(tt.c); tt.empty != empty {
t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.empty, empty)
}
}
}
func TestAssertValid(t *testing.T) {
for _, tt := range []struct {
c interface{}
err error
}{
{struct{}{}, nil},
{struct {
A, b string `valid:"1,2"`
}{}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "1", b: "2"}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "1", b: "hello"}, nil},
{struct {
A, b string `valid:"1,2"`
}{A: "hello", b: "2"}, errors.New("invalid value \"hello\" for option \"A\" (valid options: \"1,2\")")},
{struct {
A, b int `valid:"1,2"`
}{}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 1, b: 2}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 1, b: 9}, nil},
{struct {
A, b int `valid:"1,2"`
}{A: 9, b: 2}, errors.New("invalid value \"9\" for option \"A\" (valid options: \"1,2\")")},
} {
if err := AssertValid(tt.c); !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad result (%q): want %q, got %q", tt.c, tt.err, err)
}
}
}
func TestCloudConfigInvalidKeys(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("panic while instantiating CloudConfig with nil keys: %v", r)
}
}()
for _, tt := range []struct {
contents string
}{
{"coreos:"},
{"ssh_authorized_keys:"},
{"ssh_authorized_keys:\n -"},
{"ssh_authorized_keys:\n - 0:"},
{"write_files:"},
{"write_files:\n -"},
{"write_files:\n - 0:"},
{"users:"},
{"users:\n -"},
{"users:\n - 0:"},
} {
_, err := NewCloudConfig(tt.contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with invalid keys: %v", err)
}
}
}
func TestCloudConfigUnknownKeys(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
coreos_unknown:
foo: "bar"
section_unknown:
dunno:
something
bare_unknown:
bar
write_files:
- content: fun
path: /var/party
file_unknown: nofun
users:
- name: fry
passwd: somehash
user_unknown: philip
hostname:
foo
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
}
if cfg.Hostname != "foo" {
t.Fatalf("hostname not correctly set when invalid keys are present")
}
if cfg.Coreos.Etcd.Discovery != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
t.Fatalf("etcd section not correctly set when invalid keys are present")
}
if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
t.Fatalf("write_files section not correctly set when invalid keys are present")
}
if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
t.Fatalf("users section not correctly set when invalid keys are present")
}
var warnings string
catchWarn := func(f string, v ...interface{}) {
warnings += fmt.Sprintf(f, v...)
}
warnOnUnrecognizedKeys(contents, catchWarn)
if !strings.Contains(warnings, "coreos_unknown") {
t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
}
if !strings.Contains(warnings, "bare_unknown") {
t.Errorf("warnings did not catch unrecognized key bare_unknown")
}
if !strings.Contains(warnings, "section_unknown") {
t.Errorf("warnings did not catch unrecognized key section_unknown")
}
if !strings.Contains(warnings, "user_unknown") {
t.Errorf("warnings did not catch unrecognized user key user_unknown")
}
if !strings.Contains(warnings, "file_unknown") {
t.Errorf("warnings did not catch unrecognized file key file_unknown")
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
if cfg.Hostname != "" {
t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot-strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
content: '[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
write_files:
- content: |
penny
elroy
path: /etc/dogepack.conf
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 2 {
t.Error("Parsed incorrect number of SSH keys")
} else if keys[0] != "foobar" {
t.Error("Expected first SSH key to be 'foobar'")
} else if keys[1] != "foobaz" {
t.Error("Expected first SSH key to be 'foobaz'")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
wf := cfg.WriteFiles[0]
if wf.Content != "penny\nelroy\n" {
t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
}
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if wf.RawFilePermissions != "0644" {
t.Errorf("WriteFile has incorrect permissions %s", wf.RawFilePermissions)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
}
if wf.Owner != "root:dogepack" {
t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
}
}
if len(cfg.Coreos.Units) != 1 {
t.Error("Failed to parse correct number of units")
} else {
u := cfg.Coreos.Units[0]
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if u.Content != expect {
t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
}
if u.Runtime != true {
t.Errorf("Unit has incorrect runtime value")
}
if u.Name != "50-eth0.network" {
t.Errorf("Unit has incorrect name %s", u.Name)
}
if u.Type() != "network" {
t.Errorf("Unit has incorrect type '%s'", u.Type())
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.Coreos.Update.RebootStrategy != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
ssh_authorized_keys:
- foo: bar
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
t.Fatalf("Serialized config did not have expected header")
}
}
// TestDropInIgnored asserts that users are unable to set DropIn=True on units
func TestDropInIgnored(t *testing.T) {
contents := `
coreos:
units:
- name: test
dropin: true
`
cfg, err := NewCloudConfig(contents)
if err != nil || len(cfg.Coreos.Units) != 1 {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
}
if cfg.Coreos.Units[0].DropIn {
t.Errorf("dropin option on unit in cloud-config was not ignored!")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
users:
- name: elroy
passwd: somehash
ssh-authorized-keys:
- somekey
gecos: arbitrary comment
homedir: /home/place
no-create-home: yes
primary-group: things
groups:
- ping
- pong
no-user-group: true
system: y
no-log-init: True
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.PasswordHash != "somehash" {
t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
}
if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
} else {
key := user.SSHAuthorizedKeys[0]
if key != "somekey" {
t.Errorf("User SSH key is %q, expected 'somekey'", key)
}
}
if user.GECOS != "arbitrary comment" {
t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
}
if user.Homedir != "/home/place" {
t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
}
if !user.NoCreateHome {
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}
if len(user.Groups) != 2 {
t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
} else {
if user.Groups[0] != "ping" {
t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
}
if user.Groups[1] != "pong" {
t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
}
}
if !user.NoUserGroup {
t.Errorf("Failed to parse no-user-group field")
}
if !user.System {
t.Errorf("Failed to parse system field")
}
if !user.NoLogInit {
t.Errorf("Failed to parse no-log-init field")
}
}
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

3
config/etc_hosts.go Normal file
View File

@@ -0,0 +1,3 @@
package config
type EtcHosts string

32
config/etcd.go Normal file
View File

@@ -0,0 +1,32 @@
package config
type Etcd struct {
Addr string `yaml:"addr" env:"ETCD_ADDR"`
BindAddr string `yaml:"bind-addr" env:"ETCD_BIND_ADDR"`
CAFile string `yaml:"ca-file" env:"ETCD_CA_FILE"`
CertFile string `yaml:"cert-file" env:"ETCD_CERT_FILE"`
ClusterActiveSize string `yaml:"cluster-active-size" env:"ETCD_CLUSTER_ACTIVE_SIZE"`
ClusterRemoveDelay string `yaml:"cluster-remove-delay" env:"ETCD_CLUSTER_REMOVE_DELAY"`
ClusterSyncInterval string `yaml:"cluster-sync-interval" env:"ETCD_CLUSTER_SYNC_INTERVAL"`
Cors string `yaml:"cors" env:"ETCD_CORS"`
CPUProfileFile string `yaml:"cpu-profile-file" env:"ETCD_CPU_PROFILE_FILE"`
DataDir string `yaml:"data-dir" env:"ETCD_DATA_DIR"`
Discovery string `yaml:"discovery" env:"ETCD_DISCOVERY"`
HTTPReadTimeout string `yaml:"http-read-timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
HTTPWriteTimeout string `yaml:"http-write-timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
KeyFile string `yaml:"key-file" env:"ETCD_KEY_FILE"`
MaxClusterSize string `yaml:"max-cluster-size" env:"ETCD_MAX_CLUSTER_SIZE"`
MaxResultBuffer string `yaml:"max-result-buffer" env:"ETCD_MAX_RESULT_BUFFER"`
MaxRetryAttempts string `yaml:"max-retry-attempts" env:"ETCD_MAX_RETRY_ATTEMPTS"`
Name string `yaml:"name" env:"ETCD_NAME"`
PeerAddr string `yaml:"peer-addr" env:"ETCD_PEER_ADDR"`
PeerBindAddr string `yaml:"peer-bind-addr" env:"ETCD_PEER_BIND_ADDR"`
PeerCAFile string `yaml:"peer-ca-file" env:"ETCD_PEER_CA_FILE"`
PeerCertFile string `yaml:"peer-cert-file" env:"ETCD_PEER_CERT_FILE"`
PeerKeyFile string `yaml:"peer-key-file" env:"ETCD_PEER_KEY_FILE"`
Peers string `yaml:"peers" env:"ETCD_PEERS"`
PeersFile string `yaml:"peers-file" env:"ETCD_PEERS_FILE"`
Snapshot string `yaml:"snapshot" env:"ETCD_SNAPSHOT"`
Verbose string `yaml:"verbose" env:"ETCD_VERBOSE"`
VeryVerbose string `yaml:"very-verbose" env:"ETCD_VERY_VERBOSE"`
}

9
config/file.go Normal file
View File

@@ -0,0 +1,9 @@
package config
type File struct {
Encoding string `yaml:"-"`
Content string `yaml:"content"`
Owner string `yaml:"owner"`
Path string `yaml:"path"`
RawFilePermissions string `yaml:"permissions"`
}

14
config/fleet.go Normal file
View File

@@ -0,0 +1,14 @@
package config
type Fleet struct {
AgentTTL string `yaml:"agent-ttl" env:"FLEET_AGENT_TTL"`
EngineReconcileInterval string `yaml:"engine-reconcile-interval" env:"FLEET_ENGINE_RECONCILE_INTERVAL"`
EtcdCAFile string `yaml:"etcd-cafile" env:"FLEET_ETCD_CAFILE"`
EtcdCertFile string `yaml:"etcd-certfile" env:"FLEET_ETCD_CERTFILE"`
EtcdKeyFile string `yaml:"etcd-keyfile" env:"FLEET_ETCD_KEYFILE"`
EtcdRequestTimeout string `yaml:"etcd-request-timeout" env:"FLEET_ETCD_REQUEST_TIMEOUT"`
EtcdServers string `yaml:"etcd-servers" env:"FLEET_ETCD_SERVERS"`
Metadata string `yaml:"metadata" env:"FLEET_METADATA"`
PublicIP string `yaml:"public-ip" env:"FLEET_PUBLIC_IP"`
Verbosity string `yaml:"verbosity" env:"FLEET_VERBOSITY"`
}

9
config/oem.go Normal file
View File

@@ -0,0 +1,9 @@
package config
type OEM struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}

34
config/unit.go Normal file
View File

@@ -0,0 +1,34 @@
package config
import (
"path/filepath"
"strings"
)
type Unit struct {
Name string `yaml:"name"`
Mask bool `yaml:"mask"`
Enable bool `yaml:"enable"`
Runtime bool `yaml:"runtime"`
Content string `yaml:"content"`
Command string `yaml:"command"`
// For drop-in units, a cloudinit.conf is generated.
// This is currently unbound in YAML (and hence unsettable in cloud-config files)
// until the correct behaviour for multiple drop-in units is determined.
DropIn bool `yaml:"-"`
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() string {
switch u.Type() {
case "network", "netdev", "link":
return "network"
default:
return "system"
}
}

7
config/update.go Normal file
View File

@@ -0,0 +1,7 @@
package config
type Update struct {
RebootStrategy string `yaml:"reboot-strategy" env:"REBOOT_STRATEGY" valid:"best-effort,etcd-lock,reboot,off"`
Group string `yaml:"group" env:"GROUP"`
Server string `yaml:"server" env:"SERVER"`
}

17
config/user.go Normal file
View File

@@ -0,0 +1,17 @@
package config
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
SSHImportURL string `yaml:"coreos-ssh-import-url"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}

View File

@@ -3,112 +3,351 @@ package main
import (
"flag"
"fmt"
"log"
"os"
"sync"
"time"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/datasource"
"github.com/coreos/coreos-cloudinit/datasource/configdrive"
"github.com/coreos/coreos-cloudinit/datasource/file"
"github.com/coreos/coreos-cloudinit/datasource/metadata/cloudsigma"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
"github.com/coreos/coreos-cloudinit/datasource/metadata/ec2"
"github.com/coreos/coreos-cloudinit/datasource/proc_cmdline"
"github.com/coreos/coreos-cloudinit/datasource/url"
"github.com/coreos/coreos-cloudinit/initialize"
"github.com/coreos/coreos-cloudinit/pkg"
"github.com/coreos/coreos-cloudinit/system"
)
const version = "0.6.0"
const (
version = "0.10.4"
datasourceInterval = 100 * time.Millisecond
datasourceMaxInterval = 30 * time.Second
datasourceTimeout = 5 * time.Minute
)
var (
flags = struct {
printVersion bool
ignoreFailure bool
sources struct {
file string
configDrive string
metadataService bool
ec2MetadataService string
cloudSigmaMetadataService bool
digitalOceanMetadataService string
url string
procCmdLine bool
}
convertNetconf string
workspace string
sshKeyName string
oem string
}{}
)
func init() {
flag.BoolVar(&flags.printVersion, "version", false, "Print the version and exit")
flag.BoolVar(&flags.ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
flag.StringVar(&flags.sources.file, "from-file", "", "Read user-data from provided file")
flag.StringVar(&flags.sources.configDrive, "from-configdrive", "", "Read data from provided cloud-drive directory")
flag.BoolVar(&flags.sources.metadataService, "from-metadata-service", false, "[DEPRECATED - Use -from-ec2-metadata] Download data from metadata service")
flag.StringVar(&flags.sources.ec2MetadataService, "from-ec2-metadata", "", "Download EC2 data from the provided url")
flag.BoolVar(&flags.sources.cloudSigmaMetadataService, "from-cloudsigma-metadata", false, "Download data from CloudSigma server context")
flag.StringVar(&flags.sources.digitalOceanMetadataService, "from-digitalocean-metadata", "", "Download DigitalOcean data from the provided url")
flag.StringVar(&flags.sources.url, "from-url", "", "Download user-data from provided url")
flag.BoolVar(&flags.sources.procCmdLine, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", proc_cmdline.ProcCmdlineLocation, proc_cmdline.ProcCmdlineCloudConfigFlag))
flag.StringVar(&flags.oem, "oem", "", "Use the settings specific to the provided OEM")
flag.StringVar(&flags.convertNetconf, "convert-netconf", "", "Read the network config provided in cloud-drive and translate it from the specified format into networkd unit files")
flag.StringVar(&flags.workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
flag.StringVar(&flags.sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
}
type oemConfig map[string]string
var (
oemConfigs = map[string]oemConfig{
"digitalocean": oemConfig{
"from-digitalocean-metadata": "http://169.254.169.254/",
"convert-netconf": "digitalocean",
},
"ec2-compat": oemConfig{
"from-ec2-metadata": "http://169.254.169.254/",
"from-configdrive": "/media/configdrive",
},
"rackspace-onmetal": oemConfig{
"from-configdrive": "/media/configdrive",
"convert-netconf": "debian",
},
}
)
func main() {
var printVersion bool
flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
var ignoreFailure bool
flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
var file string
flag.StringVar(&file, "from-file", "", "Read user-data from provided file")
var url string
flag.StringVar(&url, "from-url", "", "Download user-data from provided url")
var useProcCmdline bool
flag.BoolVar(&useProcCmdline, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", datasource.ProcCmdlineLocation, datasource.ProcCmdlineCloudConfigFlag))
var workspace string
flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
var sshKeyName string
flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
failure := false
flag.Parse()
if printVersion == true {
if c, ok := oemConfigs[flags.oem]; ok {
for k, v := range c {
flag.Set(k, v)
}
} else if flags.oem != "" {
oems := make([]string, 0, len(oemConfigs))
for k := range oemConfigs {
oems = append(oems, k)
}
fmt.Printf("Invalid option to --oem: %q. Supported options: %q\n", flags.oem, oems)
os.Exit(2)
}
if flags.printVersion == true {
fmt.Printf("coreos-cloudinit version %s\n", version)
os.Exit(0)
}
if file != "" && url != "" && !useProcCmdline {
fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
switch flags.convertNetconf {
case "":
case "debian":
case "digitalocean":
default:
fmt.Printf("Invalid option to -convert-netconf: '%s'. Supported options: 'debian, digitalocean'\n", flags.convertNetconf)
os.Exit(2)
}
dss := getDatasources()
if len(dss) == 0 {
fmt.Println("Provide at least one of --from-file, --from-configdrive, --from-ec2-metadata, --from-cloudsigma-metadata, --from-url or --from-proc-cmdline")
os.Exit(2)
}
ds := selectDatasource(dss)
if ds == nil {
fmt.Println("No datasources available in time")
os.Exit(1)
}
var ds datasource.Datasource
if file != "" {
ds = datasource.NewLocalFile(file)
} else if url != "" {
ds = datasource.NewMetadataService(url)
} else if useProcCmdline {
ds = datasource.NewProcCmdline()
} else {
fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
os.Exit(1)
}
log.Printf("Fetching user-data from datasource of type %q", ds.Type())
userdataBytes, err := ds.Fetch()
fmt.Printf("Fetching user-data from datasource of type %q\n", ds.Type())
userdataBytes, err := ds.FetchUserdata()
if err != nil {
log.Printf("Failed fetching user-data from datasource: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
fmt.Printf("Failed fetching user-data from datasource: %v\nContinuing...\n", err)
failure = true
}
fmt.Printf("Fetching meta-data from datasource of type %q\n", ds.Type())
metadataBytes, err := ds.FetchMetadata()
if err != nil {
fmt.Printf("Failed fetching meta-data from datasource: %v\n", err)
os.Exit(1)
}
// Extract IPv4 addresses from metadata if possible
var subs map[string]string
if len(metadataBytes) > 0 {
subs, err = initialize.ExtractIPsFromMetadata(metadataBytes)
if err != nil {
fmt.Printf("Failed extracting IPs from meta-data: %v\n", err)
os.Exit(1)
}
}
if len(userdataBytes) == 0 {
log.Printf("No user data to handle, exiting.")
os.Exit(0)
}
// Apply environment to user-data
env := initialize.NewEnvironment("/", ds.ConfigRoot(), flags.workspace, flags.convertNetconf, flags.sshKeyName, subs)
userdata := env.Apply(string(userdataBytes))
env := initialize.NewEnvironment("/", workspace)
userdata := string(userdataBytes)
userdata = env.Apply(userdata)
parsed, err := initialize.ParseUserData(userdata)
if err != nil {
log.Printf("Failed parsing user-data: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
var ccm, ccu *config.CloudConfig
var script *system.Script
if ccm, err = initialize.ParseMetaData(string(metadataBytes)); err != nil {
fmt.Printf("Failed to parse meta-data: %v\n", err)
os.Exit(1)
}
}
err = initialize.PrepWorkspace(env.Workspace())
if ccm != nil && flags.convertNetconf != "" {
fmt.Printf("Fetching network config from datasource of type %q\n", ds.Type())
netconfBytes, err := ds.FetchNetworkConfig(ccm.NetworkConfigPath)
if err != nil {
log.Fatalf("Failed preparing workspace: %v", err)
fmt.Printf("Failed fetching network config from datasource: %v\n", err)
os.Exit(1)
}
ccm.NetworkConfig = string(netconfBytes)
}
switch t := parsed.(type) {
case initialize.CloudConfig:
err = initialize.Apply(t, env)
if ud, err := initialize.ParseUserData(userdata); err != nil {
fmt.Printf("Failed to parse user-data: %v\nContinuing...\n", err)
failure = true
} else {
switch t := ud.(type) {
case *config.CloudConfig:
ccu = t
case system.Script:
var path string
path, err = initialize.PersistScriptInWorkspace(t, env.Workspace())
script = &t
}
}
var cc *config.CloudConfig
if ccm != nil && ccu != nil {
fmt.Println("Merging cloud-config from meta-data and user-data")
merged := mergeCloudConfig(*ccm, *ccu)
cc = &merged
} else if ccm != nil && ccu == nil {
fmt.Println("Processing cloud-config from meta-data")
cc = ccm
} else if ccm == nil && ccu != nil {
fmt.Println("Processing cloud-config from user-data")
cc = ccu
} else {
fmt.Println("No cloud-config data to handle.")
}
if cc != nil {
if err = initialize.Apply(*cc, env); err != nil {
fmt.Printf("Failed to apply cloud-config: %v\n", err)
os.Exit(1)
}
}
if script != nil {
if err = runScript(*script, env); err != nil {
fmt.Printf("Failed to run script: %v\n", err)
os.Exit(1)
}
}
if failure && !flags.ignoreFailure {
os.Exit(1)
}
}
// mergeCloudConfig merges certain options from mdcc (a CloudConfig derived from
// meta-data) onto udcc (a CloudConfig derived from user-data), if they are
// not already set on udcc (i.e. user-data always takes precedence)
// NB: This needs to be kept in sync with ParseMetadata so that it tracks all
// elements of a CloudConfig which that function can populate.
func mergeCloudConfig(mdcc, udcc config.CloudConfig) (cc config.CloudConfig) {
if mdcc.Hostname != "" {
if udcc.Hostname != "" {
fmt.Printf("Warning: user-data hostname (%s) overrides metadata hostname (%s)\n", udcc.Hostname, mdcc.Hostname)
} else {
udcc.Hostname = mdcc.Hostname
}
}
for _, key := range mdcc.SSHAuthorizedKeys {
udcc.SSHAuthorizedKeys = append(udcc.SSHAuthorizedKeys, key)
}
if mdcc.NetworkConfigPath != "" {
if udcc.NetworkConfigPath != "" {
fmt.Printf("Warning: user-data NetworkConfigPath %s overrides metadata NetworkConfigPath %s\n", udcc.NetworkConfigPath, mdcc.NetworkConfigPath)
} else {
udcc.NetworkConfigPath = mdcc.NetworkConfigPath
}
}
if mdcc.NetworkConfig != "" {
if udcc.NetworkConfig != "" {
fmt.Printf("Warning: user-data NetworkConfig %s overrides metadata NetworkConfig %s\n", udcc.NetworkConfig, mdcc.NetworkConfig)
} else {
udcc.NetworkConfig = mdcc.NetworkConfig
}
}
return udcc
}
// getDatasources creates a slice of possible Datasources for cloudinit based
// on the different source command-line flags.
func getDatasources() []datasource.Datasource {
dss := make([]datasource.Datasource, 0, 5)
if flags.sources.file != "" {
dss = append(dss, file.NewDatasource(flags.sources.file))
}
if flags.sources.url != "" {
dss = append(dss, url.NewDatasource(flags.sources.url))
}
if flags.sources.configDrive != "" {
dss = append(dss, configdrive.NewDatasource(flags.sources.configDrive))
}
if flags.sources.metadataService {
dss = append(dss, ec2.NewDatasource(ec2.DefaultAddress))
}
if flags.sources.ec2MetadataService != "" {
dss = append(dss, ec2.NewDatasource(flags.sources.ec2MetadataService))
}
if flags.sources.cloudSigmaMetadataService {
dss = append(dss, cloudsigma.NewServerContextService())
}
if flags.sources.digitalOceanMetadataService != "" {
dss = append(dss, digitalocean.NewDatasource(flags.sources.digitalOceanMetadataService))
}
if flags.sources.procCmdLine {
dss = append(dss, proc_cmdline.NewDatasource())
}
return dss
}
// selectDatasource attempts to choose a valid Datasource to use based on its
// current availability. The first Datasource to report to be available is
// returned. Datasources will be retried if possible if they are not
// immediately available. If all Datasources are permanently unavailable or
// datasourceTimeout is reached before one becomes available, nil is returned.
func selectDatasource(sources []datasource.Datasource) datasource.Datasource {
ds := make(chan datasource.Datasource)
stop := make(chan struct{})
var wg sync.WaitGroup
for _, s := range sources {
wg.Add(1)
go func(s datasource.Datasource) {
defer wg.Done()
duration := datasourceInterval
for {
fmt.Printf("Checking availability of %q\n", s.Type())
if s.IsAvailable() {
ds <- s
return
} else if !s.AvailabilityChanges() {
return
}
select {
case <-stop:
return
case <-time.After(duration):
duration = pkg.ExpBackoff(duration, datasourceMaxInterval)
}
}
}(s)
}
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
var s datasource.Datasource
select {
case s = <-ds:
case <-done:
case <-time.After(datasourceTimeout):
}
close(stop)
return s
}
// TODO(jonboulle): this should probably be refactored and moved into a different module
func runScript(script system.Script, env *initialize.Environment) error {
err := initialize.PrepWorkspace(env.Workspace())
if err != nil {
fmt.Printf("Failed preparing workspace: %v\n", err)
return err
}
path, err := initialize.PersistScriptInWorkspace(script, env.Workspace())
if err == nil {
var name string
name, err = system.ExecuteScript(path)
initialize.PersistUnitNameInWorkspace(name, workspace)
}
}
if err != nil {
log.Fatalf("Failed resolving user-data: %v", err)
initialize.PersistUnitNameInWorkspace(name, env.Workspace())
}
return err
}

120
coreos-cloudinit_test.go Normal file
View File

@@ -0,0 +1,120 @@
package main
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestMergeCloudConfig(t *testing.T) {
simplecc := config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "foobar",
NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{}`,
}
for i, tt := range []struct {
udcc config.CloudConfig
mdcc config.CloudConfig
want config.CloudConfig
}{
{
// If mdcc is empty, udcc should be returned unchanged
simplecc,
config.CloudConfig{},
simplecc,
},
{
// If udcc is empty, mdcc should be returned unchanged(overridden)
config.CloudConfig{},
simplecc,
simplecc,
},
{
// user-data should override completely in the case of conflicts
simplecc,
config.CloudConfig{
Hostname: "meta-hostname",
NetworkConfigPath: "/path/meta",
NetworkConfig: `{"hostname":"test"}`,
},
simplecc,
},
{
// Mixed merge should succeed
config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def"},
Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{"hostname":"test"}`,
},
config.CloudConfig{
SSHAuthorizedKeys: []string{"woof", "qux"},
Hostname: "meta-hostname",
},
config.CloudConfig{
SSHAuthorizedKeys: []string{"abc", "def", "woof", "qux"},
Hostname: "user-hostname",
NetworkConfigPath: "/path/somewhere",
NetworkConfig: `{"hostname":"test"}`,
},
},
{
// Completely non-conflicting merge should be fine
config.CloudConfig{
Hostname: "supercool",
},
config.CloudConfig{
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun",
NetworkConfig: `{"hostname":"test"}`,
},
config.CloudConfig{
Hostname: "supercool",
SSHAuthorizedKeys: []string{"zaphod", "beeblebrox"},
NetworkConfigPath: "/dev/fun",
NetworkConfig: `{"hostname":"test"}`,
},
},
{
// Non-mergeable settings in user-data should not be affected
config.CloudConfig{
Hostname: "mememe",
ManageEtcHosts: config.EtcHosts("lolz"),
},
config.CloudConfig{
Hostname: "youyouyou",
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
config.CloudConfig{
Hostname: "mememe",
ManageEtcHosts: config.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
},
{
// Non-mergeable (unexpected) settings in meta-data are ignored
config.CloudConfig{
Hostname: "mememe",
},
config.CloudConfig{
ManageEtcHosts: config.EtcHosts("lolz"),
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
config.CloudConfig{
Hostname: "mememe",
NetworkConfigPath: "meta-meta-yo",
NetworkConfig: `{"hostname":"test"}`,
},
},
} {
got := mergeCloudConfig(tt.mdcc, tt.udcc)
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("case #%d: mergeCloudConfig mutated CloudConfig unexpectedly:\ngot:\n%s\nwant:\n%s", i, got, tt.want)
}
}
}

27
cover Executable file
View File

@@ -0,0 +1,27 @@
#!/bin/bash -e
#
# Generate coverage HTML for a package
# e.g. PKG=./initialize ./cover
#
if [ -z "$PKG" ]; then
echo "cover only works with a single package, sorry"
exit 255
fi
COVEROUT="coverage"
if ! [ -d "$COVEROUT" ]; then
mkdir "$COVEROUT"
fi
# strip out slashes and dots
COVERPKG=${PKG//\//}
COVERPKG=${COVERPKG//./}
# generate arg for "go test"
export COVER="-coverprofile ${COVEROUT}/${COVERPKG}.out"
source ./test
go tool cover -html=${COVEROUT}/${COVERPKG}.out

View File

@@ -0,0 +1,70 @@
package configdrive
import (
"fmt"
"io/ioutil"
"os"
"path"
)
const (
openstackApiVersion = "latest"
)
type configDrive struct {
root string
readFile func(filename string) ([]byte, error)
}
func NewDatasource(root string) *configDrive {
return &configDrive{root, ioutil.ReadFile}
}
func (cd *configDrive) IsAvailable() bool {
_, err := os.Stat(cd.root)
return !os.IsNotExist(err)
}
func (cd *configDrive) AvailabilityChanges() bool {
return true
}
func (cd *configDrive) ConfigRoot() string {
return cd.openstackRoot()
}
func (cd *configDrive) FetchMetadata() ([]byte, error) {
return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "meta_data.json"))
}
func (cd *configDrive) FetchUserdata() ([]byte, error) {
return cd.tryReadFile(path.Join(cd.openstackVersionRoot(), "user_data"))
}
func (cd *configDrive) FetchNetworkConfig(filename string) ([]byte, error) {
if filename == "" {
return []byte{}, nil
}
return cd.tryReadFile(path.Join(cd.openstackRoot(), filename))
}
func (cd *configDrive) Type() string {
return "cloud-drive"
}
func (cd *configDrive) openstackRoot() string {
return path.Join(cd.root, "openstack")
}
func (cd *configDrive) openstackVersionRoot() string {
return path.Join(cd.openstackRoot(), openstackApiVersion)
}
func (cd *configDrive) tryReadFile(filename string) ([]byte, error) {
fmt.Printf("Attempting to read from %q\n", filename)
data, err := cd.readFile(filename)
if os.IsNotExist(err) {
err = nil
}
return data, err
}

View File

@@ -0,0 +1,125 @@
package configdrive
import (
"os"
"testing"
)
type mockFilesystem []string
func (m mockFilesystem) readFile(filename string) ([]byte, error) {
for _, file := range m {
if file == filename {
return []byte(filename), nil
}
}
return nil, os.ErrNotExist
}
func TestFetchMetadata(t *testing.T) {
for _, tt := range []struct {
root string
filename string
files mockFilesystem
}{
{
"/",
"",
mockFilesystem{},
},
{
"/",
"/openstack/latest/meta_data.json",
mockFilesystem([]string{"/openstack/latest/meta_data.json"}),
},
{
"/media/configdrive",
"/media/configdrive/openstack/latest/meta_data.json",
mockFilesystem([]string{"/media/configdrive/openstack/latest/meta_data.json"}),
},
} {
cd := configDrive{tt.root, tt.files.readFile}
filename, err := cd.FetchMetadata()
if err != nil {
t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
}
if string(filename) != tt.filename {
t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
}
}
}
func TestFetchUserdata(t *testing.T) {
for _, tt := range []struct {
root string
filename string
files mockFilesystem
}{
{
"/",
"",
mockFilesystem{},
},
{
"/",
"/openstack/latest/user_data",
mockFilesystem([]string{"/openstack/latest/user_data"}),
},
{
"/media/configdrive",
"/media/configdrive/openstack/latest/user_data",
mockFilesystem([]string{"/media/configdrive/openstack/latest/user_data"}),
},
} {
cd := configDrive{tt.root, tt.files.readFile}
filename, err := cd.FetchUserdata()
if err != nil {
t.Fatalf("bad error for %q: want %q, got %q", tt, nil, err)
}
if string(filename) != tt.filename {
t.Fatalf("bad path for %q: want %q, got %q", tt, tt.filename, filename)
}
}
}
func TestConfigRoot(t *testing.T) {
for _, tt := range []struct {
root string
configRoot string
}{
{
"/",
"/openstack",
},
{
"/media/configdrive",
"/media/configdrive/openstack",
},
} {
cd := configDrive{tt.root, nil}
if configRoot := cd.ConfigRoot(); configRoot != tt.configRoot {
t.Fatalf("bad config root for %q: want %q, got %q", tt, tt.configRoot, configRoot)
}
}
}
func TestNewDatasource(t *testing.T) {
for _, tt := range []struct {
root string
expectRoot string
}{
{
root: "",
expectRoot: "",
},
{
root: "/media/configdrive",
expectRoot: "/media/configdrive",
},
} {
service := NewDatasource(tt.root)
if service.root != tt.expectRoot {
t.Fatalf("bad root (%q): want %q, got %q", tt.root, tt.expectRoot, service.root)
}
}
}

View File

@@ -1,31 +1,11 @@
package datasource
import (
"io/ioutil"
"net/http"
)
type Datasource interface {
Fetch() ([]byte, error)
IsAvailable() bool
AvailabilityChanges() bool
ConfigRoot() string
FetchMetadata() ([]byte, error)
FetchUserdata() ([]byte, error)
FetchNetworkConfig(string) ([]byte, error)
Type() string
}
func fetchURL(url string) ([]byte, error) {
client := http.Client{}
resp, err := client.Get(url)
if err != nil {
return []byte{}, err
}
defer resp.Body.Close()
if resp.StatusCode / 100 != 2 {
return []byte{}, nil
}
respBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return respBytes, nil
}

View File

@@ -1,21 +0,0 @@
package datasource
import (
"io/ioutil"
)
type localFile struct {
path string
}
func NewLocalFile(path string) *localFile {
return &localFile{path}
}
func (self *localFile) Fetch() ([]byte, error) {
return ioutil.ReadFile(self.path)
}
func (self *localFile) Type() string {
return "local-file"
}

43
datasource/file/file.go Normal file
View File

@@ -0,0 +1,43 @@
package file
import (
"io/ioutil"
"os"
)
type localFile struct {
path string
}
func NewDatasource(path string) *localFile {
return &localFile{path}
}
func (f *localFile) IsAvailable() bool {
_, err := os.Stat(f.path)
return !os.IsNotExist(err)
}
func (f *localFile) AvailabilityChanges() bool {
return true
}
func (f *localFile) ConfigRoot() string {
return ""
}
func (f *localFile) FetchMetadata() ([]byte, error) {
return []byte{}, nil
}
func (f *localFile) FetchUserdata() ([]byte, error) {
return ioutil.ReadFile(f.path)
}
func (f *localFile) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (f *localFile) Type() string {
return "local-file"
}

View File

@@ -0,0 +1,145 @@
package cloudsigma
import (
"encoding/base64"
"encoding/json"
"os"
"strings"
"github.com/coreos/coreos-cloudinit/third_party/github.com/cloudsigma/cepgo"
)
const (
userDataFieldName = "cloudinit-user-data"
)
type serverContextService struct {
client interface {
All() (interface{}, error)
Key(string) (interface{}, error)
Meta() (map[string]string, error)
FetchRaw(string) ([]byte, error)
}
}
func NewServerContextService() *serverContextService {
return &serverContextService{
client: cepgo.NewCepgo(),
}
}
func (_ *serverContextService) IsAvailable() bool {
productNameFile, err := os.Open("/sys/class/dmi/id/product_name")
if err != nil {
return false
}
productName := make([]byte, 10)
_, err = productNameFile.Read(productName)
return err == nil && string(productName) == "CloudSigma"
}
func (_ *serverContextService) AvailabilityChanges() bool {
return true
}
func (_ *serverContextService) ConfigRoot() string {
return ""
}
func (_ *serverContextService) Type() string {
return "server-context"
}
func (scs *serverContextService) FetchMetadata() ([]byte, error) {
var (
inputMetadata struct {
Name string `json:"name"`
UUID string `json:"uuid"`
Meta map[string]string `json:"meta"`
Nics []struct {
Runtime struct {
InterfaceType string `json:"interface_type"`
IPv4 struct {
IP string `json:"uuid"`
} `json:"ip_v4"`
} `json:"runtime"`
} `json:"nics"`
}
outputMetadata struct {
Hostname string `json:"name"`
PublicKeys map[string]string `json:"public_keys"`
LocalIPv4 string `json:"local-ipv4"`
PublicIPv4 string `json:"public-ipv4"`
}
)
rawMetadata, err := scs.client.FetchRaw("")
if err != nil {
return []byte{}, err
}
err = json.Unmarshal(rawMetadata, &inputMetadata)
if err != nil {
return []byte{}, err
}
if inputMetadata.Name != "" {
outputMetadata.Hostname = inputMetadata.Name
} else {
outputMetadata.Hostname = inputMetadata.UUID
}
if key, ok := inputMetadata.Meta["ssh_public_key"]; ok {
splitted := strings.Split(key, " ")
outputMetadata.PublicKeys = make(map[string]string)
outputMetadata.PublicKeys[splitted[len(splitted)-1]] = key
}
for _, nic := range inputMetadata.Nics {
if nic.Runtime.IPv4.IP != "" {
if nic.Runtime.InterfaceType == "public" {
outputMetadata.PublicIPv4 = nic.Runtime.IPv4.IP
} else {
outputMetadata.LocalIPv4 = nic.Runtime.IPv4.IP
}
}
}
return json.Marshal(outputMetadata)
}
func (scs *serverContextService) FetchUserdata() ([]byte, error) {
metadata, err := scs.client.Meta()
if err != nil {
return []byte{}, err
}
userData, ok := metadata[userDataFieldName]
if ok && isBase64Encoded(userDataFieldName, metadata) {
if decodedUserData, err := base64.StdEncoding.DecodeString(userData); err == nil {
return decodedUserData, nil
} else {
return []byte{}, nil
}
}
return []byte(userData), nil
}
func (scs *serverContextService) FetchNetworkConfig(a string) ([]byte, error) {
return nil, nil
}
func isBase64Encoded(field string, userdata map[string]string) bool {
base64Fields, ok := userdata["base64_fields"]
if !ok {
return false
}
for _, base64Field := range strings.Split(base64Fields, ",") {
if field == base64Field {
return true
}
}
return false
}

View File

@@ -0,0 +1,152 @@
package cloudsigma
import (
"encoding/json"
"reflect"
"testing"
)
type fakeCepgoClient struct {
raw []byte
meta map[string]string
keys map[string]interface{}
err error
}
func (f *fakeCepgoClient) All() (interface{}, error) {
return f.keys, f.err
}
func (f *fakeCepgoClient) Key(key string) (interface{}, error) {
return f.keys[key], f.err
}
func (f *fakeCepgoClient) Meta() (map[string]string, error) {
return f.meta, f.err
}
func (f *fakeCepgoClient) FetchRaw(key string) ([]byte, error) {
return f.raw, f.err
}
func TestServerContextFetchMetadata(t *testing.T) {
var metadata struct {
Hostname string `json:"name"`
PublicKeys map[string]string `json:"public_keys"`
LocalIPv4 string `json:"local-ipv4"`
PublicIPv4 string `json:"public-ipv4"`
}
client := new(fakeCepgoClient)
scs := NewServerContextService()
scs.client = client
client.raw = []byte(`{
"context": true,
"cpu": 4000,
"cpu_model": null,
"cpus_instead_of_cores": false,
"enable_numa": false,
"grantees": [],
"hv_relaxed": false,
"hv_tsc": false,
"jobs": [],
"mem": 4294967296,
"meta": {
"base64_fields": "cloudinit-user-data",
"cloudinit-user-data": "I2Nsb3VkLWNvbmZpZwoKaG9zdG5hbWU6IGNvcmVvczE=",
"ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe"
},
"name": "coreos",
"nics": [
{
"runtime": {
"interface_type": "public",
"ip_v4": {
"uuid": "31.171.251.74"
},
"ip_v6": null
},
"vlan": null
}
],
"smp": 2,
"status": "running",
"uuid": "20a0059b-041e-4d0c-bcc6-9b2852de48b3"
}`)
metadataBytes, err := scs.FetchMetadata()
if err != nil {
t.Error(err.Error())
}
if err := json.Unmarshal(metadataBytes, &metadata); err != nil {
t.Error(err.Error())
}
if metadata.Hostname != "coreos" {
t.Errorf("Hostname is not 'coreos' but %s instead", metadata.Hostname)
}
if metadata.PublicKeys["john@doe"] != "ssh-rsa AAAAB3NzaC1yc2E.../hQ5D5 john@doe" {
t.Error("Public SSH Keys are not being read properly")
}
if metadata.LocalIPv4 != "" {
t.Errorf("Local IP is not empty but %s instead", metadata.LocalIPv4)
}
if metadata.PublicIPv4 != "31.171.251.74" {
t.Errorf("Local IP is not 31.171.251.74 but %s instead", metadata.PublicIPv4)
}
}
func TestServerContextFetchUserdata(t *testing.T) {
client := new(fakeCepgoClient)
scs := NewServerContextService()
scs.client = client
userdataSets := []struct {
in map[string]string
err bool
out []byte
}{
{map[string]string{
"base64_fields": "cloudinit-user-data",
"cloudinit-user-data": "aG9zdG5hbWU6IGNvcmVvc190ZXN0",
}, false, []byte("hostname: coreos_test")},
{map[string]string{
"cloudinit-user-data": "#cloud-config\\nhostname: coreos1",
}, false, []byte("#cloud-config\\nhostname: coreos1")},
{map[string]string{}, false, []byte{}},
}
for i, set := range userdataSets {
client.meta = set.in
got, err := scs.FetchUserdata()
if (err != nil) != set.err {
t.Errorf("case %d: bad error state (got %t, want %t)", i, err != nil, set.err)
}
if !reflect.DeepEqual(got, set.out) {
t.Errorf("case %d: got %s, want %s", i, got, set.out)
}
}
}
func TestServerContextDecodingBase64UserData(t *testing.T) {
base64Sets := []struct {
in string
out bool
}{
{"cloudinit-user-data,foo,bar", true},
{"bar,cloudinit-user-data,foo,bar", true},
{"cloudinit-user-data", true},
{"", false},
{"foo", false},
}
for _, set := range base64Sets {
userdata := map[string]string{"base64_fields": set.in}
if isBase64Encoded("cloudinit-user-data", userdata) != set.out {
t.Errorf("isBase64Encoded(cloudinit-user-data, %s) should be %t", userdata, set.out)
}
}
}

View File

@@ -0,0 +1,107 @@
package digitalocean
import (
"encoding/json"
"strconv"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
)
const (
DefaultAddress = "http://169.254.169.254/"
apiVersion = "metadata/v1"
userdataUrl = apiVersion + "/user-data"
metadataPath = apiVersion + ".json"
)
type Address struct {
IPAddress string `json:"ip_address"`
Netmask string `json:"netmask"`
Cidr int `json:"cidr"`
Gateway string `json:"gateway"`
}
type Interface struct {
IPv4 *Address `json:"ipv4"`
IPv6 *Address `json:"ipv6"`
MAC string `json:"mac"`
Type string `json:"type"`
}
type Interfaces struct {
Public []Interface `json:"public"`
Private []Interface `json:"private"`
}
type DNS struct {
Nameservers []string `json:"nameservers"`
}
type Metadata struct {
Hostname string `json:"hostname"`
Interfaces Interfaces `json:"interfaces"`
PublicKeys []string `json:"public_keys"`
DNS DNS `json:"dns"`
}
type metadataService struct {
interfaces Interfaces
dns DNS
metadata.MetadataService
}
func NewDatasource(root string) *metadataService {
return &metadataService{MetadataService: metadata.NewDatasource(root, apiVersion, userdataUrl, metadataPath)}
}
func (ms *metadataService) FetchMetadata() ([]byte, error) {
data, err := ms.FetchData(ms.MetadataUrl())
if err != nil || len(data) == 0 {
return []byte{}, err
}
var metadata Metadata
if err := json.Unmarshal(data, &metadata); err != nil {
return []byte{}, err
}
ms.interfaces = metadata.Interfaces
ms.dns = metadata.DNS
attrs := make(map[string]interface{})
if len(metadata.Interfaces.Public) > 0 {
if metadata.Interfaces.Public[0].IPv4 != nil {
attrs["public-ipv4"] = metadata.Interfaces.Public[0].IPv4.IPAddress
}
if metadata.Interfaces.Public[0].IPv6 != nil {
attrs["public-ipv6"] = metadata.Interfaces.Public[0].IPv6.IPAddress
}
}
if len(metadata.Interfaces.Private) > 0 {
if metadata.Interfaces.Private[0].IPv4 != nil {
attrs["local-ipv4"] = metadata.Interfaces.Private[0].IPv4.IPAddress
}
if metadata.Interfaces.Private[0].IPv6 != nil {
attrs["local-ipv6"] = metadata.Interfaces.Private[0].IPv6.IPAddress
}
}
attrs["hostname"] = metadata.Hostname
keys := make(map[string]string)
for i, key := range metadata.PublicKeys {
keys[strconv.Itoa(i)] = key
}
attrs["public_keys"] = keys
return json.Marshal(attrs)
}
func (ms metadataService) FetchNetworkConfig(filename string) ([]byte, error) {
return json.Marshal(Metadata{
Interfaces: ms.interfaces,
DNS: ms.dns,
})
}
func (ms metadataService) Type() string {
return "digitalocean-metadata-service"
}

View File

@@ -0,0 +1,99 @@
package digitalocean
import (
"bytes"
"fmt"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg"
)
func TestType(t *testing.T) {
want := "digitalocean-metadata-service"
if kind := (metadataService{}).Type(); kind != want {
t.Fatalf("bad type: want %q, got %q", want, kind)
}
}
func TestFetchMetadata(t *testing.T) {
for _, tt := range []struct {
root string
metadataPath string
resources map[string]string
expect []byte
clientErr error
expectErr error
}{
{
root: "/",
metadataPath: "v1.json",
resources: map[string]string{
"/v1.json": "bad",
},
expectErr: fmt.Errorf("invalid character 'b' looking for beginning of value"),
},
{
root: "/",
metadataPath: "v1.json",
resources: map[string]string{
"/v1.json": `{
"droplet_id": 1,
"user_data": "hello",
"vendor_data": "hello",
"public_keys": [
"publickey1",
"publickey2"
],
"region": "nyc2",
"interfaces": {
"public": [
{
"ipv4": {
"ip_address": "192.168.1.2",
"netmask": "255.255.255.0",
"gateway": "192.168.1.1"
},
"ipv6": {
"ip_address": "fe00::",
"cidr": 126,
"gateway": "fe00::"
},
"mac": "ab:cd:ef:gh:ij",
"type": "public"
}
]
}
}`,
},
expect: []byte(`{"hostname":"","public-ipv4":"192.168.1.2","public-ipv6":"fe00::","public_keys":{"0":"publickey1","1":"publickey2"}}`),
},
{
clientErr: pkg.ErrTimeout{fmt.Errorf("test error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
},
} {
service := &metadataService{
MetadataService: metadata.MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
MetadataPath: tt.metadataPath,
},
}
metadata, err := service.FetchMetadata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(metadata, tt.expect) {
t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
}
}
}
func Error(err error) string {
if err != nil {
return err.Error()
}
return ""
}

View File

@@ -0,0 +1,107 @@
package ec2
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"strings"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/pkg"
)
const (
DefaultAddress = "http://169.254.169.254/"
apiVersion = "2009-04-04/"
userdataPath = apiVersion + "user-data"
metadataPath = apiVersion + "meta-data"
)
type metadataService struct {
metadata.MetadataService
}
func NewDatasource(root string) *metadataService {
return &metadataService{metadata.NewDatasource(root, apiVersion, userdataPath, metadataPath)}
}
func (ms metadataService) FetchMetadata() ([]byte, error) {
attrs := make(map[string]interface{})
if keynames, err := ms.fetchAttributes(fmt.Sprintf("%s/public-keys", ms.MetadataUrl())); err == nil {
keyIDs := make(map[string]string)
for _, keyname := range keynames {
tokens := strings.SplitN(keyname, "=", 2)
if len(tokens) != 2 {
return nil, fmt.Errorf("malformed public key: %q", keyname)
}
keyIDs[tokens[1]] = tokens[0]
}
keys := make(map[string]string)
for name, id := range keyIDs {
sshkey, err := ms.fetchAttribute(fmt.Sprintf("%s/public-keys/%s/openssh-key", ms.MetadataUrl(), id))
if err != nil {
return nil, err
}
keys[name] = sshkey
fmt.Printf("Found SSH key for %q\n", name)
}
attrs["public_keys"] = keys
} else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err
}
if hostname, err := ms.fetchAttribute(fmt.Sprintf("%s/hostname", ms.MetadataUrl())); err == nil {
attrs["hostname"] = hostname
} else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err
}
if localAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/local-ipv4", ms.MetadataUrl())); err == nil {
attrs["local-ipv4"] = localAddr
} else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err
}
if publicAddr, err := ms.fetchAttribute(fmt.Sprintf("%s/public-ipv4", ms.MetadataUrl())); err == nil {
attrs["public-ipv4"] = publicAddr
} else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err
}
if content_path, err := ms.fetchAttribute(fmt.Sprintf("%s/network_config/content_path", ms.MetadataUrl())); err == nil {
attrs["network_config"] = map[string]string{
"content_path": content_path,
}
} else if _, ok := err.(pkg.ErrNotFound); !ok {
return nil, err
}
return json.Marshal(attrs)
}
func (ms metadataService) Type() string {
return "ec2-metadata-service"
}
func (ms metadataService) fetchAttributes(url string) ([]string, error) {
resp, err := ms.FetchData(url)
if err != nil {
return nil, err
}
scanner := bufio.NewScanner(bytes.NewBuffer(resp))
data := make([]string, 0)
for scanner.Scan() {
data = append(data, scanner.Text())
}
return data, scanner.Err()
}
func (ms metadataService) fetchAttribute(url string) (string, error) {
if attrs, err := ms.fetchAttributes(url); err == nil && len(attrs) > 0 {
return attrs[0], nil
} else {
return "", err
}
}

View File

@@ -0,0 +1,185 @@
package ec2
import (
"bytes"
"fmt"
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg"
)
func TestType(t *testing.T) {
want := "ec2-metadata-service"
if kind := (metadataService{}).Type(); kind != want {
t.Fatalf("bad type: want %q, got %q", want, kind)
}
}
func TestFetchAttributes(t *testing.T) {
for _, s := range []struct {
resources map[string]string
err error
tests []struct {
path string
val []string
}
}{
{
resources: map[string]string{
"/": "a\nb\nc/",
"/c/": "d\ne/",
"/c/e/": "f",
"/a": "1",
"/b": "2",
"/c/d": "3",
"/c/e/f": "4",
},
tests: []struct {
path string
val []string
}{
{"/", []string{"a", "b", "c/"}},
{"/b", []string{"2"}},
{"/c/d", []string{"3"}},
{"/c/e/", []string{"f"}},
},
},
{
err: fmt.Errorf("test error"),
tests: []struct {
path string
val []string
}{
{"", nil},
},
},
} {
service := metadataService{metadata.MetadataService{
Client: &test.HttpClient{s.resources, s.err},
}}
for _, tt := range s.tests {
attrs, err := service.fetchAttributes(tt.path)
if err != s.err {
t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err)
}
if !reflect.DeepEqual(attrs, tt.val) {
t.Fatalf("bad fetch for %q (%q): want %q, got %q", tt.path, s.resources, tt.val, attrs)
}
}
}
}
func TestFetchAttribute(t *testing.T) {
for _, s := range []struct {
resources map[string]string
err error
tests []struct {
path string
val string
}
}{
{
resources: map[string]string{
"/": "a\nb\nc/",
"/c/": "d\ne/",
"/c/e/": "f",
"/a": "1",
"/b": "2",
"/c/d": "3",
"/c/e/f": "4",
},
tests: []struct {
path string
val string
}{
{"/a", "1"},
{"/b", "2"},
{"/c/d", "3"},
{"/c/e/f", "4"},
},
},
{
err: fmt.Errorf("test error"),
tests: []struct {
path string
val string
}{
{"", ""},
},
},
} {
service := metadataService{metadata.MetadataService{
Client: &test.HttpClient{s.resources, s.err},
}}
for _, tt := range s.tests {
attr, err := service.fetchAttribute(tt.path)
if err != s.err {
t.Fatalf("bad error for %q (%q): want %q, got %q", tt.path, s.resources, s.err, err)
}
if attr != tt.val {
t.Fatalf("bad fetch for %q (%q): want %q, got %q", tt.path, s.resources, tt.val, attr)
}
}
}
}
func TestFetchMetadata(t *testing.T) {
for _, tt := range []struct {
root string
metadataPath string
resources map[string]string
expect []byte
clientErr error
expectErr error
}{
{
root: "/",
metadataPath: "2009-04-04/meta-data",
resources: map[string]string{
"/2009-04-04/meta-data/public-keys": "bad\n",
},
expectErr: fmt.Errorf("malformed public key: \"bad\""),
},
{
root: "/",
metadataPath: "2009-04-04/meta-data",
resources: map[string]string{
"/2009-04-04/meta-data/hostname": "host",
"/2009-04-04/meta-data/local-ipv4": "1.2.3.4",
"/2009-04-04/meta-data/public-ipv4": "5.6.7.8",
"/2009-04-04/meta-data/public-keys": "0=test1\n",
"/2009-04-04/meta-data/public-keys/0": "openssh-key",
"/2009-04-04/meta-data/public-keys/0/openssh-key": "key",
"/2009-04-04/meta-data/network_config/content_path": "path",
},
expect: []byte(`{"hostname":"host","local-ipv4":"1.2.3.4","network_config":{"content_path":"path"},"public-ipv4":"5.6.7.8","public_keys":{"test1":"key"}}`),
},
{
clientErr: pkg.ErrTimeout{fmt.Errorf("test error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test error")},
},
} {
service := &metadataService{metadata.MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
MetadataPath: tt.metadataPath,
}}
metadata, err := service.FetchMetadata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(metadata, tt.expect) {
t.Fatalf("bad fetch (%q): want %q, got %q", tt.resources, tt.expect, metadata)
}
}
}
func Error(err error) string {
if err != nil {
return err.Error()
}
return ""
}

View File

@@ -0,0 +1,61 @@
package metadata
import (
"strings"
"github.com/coreos/coreos-cloudinit/pkg"
)
type MetadataService struct {
Root string
Client pkg.Getter
ApiVersion string
UserdataPath string
MetadataPath string
}
func NewDatasource(root, apiVersion, userdataPath, metadataPath string) MetadataService {
if !strings.HasSuffix(root, "/") {
root += "/"
}
return MetadataService{root, pkg.NewHttpClient(), apiVersion, userdataPath, metadataPath}
}
func (ms MetadataService) IsAvailable() bool {
_, err := ms.Client.Get(ms.Root + ms.ApiVersion)
return (err == nil)
}
func (ms MetadataService) AvailabilityChanges() bool {
return true
}
func (ms MetadataService) ConfigRoot() string {
return ms.Root
}
func (ms MetadataService) FetchUserdata() ([]byte, error) {
return ms.FetchData(ms.UserdataUrl())
}
func (ms MetadataService) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (ms MetadataService) FetchData(url string) ([]byte, error) {
if data, err := ms.Client.GetRetry(url); err == nil {
return data, err
} else if _, ok := err.(pkg.ErrNotFound); ok {
return []byte{}, nil
} else {
return data, err
}
}
func (ms MetadataService) MetadataUrl() string {
return (ms.Root + ms.MetadataPath)
}
func (ms MetadataService) UserdataUrl() string {
return (ms.Root + ms.UserdataPath)
}

View File

@@ -0,0 +1,171 @@
package metadata
import (
"bytes"
"fmt"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata/test"
"github.com/coreos/coreos-cloudinit/pkg"
)
func TestAvailabilityChanges(t *testing.T) {
want := true
if ac := (MetadataService{}).AvailabilityChanges(); ac != want {
t.Fatalf("bad AvailabilityChanges: want %q, got %q", want, ac)
}
}
func TestIsAvailable(t *testing.T) {
for _, tt := range []struct {
root string
apiVersion string
resources map[string]string
expect bool
}{
{
root: "/",
apiVersion: "2009-04-04",
resources: map[string]string{
"/2009-04-04": "",
},
expect: true,
},
{
root: "/",
resources: map[string]string{},
expect: false,
},
} {
service := &MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, nil},
ApiVersion: tt.apiVersion,
}
if a := service.IsAvailable(); a != tt.expect {
t.Fatalf("bad isAvailable (%q): want %q, got %q", tt.resources, tt.expect, a)
}
}
}
func TestFetchUserdata(t *testing.T) {
for _, tt := range []struct {
root string
userdataPath string
resources map[string]string
userdata []byte
clientErr error
expectErr error
}{
{
root: "/",
userdataPath: "2009-04-04/user-data",
resources: map[string]string{
"/2009-04-04/user-data": "hello",
},
userdata: []byte("hello"),
},
{
root: "/",
clientErr: pkg.ErrNotFound{fmt.Errorf("test not found error")},
userdata: []byte{},
},
{
root: "/",
clientErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
expectErr: pkg.ErrTimeout{fmt.Errorf("test timeout error")},
},
} {
service := &MetadataService{
Root: tt.root,
Client: &test.HttpClient{tt.resources, tt.clientErr},
UserdataPath: tt.userdataPath,
}
data, err := service.FetchUserdata()
if Error(err) != Error(tt.expectErr) {
t.Fatalf("bad error (%q): want %q, got %q", tt.resources, tt.expectErr, err)
}
if !bytes.Equal(data, tt.userdata) {
t.Fatalf("bad userdata (%q): want %q, got %q", tt.resources, tt.userdata, data)
}
}
}
func TestUrls(t *testing.T) {
for _, tt := range []struct {
root string
userdataPath string
metadataPath string
expectRoot string
userdata string
metadata string
}{
{
root: "/",
userdataPath: "2009-04-04/user-data",
metadataPath: "2009-04-04/meta-data",
expectRoot: "/",
userdata: "/2009-04-04/user-data",
metadata: "/2009-04-04/meta-data",
},
{
root: "http://169.254.169.254/",
userdataPath: "2009-04-04/user-data",
metadataPath: "2009-04-04/meta-data",
expectRoot: "http://169.254.169.254/",
userdata: "http://169.254.169.254/2009-04-04/user-data",
metadata: "http://169.254.169.254/2009-04-04/meta-data",
},
} {
service := &MetadataService{
Root: tt.root,
UserdataPath: tt.userdataPath,
MetadataPath: tt.metadataPath,
}
if url := service.UserdataUrl(); url != tt.userdata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.userdata, url)
}
if url := service.MetadataUrl(); url != tt.metadata {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.metadata, url)
}
if url := service.ConfigRoot(); url != tt.expectRoot {
t.Fatalf("bad url (%q): want %q, got %q", tt.root, tt.expectRoot, url)
}
}
}
func TestNewDatasource(t *testing.T) {
for _, tt := range []struct {
root string
expectRoot string
}{
{
root: "",
expectRoot: "/",
},
{
root: "/",
expectRoot: "/",
},
{
root: "http://169.254.169.254",
expectRoot: "http://169.254.169.254/",
},
{
root: "http://169.254.169.254/",
expectRoot: "http://169.254.169.254/",
},
} {
service := NewDatasource(tt.root, "", "", "")
if service.Root != tt.expectRoot {
t.Fatalf("bad root (%q): want %q, got %q", tt.root, tt.expectRoot, service.Root)
}
}
}
func Error(err error) string {
if err != nil {
return err.Error()
}
return ""
}

View File

@@ -0,0 +1,27 @@
package test
import (
"fmt"
"github.com/coreos/coreos-cloudinit/pkg"
)
type HttpClient struct {
Resources map[string]string
Err error
}
func (t *HttpClient) GetRetry(url string) ([]byte, error) {
if t.Err != nil {
return nil, t.Err
}
if val, ok := t.Resources[url]; ok {
return []byte(val), nil
} else {
return nil, pkg.ErrNotFound{fmt.Errorf("not found: %q", url)}
}
}
func (t *HttpClient) Get(url string) ([]byte, error) {
return t.GetRetry(url)
}

View File

@@ -1,17 +0,0 @@
package datasource
type metadataService struct {
url string
}
func NewMetadataService(url string) *metadataService {
return &metadataService{url}
}
func (ms *metadataService) Fetch() ([]byte, error) {
return fetchURL(ms.url)
}
func (ms *metadataService) Type() string {
return "metadata-service"
}

View File

@@ -1,66 +0,0 @@
package datasource
import (
"errors"
"io/ioutil"
"log"
"strings"
)
const (
ProcCmdlineLocation = "/proc/cmdline"
ProcCmdlineCloudConfigFlag = "cloud-config-url"
)
type procCmdline struct{}
func NewProcCmdline() *procCmdline {
return &procCmdline{}
}
func (self *procCmdline) Fetch() ([]byte, error) {
cmdline, err := ioutil.ReadFile(ProcCmdlineLocation)
if err != nil {
return nil, err
}
url, err := findCloudConfigURL(string(cmdline))
if err != nil {
return nil, err
}
cfg, err := fetchURL(url)
if err != nil {
return nil, err
}
return cfg, nil
}
func (self *procCmdline) Type() string {
return "proc-cmdline"
}
func findCloudConfigURL(input string) (url string, err error) {
err = errors.New("cloud-config-url not found")
for _, token := range strings.Split(input, " ") {
parts := strings.SplitN(token, "=", 2)
key := parts[0]
key = strings.Replace(key, "_", "-", -1)
if key != "cloud-config-url" {
continue
}
if len(parts) != 2 {
log.Printf("Found cloud-config-url in /proc/cmdline with no value, ignoring.")
continue
}
url = parts[1]
err = nil
}
return
}

View File

@@ -0,0 +1,99 @@
package proc_cmdline
import (
"errors"
"io/ioutil"
"log"
"strings"
"github.com/coreos/coreos-cloudinit/pkg"
)
const (
ProcCmdlineLocation = "/proc/cmdline"
ProcCmdlineCloudConfigFlag = "cloud-config-url"
)
type procCmdline struct {
Location string
}
func NewDatasource() *procCmdline {
return &procCmdline{Location: ProcCmdlineLocation}
}
func (c *procCmdline) IsAvailable() bool {
contents, err := ioutil.ReadFile(c.Location)
if err != nil {
return false
}
cmdline := strings.TrimSpace(string(contents))
_, err = findCloudConfigURL(cmdline)
return (err == nil)
}
func (c *procCmdline) AvailabilityChanges() bool {
return false
}
func (c *procCmdline) ConfigRoot() string {
return ""
}
func (c *procCmdline) FetchMetadata() ([]byte, error) {
return []byte{}, nil
}
func (c *procCmdline) FetchUserdata() ([]byte, error) {
contents, err := ioutil.ReadFile(c.Location)
if err != nil {
return nil, err
}
cmdline := strings.TrimSpace(string(contents))
url, err := findCloudConfigURL(cmdline)
if err != nil {
return nil, err
}
client := pkg.NewHttpClient()
cfg, err := client.GetRetry(url)
if err != nil {
return nil, err
}
return cfg, nil
}
func (c *procCmdline) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (c *procCmdline) Type() string {
return "proc-cmdline"
}
func findCloudConfigURL(input string) (url string, err error) {
err = errors.New("cloud-config-url not found")
for _, token := range strings.Split(input, " ") {
parts := strings.SplitN(token, "=", 2)
key := parts[0]
key = strings.Replace(key, "_", "-", -1)
if key != "cloud-config-url" {
continue
}
if len(parts) != 2 {
log.Printf("Found cloud-config-url in /proc/cmdline with no value, ignoring.")
continue
}
url = parts[1]
err = nil
}
return
}

View File

@@ -0,0 +1,88 @@
package proc_cmdline
import (
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"os"
"testing"
)
func TestParseCmdlineCloudConfigFound(t *testing.T) {
tests := []struct {
input string
expect string
}{
{
"cloud-config-url=example.com",
"example.com",
},
{
"cloud_config_url=example.com",
"example.com",
},
{
"cloud-config-url cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url= cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url=one.example.com cloud-config-url=two.example.com",
"two.example.com",
},
{
"foo=bar cloud-config-url=example.com ping=pong",
"example.com",
},
}
for i, tt := range tests {
output, err := findCloudConfigURL(tt.input)
if output != tt.expect {
t.Errorf("Test case %d failed: %s != %s", i, output, tt.expect)
}
if err != nil {
t.Errorf("Test case %d produced error: %v", i, err)
}
}
}
func TestProcCmdlineAndFetchConfig(t *testing.T) {
var (
ProcCmdlineTmpl = "foo=bar cloud-config-url=%s/config\n"
CloudConfigContent = "#cloud-config\n"
)
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" && r.RequestURI == "/config" {
fmt.Fprint(w, CloudConfigContent)
}
}))
defer ts.Close()
file, err := ioutil.TempFile(os.TempDir(), "test_proc_cmdline")
defer os.Remove(file.Name())
if err != nil {
t.Errorf("Test produced error: %v", err)
}
_, err = file.Write([]byte(fmt.Sprintf(ProcCmdlineTmpl, ts.URL)))
if err != nil {
t.Errorf("Test produced error: %v", err)
}
p := NewDatasource()
p.Location = file.Name()
cfg, err := p.FetchUserdata()
if err != nil {
t.Errorf("Test produced error: %v", err)
}
if string(cfg) != CloudConfigContent {
t.Errorf("Test failed, response body: %s != %s", cfg, CloudConfigContent)
}
}

View File

@@ -1,47 +0,0 @@
package datasource
import (
"testing"
)
func TestParseCmdlineCloudConfigFound(t *testing.T) {
tests := []struct {
input string
expect string
}{
{
"cloud-config-url=example.com",
"example.com",
},
{
"cloud_config_url=example.com",
"example.com",
},
{
"cloud-config-url cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url= cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url=one.example.com cloud-config-url=two.example.com",
"two.example.com",
},
{
"foo=bar cloud-config-url=example.com ping=pong",
"example.com",
},
}
for i, tt := range tests {
output, err := findCloudConfigURL(tt.input)
if output != tt.expect {
t.Errorf("Test case %d failed: %s != %s", i, output, tt.expect)
}
if err != nil {
t.Errorf("Test case %d produced error: %v", i, err)
}
}
}

44
datasource/url/url.go Normal file
View File

@@ -0,0 +1,44 @@
package url
import (
"github.com/coreos/coreos-cloudinit/pkg"
)
type remoteFile struct {
url string
}
func NewDatasource(url string) *remoteFile {
return &remoteFile{url}
}
func (f *remoteFile) IsAvailable() bool {
client := pkg.NewHttpClient()
_, err := client.Get(f.url)
return (err == nil)
}
func (f *remoteFile) AvailabilityChanges() bool {
return true
}
func (f *remoteFile) ConfigRoot() string {
return ""
}
func (f *remoteFile) FetchMetadata() ([]byte, error) {
return []byte{}, nil
}
func (f *remoteFile) FetchUserdata() ([]byte, error) {
client := pkg.NewHttpClient()
return client.GetRetry(f.url)
}
func (f *remoteFile) FetchNetworkConfig(filename string) ([]byte, error) {
return nil, nil
}
func (f *remoteFile) Type() string {
return "url"
}

View File

@@ -1,48 +1,34 @@
package initialize
import (
"errors"
"fmt"
"log"
"path"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/network"
"github.com/coreos/coreos-cloudinit/system"
)
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd EtcdEnvironment
Update map[string]string
Units []system.Unit
OEM OEMRelease
}
WriteFiles []system.File `yaml:"write_files"`
Hostname string
Users []system.User
ManageEtcHosts string `yaml:"manage_etc_hosts"`
// CloudConfigFile represents a CoreOS specific configuration option that can generate
// an associated system.File to be written to disk
type CloudConfigFile interface {
// File should either return (*system.File, error), or (nil, nil) if nothing
// needs to be done for this configuration option.
File() (*system.File, error)
}
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal([]byte(contents), &cfg)
return &cfg, err
// CloudConfigUnit represents a CoreOS specific configuration option that can generate
// associated system.Units to be created/enabled appropriately
type CloudConfigUnit interface {
Units() ([]system.Unit, error)
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
func Apply(cfg CloudConfig, env *Environment) error {
// Apply renders a CloudConfig to an Environment. This can involve things like
// configuring the hostname, adding new users, writing various configuration
// files to disk, and manipulating systemd services.
func Apply(cfg config.CloudConfig, env *Environment) error {
if cfg.Hostname != "" {
if err := system.SetHostname(cfg.Hostname); err != nil {
return err
@@ -50,14 +36,6 @@ func Apply(cfg CloudConfig, env *Environment) error {
log.Printf("Set hostname to %s", cfg.Hostname)
}
if cfg.Coreos.OEM.ID != "" {
if err := WriteOEMRelease(&cfg.Coreos.OEM, env.Root()); err != nil {
return err
}
log.Printf("Wrote /etc/oem-release to filesystem")
}
if len(cfg.Users) > 0 {
for _, user := range cfg.Users {
if user.Name == "" {
log.Printf("User object has no 'name' field, skipping")
@@ -100,7 +78,6 @@ func Apply(cfg CloudConfig, env *Environment) error {
}
}
}
}
if len(cfg.SSHAuthorizedKeys) > 0 {
err := system.AuthorizeSSHKeys("core", env.SSHKeyName(), cfg.SSHAuthorizedKeys)
@@ -111,47 +88,132 @@ func Apply(cfg CloudConfig, env *Environment) error {
}
}
if len(cfg.WriteFiles) > 0 {
var writeFiles []system.File
for _, file := range cfg.WriteFiles {
file.Path = path.Join(env.Root(), file.Path)
if err := system.WriteFile(&file); err != nil {
writeFiles = append(writeFiles, system.File{file})
}
for _, ccf := range []CloudConfigFile{
system.OEM{cfg.Coreos.OEM},
system.Update{cfg.Coreos.Update, system.DefaultReadConfig},
system.EtcHosts{cfg.ManageEtcHosts},
} {
f, err := ccf.File()
if err != nil {
return err
}
log.Printf("Wrote file %s to filesystem", file.Path)
if f != nil {
writeFiles = append(writeFiles, *f)
}
}
if len(cfg.Coreos.Etcd) > 0 {
if err := WriteEtcdEnvironment(cfg.Coreos.Etcd, env.Root()); err != nil {
log.Fatalf("Failed to write etcd config to filesystem: %v", err)
var units []system.Unit
for _, u := range cfg.Coreos.Units {
units = append(units, system.Unit{u})
}
log.Printf("Wrote etcd config file to filesystem")
for _, ccu := range []CloudConfigUnit{
system.Etcd{cfg.Coreos.Etcd},
system.Fleet{cfg.Coreos.Fleet},
system.Update{cfg.Coreos.Update, system.DefaultReadConfig},
} {
u, err := ccu.Units()
if err != nil {
return err
}
units = append(units, u...)
}
if s, ok := cfg.Coreos.Update["reboot-strategy"]; ok {
if err := WriteLocksmithConfig(s, env.Root()); err != nil {
log.Fatalf("Failed to write locksmith config to filesystem: %v", err)
wroteEnvironment := false
for _, file := range writeFiles {
fullPath, err := system.WriteFile(&file, env.Root())
if err != nil {
return err
}
log.Printf("Wrote locksmith config file to filesystem")
if path.Clean(file.Path) == "/etc/environment" {
wroteEnvironment = true
}
log.Printf("Wrote file %s to filesystem", fullPath)
}
if len(cfg.Coreos.Units) > 0 {
commands := make(map[string]string, 0)
for _, unit := range cfg.Coreos.Units {
dst := system.UnitDestination(&unit, env.Root())
if !wroteEnvironment {
ef := env.DefaultEnvironmentFile()
if ef != nil {
err := system.WriteEnvFile(ef, env.Root())
if err != nil {
return err
}
log.Printf("Updated /etc/environment")
}
}
if env.NetconfType() != "" {
var interfaces []network.InterfaceGenerator
var err error
switch env.NetconfType() {
case "debian":
interfaces, err = network.ProcessDebianNetconf(cfg.NetworkConfig)
case "digitalocean":
interfaces, err = network.ProcessDigitalOceanNetconf(cfg.NetworkConfig)
default:
return fmt.Errorf("Unsupported network config format %q", env.NetconfType())
}
if err != nil {
return err
}
if err := system.WriteNetworkdConfigs(interfaces); err != nil {
return err
}
if err := system.RestartNetwork(interfaces); err != nil {
return err
}
}
um := system.NewUnitManager(env.Root())
return processUnits(units, env.Root(), um)
}
// processUnits takes a set of Units and applies them to the given root using
// the given UnitManager. This can involve things like writing unit files to
// disk, masking/unmasking units, or invoking systemd
// commands against units. It returns any error encountered.
func processUnits(units []system.Unit, root string, um system.UnitManager) error {
type action struct {
unit string
command string
}
actions := make([]action, 0, len(units))
reload := false
for _, unit := range units {
dst := unit.Destination(root)
if unit.Content != "" {
log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
if err := system.PlaceUnit(&unit, dst); err != nil {
if err := um.PlaceUnit(&unit, dst); err != nil {
return err
}
log.Printf("Placed unit %s at %s", unit.Name, dst)
reload = true
}
if unit.Mask {
log.Printf("Masking unit file %s", unit.Name)
if err := um.MaskUnit(&unit); err != nil {
return err
}
} else if unit.Runtime {
log.Printf("Ensuring runtime unit file %s is unmasked", unit.Name)
if err := um.UnmaskUnit(&unit); err != nil {
return err
}
}
if unit.Enable {
if unit.Group() != "network" {
log.Printf("Enabling unit file %s", dst)
if err := system.EnableUnitFile(dst, unit.Runtime); err != nil {
log.Printf("Enabling unit file %s", unit.Name)
if err := um.EnableUnitFile(unit.Name, unit.Runtime); err != nil {
return err
}
log.Printf("Enabled unit %s", unit.Name)
@@ -161,36 +223,25 @@ func Apply(cfg CloudConfig, env *Environment) error {
}
if unit.Group() == "network" {
commands["systemd-networkd.service"] = "restart"
} else {
if unit.Command != "" {
commands[unit.Name] = unit.Command
}
actions = append(actions, action{"systemd-networkd.service", "restart"})
} else if unit.Command != "" {
actions = append(actions, action{unit.Name, unit.Command})
}
}
if err := system.DaemonReload(); err != nil {
log.Fatalf("Failed systemd daemon-reload: %v", err)
if reload {
if err := um.DaemonReload(); err != nil {
return errors.New(fmt.Sprintf("failed systemd daemon-reload: %v", err))
}
}
for unit, command := range commands {
log.Printf("Calling unit command '%s %s'", command, unit)
res, err := system.RunUnitCommand(command, unit)
for _, action := range actions {
log.Printf("Calling unit command '%s %s'", action.command, action.unit)
res, err := um.RunUnitCommand(action.command, action.unit)
if err != nil {
return err
}
log.Printf("Result of '%s %s': %s", command, unit, res)
}
}
if cfg.ManageEtcHosts != "" {
if err := WriteEtcHosts(cfg.ManageEtcHosts, env.Root()); err != nil {
log.Fatalf("Failed to write /etc/hosts to filesystem: %v", err)
}
log.Printf("Wrote /etc/hosts file to filesystem")
log.Printf("Result of '%s %s': %s", action.command, action.unit, res)
}
return nil

View File

@@ -1,249 +1,114 @@
package initialize
import (
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
if cfg.Hostname != "" {
t.Errorf("Expected hostname to be empty, got '%s'", cfg.Hostname)
}
type TestUnitManager struct {
placed []string
enabled []string
masked []string
unmasked []string
commands map[string]string
reload bool
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot-strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
content: '[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
write_files:
- content: |
penny
elroy
path: /etc/dogepack.conf
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 2 {
t.Error("Parsed incorrect number of SSH keys")
} else if keys[0] != "foobar" {
t.Error("Expected first SSH key to be 'foobar'")
} else if keys[1] != "foobaz" {
t.Error("Expected first SSH key to be 'foobaz'")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
wf := cfg.WriteFiles[0]
if wf.Content != "penny\nelroy\n" {
t.Errorf("WriteFile has incorrect contents '%s'", wf.Content)
}
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if perm, _ := wf.Permissions(); perm != 0644 {
t.Errorf("WriteFile has incorrect permissions %s", perm)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
}
if wf.Owner != "root:dogepack" {
t.Errorf("WriteFile has incorrect owner %s", wf.Owner)
}
}
if len(cfg.Coreos.Units) != 1 {
t.Error("Failed to parse correct number of units")
} else {
u := cfg.Coreos.Units[0]
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if u.Content != expect {
t.Errorf("Unit has incorrect contents '%s'.\nExpected '%s'.", u.Content, expect)
}
if u.Runtime != true {
t.Errorf("Unit has incorrect runtime value")
}
if u.Name != "50-eth0.network" {
t.Errorf("Unit has incorrect name %s", u.Name)
}
if u.Type() != "network" {
t.Errorf("Unit has incorrect type '%s'", u.Type())
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.Coreos.Update["reboot-strategy"] != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
func (tum *TestUnitManager) PlaceUnit(unit *system.Unit, dst string) error {
tum.placed = append(tum.placed, unit.Name)
return nil
}
func (tum *TestUnitManager) EnableUnitFile(unit string, runtime bool) error {
tum.enabled = append(tum.enabled, unit)
return nil
}
func (tum *TestUnitManager) RunUnitCommand(command, unit string) (string, error) {
tum.commands = make(map[string]string)
tum.commands[unit] = command
return "", nil
}
func (tum *TestUnitManager) DaemonReload() error {
tum.reload = true
return nil
}
func (tum *TestUnitManager) MaskUnit(unit *system.Unit) error {
tum.masked = append(tum.masked, unit.Name)
return nil
}
func (tum *TestUnitManager) UnmaskUnit(unit *system.Unit) error {
tum.unmasked = append(tum.unmasked, unit.Name)
return nil
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := `
ssh_authorized_keys:
- foo: bar
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
func TestProcessUnits(t *testing.T) {
tum := &TestUnitManager{}
units := []system.Unit{
system.Unit{config.Unit{
Name: "foo",
Mask: true,
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
}
if len(tum.masked) != 1 || tum.masked[0] != "foo" {
t.Errorf("expected foo to be masked, but found %v", tum.masked)
}
keys := cfg.SSHAuthorizedKeys
if len(keys) != 0 {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
t.Fatalf("Serialized config did not have expected header")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := `
users:
- name: elroy
passwd: somehash
ssh-authorized-keys:
- somekey
gecos: arbitrary comment
homedir: /home/place
no-create-home: yes
primary-group: things
groups:
- ping
- pong
no-user-group: true
system: y
no-log-init: True
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.PasswordHash != "somehash" {
t.Errorf("User passwd is %q, expected 'somehash'", user.PasswordHash)
}
if keys := user.SSHAuthorizedKeys; len(keys) != 1 {
t.Errorf("Parsed %d ssh keys, expected 1", len(keys))
} else {
key := user.SSHAuthorizedKeys[0]
if key != "somekey" {
t.Errorf("User SSH key is %q, expected 'somekey'", key)
}
}
if user.GECOS != "arbitrary comment" {
t.Errorf("Failed to parse gecos field, got %q", user.GECOS)
}
if user.Homedir != "/home/place" {
t.Errorf("Failed to parse homedir field, got %q", user.Homedir)
}
if !user.NoCreateHome {
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}
if len(user.Groups) != 2 {
t.Errorf("Failed to parse 2 goups, got %d", len(user.Groups))
} else {
if user.Groups[0] != "ping" {
t.Errorf("First group was %q, not expected value 'ping'", user.Groups[0])
}
if user.Groups[1] != "pong" {
t.Errorf("First group was %q, not expected value 'pong'", user.Groups[1])
}
}
if !user.NoUserGroup {
t.Errorf("Failed to parse no-user-group field")
}
if !user.System {
t.Errorf("Failed to parse system field")
}
if !user.NoLogInit {
t.Errorf("Failed to parse no-log-init field")
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{config.Unit{
Name: "bar.network",
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
}
if _, ok := tum.commands["systemd-networkd.service"]; !ok {
t.Errorf("expected systemd-networkd.service to be reloaded!")
}
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{config.Unit{
Name: "baz.service",
Content: "[Service]\nExecStart=/bin/true",
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
}
if len(tum.placed) != 1 || tum.placed[0] != "baz.service" {
t.Fatalf("expected baz.service to be written, but got %v", tum.placed)
}
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{config.Unit{
Name: "locksmithd.service",
Runtime: true,
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
}
if len(tum.unmasked) != 1 || tum.unmasked[0] != "locksmithd.service" {
t.Fatalf("expected locksmithd.service to be unmasked, but got %v", tum.unmasked)
}
tum = &TestUnitManager{}
units = []system.Unit{
system.Unit{config.Unit{
Name: "woof",
Enable: true,
}},
}
if err := processUnits(units, "", tum); err != nil {
t.Fatalf("unexpected error calling processUnits: %v", err)
}
if len(tum.enabled) != 1 || tum.enabled[0] != "woof" {
t.Fatalf("expected woof to be enabled, but got %v", tum.enabled)
}
}

View File

@@ -3,45 +3,105 @@ package initialize
import (
"os"
"path"
"regexp"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
const DefaultSSHKeyName = "coreos-cloudinit"
type Environment struct {
root string
configRoot string
workspace string
netconfType string
sshKeyName string
substitutions map[string]string
}
func NewEnvironment(root, workspace string) *Environment {
substitutions := map[string]string{
// TODO(jonboulle): this is getting unwieldy, should be able to simplify the interface somehow
func NewEnvironment(root, configRoot, workspace, netconfType, sshKeyName string, substitutions map[string]string) *Environment {
if substitutions == nil {
substitutions = make(map[string]string)
}
// If certain values are not in the supplied substitution, fall back to retrieving them from the environment
for k, v := range map[string]string{
"$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"),
"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
"$public_ipv6": os.Getenv("COREOS_PUBLIC_IPV6"),
"$private_ipv6": os.Getenv("COREOS_PRIVATE_IPV6"),
} {
if _, ok := substitutions[k]; !ok {
substitutions[k] = v
}
return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
}
return &Environment{root, configRoot, workspace, netconfType, sshKeyName, substitutions}
}
func (self *Environment) Workspace() string {
return path.Join(self.root, self.workspace)
func (e *Environment) Workspace() string {
return path.Join(e.root, e.workspace)
}
func (self *Environment) Root() string {
return self.root
func (e *Environment) Root() string {
return e.root
}
func (self *Environment) SSHKeyName() string {
return self.sshKeyName
func (e *Environment) ConfigRoot() string {
return e.configRoot
}
func (self *Environment) SetSSHKeyName(name string) {
self.sshKeyName = name
func (e *Environment) NetconfType() string {
return e.netconfType
}
func (self *Environment) Apply(data string) string {
for key, val := range self.substitutions {
data = strings.Replace(data, key, val, -1)
func (e *Environment) SSHKeyName() string {
return e.sshKeyName
}
func (e *Environment) SetSSHKeyName(name string) {
e.sshKeyName = name
}
// Apply goes through the map of substitutions and replaces all instances of
// the keys with their respective values. It supports escaping substitutions
// with a leading '\'.
func (e *Environment) Apply(data string) string {
for key, val := range e.substitutions {
matchKey := strings.Replace(key, `$`, `\$`, -1)
replKey := strings.Replace(key, `$`, `$$`, -1)
// "key" -> "val"
data = regexp.MustCompile(`([^\\]|^)`+matchKey).ReplaceAllString(data, `${1}`+val)
// "\key" -> "key"
data = regexp.MustCompile(`\\`+matchKey).ReplaceAllString(data, replKey)
}
return data
}
func (e *Environment) DefaultEnvironmentFile() *system.EnvFile {
ef := system.EnvFile{
File: &system.File{config.File{
Path: "/etc/environment",
}},
Vars: map[string]string{},
}
if ip, ok := e.substitutions["$public_ipv4"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PUBLIC_IPV4"] = ip
}
if ip, ok := e.substitutions["$private_ipv4"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV4"] = ip
}
if ip, ok := e.substitutions["$public_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PUBLIC_IPV6"] = ip
}
if ip, ok := e.substitutions["$private_ipv6"]; ok && len(ip) > 0 {
ef.Vars["COREOS_PRIVATE_IPV6"] = ip
}
if len(ef.Vars) == 0 {
return nil
} else {
return &ef
}
}

View File

@@ -1,27 +1,132 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestEnvironmentApply(t *testing.T) {
os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
env := NewEnvironment("./", "./")
input := `[Service]
ExecStart=/usr/bin/echo "$public_ipv4"
ExecStop=/usr/bin/echo $private_ipv4
ExecStop=/usr/bin/echo $unknown
`
expected := `[Service]
ExecStart=/usr/bin/echo "192.0.2.3"
ExecStop=/usr/bin/echo 192.0.2.203
ExecStop=/usr/bin/echo $unknown
`
os.Setenv("COREOS_PUBLIC_IPV4", "1.2.3.4")
os.Setenv("COREOS_PRIVATE_IPV4", "5.6.7.8")
os.Setenv("COREOS_PUBLIC_IPV6", "1234::")
os.Setenv("COREOS_PRIVATE_IPV6", "5678::")
for _, tt := range []struct {
subs map[string]string
input string
out string
}{
{
// Substituting both values directly should always take precedence
// over environment variables
map[string]string{
"$public_ipv4": "192.0.2.3",
"$private_ipv4": "192.0.2.203",
"$public_ipv6": "fe00:1234::",
"$private_ipv6": "fe00:5678::",
},
`[Service]
ExecStart=/usr/bin/echo "$public_ipv4 $public_ipv6"
ExecStop=/usr/bin/echo $private_ipv4 $private_ipv6
ExecStop=/usr/bin/echo $unknown`,
`[Service]
ExecStart=/usr/bin/echo "192.0.2.3 fe00:1234::"
ExecStop=/usr/bin/echo 192.0.2.203 fe00:5678::
ExecStop=/usr/bin/echo $unknown`,
},
{
// Substituting one value directly while falling back with the other
map[string]string{"$private_ipv4": "127.0.0.1"},
"$private_ipv4\n$public_ipv4",
"127.0.0.1\n1.2.3.4",
},
{
// Falling back to environment variables for both values
map[string]string{"foo": "bar"},
"$private_ipv4\n$public_ipv4",
"5.6.7.8\n1.2.3.4",
},
{
// No substitutions
nil,
"$private_ipv4\nfoobar",
"5.6.7.8\nfoobar",
},
{
// Escaping substitutions
map[string]string{"$private_ipv4": "127.0.0.1"},
`\$private_ipv4
$private_ipv4
addr: \$private_ipv4
\\$private_ipv4`,
`$private_ipv4
127.0.0.1
addr: $private_ipv4
\$private_ipv4`,
},
{
// No substitutions with escaping
nil,
"\\$test\n$test",
"\\$test\n$test",
},
} {
output := env.Apply(input)
if output != expected {
t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
env := NewEnvironment("./", "./", "./", "", "", tt.subs)
got := env.Apply(tt.input)
if got != tt.out {
t.Fatalf("Environment incorrectly applied.\ngot:\n%s\nwant:\n%s", got, tt.out)
}
}
}
func TestEnvironmentFile(t *testing.T) {
subs := map[string]string{
"$public_ipv4": "1.2.3.4",
"$private_ipv4": "5.6.7.8",
"$public_ipv6": "1234::",
"$private_ipv6": "5678::",
}
expect := "COREOS_PRIVATE_IPV4=5.6.7.8\nCOREOS_PRIVATE_IPV6=5678::\nCOREOS_PUBLIC_IPV4=1.2.3.4\nCOREOS_PUBLIC_IPV6=1234::\n"
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
env := NewEnvironment("./", "./", "./", "", "", subs)
ef := env.DefaultEnvironmentFile()
err = system.WriteEnvFile(ef, dir)
if err != nil {
t.Fatalf("WriteEnvFile failed: %v", err)
}
fullPath := path.Join(dir, "etc", "environment")
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expect {
t.Fatalf("File has incorrect contents: %q", contents)
}
}
func TestEnvironmentFileNil(t *testing.T) {
subs := map[string]string{
"$public_ipv4": "",
"$private_ipv4": "",
"$public_ipv6": "",
"$private_ipv6": "",
}
env := NewEnvironment("./", "./", "./", "", "", subs)
ef := env.DefaultEnvironmentFile()
if ef != nil {
t.Fatalf("Environment file not nil: %v", ef)
}
}

View File

@@ -1,62 +0,0 @@
package initialize
import (
"errors"
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type EtcdEnvironment map[string]string
func (ec EtcdEnvironment) normalized() map[string]string {
out := make(map[string]string, len(ec))
for key, val := range ec {
key = strings.ToUpper(key)
key = strings.Replace(key, "-", "_", -1)
out[key] = val
}
return out
}
func (ec EtcdEnvironment) String() (out string) {
norm := ec.normalized()
if val, ok := norm["DISCOVERY_URL"]; ok {
delete(norm, "DISCOVERY_URL")
if _, ok := norm["DISCOVERY"]; !ok {
norm["DISCOVERY"] = val
}
}
out += "[Service]\n"
for key, val := range norm {
out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
}
return
}
// Write an EtcdEnvironment to the appropriate path on disk for etcd.service
func WriteEtcdEnvironment(env EtcdEnvironment, root string) error {
if _, ok := env["name"]; !ok {
if machineID := system.MachineID(root); machineID != "" {
env["name"] = machineID
} else if hostname, err := system.Hostname(); err == nil {
env["name"] = hostname
} else {
return errors.New("Unable to determine default etcd name")
}
}
file := system.File{
Path: path.Join(root, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf"),
RawFilePermissions: "0644",
Content: env.String(),
}
return system.WriteFile(&file)
}

View File

@@ -1,138 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"os/exec"
"path"
"testing"
)
func TestEtcdEnvironment(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "ping"
cfg["discovery"] = "pong"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=pong"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
ec := EtcdEnvironment{
"name": "node001",
"discovery": "http://disco.example.com/foobar",
"peer-bind-addr": "127.0.0.1:7002",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
if err := WriteEtcdEnvironment(ec, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
ec := EtcdEnvironment{}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
if err != nil {
t.Fatalf("Failed writing out /etc/machine-id: %v", err)
}
if err := WriteEtcdEnvironment(ec, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node007"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func rmdir(path string) error {
cmd := exec.Command("rm", "-rf", path)
return cmd.Run()
}

View File

@@ -1,32 +0,0 @@
package initialize
import (
"testing"
)
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}

View File

@@ -1,82 +0,0 @@
package initialize
import (
"bufio"
"fmt"
"io/ioutil"
"os"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
const locksmithUnit = "locksmithd.service"
// addStrategy creates an `/etc/coreos/update.conf` file with the requested
// strategy via rewriting the file on disk or by starting from
// `/usr/share/coreos/update.conf`.
func addStrategy(strategy string, root string) error {
etcUpdate := path.Join(root, "etc", "coreos", "update.conf")
usrUpdate := path.Join(root, "usr", "share", "coreos", "update.conf")
tmp, err := ioutil.TempFile(path.Join(root, "etc", "coreos"), ".update.conf")
if err != nil {
return err
}
err = tmp.Chmod(0644)
if err != nil {
return err
}
conf, err := os.Open(etcUpdate)
if os.IsNotExist(err) {
conf, err = os.Open(usrUpdate)
if err != nil {
return err
}
}
scanner := bufio.NewScanner(conf)
sawStrat := false
stratLine := "REBOOT_STRATEGY="+strategy
for scanner.Scan() {
line := scanner.Text()
if strings.HasPrefix(line, "REBOOT_STRATEGY=") {
line = stratLine
sawStrat = true
}
fmt.Fprintln(tmp, line)
if err := scanner.Err(); err != nil {
return err
}
}
if !sawStrat {
fmt.Fprintln(tmp, stratLine)
}
return os.Rename(tmp.Name(), etcUpdate)
}
// WriteLocksmithConfig updates the `update.conf` file with a REBOOT_STRATEGY for locksmith.
func WriteLocksmithConfig(strategy string, root string) error {
cmd := "restart"
if strategy == "off" {
err := system.MaskUnit(locksmithUnit, root)
if err != nil {
return err
}
cmd = "stop"
} else {
return addStrategy(strategy, root)
}
if err := system.DaemonReload(); err != nil {
return err
}
if _, err := system.RunUnitCommand(cmd, locksmithUnit); err != nil {
return err
}
return nil
}

View File

@@ -1,92 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
)
const (
base = `SERVER=https://example.com
GROUP=thegroupc`
configured = base + `
REBOOT_STRATEGY=awesome
`
expected = base + `
REBOOT_STRATEGY=etcd-lock
`
)
func setupFixtures(dir string) {
os.MkdirAll(path.Join(dir, "usr", "share", "coreos"), 0755)
os.MkdirAll(path.Join(dir, "etc", "coreos"), 0755)
os.MkdirAll(path.Join(dir, "run", "systemd", "system"), 0755)
ioutil.WriteFile(path.Join(dir, "usr", "share", "coreos", "update.conf"), []byte(base), 0644)
}
func TestLocksmithEnvironmentWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
for i := 0; i < 2; i++ {
if i == 1 {
err = ioutil.WriteFile(path.Join(dir, "etc", "coreos", "update.conf"), []byte(configured), 0644)
if err != nil {
t.Fatal(err)
}
}
if err := WriteLocksmithConfig("etcd-lock", dir); err != nil {
t.Fatalf("Processing of LocksmithEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "etc", "coreos", "update.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expected {
t.Fatalf("File has incorrect contents, got %v, wanted %v", string(contents), expected)
}
}
}
func TestLocksmithEnvironmentMasked(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
if err := WriteLocksmithConfig("off", dir); err != nil {
t.Fatalf("Processing of LocksmithEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "etc", "systemd", "system", "locksmithd.service")
target, err := os.Readlink(fullPath)
if err != nil {
t.Fatalf("Unable to read link %v", err)
}
if target != "/dev/null" {
t.Fatalf("Locksmith not masked, unit target %v", target)
}
}

View File

@@ -1,44 +0,0 @@
package initialize
import (
"errors"
"fmt"
"os"
"path"
"github.com/coreos/coreos-cloudinit/system"
)
const DefaultIpv4Address = "127.0.0.1"
func generateEtcHosts(option string) (out string, err error) {
if option != "localhost" {
return "", errors.New("Invalid option to manage_etc_hosts")
}
// use the operating system hostname
hostname, err := os.Hostname()
if err != nil {
return "", err
}
return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
}
// Write an /etc/hosts file
func WriteEtcHosts(option string, root string) error {
etcHosts, err := generateEtcHosts(option)
if err != nil {
return err
}
file := system.File{
Path: path.Join(root, "etc", "hosts"),
RawFilePermissions: "0644",
Content: etcHosts,
}
return system.WriteFile(&file)
}

View File

@@ -1,76 +0,0 @@
package initialize
import (
"fmt"
"io/ioutil"
"os"
"path"
"testing"
)
func TestCloudConfigManageEtcHosts(t *testing.T) {
contents := `
manage_etc_hosts: localhost
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
manageEtcHosts := cfg.ManageEtcHosts
if manageEtcHosts != "localhost" {
t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
}
}
func TestManageEtcHostsInvalidValue(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer rmdir(dir)
if err := WriteEtcHosts("invalid", dir); err == nil {
t.Fatalf("WriteEtcHosts succeeded with invalid value: %v", err)
}
}
func TestEtcHostsWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer rmdir(dir)
if err := WriteEtcHosts("localhost", dir); err != nil {
t.Fatalf("WriteEtcHosts failed: %v", err)
}
fullPath := path.Join(dir, "etc", "hosts")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("Unable to read OS hostname: %v", err)
}
expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

74
initialize/meta_data.go Normal file
View File

@@ -0,0 +1,74 @@
package initialize
import (
"encoding/json"
"sort"
"github.com/coreos/coreos-cloudinit/config"
)
// ParseMetaData parses a JSON blob in the OpenStack metadata service format,
// and converts it to a partially hydrated CloudConfig.
func ParseMetaData(contents string) (*config.CloudConfig, error) {
if len(contents) == 0 {
return nil, nil
}
var metadata struct {
SSHAuthorizedKeyMap map[string]string `json:"public_keys"`
Hostname string `json:"hostname"`
NetworkConfig struct {
ContentPath string `json:"content_path"`
} `json:"network_config"`
}
if err := json.Unmarshal([]byte(contents), &metadata); err != nil {
return nil, err
}
var cfg config.CloudConfig
if len(metadata.SSHAuthorizedKeyMap) > 0 {
cfg.SSHAuthorizedKeys = make([]string, 0, len(metadata.SSHAuthorizedKeyMap))
for _, name := range sortedKeys(metadata.SSHAuthorizedKeyMap) {
cfg.SSHAuthorizedKeys = append(cfg.SSHAuthorizedKeys, metadata.SSHAuthorizedKeyMap[name])
}
}
cfg.Hostname = metadata.Hostname
cfg.NetworkConfigPath = metadata.NetworkConfig.ContentPath
return &cfg, nil
}
// ExtractIPsFromMetaData parses a JSON blob in the OpenStack metadata service
// format and returns a substitution map possibly containing private_ipv4,
// public_ipv4, private_ipv6, and public_ipv6 addresses.
func ExtractIPsFromMetadata(contents []byte) (map[string]string, error) {
var ips struct {
PublicIPv4 string `json:"public-ipv4"`
PrivateIPv4 string `json:"local-ipv4"`
PublicIPv6 string `json:"public-ipv6"`
PrivateIPv6 string `json:"local-ipv6"`
}
if err := json.Unmarshal(contents, &ips); err != nil {
return nil, err
}
m := make(map[string]string)
if ips.PrivateIPv4 != "" {
m["$private_ipv4"] = ips.PrivateIPv4
}
if ips.PublicIPv4 != "" {
m["$public_ipv4"] = ips.PublicIPv4
}
if ips.PrivateIPv6 != "" {
m["$private_ipv6"] = ips.PrivateIPv6
}
if ips.PublicIPv6 != "" {
m["$public_ipv6"] = ips.PublicIPv6
}
return m, nil
}
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

View File

@@ -0,0 +1,73 @@
package initialize
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestParseMetadata(t *testing.T) {
for i, tt := range []struct {
in string
want *config.CloudConfig
err bool
}{
{"", nil, false},
{`garbage, invalid json`, nil, true},
{`{"foo": "bar"}`, &config.CloudConfig{}, false},
{`{"network_config": {"content_path": "asdf"}}`, &config.CloudConfig{NetworkConfigPath: "asdf"}, false},
{`{"hostname": "turkleton"}`, &config.CloudConfig{Hostname: "turkleton"}, false},
{`{"public_keys": {"jack": "jill", "bob": "alice"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"alice", "jill"}}, false},
{`{"unknown": "thing", "hostname": "my_host", "public_keys": {"do": "re", "mi": "fa"}, "network_config": {"content_path": "/root", "blah": "zzz"}}`, &config.CloudConfig{SSHAuthorizedKeys: []string{"re", "fa"}, Hostname: "my_host", NetworkConfigPath: "/root"}, false},
} {
got, err := ParseMetaData(tt.in)
if tt.err != (err != nil) {
t.Errorf("case #%d: bad error state: got %t, want %t (err=%v)", i, (err != nil), tt.err, err)
}
if got == nil {
if tt.want != nil {
t.Errorf("case #%d: unexpected nil output", i)
}
} else if tt.want == nil {
t.Errorf("case #%d: unexpected non-nil output", i)
} else {
if !reflect.DeepEqual(*got, *tt.want) {
t.Errorf("case #%d: bad output:\ngot\n%v\nwant\n%v", i, *got, *tt.want)
}
}
}
}
func TestExtractIPsFromMetadata(t *testing.T) {
for i, tt := range []struct {
in []byte
err bool
out map[string]string
}{
{
[]byte(`{"public-ipv4": "12.34.56.78", "local-ipv4": "1.2.3.4", "public-ipv6": "1234::", "local-ipv6": "5678::"}`),
false,
map[string]string{"$public_ipv4": "12.34.56.78", "$private_ipv4": "1.2.3.4", "$public_ipv6": "1234::", "$private_ipv6": "5678::"},
},
{
[]byte(`{"local-ipv4": "127.0.0.1", "something_else": "don't care"}`),
false,
map[string]string{"$private_ipv4": "127.0.0.1"},
},
{
[]byte(`garbage`),
true,
nil,
},
} {
got, err := ExtractIPsFromMetadata(tt.in)
if (err != nil) != tt.err {
t.Errorf("bad error state (got %t, want %t)", err != nil, tt.err)
}
if !reflect.DeepEqual(got, tt.out) {
t.Errorf("case %d: got %s, want %s", i, got, tt.out)
}
}
}

View File

@@ -1,39 +0,0 @@
package initialize
import (
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type OEMRelease struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}
func (oem *OEMRelease) String() string {
fields := []string{
fmt.Sprintf("ID=%s", oem.ID),
fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
fmt.Sprintf("NAME=%q", oem.Name),
fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
}
return strings.Join(fields, "\n") + "\n"
}
func WriteOEMRelease(oem *OEMRelease, root string) error {
file := system.File{
Path: path.Join(root, "etc", "oem-release"),
RawFilePermissions: "0644",
Content: oem.String(),
}
return system.WriteFile(&file)
}

View File

@@ -1,53 +0,0 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
)
func TestOEMReleaseWrittenToDisk(t *testing.T) {
oem := OEMRelease{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
if err := WriteOEMRelease(&oem, dir); err != nil {
t.Fatalf("Processing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "etc", "oem-release")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `ID=rackspace
VERSION_ID=168.0.0
NAME="Rackspace Cloud Servers"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

View File

@@ -3,9 +3,8 @@ package initialize
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"github.com/coreos/coreos-cloudinit/pkg"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -25,22 +24,19 @@ func SSHImportKeysFromURL(system_user string, url string) error {
}
func fetchUserKeys(url string) ([]string, error) {
res, err := http.Get(url)
defer res.Body.Close()
client := pkg.NewHttpClient()
data, err := client.GetRetry(url)
if err != nil {
return nil, err
}
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
var data []UserKey
err = json.Unmarshal(body, &data)
var userKeys []UserKey
err = json.Unmarshal(data, &userKeys)
if err != nil {
return nil, err
}
keys := make([]string, 0)
for _, key := range data {
for _, key := range userKeys {
keys = append(keys, key.Key)
}
return keys, err

View File

@@ -39,31 +39,4 @@ func TestCloudConfigUsersUrlMarshal(t *testing.T) {
if keys[2] != expected {
t.Fatalf("expected %s, got %s", expected, keys[2])
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

View File

@@ -5,28 +5,27 @@ import (
"log"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
func ParseUserData(contents string) (interface{}, error) {
if len(contents) == 0 {
return nil, nil
}
header := strings.SplitN(contents, "\n", 2)[0]
// Explicitly trim the header so we can handle user-data from
// non-unix operating systems. The rest of the file is parsed
// by goyaml, which correctly handles CRLF.
// by yaml, which correctly handles CRLF.
header = strings.TrimSpace(header)
if strings.HasPrefix(header, "#!") {
log.Printf("Parsing user-data as script")
return system.Script(contents), nil
} else if header == "#cloud-config" {
log.Printf("Parsing user-data as cloud-config")
cfg, err := NewCloudConfig(contents)
if err != nil {
log.Fatal(err.Error())
}
return *cfg, nil
return config.NewCloudConfig(contents)
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}

View File

@@ -2,6 +2,8 @@ package initialize
import (
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestParseHeaderCRLF(t *testing.T) {
@@ -37,7 +39,7 @@ func TestParseConfigCRLF(t *testing.T) {
t.Fatalf("Failed parsing config: %v", err)
}
cfg := ud.(CloudConfig)
cfg := ud.(*config.CloudConfig)
if cfg.Hostname != "foo" {
t.Error("Failed parsing hostname from config")
@@ -47,3 +49,12 @@ func TestParseConfigCRLF(t *testing.T) {
t.Error("Parsed incorrect number of SSH keys")
}
}
func TestParseConfigEmpty(t *testing.T) {
i, e := ParseUserData(``)
if i != nil {
t.Error("ParseUserData of empty string returned non-nil unexpectedly")
} else if e != nil {
t.Error("ParseUserData of empty string returned error unexpectedly")
}
}

View File

@@ -3,7 +3,9 @@ package initialize
import (
"io/ioutil"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/system"
)
@@ -28,21 +30,23 @@ func PersistScriptInWorkspace(script system.Script, workspace string) (string, e
}
tmp.Close()
file := system.File{
Path: tmp.Name(),
relpath := strings.TrimPrefix(tmp.Name(), workspace)
file := system.File{config.File{
Path: relpath,
RawFilePermissions: "0744",
Content: string(script),
}
}}
err = system.WriteFile(&file)
return file.Path, err
return system.WriteFile(&file, workspace)
}
func PersistUnitNameInWorkspace(name string, workspace string) error {
file := system.File{
Path: path.Join(workspace, "scripts", "unit-name"),
file := system.File{config.File{
Path: path.Join("scripts", "unit-name"),
RawFilePermissions: "0644",
Content: name,
}
return system.WriteFile(&file)
}}
_, err := system.WriteFile(&file, workspace)
return err
}

49
network/debian.go Normal file
View File

@@ -0,0 +1,49 @@
package network
import (
"log"
"strings"
)
func ProcessDebianNetconf(config string) ([]InterfaceGenerator, error) {
log.Println("Processing Debian network config")
lines := formatConfig(config)
stanzas, err := parseStanzas(lines)
if err != nil {
return nil, err
}
interfaces := make([]*stanzaInterface, 0, len(stanzas))
for _, stanza := range stanzas {
switch s := stanza.(type) {
case *stanzaInterface:
interfaces = append(interfaces, s)
}
}
log.Printf("Parsed %d network interfaces\n", len(interfaces))
log.Println("Processed Debian network config")
return buildInterfaces(interfaces), nil
}
func formatConfig(config string) []string {
lines := []string{}
config = strings.Replace(config, "\\\n", "", -1)
for config != "" {
split := strings.SplitN(config, "\n", 2)
line := strings.TrimSpace(split[0])
if len(split) == 2 {
config = split[1]
} else {
config = ""
}
if strings.HasPrefix(line, "#") || line == "" {
continue
}
lines = append(lines, line)
}
return lines
}

42
network/debian_test.go Normal file
View File

@@ -0,0 +1,42 @@
package network
import (
"testing"
)
func TestFormatConfigs(t *testing.T) {
for in, n := range map[string]int{
"": 0,
"line1\\\nis long": 1,
"#comment": 0,
"#comment\\\ncomment": 0,
" #comment \\\n comment\nline 1\nline 2\\\n is long": 2,
} {
lines := formatConfig(in)
if len(lines) != n {
t.Fatalf("bad number of lines for config %q: got %d, want %d", in, len(lines), n)
}
}
}
func TestProcessDebianNetconf(t *testing.T) {
for _, tt := range []struct {
in string
fail bool
n int
}{
{"", false, 0},
{"iface", true, -1},
{"auto eth1\nauto eth2", false, 0},
{"iface eth1 inet manual", false, 1},
} {
interfaces, err := ProcessDebianNetconf(tt.in)
failed := err != nil
if tt.fail != failed {
t.Fatalf("bad failure state for %q: got %b, want %b", failed, tt.fail)
}
if tt.n != -1 && tt.n != len(interfaces) {
t.Fatalf("bad number of interfaces for %q: got %d, want %q", tt.in, len(interfaces), tt.n)
}
}
}

142
network/digitalocean.go Normal file
View File

@@ -0,0 +1,142 @@
package network
import (
"encoding/json"
"fmt"
"log"
"net"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
)
func ProcessDigitalOceanNetconf(config string) ([]InterfaceGenerator, error) {
log.Println("Processing DigitalOcean network config")
if config == "" {
return nil, nil
}
var cfg digitalocean.Metadata
if err := json.Unmarshal([]byte(config), &cfg); err != nil {
return nil, err
}
log.Println("Parsing nameservers")
nameservers, err := parseNameservers(cfg.DNS)
if err != nil {
return nil, err
}
log.Printf("Parsed %d nameservers\n", len(nameservers))
log.Println("Parsing interfaces")
generators, err := parseInterfaces(cfg.Interfaces, nameservers)
if err != nil {
return nil, err
}
log.Printf("Parsed %d network interfaces\n", len(generators))
log.Println("Processed DigitalOcean network config")
return generators, nil
}
func parseNameservers(cfg digitalocean.DNS) ([]net.IP, error) {
nameservers := make([]net.IP, 0, len(cfg.Nameservers))
for _, ns := range cfg.Nameservers {
if ip := net.ParseIP(ns); ip == nil {
return nil, fmt.Errorf("could not parse %q as nameserver IP address", ns)
} else {
nameservers = append(nameservers, ip)
}
}
return nameservers, nil
}
func parseInterfaces(cfg digitalocean.Interfaces, nameservers []net.IP) ([]InterfaceGenerator, error) {
generators := make([]InterfaceGenerator, 0, len(cfg.Public)+len(cfg.Private))
for _, iface := range cfg.Public {
if generator, err := parseInterface(iface, nameservers, true); err == nil {
generators = append(generators, &physicalInterface{*generator})
} else {
return nil, err
}
}
for _, iface := range cfg.Private {
if generator, err := parseInterface(iface, []net.IP{}, false); err == nil {
generators = append(generators, &physicalInterface{*generator})
} else {
return nil, err
}
}
return generators, nil
}
func parseInterface(iface digitalocean.Interface, nameservers []net.IP, useRoute bool) (*logicalInterface, error) {
routes := make([]route, 0)
addresses := make([]net.IPNet, 0)
if iface.IPv4 != nil {
var ip, mask, gateway net.IP
if ip = net.ParseIP(iface.IPv4.IPAddress); ip == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 address", iface.IPv4.IPAddress)
}
if mask = net.ParseIP(iface.IPv4.Netmask); mask == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 mask", iface.IPv4.Netmask)
}
addresses = append(addresses, net.IPNet{
IP: ip,
Mask: net.IPMask(mask),
})
if useRoute {
if gateway = net.ParseIP(iface.IPv4.Gateway); gateway == nil {
return nil, fmt.Errorf("could not parse %q as IPv4 gateway", iface.IPv4.Gateway)
}
routes = append(routes, route{
destination: net.IPNet{
IP: net.IPv4zero,
Mask: net.IPMask(net.IPv4zero),
},
gateway: gateway,
})
}
}
if iface.IPv6 != nil {
var ip, gateway net.IP
if ip = net.ParseIP(iface.IPv6.IPAddress); ip == nil {
return nil, fmt.Errorf("could not parse %q as IPv6 address", iface.IPv6.IPAddress)
}
addresses = append(addresses, net.IPNet{
IP: ip,
Mask: net.CIDRMask(iface.IPv6.Cidr, net.IPv6len*8),
})
if useRoute {
if gateway = net.ParseIP(iface.IPv6.Gateway); gateway == nil {
return nil, fmt.Errorf("could not parse %q as IPv6 gateway", iface.IPv6.Gateway)
}
routes = append(routes, route{
destination: net.IPNet{
IP: net.IPv6zero,
Mask: net.IPMask(net.IPv6zero),
},
gateway: gateway,
})
}
}
hwaddr, err := net.ParseMAC(iface.MAC)
if err != nil {
return nil, err
}
if nameservers == nil {
nameservers = []net.IP{}
}
return &logicalInterface{
hwaddr: hwaddr,
config: configMethodStatic{
addresses: addresses,
nameservers: nameservers,
routes: routes,
},
}, nil
}

View File

@@ -0,0 +1,367 @@
package network
import (
"errors"
"net"
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/datasource/metadata/digitalocean"
)
func TestParseNameservers(t *testing.T) {
for _, tt := range []struct {
dns digitalocean.DNS
nss []net.IP
err error
}{
{
dns: digitalocean.DNS{},
nss: []net.IP{},
},
{
dns: digitalocean.DNS{[]string{"1.2.3.4"}},
nss: []net.IP{net.ParseIP("1.2.3.4")},
},
{
dns: digitalocean.DNS{[]string{"bad"}},
err: errors.New("could not parse \"bad\" as nameserver IP address"),
},
} {
nss, err := parseNameservers(tt.dns)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.dns, tt.err, err)
}
if !reflect.DeepEqual(tt.nss, nss) {
t.Fatalf("bad nameservers (%+v): want %#v, got %#v", tt.dns, tt.nss, nss)
}
}
}
func TestParseInterface(t *testing.T) {
for _, tt := range []struct {
cfg digitalocean.Interface
nss []net.IP
useRoute bool
iface *logicalInterface
err error
}{
{
cfg: digitalocean.Interface{
MAC: "bad",
},
err: errors.New("invalid MAC address: bad"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
},
useRoute: true,
nss: []net.IP{net.ParseIP("1.2.3.4")},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{net.ParseIP("1.2.3.4")},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "bad",
Netmask: "255.255.0.0",
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 address"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "bad",
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 mask"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "ignoreme",
},
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "bad",
},
},
useRoute: true,
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv4 gateway"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv4: &digitalocean.Address{
IPAddress: "1.2.3.4",
Netmask: "255.255.0.0",
Gateway: "5.6.7.8",
},
},
useRoute: true,
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("1.2.3.4"), net.IPMask(net.ParseIP("255.255.0.0"))}},
nameservers: []net.IP{},
routes: []route{route{net.IPNet{net.IPv4zero, net.IPMask(net.IPv4zero)}, net.ParseIP("5.6.7.8")}},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "bad",
Cidr: 16,
},
},
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv6 address"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "ignoreme",
},
},
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
nameservers: []net.IP{},
routes: []route{},
},
},
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "bad",
},
},
useRoute: true,
nss: []net.IP{},
err: errors.New("could not parse \"bad\" as IPv6 gateway"),
},
{
cfg: digitalocean.Interface{
MAC: "01:23:45:67:89:AB",
IPv6: &digitalocean.Address{
IPAddress: "fe00::",
Cidr: 16,
Gateway: "fe00:1234::",
},
},
useRoute: true,
nss: []net.IP{},
iface: &logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{net.IPNet{net.ParseIP("fe00::"), net.IPMask(net.ParseIP("ffff::"))}},
nameservers: []net.IP{},
routes: []route{route{net.IPNet{net.IPv6zero, net.IPMask(net.IPv6zero)}, net.ParseIP("fe00:1234::")}},
},
},
},
} {
iface, err := parseInterface(tt.cfg, tt.nss, tt.useRoute)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.iface, iface) {
t.Fatalf("bad interface (%+v): want %#v, got %#v", tt.cfg, tt.iface, iface)
}
}
}
func TestParseInterfaces(t *testing.T) {
for _, tt := range []struct {
cfg digitalocean.Interfaces
nss []net.IP
ifaces []InterfaceGenerator
err error
}{
{
ifaces: []InterfaceGenerator{},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
nss: []net.IP{net.ParseIP("1.2.3.4")},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{net.ParseIP("1.2.3.4")},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "01:23:45:67:89:AB"}},
},
nss: []net.IP{net.ParseIP("1.2.3.4")},
ifaces: []InterfaceGenerator{
&physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}),
config: configMethodStatic{
addresses: []net.IPNet{},
nameservers: []net.IP{},
routes: []route{},
},
}},
},
},
{
cfg: digitalocean.Interfaces{
Public: []digitalocean.Interface{{MAC: "bad"}},
},
err: errors.New("invalid MAC address: bad"),
},
{
cfg: digitalocean.Interfaces{
Private: []digitalocean.Interface{{MAC: "bad"}},
},
err: errors.New("invalid MAC address: bad"),
},
} {
ifaces, err := parseInterfaces(tt.cfg, tt.nss)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%+v): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.ifaces, ifaces) {
t.Fatalf("bad interfaces (%+v): want %#v, got %#v", tt.cfg, tt.ifaces, ifaces)
}
}
}
func TestProcessDigitalOceanNetconf(t *testing.T) {
for _, tt := range []struct {
cfg string
ifaces []InterfaceGenerator
err error
}{
{
cfg: ``,
},
{
cfg: `{"dns":{"nameservers":["bad"]}}`,
err: errors.New("could not parse \"bad\" as nameserver IP address"),
},
{
cfg: `{"interfaces":{"public":[{"ipv4":{"ip_address":"bad"}}]}}`,
err: errors.New("could not parse \"bad\" as IPv4 address"),
},
{
cfg: `{}`,
ifaces: []InterfaceGenerator{},
},
} {
ifaces, err := ProcessDigitalOceanNetconf(tt.cfg)
if !errorsEqual(tt.err, err) {
t.Fatalf("bad error (%q): want %q, got %q", tt.cfg, tt.err, err)
}
if !reflect.DeepEqual(tt.ifaces, ifaces) {
t.Fatalf("bad interfaces (%q): want %#v, got %#v", tt.cfg, tt.ifaces, ifaces)
}
}
}
func errorsEqual(a, b error) bool {
if a == nil && b == nil {
return true
}
if (a != nil && b == nil) || (a == nil && b != nil) {
return false
}
return (a.Error() == b.Error())
}

316
network/interface.go Normal file
View File

@@ -0,0 +1,316 @@
package network
import (
"fmt"
"net"
"sort"
"strconv"
"strings"
)
type InterfaceGenerator interface {
Name() string
Filename() string
Netdev() string
Link() string
Network() string
Type() string
ModprobeParams() string
}
type networkInterface interface {
InterfaceGenerator
Children() []networkInterface
setConfigDepth(int)
}
type logicalInterface struct {
name string
hwaddr net.HardwareAddr
config configMethod
children []networkInterface
configDepth int
}
func (i *logicalInterface) Name() string {
return i.name
}
func (i *logicalInterface) Network() string {
config := fmt.Sprintln("[Match]")
if i.name != "" {
config += fmt.Sprintf("Name=%s\n", i.name)
}
if i.hwaddr != nil {
config += fmt.Sprintf("MACAddress=%s\n", i.hwaddr)
}
config += "\n[Network]\n"
for _, child := range i.children {
switch iface := child.(type) {
case *vlanInterface:
config += fmt.Sprintf("VLAN=%s\n", iface.name)
case *bondInterface:
config += fmt.Sprintf("Bond=%s\n", iface.name)
}
}
switch conf := i.config.(type) {
case configMethodStatic:
for _, nameserver := range conf.nameservers {
config += fmt.Sprintf("DNS=%s\n", nameserver)
}
for _, addr := range conf.addresses {
config += fmt.Sprintf("\n[Address]\nAddress=%s\n", addr.String())
}
for _, route := range conf.routes {
config += fmt.Sprintf("\n[Route]\nDestination=%s\nGateway=%s\n", route.destination.String(), route.gateway)
}
case configMethodDHCP:
config += "DHCP=true\n"
}
return config
}
func (i *logicalInterface) Link() string {
return ""
}
func (i *logicalInterface) Netdev() string {
return ""
}
func (i *logicalInterface) Filename() string {
name := i.name
if name == "" {
name = i.hwaddr.String()
}
return fmt.Sprintf("%02x-%s", i.configDepth, name)
}
func (i *logicalInterface) Children() []networkInterface {
return i.children
}
func (i *logicalInterface) ModprobeParams() string {
return ""
}
func (i *logicalInterface) setConfigDepth(depth int) {
i.configDepth = depth
}
type physicalInterface struct {
logicalInterface
}
func (p *physicalInterface) Type() string {
return "physical"
}
type bondInterface struct {
logicalInterface
slaves []string
options map[string]string
}
func (b *bondInterface) Netdev() string {
return fmt.Sprintf("[NetDev]\nKind=bond\nName=%s\n", b.name)
}
func (b *bondInterface) Type() string {
return "bond"
}
func (b *bondInterface) ModprobeParams() string {
params := ""
for _, name := range sortedKeys(b.options) {
params += fmt.Sprintf("%s=%s ", name, b.options[name])
}
params = strings.TrimSuffix(params, " ")
return params
}
type vlanInterface struct {
logicalInterface
id int
rawDevice string
}
func (v *vlanInterface) Netdev() string {
config := fmt.Sprintf("[NetDev]\nKind=vlan\nName=%s\n", v.name)
switch c := v.config.(type) {
case configMethodStatic:
if c.hwaddress != nil {
config += fmt.Sprintf("MACAddress=%s\n", c.hwaddress)
}
case configMethodDHCP:
if c.hwaddress != nil {
config += fmt.Sprintf("MACAddress=%s\n", c.hwaddress)
}
}
config += fmt.Sprintf("\n[VLAN]\nId=%d\n", v.id)
return config
}
func (v *vlanInterface) Type() string {
return "vlan"
}
func buildInterfaces(stanzas []*stanzaInterface) []InterfaceGenerator {
interfaceMap := createInterfaces(stanzas)
linkAncestors(interfaceMap)
markConfigDepths(interfaceMap)
interfaces := make([]InterfaceGenerator, 0, len(interfaceMap))
for _, name := range sortedInterfaces(interfaceMap) {
interfaces = append(interfaces, interfaceMap[name])
}
return interfaces
}
func createInterfaces(stanzas []*stanzaInterface) map[string]networkInterface {
interfaceMap := make(map[string]networkInterface)
for _, iface := range stanzas {
switch iface.kind {
case interfaceBond:
bondOptions := make(map[string]string)
for _, k := range []string{"mode", "miimon", "lacp-rate"} {
if v, ok := iface.options["bond-"+k]; ok && len(v) > 0 {
bondOptions[k] = v[0]
}
}
interfaceMap[iface.name] = &bondInterface{
logicalInterface{
name: iface.name,
config: iface.configMethod,
children: []networkInterface{},
},
iface.options["bond-slaves"],
bondOptions,
}
for _, slave := range iface.options["bond-slaves"] {
if _, ok := interfaceMap[slave]; !ok {
interfaceMap[slave] = &physicalInterface{
logicalInterface{
name: slave,
config: configMethodManual{},
children: []networkInterface{},
},
}
}
}
case interfacePhysical:
if _, ok := iface.configMethod.(configMethodLoopback); ok {
continue
}
interfaceMap[iface.name] = &physicalInterface{
logicalInterface{
name: iface.name,
config: iface.configMethod,
children: []networkInterface{},
},
}
case interfaceVLAN:
var rawDevice string
id, _ := strconv.Atoi(iface.options["id"][0])
if device := iface.options["raw_device"]; len(device) == 1 {
rawDevice = device[0]
if _, ok := interfaceMap[rawDevice]; !ok {
interfaceMap[rawDevice] = &physicalInterface{
logicalInterface{
name: rawDevice,
config: configMethodManual{},
children: []networkInterface{},
},
}
}
}
interfaceMap[iface.name] = &vlanInterface{
logicalInterface{
name: iface.name,
config: iface.configMethod,
children: []networkInterface{},
},
id,
rawDevice,
}
}
}
return interfaceMap
}
func linkAncestors(interfaceMap map[string]networkInterface) {
for _, name := range sortedInterfaces(interfaceMap) {
iface := interfaceMap[name]
switch i := iface.(type) {
case *vlanInterface:
if parent, ok := interfaceMap[i.rawDevice]; ok {
switch p := parent.(type) {
case *physicalInterface:
p.children = append(p.children, iface)
case *bondInterface:
p.children = append(p.children, iface)
}
}
case *bondInterface:
for _, slave := range i.slaves {
if parent, ok := interfaceMap[slave]; ok {
switch p := parent.(type) {
case *physicalInterface:
p.children = append(p.children, iface)
case *bondInterface:
p.children = append(p.children, iface)
}
}
}
}
}
}
func markConfigDepths(interfaceMap map[string]networkInterface) {
rootInterfaceMap := make(map[string]networkInterface)
for k, v := range interfaceMap {
rootInterfaceMap[k] = v
}
for _, iface := range interfaceMap {
for _, child := range iface.Children() {
delete(rootInterfaceMap, child.Name())
}
}
for _, iface := range rootInterfaceMap {
setDepth(iface)
}
}
func setDepth(iface networkInterface) int {
maxDepth := 0
for _, child := range iface.Children() {
if depth := setDepth(child); depth > maxDepth {
maxDepth = depth
}
}
iface.setConfigDepth(maxDepth)
return (maxDepth + 1)
}
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}
func sortedInterfaces(m map[string]networkInterface) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

354
network/interface_test.go Normal file
View File

@@ -0,0 +1,354 @@
package network
import (
"net"
"reflect"
"testing"
)
func TestInterfaceGenerators(t *testing.T) {
for _, tt := range []struct {
name string
netdev string
link string
network string
kind string
iface InterfaceGenerator
}{
{
name: "",
network: "[Match]\nMACAddress=00:01:02:03:04:05\n\n[Network]\n",
kind: "physical",
iface: &physicalInterface{logicalInterface{
hwaddr: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
}},
},
{
name: "testname",
network: "[Match]\nName=testname\n\n[Network]\nBond=testbond1\nVLAN=testvlan1\nVLAN=testvlan2\n",
kind: "physical",
iface: &physicalInterface{logicalInterface{
name: "testname",
children: []networkInterface{
&bondInterface{logicalInterface: logicalInterface{name: "testbond1"}},
&vlanInterface{logicalInterface: logicalInterface{name: "testvlan1"}, id: 1},
&vlanInterface{logicalInterface: logicalInterface{name: "testvlan2"}, id: 1},
},
}},
},
{
name: "testname",
netdev: "[NetDev]\nKind=bond\nName=testname\n",
network: "[Match]\nName=testname\n\n[Network]\nBond=testbond1\nVLAN=testvlan1\nVLAN=testvlan2\nDHCP=true\n",
kind: "bond",
iface: &bondInterface{logicalInterface: logicalInterface{
name: "testname",
config: configMethodDHCP{},
children: []networkInterface{
&bondInterface{logicalInterface: logicalInterface{name: "testbond1"}},
&vlanInterface{logicalInterface: logicalInterface{name: "testvlan1"}, id: 1},
&vlanInterface{logicalInterface: logicalInterface{name: "testvlan2"}, id: 1},
},
}},
},
{
name: "testname",
netdev: "[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=1\n",
network: "[Match]\nName=testname\n\n[Network]\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface{name: "testname"}, 1, ""},
},
{
name: "testname",
netdev: "[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
network: "[Match]\nName=testname\n\n[Network]\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface{name: "testname", config: configMethodStatic{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
},
{
name: "testname",
netdev: "[NetDev]\nKind=vlan\nName=testname\nMACAddress=00:01:02:03:04:05\n\n[VLAN]\nId=1\n",
network: "[Match]\nName=testname\n\n[Network]\nDHCP=true\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface{name: "testname", config: configMethodDHCP{hwaddress: net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5})}}, 1, ""},
},
{
name: "testname",
netdev: "[NetDev]\nKind=vlan\nName=testname\n\n[VLAN]\nId=0\n",
network: "[Match]\nName=testname\n\n[Network]\nDNS=8.8.8.8\n\n[Address]\nAddress=192.168.1.100/24\n\n[Route]\nDestination=0.0.0.0/0\nGateway=1.2.3.4\n",
kind: "vlan",
iface: &vlanInterface{logicalInterface: logicalInterface{
name: "testname",
config: configMethodStatic{
addresses: []net.IPNet{{IP: []byte{192, 168, 1, 100}, Mask: []byte{255, 255, 255, 0}}},
nameservers: []net.IP{[]byte{8, 8, 8, 8}},
routes: []route{route{destination: net.IPNet{IP: []byte{0, 0, 0, 0}, Mask: []byte{0, 0, 0, 0}}, gateway: []byte{1, 2, 3, 4}}},
},
}},
},
} {
if name := tt.iface.Name(); name != tt.name {
t.Fatalf("bad name (%q): want %q, got %q", tt.iface, tt.name, name)
}
if netdev := tt.iface.Netdev(); netdev != tt.netdev {
t.Fatalf("bad netdev (%q): want %q, got %q", tt.iface, tt.netdev, netdev)
}
if link := tt.iface.Link(); link != tt.link {
t.Fatalf("bad link (%q): want %q, got %q", tt.iface, tt.link, link)
}
if network := tt.iface.Network(); network != tt.network {
t.Fatalf("bad network (%q): want %q, got %q", tt.iface, tt.network, network)
}
if kind := tt.iface.Type(); kind != tt.kind {
t.Fatalf("bad type (%q): want %q, got %q", tt.iface, tt.kind, kind)
}
}
}
func TestModprobeParams(t *testing.T) {
for _, tt := range []struct {
i InterfaceGenerator
p string
}{
{
i: &physicalInterface{},
p: "",
},
{
i: &vlanInterface{},
p: "",
},
{
i: &bondInterface{
logicalInterface{},
nil,
map[string]string{
"a": "1",
"b": "2",
},
},
p: "a=1 b=2",
},
} {
if p := tt.i.ModprobeParams(); p != tt.p {
t.Fatalf("bad params (%q): got %s, want %s", tt.i, p, tt.p)
}
}
}
func TestBuildInterfacesLo(t *testing.T) {
stanzas := []*stanzaInterface{
&stanzaInterface{
name: "lo",
kind: interfacePhysical,
auto: false,
configMethod: configMethodLoopback{},
options: map[string][]string{},
},
}
interfaces := buildInterfaces(stanzas)
if len(interfaces) != 0 {
t.FailNow()
}
}
func TestBuildInterfacesBlindBond(t *testing.T) {
stanzas := []*stanzaInterface{
{
name: "bond0",
kind: interfaceBond,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"bond-slaves": []string{"eth0"},
},
},
}
interfaces := buildInterfaces(stanzas)
bond0 := &bondInterface{
logicalInterface{
name: "bond0",
config: configMethodManual{},
children: []networkInterface{},
configDepth: 0,
},
[]string{"eth0"},
map[string]string{},
}
eth0 := &physicalInterface{
logicalInterface{
name: "eth0",
config: configMethodManual{},
children: []networkInterface{bond0},
configDepth: 1,
},
}
expect := []InterfaceGenerator{bond0, eth0}
if !reflect.DeepEqual(interfaces, expect) {
t.FailNow()
}
}
func TestBuildInterfacesBlindVLAN(t *testing.T) {
stanzas := []*stanzaInterface{
{
name: "vlan0",
kind: interfaceVLAN,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"id": []string{"0"},
"raw_device": []string{"eth0"},
},
},
}
interfaces := buildInterfaces(stanzas)
vlan0 := &vlanInterface{
logicalInterface{
name: "vlan0",
config: configMethodManual{},
children: []networkInterface{},
configDepth: 0,
},
0,
"eth0",
}
eth0 := &physicalInterface{
logicalInterface{
name: "eth0",
config: configMethodManual{},
children: []networkInterface{vlan0},
configDepth: 1,
},
}
expect := []InterfaceGenerator{eth0, vlan0}
if !reflect.DeepEqual(interfaces, expect) {
t.FailNow()
}
}
func TestBuildInterfaces(t *testing.T) {
stanzas := []*stanzaInterface{
&stanzaInterface{
name: "eth0",
kind: interfacePhysical,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{},
},
&stanzaInterface{
name: "bond0",
kind: interfaceBond,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"bond-slaves": []string{"eth0"},
"bond-mode": []string{"4"},
"bond-miimon": []string{"100"},
},
},
&stanzaInterface{
name: "bond1",
kind: interfaceBond,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"bond-slaves": []string{"bond0"},
},
},
&stanzaInterface{
name: "vlan0",
kind: interfaceVLAN,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"id": []string{"0"},
"raw_device": []string{"eth0"},
},
},
&stanzaInterface{
name: "vlan1",
kind: interfaceVLAN,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{
"id": []string{"1"},
"raw_device": []string{"bond0"},
},
},
}
interfaces := buildInterfaces(stanzas)
vlan1 := &vlanInterface{
logicalInterface{
name: "vlan1",
config: configMethodManual{},
children: []networkInterface{},
configDepth: 0,
},
1,
"bond0",
}
vlan0 := &vlanInterface{
logicalInterface{
name: "vlan0",
config: configMethodManual{},
children: []networkInterface{},
configDepth: 0,
},
0,
"eth0",
}
bond1 := &bondInterface{
logicalInterface{
name: "bond1",
config: configMethodManual{},
children: []networkInterface{},
configDepth: 0,
},
[]string{"bond0"},
map[string]string{},
}
bond0 := &bondInterface{
logicalInterface{
name: "bond0",
config: configMethodManual{},
children: []networkInterface{bond1, vlan1},
configDepth: 1,
},
[]string{"eth0"},
map[string]string{
"mode": "4",
"miimon": "100",
},
}
eth0 := &physicalInterface{
logicalInterface{
name: "eth0",
config: configMethodManual{},
children: []networkInterface{bond0, vlan0},
configDepth: 2,
},
}
expect := []InterfaceGenerator{bond0, bond1, eth0, vlan0, vlan1}
if !reflect.DeepEqual(interfaces, expect) {
t.FailNow()
}
}
func TestFilename(t *testing.T) {
for _, tt := range []struct {
i logicalInterface
f string
}{
{logicalInterface{name: "iface", configDepth: 0}, "00-iface"},
{logicalInterface{name: "iface", configDepth: 9}, "09-iface"},
{logicalInterface{name: "iface", configDepth: 10}, "0a-iface"},
{logicalInterface{name: "iface", configDepth: 53}, "35-iface"},
{logicalInterface{hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}), configDepth: 1}, "01-01:23:45:67:89:ab"},
{logicalInterface{name: "iface", hwaddr: net.HardwareAddr([]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xab}), configDepth: 1}, "01-iface"},
} {
if tt.i.Filename() != tt.f {
t.Fatalf("bad filename (%q): got %q, want %q", tt.i, tt.i.Filename(), tt.f)
}
}
}

326
network/stanza.go Normal file
View File

@@ -0,0 +1,326 @@
package network
import (
"fmt"
"net"
"strconv"
"strings"
)
type stanza interface{}
type stanzaAuto struct {
interfaces []string
}
type stanzaInterface struct {
name string
kind interfaceKind
auto bool
configMethod configMethod
options map[string][]string
}
type interfaceKind int
const (
interfaceBond = interfaceKind(iota)
interfacePhysical
interfaceVLAN
)
type route struct {
destination net.IPNet
gateway net.IP
}
type configMethod interface{}
type configMethodStatic struct {
addresses []net.IPNet
nameservers []net.IP
routes []route
hwaddress net.HardwareAddr
}
type configMethodLoopback struct{}
type configMethodManual struct{}
type configMethodDHCP struct {
hwaddress net.HardwareAddr
}
func parseStanzas(lines []string) (stanzas []stanza, err error) {
rawStanzas, err := splitStanzas(lines)
if err != nil {
return nil, err
}
stanzas = make([]stanza, 0, len(rawStanzas))
for _, rawStanza := range rawStanzas {
if stanza, err := parseStanza(rawStanza); err == nil {
stanzas = append(stanzas, stanza)
} else {
return nil, err
}
}
autos := make([]string, 0)
interfaceMap := make(map[string]*stanzaInterface)
for _, stanza := range stanzas {
switch c := stanza.(type) {
case *stanzaAuto:
autos = append(autos, c.interfaces...)
case *stanzaInterface:
interfaceMap[c.name] = c
}
}
// Apply the auto attribute
for _, auto := range autos {
if iface, ok := interfaceMap[auto]; ok {
iface.auto = true
}
}
return stanzas, nil
}
func splitStanzas(lines []string) ([][]string, error) {
var curStanza []string
stanzas := make([][]string, 0)
for _, line := range lines {
if isStanzaStart(line) {
if curStanza != nil {
stanzas = append(stanzas, curStanza)
}
curStanza = []string{line}
} else if curStanza != nil {
curStanza = append(curStanza, line)
} else {
return nil, fmt.Errorf("missing stanza start %q", line)
}
}
if curStanza != nil {
stanzas = append(stanzas, curStanza)
}
return stanzas, nil
}
func isStanzaStart(line string) bool {
switch strings.Split(line, " ")[0] {
case "auto":
fallthrough
case "iface":
fallthrough
case "mapping":
return true
}
if strings.HasPrefix(line, "allow-") {
return true
}
return false
}
func parseStanza(rawStanza []string) (stanza, error) {
if len(rawStanza) == 0 {
panic("empty stanza")
}
tokens := strings.Fields(rawStanza[0])
if len(tokens) < 2 {
return nil, fmt.Errorf("malformed stanza start %q", rawStanza[0])
}
kind := tokens[0]
attributes := tokens[1:]
switch kind {
case "auto":
return parseAutoStanza(attributes, rawStanza[1:])
case "iface":
return parseInterfaceStanza(attributes, rawStanza[1:])
default:
return nil, fmt.Errorf("unknown stanza %q", kind)
}
}
func parseAutoStanza(attributes []string, options []string) (*stanzaAuto, error) {
return &stanzaAuto{interfaces: attributes}, nil
}
func parseInterfaceStanza(attributes []string, options []string) (*stanzaInterface, error) {
if len(attributes) != 3 {
return nil, fmt.Errorf("incorrect number of attributes")
}
iface := attributes[0]
confMethod := attributes[2]
optionMap := make(map[string][]string, 0)
for _, option := range options {
if strings.HasPrefix(option, "post-up") {
tokens := strings.SplitAfterN(option, " ", 2)
if len(tokens) != 2 {
continue
}
if v, ok := optionMap["post-up"]; ok {
optionMap["post-up"] = append(v, tokens[1])
} else {
optionMap["post-up"] = []string{tokens[1]}
}
} else if strings.HasPrefix(option, "pre-down") {
tokens := strings.SplitAfterN(option, " ", 2)
if len(tokens) != 2 {
continue
}
if v, ok := optionMap["pre-down"]; ok {
optionMap["pre-down"] = append(v, tokens[1])
} else {
optionMap["pre-down"] = []string{tokens[1]}
}
} else {
tokens := strings.Fields(option)
optionMap[tokens[0]] = tokens[1:]
}
}
var conf configMethod
switch confMethod {
case "static":
config := configMethodStatic{
addresses: make([]net.IPNet, 1),
routes: make([]route, 0),
nameservers: make([]net.IP, 0),
}
if addresses, ok := optionMap["address"]; ok {
if len(addresses) == 1 {
config.addresses[0].IP = net.ParseIP(addresses[0])
}
}
if netmasks, ok := optionMap["netmask"]; ok {
if len(netmasks) == 1 {
config.addresses[0].Mask = net.IPMask(net.ParseIP(netmasks[0]).To4())
}
}
if config.addresses[0].IP == nil || config.addresses[0].Mask == nil {
return nil, fmt.Errorf("malformed static network config for %q", iface)
}
if gateways, ok := optionMap["gateway"]; ok {
if len(gateways) == 1 {
config.routes = append(config.routes, route{
destination: net.IPNet{
IP: net.IPv4(0, 0, 0, 0),
Mask: net.IPv4Mask(0, 0, 0, 0),
},
gateway: net.ParseIP(gateways[0]),
})
}
}
if hwaddress, err := parseHwaddress(optionMap, iface); err == nil {
config.hwaddress = hwaddress
} else {
return nil, err
}
for _, nameserver := range optionMap["dns-nameservers"] {
config.nameservers = append(config.nameservers, net.ParseIP(nameserver))
}
for _, postup := range optionMap["post-up"] {
if strings.HasPrefix(postup, "route add") {
route := route{}
fields := strings.Fields(postup)
for i, field := range fields[:len(fields)-1] {
switch field {
case "-net":
if _, dst, err := net.ParseCIDR(fields[i+1]); err == nil {
route.destination = *dst
} else {
route.destination.IP = net.ParseIP(fields[i+1])
}
case "netmask":
route.destination.Mask = net.IPMask(net.ParseIP(fields[i+1]).To4())
case "gw":
route.gateway = net.ParseIP(fields[i+1])
}
}
if route.destination.IP != nil && route.destination.Mask != nil && route.gateway != nil {
config.routes = append(config.routes, route)
}
}
}
conf = config
case "loopback":
conf = configMethodLoopback{}
case "manual":
conf = configMethodManual{}
case "dhcp":
config := configMethodDHCP{}
if hwaddress, err := parseHwaddress(optionMap, iface); err == nil {
config.hwaddress = hwaddress
} else {
return nil, err
}
conf = config
default:
return nil, fmt.Errorf("invalid config method %q", confMethod)
}
if _, ok := optionMap["vlan_raw_device"]; ok {
return parseVLANStanza(iface, conf, attributes, optionMap)
}
if strings.Contains(iface, ".") {
return parseVLANStanza(iface, conf, attributes, optionMap)
}
if _, ok := optionMap["bond-slaves"]; ok {
return parseBondStanza(iface, conf, attributes, optionMap)
}
return parsePhysicalStanza(iface, conf, attributes, optionMap)
}
func parseHwaddress(options map[string][]string, iface string) (net.HardwareAddr, error) {
if hwaddress, ok := options["hwaddress"]; ok && len(hwaddress) == 2 {
switch hwaddress[0] {
case "ether":
if address, err := net.ParseMAC(hwaddress[1]); err == nil {
return address, nil
}
return nil, fmt.Errorf("malformed hwaddress option for %q", iface)
}
}
return nil, nil
}
func parseBondStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
return &stanzaInterface{name: iface, kind: interfaceBond, configMethod: conf, options: options}, nil
}
func parsePhysicalStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
return &stanzaInterface{name: iface, kind: interfacePhysical, configMethod: conf, options: options}, nil
}
func parseVLANStanza(iface string, conf configMethod, attributes []string, options map[string][]string) (*stanzaInterface, error) {
var id string
if strings.Contains(iface, ".") {
tokens := strings.Split(iface, ".")
id = tokens[len(tokens)-1]
} else if strings.HasPrefix(iface, "vlan") {
id = strings.TrimPrefix(iface, "vlan")
} else {
return nil, fmt.Errorf("malformed vlan name %q", iface)
}
if _, err := strconv.Atoi(id); err != nil {
return nil, fmt.Errorf("malformed vlan name %q", iface)
}
options["id"] = []string{id}
options["raw_device"] = options["vlan_raw_device"]
return &stanzaInterface{name: iface, kind: interfaceVLAN, configMethod: conf, options: options}, nil
}

568
network/stanza_test.go Normal file
View File

@@ -0,0 +1,568 @@
package network
import (
"net"
"reflect"
"strings"
"testing"
)
func TestSplitStanzasNoParent(t *testing.T) {
in := []string{"test"}
e := "missing stanza start"
_, err := splitStanzas(in)
if err == nil || !strings.HasPrefix(err.Error(), e) {
t.Fatalf("bad error for splitStanzas(%q): got %q, want %q", in, err, e)
}
}
func TestBadParseStanzas(t *testing.T) {
for in, e := range map[string]string{
"": "missing stanza start",
"iface": "malformed stanza start",
"allow-?? unknown": "unknown stanza",
} {
_, err := parseStanzas([]string{in})
if err == nil || !strings.HasPrefix(err.Error(), e) {
t.Fatalf("bad error for parseStanzas(%q): got %q, want %q", in, err, e)
}
}
}
func TestBadParseInterfaceStanza(t *testing.T) {
for _, tt := range []struct {
in []string
opts []string
e string
}{
{[]string{}, nil, "incorrect number of attributes"},
{[]string{"eth", "inet", "invalid"}, nil, "invalid config method"},
{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100"}, "malformed static network config"},
{[]string{"eth", "inet", "static"}, []string{"netmask 255.255.255.0"}, "malformed static network config"},
{[]string{"eth", "inet", "static"}, []string{"address invalid", "netmask 255.255.255.0"}, "malformed static network config"},
{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100", "netmask invalid"}, "malformed static network config"},
{[]string{"eth", "inet", "static"}, []string{"address 192.168.1.100", "netmask 255.255.255.0", "hwaddress ether NotAnAddress"}, "malformed hwaddress option"},
{[]string{"eth", "inet", "dhcp"}, []string{"hwaddress ether NotAnAddress"}, "malformed hwaddress option"},
} {
_, err := parseInterfaceStanza(tt.in, tt.opts)
if err == nil || !strings.HasPrefix(err.Error(), tt.e) {
t.Fatalf("bad error parsing interface stanza %q: got %q, want %q", tt.in, err.Error(), tt.e)
}
}
}
func TestBadParseVLANStanzas(t *testing.T) {
conf := configMethodManual{}
options := map[string][]string{}
for _, in := range []string{"myvlan", "eth.vlan"} {
_, err := parseVLANStanza(in, conf, nil, options)
if err == nil || !strings.HasPrefix(err.Error(), "malformed vlan name") {
t.Fatalf("did not error on bad vlan %q", in)
}
}
}
func TestSplitStanzas(t *testing.T) {
expect := [][]string{
{"auto lo"},
{"iface eth1", "option: 1"},
{"mapping"},
{"allow-"},
}
lines := make([]string, 0, 5)
for _, stanza := range expect {
for _, line := range stanza {
lines = append(lines, line)
}
}
stanzas, err := splitStanzas(lines)
if err != nil {
t.FailNow()
}
for i, stanza := range stanzas {
if len(stanza) != len(expect[i]) {
t.FailNow()
}
for j, line := range stanza {
if line != expect[i][j] {
t.FailNow()
}
}
}
}
func TestParseStanzaNil(t *testing.T) {
defer func() {
if r := recover(); r == nil {
t.Fatal("parseStanza(nil) did not panic")
}
}()
parseStanza(nil)
}
func TestParseStanzaSuccess(t *testing.T) {
for _, in := range []string{
"auto a",
"iface a inet manual",
} {
if _, err := parseStanza([]string{in}); err != nil {
t.Fatalf("unexpected error parsing stanza %q: %s", in, err)
}
}
}
func TestParseAutoStanza(t *testing.T) {
interfaces := []string{"test", "attribute"}
stanza, err := parseAutoStanza(interfaces, nil)
if err != nil {
t.Fatalf("unexpected error parsing auto stanza %q: %s", interfaces, err)
}
if !reflect.DeepEqual(stanza.interfaces, interfaces) {
t.FailNow()
}
}
func TestParseBondStanzaNoSlaves(t *testing.T) {
bond, err := parseBondStanza("", nil, nil, map[string][]string{})
if err != nil {
t.FailNow()
}
if bond.options["bond-slaves"] != nil {
t.FailNow()
}
}
func TestParseBondStanza(t *testing.T) {
conf := configMethodManual{}
options := map[string][]string{
"bond-slaves": []string{"1", "2"},
}
bond, err := parseBondStanza("test", conf, nil, options)
if err != nil {
t.FailNow()
}
if bond.name != "test" {
t.FailNow()
}
if bond.kind != interfaceBond {
t.FailNow()
}
if bond.configMethod != conf {
t.FailNow()
}
}
func TestParsePhysicalStanza(t *testing.T) {
conf := configMethodManual{}
options := map[string][]string{
"a": []string{"1", "2"},
"b": []string{"1"},
}
physical, err := parsePhysicalStanza("test", conf, nil, options)
if err != nil {
t.FailNow()
}
if physical.name != "test" {
t.FailNow()
}
if physical.kind != interfacePhysical {
t.FailNow()
}
if physical.configMethod != conf {
t.FailNow()
}
if !reflect.DeepEqual(physical.options, options) {
t.FailNow()
}
}
func TestParseVLANStanzas(t *testing.T) {
conf := configMethodManual{}
options := map[string][]string{}
for _, in := range []string{"vlan25", "eth.25"} {
vlan, err := parseVLANStanza(in, conf, nil, options)
if err != nil {
t.Fatalf("unexpected error from parseVLANStanza(%q): %s", in, err)
}
if !reflect.DeepEqual(vlan.options["id"], []string{"25"}) {
t.FailNow()
}
}
}
func TestParseInterfaceStanzaStaticAddress(t *testing.T) {
options := []string{"address 192.168.1.100", "netmask 255.255.255.0"}
expect := []net.IPNet{
{
IP: net.IPv4(192, 168, 1, 100),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
if err != nil {
t.FailNow()
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.FailNow()
}
if !reflect.DeepEqual(static.addresses, expect) {
t.FailNow()
}
}
func TestParseInterfaceStanzaStaticGateway(t *testing.T) {
options := []string{"address 192.168.1.100", "netmask 255.255.255.0", "gateway 192.168.1.1"}
expect := []route{
{
destination: net.IPNet{
IP: net.IPv4(0, 0, 0, 0),
Mask: net.IPv4Mask(0, 0, 0, 0),
},
gateway: net.IPv4(192, 168, 1, 1),
},
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
if err != nil {
t.FailNow()
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.FailNow()
}
if !reflect.DeepEqual(static.routes, expect) {
t.FailNow()
}
}
func TestParseInterfaceStanzaStaticDNS(t *testing.T) {
options := []string{"address 192.168.1.100", "netmask 255.255.255.0", "dns-nameservers 192.168.1.10 192.168.1.11 192.168.1.12"}
expect := []net.IP{
net.IPv4(192, 168, 1, 10),
net.IPv4(192, 168, 1, 11),
net.IPv4(192, 168, 1, 12),
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
if err != nil {
t.FailNow()
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.FailNow()
}
if !reflect.DeepEqual(static.nameservers, expect) {
t.FailNow()
}
}
func TestBadParseInterfaceStanzasStaticPostUp(t *testing.T) {
for _, in := range []string{
"post-up invalid",
"post-up route add",
"post-up route add -net",
"post-up route add gw",
"post-up route add netmask",
"gateway",
"gateway 192.168.1.1 192.168.1.2",
} {
options := []string{"address 192.168.1.100", "netmask 255.255.255.0", in}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, options)
if err != nil {
t.Fatalf("parseInterfaceStanza with options %s got unexpected error", options)
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.Fatalf("parseInterfaceStanza with options %s did not return configMethodStatic", options)
}
if len(static.routes) != 0 {
t.Fatalf("parseInterfaceStanza with options %s did not return zero-length static routes", options)
}
}
}
func TestParseInterfaceStanzaStaticPostUp(t *testing.T) {
for _, tt := range []struct {
options []string
expect []route
}{
{
options: []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0 netmask 255.255.255.0",
},
expect: []route{
{
destination: net.IPNet{
IP: net.IPv4(192, 168, 1, 0),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
gateway: net.IPv4(192, 168, 1, 1),
},
},
},
{
options: []string{
"address 192.168.1.100",
"netmask 255.255.255.0",
"post-up route add gw 192.168.1.1 -net 192.168.1.0/24 || true",
},
expect: []route{
{
destination: func() net.IPNet {
if _, net, err := net.ParseCIDR("192.168.1.0/24"); err == nil {
return *net
} else {
panic(err)
}
}(),
gateway: net.IPv4(192, 168, 1, 1),
},
},
},
} {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "static"}, tt.options)
if err != nil {
t.Fatalf("bad error (%+v): want nil, got %s\n", tt, err)
}
static, ok := iface.configMethod.(configMethodStatic)
if !ok {
t.Fatalf("bad config method (%+v): want configMethodStatic, got %T\n", tt, iface.configMethod)
}
if !reflect.DeepEqual(static.routes, tt.expect) {
t.Fatalf("bad routes (%+v): want %#v, got %#v\n", tt, tt.expect, static.routes)
}
}
}
func TestParseInterfaceStanzaLoopback(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "loopback"}, nil)
if err != nil {
t.FailNow()
}
if _, ok := iface.configMethod.(configMethodLoopback); !ok {
t.FailNow()
}
}
func TestParseInterfaceStanzaManual(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, nil)
if err != nil {
t.FailNow()
}
if _, ok := iface.configMethod.(configMethodManual); !ok {
t.FailNow()
}
}
func TestParseInterfaceStanzaDHCP(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"eth", "inet", "dhcp"}, nil)
if err != nil {
t.FailNow()
}
if _, ok := iface.configMethod.(configMethodDHCP); !ok {
t.FailNow()
}
}
func TestParseInterfaceStanzaPostUpOption(t *testing.T) {
options := []string{
"post-up",
"post-up 1 2",
"post-up 3 4",
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
if err != nil {
t.FailNow()
}
if !reflect.DeepEqual(iface.options["post-up"], []string{"1 2", "3 4"}) {
t.Log(iface.options["post-up"])
t.FailNow()
}
}
func TestParseInterfaceStanzaPreDownOption(t *testing.T) {
options := []string{
"pre-down",
"pre-down 3",
"pre-down 4",
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
if err != nil {
t.FailNow()
}
if !reflect.DeepEqual(iface.options["pre-down"], []string{"3", "4"}) {
t.Log(iface.options["pre-down"])
t.FailNow()
}
}
func TestParseInterfaceStanzaEmptyOption(t *testing.T) {
options := []string{
"test",
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
if err != nil {
t.FailNow()
}
if !reflect.DeepEqual(iface.options["test"], []string{}) {
t.FailNow()
}
}
func TestParseInterfaceStanzaOptions(t *testing.T) {
options := []string{
"test1 1",
"test2 2 3",
"test1 5 6",
}
iface, err := parseInterfaceStanza([]string{"eth", "inet", "manual"}, options)
if err != nil {
t.FailNow()
}
if !reflect.DeepEqual(iface.options["test1"], []string{"5", "6"}) {
t.Log(iface.options["test1"])
t.FailNow()
}
if !reflect.DeepEqual(iface.options["test2"], []string{"2", "3"}) {
t.Log(iface.options["test2"])
t.FailNow()
}
}
func TestParseInterfaceStanzaHwaddress(t *testing.T) {
for _, tt := range []struct {
attr []string
opt []string
hw net.HardwareAddr
}{
{
[]string{"mybond", "inet", "dhcp"},
[]string{},
nil,
},
{
[]string{"mybond", "inet", "dhcp"},
[]string{"hwaddress ether 00:01:02:03:04:05"},
net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
},
{
[]string{"mybond", "inet", "static"},
[]string{"hwaddress ether 00:01:02:03:04:05", "address 192.168.1.100", "netmask 255.255.255.0"},
net.HardwareAddr([]byte{0, 1, 2, 3, 4, 5}),
},
} {
iface, err := parseInterfaceStanza(tt.attr, tt.opt)
if err != nil {
t.Fatalf("error in parseInterfaceStanza (%q, %q): %q", tt.attr, tt.opt, err)
}
switch c := iface.configMethod.(type) {
case configMethodStatic:
if !reflect.DeepEqual(c.hwaddress, tt.hw) {
t.Fatalf("bad hwaddress (%q, %q): got %q, want %q", tt.attr, tt.opt, c.hwaddress, tt.hw)
}
case configMethodDHCP:
if !reflect.DeepEqual(c.hwaddress, tt.hw) {
t.Fatalf("bad hwaddress (%q, %q): got %q, want %q", tt.attr, tt.opt, c.hwaddress, tt.hw)
}
}
}
}
func TestParseInterfaceStanzaBond(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"mybond", "inet", "manual"}, []string{"bond-slaves eth"})
if err != nil {
t.FailNow()
}
if iface.kind != interfaceBond {
t.FailNow()
}
}
func TestParseInterfaceStanzaVLANName(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"eth0.1", "inet", "manual"}, nil)
if err != nil {
t.FailNow()
}
if iface.kind != interfaceVLAN {
t.FailNow()
}
}
func TestParseInterfaceStanzaVLANOption(t *testing.T) {
iface, err := parseInterfaceStanza([]string{"vlan1", "inet", "manual"}, []string{"vlan_raw_device eth"})
if err != nil {
t.FailNow()
}
if iface.kind != interfaceVLAN {
t.FailNow()
}
}
func TestParseStanzasNone(t *testing.T) {
stanzas, err := parseStanzas(nil)
if err != nil {
t.FailNow()
}
if len(stanzas) != 0 {
t.FailNow()
}
}
func TestParseStanzas(t *testing.T) {
lines := []string{
"auto lo",
"iface lo inet loopback",
"iface eth1 inet manual",
"iface eth2 inet manual",
"iface eth3 inet manual",
"auto eth1 eth3",
}
expect := []stanza{
&stanzaAuto{
interfaces: []string{"lo"},
},
&stanzaInterface{
name: "lo",
kind: interfacePhysical,
auto: true,
configMethod: configMethodLoopback{},
options: map[string][]string{},
},
&stanzaInterface{
name: "eth1",
kind: interfacePhysical,
auto: true,
configMethod: configMethodManual{},
options: map[string][]string{},
},
&stanzaInterface{
name: "eth2",
kind: interfacePhysical,
auto: false,
configMethod: configMethodManual{},
options: map[string][]string{},
},
&stanzaInterface{
name: "eth3",
kind: interfacePhysical,
auto: true,
configMethod: configMethodManual{},
options: map[string][]string{},
},
&stanzaAuto{
interfaces: []string{"eth1", "eth3"},
},
}
stanzas, err := parseStanzas(lines)
if err != err {
t.FailNow()
}
if !reflect.DeepEqual(stanzas, expect) {
t.FailNow()
}
}

161
pkg/http_client.go Normal file
View File

@@ -0,0 +1,161 @@
package pkg
import (
"crypto/tls"
"errors"
"fmt"
"io/ioutil"
"log"
"net"
"net/http"
neturl "net/url"
"strings"
"time"
)
const (
HTTP_2xx = 2
HTTP_4xx = 4
)
type Err error
type ErrTimeout struct {
Err
}
type ErrNotFound struct {
Err
}
type ErrInvalid struct {
Err
}
type ErrServer struct {
Err
}
type ErrNetwork struct {
Err
}
type HttpClient struct {
// Maximum exp backoff duration. Defaults to 5 seconds
MaxBackoff time.Duration
// Maximum number of connection retries. Defaults to 15
MaxRetries int
// HTTP client timeout, this is suggested to be low since exponential
// backoff will kick off too. Defaults to 2 seconds
Timeout time.Duration
// Whether or not to skip TLS verification. Defaults to false
SkipTLS bool
client *http.Client
}
type Getter interface {
Get(string) ([]byte, error)
GetRetry(string) ([]byte, error)
}
func NewHttpClient() *HttpClient {
hc := &HttpClient{
MaxBackoff: time.Second * 5,
MaxRetries: 15,
Timeout: time.Duration(2) * time.Second,
SkipTLS: false,
}
// We need to create our own client in order to add timeout support.
// TODO(c4milo) Replace it once Go 1.3 is officially used by CoreOS
// More info: https://code.google.com/p/go/source/detail?r=ada6f2d5f99f
hc.client = &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: hc.SkipTLS,
},
Dial: func(network, addr string) (net.Conn, error) {
deadline := time.Now().Add(hc.Timeout)
c, err := net.DialTimeout(network, addr, hc.Timeout)
if err != nil {
return nil, err
}
c.SetDeadline(deadline)
return c, nil
},
},
}
return hc
}
func ExpBackoff(interval, max time.Duration) time.Duration {
interval = interval * 2
if interval > max {
interval = max
}
return interval
}
// GetRetry fetches a given URL with support for exponential backoff and maximum retries
func (h *HttpClient) GetRetry(rawurl string) ([]byte, error) {
if rawurl == "" {
return nil, ErrInvalid{errors.New("URL is empty. Skipping.")}
}
url, err := neturl.Parse(rawurl)
if err != nil {
return nil, ErrInvalid{err}
}
// Unfortunately, url.Parse is too generic to throw errors if a URL does not
// have a valid HTTP scheme. So, we have to do this extra validation
if !strings.HasPrefix(url.Scheme, "http") {
return nil, ErrInvalid{fmt.Errorf("URL %s does not have a valid HTTP scheme. Skipping.", rawurl)}
}
dataURL := url.String()
duration := 50 * time.Millisecond
for retry := 1; retry <= h.MaxRetries; retry++ {
log.Printf("Fetching data from %s. Attempt #%d", dataURL, retry)
data, err := h.Get(dataURL)
switch err.(type) {
case ErrNetwork:
log.Printf(err.Error())
case ErrServer:
log.Printf(err.Error())
case ErrNotFound:
return data, err
default:
return data, err
}
duration = ExpBackoff(duration, h.MaxBackoff)
log.Printf("Sleeping for %v...", duration)
time.Sleep(duration)
}
return nil, ErrTimeout{fmt.Errorf("Unable to fetch data. Maximum retries reached: %d", h.MaxRetries)}
}
func (h *HttpClient) Get(dataURL string) ([]byte, error) {
if resp, err := h.client.Get(dataURL); err == nil {
defer resp.Body.Close()
switch resp.StatusCode / 100 {
case HTTP_2xx:
return ioutil.ReadAll(resp.Body)
case HTTP_4xx:
return nil, ErrNotFound{fmt.Errorf("Not found. HTTP status code: %d", resp.StatusCode)}
default:
return nil, ErrServer{fmt.Errorf("Server error. HTTP status code: %d", resp.StatusCode)}
}
} else {
return nil, ErrNetwork{fmt.Errorf("Unable to fetch data: %s", err.Error())}
}
}

140
pkg/http_client_test.go Normal file
View File

@@ -0,0 +1,140 @@
package pkg
import (
"fmt"
"io"
"math"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestExpBackoff(t *testing.T) {
duration := time.Millisecond
max := time.Hour
for i := 0; i < math.MaxUint16; i++ {
duration = ExpBackoff(duration, max)
if duration < 0 {
t.Fatalf("duration too small: %v %v", duration, i)
}
if duration > max {
t.Fatalf("duration too large: %v %v", duration, i)
}
}
}
// Test exponential backoff and that it continues retrying if a 5xx response is
// received
func TestGetURLExpBackOff(t *testing.T) {
var expBackoffTests = []struct {
count int
body string
}{
{0, "number of attempts: 0"},
{1, "number of attempts: 1"},
{2, "number of attempts: 2"},
}
client := NewHttpClient()
for i, tt := range expBackoffTests {
mux := http.NewServeMux()
count := 0
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if count == tt.count {
io.WriteString(w, fmt.Sprintf("number of attempts: %d", count))
return
}
count++
http.Error(w, "", 500)
})
ts := httptest.NewServer(mux)
defer ts.Close()
data, err := client.GetRetry(ts.URL)
if err != nil {
t.Errorf("Test case %d produced error: %v", i, err)
}
if count != tt.count {
t.Errorf("Test case %d failed: %d != %d", i, count, tt.count)
}
if string(data) != tt.body {
t.Errorf("Test case %d failed: %s != %s", i, tt.body, data)
}
}
}
// Test that it stops retrying if a 4xx response comes back
func TestGetURL4xx(t *testing.T) {
client := NewHttpClient()
retries := 0
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
retries++
http.Error(w, "", 404)
}))
defer ts.Close()
_, err := client.GetRetry(ts.URL)
if err == nil {
t.Errorf("Incorrect result\ngot: %s\nwant: %s", err.Error(), "Not found. HTTP status code: 404")
}
if retries > 1 {
t.Errorf("Number of retries:\n%d\nExpected number of retries:\n%s", retries, 1)
}
}
// Test that it fetches and returns user-data just fine
func TestGetURL2xx(t *testing.T) {
var cloudcfg = `
#cloud-config
coreos:
oem:
id: test
name: CoreOS.box for Test
version-id: %VERSION_ID%+%BUILD_ID%
home-url: https://github.com/coreos/coreos-cloudinit
bug-report-url: https://github.com/coreos/coreos-cloudinit
update:
reboot-strategy: best-effort
`
client := NewHttpClient()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, cloudcfg)
}))
defer ts.Close()
data, err := client.GetRetry(ts.URL)
if err != nil {
t.Errorf("Incorrect result\ngot: %v\nwant: %v", err, nil)
}
if string(data) != cloudcfg {
t.Errorf("Incorrect result\ngot: %s\nwant: %s", string(data), cloudcfg)
}
}
// Test attempt to fetching using malformed URL
func TestGetMalformedURL(t *testing.T) {
client := NewHttpClient()
var tests = []struct {
url string
want string
}{
{"boo", "URL boo does not have a valid HTTP scheme. Skipping."},
{"mailto://boo", "URL mailto://boo does not have a valid HTTP scheme. Skipping."},
{"ftp://boo", "URL ftp://boo does not have a valid HTTP scheme. Skipping."},
{"", "URL is empty. Skipping."},
}
for _, test := range tests {
_, err := client.GetRetry(test.url)
if err == nil || err.Error() != test.want {
t.Errorf("Incorrect result\ngot: %v\nwant: %v", err, test.want)
}
}
}

View File

@@ -1 +0,0 @@
../../../

26
system/env.go Normal file
View File

@@ -0,0 +1,26 @@
package system
import (
"fmt"
"reflect"
)
// dropinContents generates the contents for a drop-in unit given the config.
// The argument must be a struct from the 'config' package.
func dropinContents(e interface{}) string {
et := reflect.TypeOf(e)
ev := reflect.ValueOf(e)
var out string
for i := 0; i < et.NumField(); i++ {
if val := ev.Field(i).String(); val != "" {
key := et.Field(i).Tag.Get("env")
out += fmt.Sprintf("Environment=\"%s=%s\"\n", key, val)
}
}
if out == "" {
return ""
}
return "[Service]\n" + out
}

100
system/env_file.go Normal file
View File

@@ -0,0 +1,100 @@
package system
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"path"
"regexp"
"sort"
)
type EnvFile struct {
Vars map[string]string
// mask File.Content, it shouldn't be used.
Content interface{} `json:"-" yaml:"-"`
*File
}
// only allow sh compatible identifiers
var validKey = regexp.MustCompile(`^[a-zA-Z0-9_]+$`)
// match each line, optionally capturing valid identifiers, discarding dos line endings
var lineLexer = regexp.MustCompile(`(?m)^((?:([a-zA-Z0-9_]+)=)?.*?)\r?\n`)
// mergeEnvContents: Update the existing file contents with new values,
// preserving variable ordering and all content this code doesn't understand.
// All new values are appended to the bottom of the old, sorted by key.
func mergeEnvContents(old []byte, pending map[string]string) []byte {
var buf bytes.Buffer
var match [][]byte
// it is awkward for the regex to handle a missing newline gracefully
if len(old) != 0 && !bytes.HasSuffix(old, []byte{'\n'}) {
old = append(old, byte('\n'))
}
for _, match = range lineLexer.FindAllSubmatch(old, -1) {
key := string(match[2])
if value, ok := pending[key]; ok {
fmt.Fprintf(&buf, "%s=%s\n", key, value)
delete(pending, key)
} else {
fmt.Fprintf(&buf, "%s\n", match[1])
}
}
for _, key := range keys(pending) {
value := pending[key]
fmt.Fprintf(&buf, "%s=%s\n", key, value)
}
return buf.Bytes()
}
// WriteEnvFile updates an existing env `KEY=value` formated file with
// new values provided in EnvFile.Vars; File.Content is ignored.
// Existing ordering and any unknown formatting such as comments are
// preserved. If no changes are required the file is untouched.
func WriteEnvFile(ef *EnvFile, root string) error {
// validate new keys, mergeEnvContents uses pending to track writes
pending := make(map[string]string, len(ef.Vars))
for key, value := range ef.Vars {
if !validKey.MatchString(key) {
return fmt.Errorf("Invalid name %q for %s", key, ef.Path)
}
pending[key] = value
}
if len(pending) == 0 {
return nil
}
oldContent, err := ioutil.ReadFile(path.Join(root, ef.Path))
if err != nil {
if os.IsNotExist(err) {
oldContent = []byte{}
} else {
return err
}
}
newContent := mergeEnvContents(oldContent, pending)
if bytes.Equal(oldContent, newContent) {
return nil
}
ef.File.Content = string(newContent)
_, err = WriteFile(ef.File, root)
return err
}
// keys returns the keys of a map in sorted order
func keys(m map[string]string) (s []string) {
for k, _ := range m {
s = append(s, k)
}
sort.Strings(s)
return
}

428
system/env_file_test.go Normal file
View File

@@ -0,0 +1,428 @@
package system
import (
"io/ioutil"
"os"
"path"
"strings"
"syscall"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
const (
base = "# a file\nFOO=base\n\nBAR= hi there\n"
baseNoNewline = "# a file\nFOO=base\n\nBAR= hi there"
baseDos = "# a file\r\nFOO=base\r\n\r\nBAR= hi there\r\n"
expectUpdate = "# a file\nFOO=test\n\nBAR= hi there\nNEW=a value\n"
expectCreate = "FOO=test\nNEW=a value\n"
)
var (
valueUpdate = map[string]string{
"FOO": "test",
"NEW": "a value",
}
valueNoop = map[string]string{
"FOO": "base",
}
valueEmpty = map[string]string{}
valueInvalid = map[string]string{
"FOO-X": "test",
}
)
func TestWriteEnvFileUpdate(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(base), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueUpdate,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expectUpdate {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was not replaced: %s", fullPath)
}
}
func TestWriteEnvFileUpdateNoNewline(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(baseNoNewline), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueUpdate,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expectUpdate {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was not replaced: %s", fullPath)
}
}
func TestWriteEnvFileCreate(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueUpdate,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expectCreate {
t.Fatalf("File has incorrect contents: %q", contents)
}
}
func TestWriteEnvFileNoop(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(base), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueNoop,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != base {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino != newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was replaced: %s", fullPath)
}
}
func TestWriteEnvFileUpdateDos(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueUpdate,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expectUpdate {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was not replaced: %s", fullPath)
}
}
// A middle ground noop, values are unchanged but we did have a value.
// Seems reasonable to rewrite the file in Unix format anyway.
func TestWriteEnvFileDos2Unix(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueNoop,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != base {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino == newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was not replaced: %s", fullPath)
}
}
// If it really is a noop (structure is empty) don't even do dos2unix
func TestWriteEnvFileEmpty(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(baseDos), 0644)
oldStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueEmpty,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != baseDos {
t.Fatalf("File has incorrect contents: %q", contents)
}
newStat, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if oldStat.Sys().(*syscall.Stat_t).Ino != newStat.Sys().(*syscall.Stat_t).Ino {
t.Fatalf("File was replaced: %s", fullPath)
}
}
// no point in creating empty files
func TestWriteEnvFileEmptyNoCreate(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueEmpty,
}
err = WriteEnvFile(&ef, dir)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
contents, err := ioutil.ReadFile(fullPath)
if err == nil {
t.Fatalf("File has incorrect contents: %q", contents)
} else if !os.IsNotExist(err) {
t.Fatalf("Unexpected error while reading file: %v", err)
}
}
func TestWriteEnvFilePermFailure(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
fullPath := path.Join(dir, name)
ioutil.WriteFile(fullPath, []byte(base), 0000)
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueUpdate,
}
err = WriteEnvFile(&ef, dir)
if !os.IsPermission(err) {
t.Fatalf("Not a pemission denied error: %v", err)
}
}
func TestWriteEnvFileNameFailure(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
name := "foo.conf"
ef := EnvFile{
File: &File{config.File{
Path: name,
}},
Vars: valueInvalid,
}
err = WriteEnvFile(&ef, dir)
if err == nil || !strings.HasPrefix(err.Error(), "Invalid name") {
t.Fatalf("Not an invalid name error: %v", err)
}
}

48
system/etc_hosts.go Normal file
View File

@@ -0,0 +1,48 @@
package system
import (
"errors"
"fmt"
"os"
"path"
"github.com/coreos/coreos-cloudinit/config"
)
const DefaultIpv4Address = "127.0.0.1"
type EtcHosts struct {
Config config.EtcHosts
}
func (eh EtcHosts) generateEtcHosts() (out string, err error) {
if eh.Config != "localhost" {
return "", errors.New("Invalid option to manage_etc_hosts")
}
// use the operating system hostname
hostname, err := os.Hostname()
if err != nil {
return "", err
}
return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
}
func (eh EtcHosts) File() (*File, error) {
if eh.Config == "" {
return nil, nil
}
etcHosts, err := eh.generateEtcHosts()
if err != nil {
return nil, err
}
return &File{config.File{
Path: path.Join("etc", "hosts"),
RawFilePermissions: "0644",
Content: etcHosts,
}}, nil
}

46
system/etc_hosts_test.go Normal file
View File

@@ -0,0 +1,46 @@
package system
import (
"fmt"
"os"
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestEtcdHostsFile(t *testing.T) {
hostname, err := os.Hostname()
if err != nil {
panic(err)
}
for _, tt := range []struct {
config config.EtcHosts
file *File
err error
}{
{
"invalid",
nil,
fmt.Errorf("Invalid option to manage_etc_hosts"),
},
{
"localhost",
&File{config.File{
Content: fmt.Sprintf("127.0.0.1 %s\n", hostname),
Path: "etc/hosts",
RawFilePermissions: "0644",
}},
nil,
},
} {
file, err := EtcHosts{tt.config}.File()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

25
system/etcd.go Normal file
View File

@@ -0,0 +1,25 @@
package system
import (
"github.com/coreos/coreos-cloudinit/config"
)
// Etcd is a top-level structure which embeds its underlying configuration,
// config.Etcd, and provides the system-specific Unit().
type Etcd struct {
config.Etcd
}
// Units creates a Unit file drop-in for etcd, using any configured options.
func (ee Etcd) Units() ([]Unit, error) {
content := dropinContents(ee.Etcd)
if content == "" {
return nil, nil
}
return []Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: content,
}}}, nil
}

60
system/etcd_test.go Normal file
View File

@@ -0,0 +1,60 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestEtcdUnits(t *testing.T) {
for _, tt := range []struct {
config config.Etcd
units []Unit
}{
{
config.Etcd{},
nil,
},
{
config.Etcd{
Discovery: "http://disco.example.com/foobar",
PeerBindAddr: "127.0.0.1:7002",
},
[]Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`,
}}},
},
{
config.Etcd{
Name: "node001",
Discovery: "http://disco.example.com/foobar",
PeerBindAddr: "127.0.0.1:7002",
},
[]Unit{{config.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_NAME=node001"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`,
}}},
},
} {
units, err := Etcd{tt.config}.Units()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(tt.units, units) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.units, units)
}
}
}

View File

@@ -8,14 +8,14 @@ import (
"os/exec"
"path"
"strconv"
"github.com/coreos/coreos-cloudinit/config"
)
// File is a top-level structure which embeds its underlying configuration,
// config.File, and provides the system-specific Permissions().
type File struct {
Encoding string
Content string
Owner string
Path string
RawFilePermissions string `yaml:"permissions"`
config.File
}
func (f *File) Permissions() (os.FileMode, error) {
@@ -31,33 +31,55 @@ func (f *File) Permissions() (os.FileMode, error) {
return os.FileMode(perm), nil
}
func WriteFile(f *File) error {
func WriteFile(f *File, root string) (string, error) {
if f.Encoding != "" {
return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
return "", fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
}
if err := os.MkdirAll(path.Dir(f.Path), os.FileMode(0755)); err != nil {
return err
fullpath := path.Join(root, f.Path)
dir := path.Dir(fullpath)
if err := EnsureDirectoryExists(dir); err != nil {
return "", err
}
perm, err := f.Permissions()
if err != nil {
return err
return "", err
}
if err := ioutil.WriteFile(f.Path, []byte(f.Content), perm); err != nil {
return err
var tmp *os.File
// Create a temporary file in the same directory to ensure it's on the same filesystem
if tmp, err = ioutil.TempFile(dir, "cloudinit-temp"); err != nil {
return "", err
}
if err := ioutil.WriteFile(tmp.Name(), []byte(f.Content), perm); err != nil {
return "", err
}
if err := tmp.Close(); err != nil {
return "", err
}
// Ensure the permissions are as requested (since WriteFile can be affected by sticky bit)
if err := os.Chmod(tmp.Name(), perm); err != nil {
return "", err
}
if f.Owner != "" {
// We shell out since we don't have a way to look up unix groups natively
cmd := exec.Command("chown", f.Owner, f.Path)
cmd := exec.Command("chown", f.Owner, tmp.Name())
if err := cmd.Run(); err != nil {
return err
return "", err
}
}
return nil
if err := os.Rename(tmp.Name(), fullpath); err != nil {
return "", err
}
return fullpath, nil
}
func EnsureDirectoryExists(dir string) error {

View File

@@ -4,8 +4,9 @@ import (
"io/ioutil"
"os"
"path"
"syscall"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestWriteFileUnencodedContent(t *testing.T) {
@@ -13,18 +14,22 @@ func TestWriteFileUnencodedContent(t *testing.T) {
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
defer os.RemoveAll(dir)
fullPath := path.Join(dir, "tmp", "foo")
fn := "foo"
fullPath := path.Join(dir, fn)
wf := File{
Path: fullPath,
wf := File{config.File{
Path: fn,
Content: "bar",
RawFilePermissions: "0644",
}
}}
if err := WriteFile(&wf); err != nil {
path, err := WriteFile(&wf, dir)
if err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
} else if path != fullPath {
t.Fatalf("WriteFile returned bad path: want %s, got %s", fullPath, path)
}
fi, err := os.Stat(fullPath)
@@ -51,15 +56,15 @@ func TestWriteFileInvalidPermission(t *testing.T) {
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
defer os.RemoveAll(dir)
wf := File{
wf := File{config.File{
Path: path.Join(dir, "tmp", "foo"),
Content: "bar",
RawFilePermissions: "pants",
}
}}
if err := WriteFile(&wf); err == nil {
if _, err := WriteFile(&wf, dir); err == nil {
t.Fatalf("Expected error to be raised when writing file with invalid permission")
}
}
@@ -69,17 +74,21 @@ func TestWriteFilePermissions(t *testing.T) {
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
defer os.RemoveAll(dir)
fullPath := path.Join(dir, "tmp", "foo")
fn := "foo"
fullPath := path.Join(dir, fn)
wf := File{
Path: fullPath,
wf := File{config.File{
Path: fn,
RawFilePermissions: "0755",
}
}}
if err := WriteFile(&wf); err != nil {
path, err := WriteFile(&wf, dir)
if err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
} else if path != fullPath {
t.Fatalf("WriteFile returned bad path: want %s, got %s", fullPath, path)
}
fi, err := os.Stat(fullPath)
@@ -97,15 +106,15 @@ func TestWriteFileEncodedContent(t *testing.T) {
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
defer os.RemoveAll(dir)
wf := File{
wf := File{config.File{
Path: path.Join(dir, "tmp", "foo"),
Content: "",
Encoding: "base64",
}
}}
if err := WriteFile(&wf); err == nil {
if _, err := WriteFile(&wf, dir); err == nil {
t.Fatalf("Expected error to be raised when writing file with encoding")
}
}

26
system/fleet.go Normal file
View File

@@ -0,0 +1,26 @@
package system
import (
"github.com/coreos/coreos-cloudinit/config"
)
// Fleet is a top-level structure which embeds its underlying configuration,
// config.Fleet, and provides the system-specific Unit().
type Fleet struct {
config.Fleet
}
// Units generates a Unit file drop-in for fleet, if any fleet options were
// configured in cloud-config
func (fe Fleet) Units() ([]Unit, error) {
content := dropinContents(fe.Fleet)
if content == "" {
return nil, nil
}
return []Unit{{config.Unit{
Name: "fleet.service",
Runtime: true,
DropIn: true,
Content: content,
}}}, nil
}

41
system/fleet_test.go Normal file
View File

@@ -0,0 +1,41 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestFleetUnits(t *testing.T) {
for _, tt := range []struct {
config config.Fleet
units []Unit
}{
{
config.Fleet{},
nil,
},
{
config.Fleet{
PublicIP: "12.34.56.78",
},
[]Unit{{config.Unit{
Name: "fleet.service",
Content: `[Service]
Environment="FLEET_PUBLIC_IP=12.34.56.78"
`,
Runtime: true,
DropIn: true,
}}},
},
} {
units, err := Fleet{tt.config}.Units()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(units, tt.units) {
t.Errorf("bad units (%q): want %q, got %q", tt.config, tt.units, units)
}
}
}

119
system/networkd.go Normal file
View File

@@ -0,0 +1,119 @@
package system
import (
"fmt"
"log"
"net"
"os/exec"
"strings"
"time"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/network"
"github.com/coreos/coreos-cloudinit/third_party/github.com/dotcloud/docker/pkg/netlink"
)
const (
runtimeNetworkPath = "/run/systemd/network"
)
func RestartNetwork(interfaces []network.InterfaceGenerator) (err error) {
defer func() {
if e := restartNetworkd(); e != nil {
err = e
return
}
// TODO(crawford): Get rid of this once networkd fixes the race
// https://bugs.freedesktop.org/show_bug.cgi?id=76077
time.Sleep(5 * time.Second)
if e := restartNetworkd(); e != nil {
err = e
}
}()
if err = downNetworkInterfaces(interfaces); err != nil {
return
}
if err = maybeProbe8012q(interfaces); err != nil {
return
}
return maybeProbeBonding(interfaces)
}
func downNetworkInterfaces(interfaces []network.InterfaceGenerator) error {
sysInterfaceMap := make(map[string]*net.Interface)
if systemInterfaces, err := net.Interfaces(); err == nil {
for _, iface := range systemInterfaces {
iface := iface
sysInterfaceMap[iface.Name] = &iface
}
} else {
return err
}
for _, iface := range interfaces {
if systemInterface, ok := sysInterfaceMap[iface.Name()]; ok {
log.Printf("Taking down interface %q\n", systemInterface.Name)
if err := netlink.NetworkLinkDown(systemInterface); err != nil {
fmt.Printf("Error while downing interface %q (%s). Continuing...\n", systemInterface.Name, err)
}
}
}
return nil
}
func maybeProbe8012q(interfaces []network.InterfaceGenerator) error {
for _, iface := range interfaces {
if iface.Type() == "vlan" {
log.Printf("Probing LKM %q (%q)\n", "8021q", "8021q")
return exec.Command("modprobe", "8021q").Run()
}
}
return nil
}
func maybeProbeBonding(interfaces []network.InterfaceGenerator) error {
for _, iface := range interfaces {
if iface.Type() == "bond" {
args := append([]string{"bonding"}, strings.Split(iface.ModprobeParams(), " ")...)
log.Printf("Probing LKM %q (%q)\n", "bonding", args)
return exec.Command("modprobe", args...).Run()
}
}
return nil
}
func restartNetworkd() error {
log.Printf("Restarting networkd.service\n")
_, err := NewUnitManager("").RunUnitCommand("restart", "systemd-networkd.service")
return err
}
func WriteNetworkdConfigs(interfaces []network.InterfaceGenerator) error {
for _, iface := range interfaces {
filename := fmt.Sprintf("%s.netdev", iface.Filename())
if err := writeConfig(filename, iface.Netdev()); err != nil {
return err
}
filename = fmt.Sprintf("%s.link", iface.Filename())
if err := writeConfig(filename, iface.Link()); err != nil {
return err
}
filename = fmt.Sprintf("%s.network", iface.Filename())
if err := writeConfig(filename, iface.Network()); err != nil {
return err
}
}
return nil
}
func writeConfig(filename string, content string) error {
if content == "" {
return nil
}
log.Printf("Writing networkd unit %q\n", filename)
_, err := WriteFile(&File{config.File{Content: content, Path: filename}}, runtimeNetworkPath)
return err
}

32
system/oem.go Normal file
View File

@@ -0,0 +1,32 @@
package system
import (
"fmt"
"path"
"github.com/coreos/coreos-cloudinit/config"
)
// OEM is a top-level structure which embeds its underlying configuration,
// config.OEM, and provides the system-specific File().
type OEM struct {
config.OEM
}
func (oem OEM) File() (*File, error) {
if oem.ID == "" {
return nil, nil
}
content := fmt.Sprintf("ID=%s\n", oem.ID)
content += fmt.Sprintf("VERSION_ID=%s\n", oem.VersionID)
content += fmt.Sprintf("NAME=%q\n", oem.Name)
content += fmt.Sprintf("HOME_URL=%q\n", oem.HomeURL)
content += fmt.Sprintf("BUG_REPORT_URL=%q\n", oem.BugReportURL)
return &File{config.File{
Path: path.Join("etc", "oem-release"),
RawFilePermissions: "0644",
Content: content,
}}, nil
}

47
system/oem_test.go Normal file
View File

@@ -0,0 +1,47 @@
package system
import (
"reflect"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestOEMFile(t *testing.T) {
for _, tt := range []struct {
config config.OEM
file *File
}{
{
config.OEM{},
nil,
},
{
config.OEM{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
},
&File{config.File{
Path: "etc/oem-release",
RawFilePermissions: "0644",
Content: `ID=rackspace
VERSION_ID=168.0.0
NAME="Rackspace Cloud Servers"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`,
}},
},
} {
file, err := OEM{tt.config}.File()
if err != nil {
t.Errorf("bad error (%q): want %q, got %q", tt.config, nil, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad file (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

View File

@@ -10,53 +10,25 @@ import (
"path/filepath"
"strings"
"github.com/coreos/coreos-cloudinit/config"
"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
)
func NewUnitManager(root string) UnitManager {
return &systemd{root}
}
type systemd struct {
root string
}
// fakeMachineID is placed on non-usr CoreOS images and should
// never be used as a true MachineID
const fakeMachineID = "42000000000000000000000000000042"
type Unit struct {
Name string
Enable bool
Runtime bool
Content string
Command string
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() (group string) {
t := u.Type()
if t == "network" || t == "netdev" || t == "link" {
group = "network"
} else {
group = "system"
}
return
}
type Script []byte
// UnitDestination builds the appropriate absolte file path for
// the given unit. The root argument indicates the effective base
// directory of the system (similar to a chroot).
func UnitDestination(u *Unit, root string) string {
dir := "etc"
if u.Runtime {
dir = "run"
}
return path.Join(root, dir, "systemd", u.Group(), u.Name)
}
// PlaceUnit writes a unit file at the provided destination, creating
// parent directories as necessary.
func PlaceUnit(u *Unit, dst string) error {
func (s *systemd) PlaceUnit(u *Unit, dst string) error {
dir := filepath.Dir(dst)
if _, err := os.Stat(dir); os.IsNotExist(err) {
if err := os.MkdirAll(dir, os.FileMode(0755)); err != nil {
@@ -64,13 +36,13 @@ func PlaceUnit(u *Unit, dst string) error {
}
}
file := File{
Path: dst,
file := File{config.File{
Path: filepath.Base(dst),
Content: u.Content,
RawFilePermissions: "0644",
}
}}
err := WriteFile(&file)
_, err := WriteFile(&file, dir)
if err != nil {
return err
}
@@ -78,18 +50,18 @@ func PlaceUnit(u *Unit, dst string) error {
return nil
}
func EnableUnitFile(file string, runtime bool) error {
func (s *systemd) EnableUnitFile(unit string, runtime bool) error {
conn, err := dbus.New()
if err != nil {
return err
}
files := []string{file}
_, _, err = conn.EnableUnitFiles(files, runtime, true)
units := []string{unit}
_, _, err = conn.EnableUnitFiles(units, runtime, true)
return err
}
func RunUnitCommand(command, unit string) (string, error) {
func (s *systemd) RunUnitCommand(command, unit string) (string, error) {
conn, err := dbus.New()
if err != nil {
return "", err
@@ -118,7 +90,7 @@ func RunUnitCommand(command, unit string) (string, error) {
return fn(unit, "replace")
}
func DaemonReload() error {
func (s *systemd) DaemonReload() error {
conn, err := dbus.New()
if err != nil {
return err
@@ -127,6 +99,57 @@ func DaemonReload() error {
return conn.Reload()
}
// MaskUnit masks the given Unit by symlinking its unit file to
// /dev/null, analogous to `systemctl mask`.
// N.B.: Unlike `systemctl mask`, this function will *remove any existing unit
// file at the location*, to ensure that the mask will succeed.
func (s *systemd) MaskUnit(unit *Unit) error {
masked := unit.Destination(s.root)
if _, err := os.Stat(masked); os.IsNotExist(err) {
if err := os.MkdirAll(path.Dir(masked), os.FileMode(0755)); err != nil {
return err
}
} else if err := os.Remove(masked); err != nil {
return err
}
return os.Symlink("/dev/null", masked)
}
// UnmaskUnit is analogous to systemd's unit_file_unmask. If the file
// associated with the given Unit is empty or appears to be a symlink to
// /dev/null, it is removed.
func (s *systemd) UnmaskUnit(unit *Unit) error {
masked := unit.Destination(s.root)
ne, err := nullOrEmpty(masked)
if os.IsNotExist(err) {
return nil
} else if err != nil {
return err
}
if !ne {
log.Printf("%s is not null or empty, refusing to unmask", masked)
return nil
}
return os.Remove(masked)
}
// nullOrEmpty checks whether a given path appears to be an empty regular file
// or a symlink to /dev/null
func nullOrEmpty(path string) (bool, error) {
fi, err := os.Stat(path)
if err != nil {
return false, err
}
m := fi.Mode()
if m.IsRegular() && fi.Size() <= 0 {
return true, nil
}
if m&os.ModeCharDevice > 0 {
return true, nil
}
return false, nil
}
func ExecuteScript(scriptPath string) (string, error) {
props := []dbus.Property{
dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
@@ -165,11 +188,3 @@ func MachineID(root string) string {
return id
}
func MaskUnit(unit string, root string) error {
masked := path.Join(root, "etc", "systemd", "system", unit)
if err := os.MkdirAll(path.Dir(masked), os.FileMode(0755)); err != nil {
return err
}
return os.Symlink("/dev/null", masked)
}

View File

@@ -5,10 +5,12 @@ import (
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func TestPlaceNetworkUnit(t *testing.T) {
u := Unit{
u := Unit{config.Unit{
Name: "50-eth0.network",
Runtime: true,
Content: `[Match]
@@ -17,7 +19,7 @@ Name=eth47
[Network]
Address=10.209.171.177/19
`,
}
}}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
@@ -25,13 +27,15 @@ Address=10.209.171.177/19
}
defer os.RemoveAll(dir)
dst := UnitDestination(&u, dir)
sd := &systemd{dir}
dst := u.Destination(dir)
expectDst := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
if dst != expectDst {
t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
t.Fatalf("unit.Destination returned %s, expected %s", dst, expectDst)
}
if err := PlaceUnit(&u, dst); err != nil {
if err := sd.PlaceUnit(&u, dst); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
@@ -60,15 +64,39 @@ Address=10.209.171.177/19
}
}
func TestUnitDestination(t *testing.T) {
dir := "/some/dir"
name := "foobar.service"
u := Unit{config.Unit{
Name: name,
DropIn: false,
}}
dst := u.Destination(dir)
expectDst := path.Join(dir, "etc", "systemd", "system", "foobar.service")
if dst != expectDst {
t.Errorf("unit.Destination returned %s, expected %s", dst, expectDst)
}
u.DropIn = true
dst = u.Destination(dir)
expectDst = path.Join(dir, "etc", "systemd", "system", "foobar.service.d", cloudConfigDropIn)
if dst != expectDst {
t.Errorf("unit.Destination returned %s, expected %s", dst, expectDst)
}
}
func TestPlaceMountUnit(t *testing.T) {
u := Unit{
u := Unit{config.Unit{
Name: "media-state.mount",
Runtime: false,
Content: `[Mount]
What=/dev/sdb1
Where=/media/state
`,
}
}}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
@@ -76,13 +104,15 @@ Where=/media/state
}
defer os.RemoveAll(dir)
dst := UnitDestination(&u, dir)
sd := &systemd{dir}
dst := u.Destination(dir)
expectDst := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
if dst != expectDst {
t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
t.Fatalf("unit.Destination returned %s, expected %s", dst, expectDst)
}
if err := PlaceUnit(&u, dst); err != nil {
if err := sd.PlaceUnit(&u, dst); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
@@ -123,22 +153,137 @@ func TestMachineID(t *testing.T) {
t.Fatalf("File has incorrect contents")
}
}
func TestMaskUnit(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
if err := MaskUnit("foo.service", dir); err != nil {
t.Fatalf("Unable to mask unit: %v", err)
}
fullPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
target, err := os.Readlink(fullPath)
sd := &systemd{dir}
// Ensure mask works with units that do not currently exist
uf := &Unit{config.Unit{Name: "foo.service"}}
if err := sd.MaskUnit(uf); err != nil {
t.Fatalf("Unable to mask new unit: %v", err)
}
fooPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
fooTgt, err := os.Readlink(fooPath)
if err != nil {
t.Fatalf("Unable to read link", err)
}
if target != "/dev/null" {
t.Fatalf("unit not masked, got unit target", target)
if fooTgt != "/dev/null" {
t.Fatalf("unit not masked, got unit target", fooTgt)
}
// Ensure mask works with unit files that already exist
ub := &Unit{config.Unit{Name: "bar.service"}}
barPath := path.Join(dir, "etc", "systemd", "system", "bar.service")
if _, err := os.Create(barPath); err != nil {
t.Fatalf("Error creating new unit file: %v", err)
}
if err := sd.MaskUnit(ub); err != nil {
t.Fatalf("Unable to mask existing unit: %v", err)
}
barTgt, err := os.Readlink(barPath)
if err != nil {
t.Fatalf("Unable to read link", err)
}
if barTgt != "/dev/null" {
t.Fatalf("unit not masked, got unit target", barTgt)
}
}
func TestUnmaskUnit(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
sd := &systemd{dir}
nilUnit := &Unit{config.Unit{Name: "null.service"}}
if err := sd.UnmaskUnit(nilUnit); err != nil {
t.Errorf("unexpected error from unmasking nonexistent unit: %v", err)
}
uf := &Unit{config.Unit{Name: "foo.service", Content: "[Service]\nExecStart=/bin/true"}}
dst := uf.Destination(dir)
if err := os.MkdirAll(path.Dir(dst), os.FileMode(0755)); err != nil {
t.Fatalf("Unable to create unit directory: %v", err)
}
if _, err := os.Create(dst); err != nil {
t.Fatalf("Unable to write unit file: %v", err)
}
if err := ioutil.WriteFile(dst, []byte(uf.Content), 700); err != nil {
t.Fatalf("Unable to write unit file: %v", err)
}
if err := sd.UnmaskUnit(uf); err != nil {
t.Errorf("unmask of non-empty unit returned unexpected error: %v", err)
}
got, _ := ioutil.ReadFile(dst)
if string(got) != uf.Content {
t.Errorf("unmask of non-empty unit mutated unit contents unexpectedly")
}
ub := &Unit{config.Unit{Name: "bar.service"}}
dst = ub.Destination(dir)
if err := os.Symlink("/dev/null", dst); err != nil {
t.Fatalf("Unable to create masked unit: %v", err)
}
if err := sd.UnmaskUnit(ub); err != nil {
t.Errorf("unmask of unit returned unexpected error: %v", err)
}
if _, err := os.Stat(dst); !os.IsNotExist(err) {
t.Errorf("expected %s to not exist after unmask, but got err: %s", err)
}
}
func TestNullOrEmpty(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
non := path.Join(dir, "does_not_exist")
ne, err := nullOrEmpty(non)
if !os.IsNotExist(err) {
t.Errorf("nullOrEmpty on nonexistent file returned bad error: %v", err)
}
if ne {
t.Errorf("nullOrEmpty returned true unxpectedly")
}
regEmpty := path.Join(dir, "regular_empty_file")
_, err = os.Create(regEmpty)
if err != nil {
t.Fatalf("Unable to create tempfile: %v", err)
}
gotNe, gotErr := nullOrEmpty(regEmpty)
if !gotNe || gotErr != nil {
t.Errorf("nullOrEmpty of regular empty file returned %t, %v - want true, nil", gotNe, gotErr)
}
reg := path.Join(dir, "regular_file")
if err := ioutil.WriteFile(reg, []byte("asdf"), 700); err != nil {
t.Fatalf("Unable to create tempfile: %v", err)
}
gotNe, gotErr = nullOrEmpty(reg)
if gotNe || gotErr != nil {
t.Errorf("nullOrEmpty of regular file returned %t, %v - want false, nil", gotNe, gotErr)
}
null := path.Join(dir, "null")
if err := os.Symlink(os.DevNull, null); err != nil {
t.Fatalf("Unable to create /dev/null link: %s", err)
}
gotNe, gotErr = nullOrEmpty(null)
if !gotNe || gotErr != nil {
t.Errorf("nullOrEmpty of null symlink returned %t, %v - want true, nil", gotNe, gotErr)
}
}

44
system/unit.go Normal file
View File

@@ -0,0 +1,44 @@
package system
import (
"fmt"
"path"
"github.com/coreos/coreos-cloudinit/config"
)
// Name for drop-in service configuration files created by cloudconfig
const cloudConfigDropIn = "20-cloudinit.conf"
type UnitManager interface {
PlaceUnit(unit *Unit, dst string) error
EnableUnitFile(unit string, runtime bool) error
RunUnitCommand(command, unit string) (string, error)
DaemonReload() error
MaskUnit(unit *Unit) error
UnmaskUnit(unit *Unit) error
}
// Unit is a top-level structure which embeds its underlying configuration,
// config.Unit, and provides the system-specific Destination().
type Unit struct {
config.Unit
}
type Script []byte
// Destination builds the appropriate absolute file path for
// the Unit. The root argument indicates the effective base
// directory of the system (similar to a chroot).
func (u *Unit) Destination(root string) string {
dir := "etc"
if u.Runtime {
dir = "run"
}
if u.DropIn {
return path.Join(root, dir, "systemd", u.Group(), fmt.Sprintf("%s.d", u.Name), cloudConfigDropIn)
} else {
return path.Join(root, dir, "systemd", u.Group(), u.Name)
}
}

137
system/update.go Normal file
View File

@@ -0,0 +1,137 @@
package system
import (
"bufio"
"fmt"
"io"
"os"
"path"
"reflect"
"sort"
"strings"
"github.com/coreos/coreos-cloudinit/config"
)
const (
locksmithUnit = "locksmithd.service"
updateEngineUnit = "update-engine.service"
)
// Update is a top-level structure which contains its underlying configuration,
// config.Update, a function for reading the configuration (the default
// implementation reading from the filesystem), and provides the system-specific
// File() and Unit().
type Update struct {
Config config.Update
ReadConfig func() (io.Reader, error)
}
func DefaultReadConfig() (io.Reader, error) {
etcUpdate := path.Join("/etc", "coreos", "update.conf")
usrUpdate := path.Join("/usr", "share", "coreos", "update.conf")
f, err := os.Open(etcUpdate)
if os.IsNotExist(err) {
f, err = os.Open(usrUpdate)
}
return f, err
}
// File generates an `/etc/coreos/update.conf` file (if any update
// configuration options are set in cloud-config) by either rewriting the
// existing file on disk, or starting from `/usr/share/coreos/update.conf`
func (uc Update) File() (*File, error) {
if config.IsZero(uc.Config) {
return nil, nil
}
if err := config.AssertValid(uc.Config); err != nil {
return nil, err
}
// Generate the list of possible substitutions to be performed based on the options that are configured
subs := map[string]string{}
uct := reflect.TypeOf(uc.Config)
ucv := reflect.ValueOf(uc.Config)
for i := 0; i < uct.NumField(); i++ {
val := ucv.Field(i).String()
if val == "" {
continue
}
env := uct.Field(i).Tag.Get("env")
subs[env] = fmt.Sprintf("%s=%s", env, val)
}
conf, err := uc.ReadConfig()
if err != nil {
return nil, err
}
scanner := bufio.NewScanner(conf)
var out string
for scanner.Scan() {
line := scanner.Text()
for env, value := range subs {
if strings.HasPrefix(line, env) {
line = value
delete(subs, env)
break
}
}
out += line
out += "\n"
if err := scanner.Err(); err != nil {
return nil, err
}
}
for _, key := range sortedKeys(subs) {
out += subs[key]
out += "\n"
}
return &File{config.File{
Path: path.Join("etc", "coreos", "update.conf"),
RawFilePermissions: "0644",
Content: out,
}}, nil
}
// Units generates units for the cloud-init initializer to act on:
// - a locksmith Unit, if "reboot-strategy" was set in cloud-config
// - an update_engine Unit, if "group" or "server" was set in cloud-config
func (uc Update) Units() ([]Unit, error) {
var units []Unit
if uc.Config.RebootStrategy != "" {
ls := &Unit{config.Unit{
Name: locksmithUnit,
Command: "restart",
Mask: false,
Runtime: true,
}}
if uc.Config.RebootStrategy == "off" {
ls.Command = "stop"
ls.Mask = true
}
units = append(units, *ls)
}
if uc.Config.Group != "" || uc.Config.Server != "" {
ue := Unit{config.Unit{
Name: updateEngineUnit,
Command: "restart",
}}
units = append(units, ue)
}
return units, nil
}
func sortedKeys(m map[string]string) (keys []string) {
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
return
}

151
system/update_test.go Normal file
View File

@@ -0,0 +1,151 @@
package system
import (
"errors"
"io"
"reflect"
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/config"
)
func testReadConfig(config string) func() (io.Reader, error) {
return func() (io.Reader, error) {
return strings.NewReader(config), nil
}
}
func TestUpdateUnits(t *testing.T) {
for _, tt := range []struct {
config config.Update
units []Unit
err error
}{
{
config: config.Update{},
},
{
config: config.Update{Group: "master", Server: "http://foo.com"},
units: []Unit{{config.Unit{
Name: "update-engine.service",
Command: "restart",
}}},
},
{
config: config.Update{RebootStrategy: "best-effort"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "reboot"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "restart",
Runtime: true,
}}},
},
{
config: config.Update{RebootStrategy: "off"},
units: []Unit{{config.Unit{
Name: "locksmithd.service",
Command: "stop",
Runtime: true,
Mask: true,
}}},
},
} {
units, err := Update{tt.config, testReadConfig("")}.Units()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.units, units) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.units, units)
}
}
}
func TestUpdateFile(t *testing.T) {
for _, tt := range []struct {
config config.Update
orig string
file *File
err error
}{
{
config: config.Update{},
},
{
config: config.Update{RebootStrategy: "wizzlewazzle"},
err: errors.New("invalid value \"wizzlewazzle\" for option \"RebootStrategy\" (valid options: \"best-effort,etcd-lock,reboot,off\")"),
},
{
config: config.Update{Group: "master", Server: "http://foo.com"},
file: &File{config.File{
Content: "GROUP=master\nSERVER=http://foo.com\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "best-effort"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=best-effort\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=etcd-lock\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "reboot"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=reboot\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "off"},
file: &File{config.File{
Content: "REBOOT_STRATEGY=off\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
{
config: config.Update{RebootStrategy: "etcd-lock"},
orig: "SERVER=https://example.com\nGROUP=thegroupc\nREBOOT_STRATEGY=awesome",
file: &File{config.File{
Content: "SERVER=https://example.com\nGROUP=thegroupc\nREBOOT_STRATEGY=etcd-lock\n",
Path: "etc/coreos/update.conf",
RawFilePermissions: "0644",
}},
},
} {
file, err := Update{tt.config, testReadConfig(tt.orig)}.File()
if !reflect.DeepEqual(tt.err, err) {
t.Errorf("bad error (%q): want %q, got %q", tt.config, tt.err, err)
}
if !reflect.DeepEqual(tt.file, file) {
t.Errorf("bad units (%q): want %#v, got %#v", tt.config, tt.file, file)
}
}
}

View File

@@ -6,30 +6,16 @@ import (
"os/exec"
"os/user"
"strings"
"github.com/coreos/coreos-cloudinit/config"
)
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
SSHImportURL string `yaml:"coreos-ssh-import-url"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}
func UserExists(u *User) bool {
func UserExists(u *config.User) bool {
_, err := user.Lookup(u.Name)
return err == nil
}
func CreateUser(u *User) error {
func CreateUser(u *config.User) error {
args := []string{}
if u.PasswordHash != "" {
@@ -53,7 +39,7 @@ func CreateUser(u *User) error {
}
if u.PrimaryGroup != "" {
args = append(args, "--primary-group", u.PrimaryGroup)
args = append(args, "--gid", u.PrimaryGroup)
}
if len(u.Groups) > 0 {

58
test
View File

@@ -1,10 +1,56 @@
#!/bin/bash -e
#
# Run all coreos-cloudinit tests
# ./test
# ./test -v
#
# Run tests for one package
# PKG=initialize ./test
#
echo "Building bin/coreos-cloudinit"
. build
# Invoke ./cover for HTML output
COVER=${COVER:-"-cover"}
source ./build
declare -a TESTPKGS=(
config
datasource
datasource/configdrive
datasource/file
datasource/metadata
datasource/metadata/cloudsigma
datasource/metadata/digitalocean
datasource/metadata/ec2
datasource/proc_cmdline
datasource/url
initialize
network
pkg
system
)
if [ -z "$PKG" ]; then
GOFMTPATH="${TESTPKGS[*]} coreos-cloudinit.go"
# prepend repo path to each package
TESTPKGS="${TESTPKGS[*]/#/${REPO_PATH}/} ./"
else
GOFMTPATH="$TESTPKGS"
# strip out slashes and dots from PKG=./foo/
TESTPKGS=${PKG//\//}
TESTPKGS=${TESTPKGS//./}
TESTPKGS=${TESTPKGS/#/${REPO_PATH}/}
fi
echo "Running tests..."
for pkg in "./initialize ./system ./datasource"; do
go test -i $pkg
go test -v $pkg
done
go test -i ${TESTPKGS}
go test ${COVER} $@ ${TESTPKGS}
echo "Checking gofmt..."
fmtRes=$(gofmt -l $GOFMTPATH)
if [ -n "$fmtRes" ]; then
echo "$fmtRes"
exit 1
fi
echo "Success"

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,43 @@
cepgo
=====
Cepko implements easy-to-use communication with CloudSigma's VMs through a
virtual serial port without bothering with formatting the messages properly nor
parsing the output with the specific and sometimes confusing shell tools for
that purpose.
Having the server definition accessible by the VM can be useful in various
ways. For example it is possible to easily determine from within the VM, which
network interfaces are connected to public and which to private network.
Another use is to pass some data to initial VM setup scripts, like setting the
hostname to the VM name or passing ssh public keys through server meta.
Example usage:
package main
import (
"fmt"
"github.com/cloudsigma/cepgo"
)
func main() {
c := cepgo.NewCepgo()
result, err := c.Meta()
if err != nil {
panic(err)
}
fmt.Printf("%#v", result)
}
Output:
map[string]interface {}{
"optimize_for":"custom",
"ssh_public_key":"ssh-rsa AAA...",
"description":"[...]",
}
For more information take a look at the Server Context section of CloudSigma
API Docs: http://cloudsigma-docs.readthedocs.org/en/latest/server_context.html

View File

@@ -0,0 +1,186 @@
// Cepko implements easy-to-use communication with CloudSigma's VMs through a
// virtual serial port without bothering with formatting the messages properly
// nor parsing the output with the specific and sometimes confusing shell tools
// for that purpose.
//
// Having the server definition accessible by the VM can be useful in various
// ways. For example it is possible to easily determine from within the VM,
// which network interfaces are connected to public and which to private
// network. Another use is to pass some data to initial VM setup scripts, like
// setting the hostname to the VM name or passing ssh public keys through
// server meta.
//
// Example usage:
//
// package main
//
// import (
// "fmt"
//
// "github.com/cloudsigma/cepgo"
// )
//
// func main() {
// c := cepgo.NewCepgo()
// result, err := c.Meta()
// if err != nil {
// panic(err)
// }
// fmt.Printf("%#v", result)
// }
//
// Output:
//
// map[string]string{
// "optimize_for":"custom",
// "ssh_public_key":"ssh-rsa AAA...",
// "description":"[...]",
// }
//
// For more information take a look at the Server Context section API Docs:
// http://cloudsigma-docs.readthedocs.org/en/latest/server_context.html
package cepgo
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"runtime"
"github.com/coreos/coreos-cloudinit/third_party/github.com/tarm/goserial"
)
const (
requestPattern = "<\n%s\n>"
EOT = '\x04' // End Of Transmission
)
var (
SerialPort string = "/dev/ttyS1"
Baud int = 115200
)
// Sets the serial port. If the operating system is windows CloudSigma's server
// context is at COM2 port, otherwise (linux, freebsd, darwin) the port is
// being left to the default /dev/ttyS1.
func init() {
if runtime.GOOS == "windows" {
SerialPort = "COM2"
}
}
// The default fetcher makes the connection to the serial port,
// writes given query and reads until the EOT symbol.
func fetchViaSerialPort(key string) ([]byte, error) {
config := &serial.Config{Name: SerialPort, Baud: Baud}
connection, err := serial.OpenPort(config)
if err != nil {
return nil, err
}
query := fmt.Sprintf(requestPattern, key)
if _, err := connection.Write([]byte(query)); err != nil {
return nil, err
}
reader := bufio.NewReader(connection)
answer, err := reader.ReadBytes(EOT)
if err != nil {
return nil, err
}
return answer[0 : len(answer)-1], nil
}
// Queries to the serial port can be executed only from instance of this type.
// The result from each of them can be either interface{}, map[string]string or
// a single in case of single value is returned. There is also a public metod
// who directly calls the fetcher and returns raw []byte from the serial port.
type Cepgo struct {
fetcher func(string) ([]byte, error)
}
// Creates a Cepgo instance with the default serial port fetcher.
func NewCepgo() *Cepgo {
cepgo := new(Cepgo)
cepgo.fetcher = fetchViaSerialPort
return cepgo
}
// Creates a Cepgo instance with custom fetcher.
func NewCepgoFetcher(fetcher func(string) ([]byte, error)) *Cepgo {
cepgo := new(Cepgo)
cepgo.fetcher = fetcher
return cepgo
}
// Fetches raw []byte from the serial port using directly the fetcher member.
func (c *Cepgo) FetchRaw(key string) ([]byte, error) {
return c.fetcher(key)
}
// Fetches a single key and tries to unmarshal the result to json and returns
// it. If the unmarshalling fails it's safe to assume the result it's just a
// string and returns it.
func (c *Cepgo) Key(key string) (interface{}, error) {
var result interface{}
fetched, err := c.FetchRaw(key)
if err != nil {
return nil, err
}
err = json.Unmarshal(fetched, &result)
if err != nil {
return string(fetched), nil
}
return result, nil
}
// Fetches all the server context. Equivalent of c.Key("")
func (c *Cepgo) All() (interface{}, error) {
return c.Key("")
}
// Fetches only the object meta field and makes sure to return a proper
// map[string]string
func (c *Cepgo) Meta() (map[string]string, error) {
rawMeta, err := c.Key("/meta/")
if err != nil {
return nil, err
}
return typeAssertToMapOfStrings(rawMeta)
}
// Fetches only the global context and makes sure to return a proper
// map[string]string
func (c *Cepgo) GlobalContext() (map[string]string, error) {
rawContext, err := c.Key("/global_context/")
if err != nil {
return nil, err
}
return typeAssertToMapOfStrings(rawContext)
}
// Just a little helper function that uses type assertions in order to convert
// a interface{} to map[string]string if this is possible.
func typeAssertToMapOfStrings(raw interface{}) (map[string]string, error) {
result := make(map[string]string)
dictionary, ok := raw.(map[string]interface{})
if !ok {
return nil, errors.New("Received bytes are formatted badly")
}
for key, rawValue := range dictionary {
if value, ok := rawValue.(string); ok {
result[key] = value
} else {
return nil, errors.New("Server context metadata is formatted badly")
}
}
return result, nil
}

View File

@@ -0,0 +1,122 @@
package cepgo
import (
"encoding/json"
"testing"
)
func fetchMock(key string) ([]byte, error) {
context := []byte(`{
"context": true,
"cpu": 4000,
"cpu_model": null,
"cpus_instead_of_cores": false,
"enable_numa": false,
"global_context": {
"some_global_key": "some_global_val"
},
"grantees": [],
"hv_relaxed": false,
"hv_tsc": false,
"jobs": [],
"mem": 4294967296,
"meta": {
"base64_fields": "cloudinit-user-data",
"cloudinit-user-data": "I2Nsb3VkLWNvbmZpZwoKaG9zdG5hbWU6IGNvcmVvczE=",
"ssh_public_key": "ssh-rsa AAAAB2NzaC1yc2E.../hQ5D5 john@doe"
},
"name": "coreos",
"nics": [
{
"runtime": {
"interface_type": "public",
"ip_v4": {
"uuid": "31.171.251.74"
},
"ip_v6": null
},
"vlan": null
}
],
"smp": 2,
"status": "running",
"uuid": "20a0059b-041e-4d0c-bcc6-9b2852de48b3"
}`)
if key == "" {
return context, nil
}
var marshalledContext map[string]interface{}
err := json.Unmarshal(context, &marshalledContext)
if err != nil {
return nil, err
}
if key[0] == '/' {
key = key[1:]
}
if key[len(key)-1] == '/' {
key = key[:len(key)-1]
}
return json.Marshal(marshalledContext[key])
}
func TestAll(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.All()
if err != nil {
t.Error(err)
}
for _, key := range []string{"meta", "name", "uuid", "global_context"} {
if _, ok := result.(map[string]interface{})[key]; !ok {
t.Errorf("%s not in all keys", key)
}
}
}
func TestKey(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.Key("uuid")
if err != nil {
t.Error(err)
}
if _, ok := result.(string); !ok {
t.Errorf("%#v\n", result)
t.Error("Fetching the uuid did not return a string")
}
}
func TestMeta(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
meta, err := cepgo.Meta()
if err != nil {
t.Errorf("%#v\n", meta)
t.Error(err)
}
if _, ok := meta["ssh_public_key"]; !ok {
t.Error("ssh_public_key is not in the meta")
}
}
func TestGlobalContext(t *testing.T) {
cepgo := NewCepgoFetcher(fetchMock)
result, err := cepgo.GlobalContext()
if err != nil {
t.Error(err)
}
if _, ok := result["some_global_key"]; !ok {
t.Error("some_global_key is not in the global context")
}
}

View File

@@ -0,0 +1,2 @@
Michael Crosby <michael@crosbymichael.com> (@crosbymichael)
Guillaume J. Charmes <guillaume@docker.com> (@creack)

Some files were not shown because too many files have changed in this diff Show More