Compare commits

..

209 Commits

Author SHA1 Message Date
Jonathan Boulle
a9a910b5c4 chore(coreos-cloudinit): bump to 0.7.3 2014-05-29 14:52:58 -07:00
Jonathan Boulle
8e94b4140a Merge pull request #122 from jonboulle/122
ec2-cloudinit service fails after reboot with "reboot-strategy: off"
2014-05-29 14:25:58 -07:00
Jonathan Boulle
cd322863e9 Merge pull request #129 from jonboulle/exp
fix(pkg): simplify exponential backoff to avoid overflows
2014-05-29 14:02:47 -07:00
Jonathan Boulle
786e4bef65 fix(systemd): remove any existing unit when calling mask 2014-05-29 13:59:55 -07:00
Jonathan Boulle
269a658d4b fix(pkg): simplify exponential backoff to avoid overflows 2014-05-29 11:11:18 -07:00
Michael Marineau
e317c7eb9a chore(coreos-cloudinit): bump to 0.7.2+git 2014-05-27 14:02:11 -07:00
Michael Marineau
974de943e0 chore(coreos-cloudinit): bump to 0.7.2 2014-05-27 13:37:58 -07:00
Jonathan Boulle
db3f008543 Merge pull request #127 from jonboulle/127
"Enable" option does not support units in /usr/lib64/systemd
2014-05-26 15:24:30 -07:00
Jonathan Boulle
b04509ae54 fix(systemd): EnableUnitFile unit name rather than absolute destination 2014-05-26 15:16:24 -07:00
Jonathan Boulle
6c07e8784f Merge pull request #125 from jonboulle/no_locksmith_enable
Dies trying to enable non-existent /etc/systemd/system/locksmithd.service
2014-05-26 13:11:47 -07:00
Jonathan Boulle
60ab4222de fix(update): locksmith service does not need disabling/enabling 2014-05-26 12:33:23 -07:00
Brandon Philips
1a295f65c7 Merge pull request #123 from c4milo/shared-http-client
feat(util/http_client): Adds generic HTTP client
2014-05-22 14:37:32 -07:00
Camilo Aguilar
cec0926c5c fix(pkg/http_client): Printf is smarter than you think
Printf determines what the duration unit is
and prints it accordingly.
2014-05-22 14:53:54 -04:00
Camilo Aguilar
8ca3c2ed1f style(httpbackoff -> pkg): Adjusts package name to follow convention 2014-05-22 14:37:19 -04:00
Camilo Aguilar
2cedebb4eb style(util->httpbackoff): Changes package as per @philips suggestion 2014-05-21 21:12:16 -04:00
Camilo Aguilar
3e00a37ef5 feat(util/http_client): Adds generic HTTP client
Supports retries with exponential backoff as well as connection
timeouts and the ability to skip SSL/TLS verification.

This commit also refactors datasource and initialize packages
in order to use the new HTTP client.
2014-05-21 13:31:50 -04:00
Jonathan Boulle
59d1eba423 Merge pull request #111 from namsral/patch-1
Trim newlines from the cloud-config-url option
2014-05-21 10:18:24 -07:00
Jonathan Boulle
af69149260 Merge pull request #120 from brianredbeard/pr20-fix
fix(docs) Clear description of update server changes
2014-05-21 10:01:25 -07:00
Brandon Philips
5fa2ad8dfd Merge pull request #121 from iamveen/master
removed tricky space from cloud-config header
2014-05-21 05:33:05 -07:00
Lars Wiegman
513a1eb602 Trim newlines from the cloud-config-url kernel parameter and added a test
- In the Fetch function trim whitespace from /proc/cmdline
- New test for Fetch function
- Added Location field to the procCmdline struct for testing
2014-05-21 11:09:39 +02:00
Gavin Dunne
5189e1594e removed tricky space from cloud-config header 2014-05-21 01:22:09 -07:00
Brian 'Redbeard' Harrington
8b5bc47429 fix(doc) more sensible ordering
It makes a bit more sense to specify the scope of the section
before getting into details about how it's done.
2014-05-20 23:29:56 -07:00
Brian 'Redbeard' Harrington
a64fcd2893 fix(docs) Clear description of update server changes TBD
Pulling in @philips' changes from coreos/coreos-cloudinit#6 after
trashing PR coreos/coreos-cloudinit#20.  Cleanup of that PR was
beyond my git-fu.

cc @jonboulle
2014-05-20 22:53:29 -07:00
Brandon Philips
5b1145c044 Merge pull request #118 from c4milo/log-timestamp-fix
chore(logging): Removes duplicated timestamp during booting
2014-05-17 16:31:07 -07:00
Michael Marineau
a49877b99f chore(coreos-cloudinit): bump to 0.7.1+git 2014-05-16 21:23:34 -07:00
Michael Marineau
24f181f7a3 chore(coreos-cloudinit): bump to 0.7.1 2014-05-16 21:21:47 -07:00
Michael Marineau
61e70fcce8 Merge pull request #119 from marineam/container
container and panic fixes
2014-05-16 21:19:43 -07:00
Michael Marineau
ea6262f0ae fix(etcd): fix runtime panic when etcd section is missing.
The etcd code tries to assign ee["name"] even when the map was never
defined and assigning to an uninitialized map causes a panic.
2014-05-16 20:38:49 -07:00
Michael Marineau
f83ce07416 feat(units): Add generic cloudinit path unit
Switch to triggering common user configs via a path unit. This is
particularly useful for config drive so that a config drive can be
mounted by something other than the udev triggered services, a bind
mount when running in a container for example.
2014-05-16 20:38:49 -07:00
Brandon Philips
140682350d chore(coreos-cloudinit): bump to 0.7.0+git 2014-05-16 18:22:22 -07:00
Brandon Philips
289ada4668 chore(coreos-cloudinit): bump to 0.7.0 2014-05-16 18:22:22 -07:00
Camilo Aguilar
5d58c6c1c1 chore(logging): Removes duplicated timestamp during booting 2014-05-16 17:35:31 -04:00
Jonathan Boulle
d95df78c6d Merge pull request #117 from c4milo/travis-support
chore(travis): Adds travis yaml file as well as badge in README
2014-05-16 14:11:37 -07:00
Camilo Aguilar
ac4c969454 chore(travis): Adds travis yaml file and badge in README 2014-05-16 17:09:59 -04:00
Jonathan Boulle
04fcd3935f Merge pull request #114 from c4milo/fetch-url-refactor
refactor(datastore/fetch): Makes more failure proof fetching user-data files.
2014-05-16 14:03:54 -07:00
Camilo Aguilar
36efcc9d69 test(datastore/fetch): Makes sure err is not nil 2014-05-16 16:57:58 -04:00
Jonathan Boulle
f7ecc2461c Merge pull request #109 from jonboulle/fleet
fix(docs): add documentation for fleet section
2014-05-16 13:38:12 -07:00
Jonathan Boulle
8df9ee3ca2 Merge pull request #115 from burke/master
Response body must not be closed if request error'd.
2014-05-16 13:20:27 -07:00
Burke Libbey
321ceaa0da Response body must not be closed if request error'd. 2014-05-16 15:42:11 -04:00
Jonathan Boulle
05daad692e fix(docs): add documentation for fleet section 2014-05-16 12:10:21 -07:00
Camilo Aguilar
4b6fc63e8c fix(datastore/fetch): off-by-one oversight 2014-05-16 12:36:05 -04:00
Camilo Aguilar
fcccfb085f style(datastore/fetch): Adjusts comments formatting 2014-05-16 12:35:39 -04:00
Camilo Aguilar
ebf134f181 refactor(datastore/fetch): Makes more failure proof fetching user-data files
- Adds URL validations
- Adds timeout support for http client
- Limits the amount of retries to not spin forever
- Fails faster if response status code is 4xx
- Does a little bit more of logging
- Adds more tests
2014-05-16 12:35:06 -04:00
Jonathan Boulle
51d77516a5 Merge pull request #90 from jonboulle/90
Warn or error on unrecognized keys in cloud-config.yml
2014-05-15 18:53:48 -07:00
Jonathan Boulle
98f5ead730 fix(*): catch more unknown keys in user and file sections 2014-05-15 18:53:17 -07:00
Jonathan Boulle
81fe0dc9e0 fix(initialize): also check for unknown coreos keys 2014-05-15 18:53:17 -07:00
Jonathan Boulle
e852be65f7 feat(*): warn on encountering unrecognized keys in cloud-config 2014-05-15 18:53:17 -07:00
Brandon Philips
0a16532d4b Merge pull request #113 from c4milo/exponential_backoff
Exponential backoff with sleep capping
2014-05-15 10:16:42 -07:00
Camilo Aguilar
ff70a60fbc Adds sleep cap to exponential backoff so it does not go too high 2014-05-15 13:04:37 -04:00
Kelsey Hightower
31f61d7531 Use exponential backoff when fetching user-data from an URL.
The user-cloudinit-proc-cmdline systemd unit is responsible for
fetching user-data from various sources during the cloud-init
process. When fetching user-data from an URL datasource we face
a race condition since the network may not be available, which
can cause the job to fail and no further attempts to fetch the
user-data are made.

Eliminate the race condition when fetching user-data from an URL
datasource. Retry the fetch using an exponential backoff until
the user-data is retrieved.

Fixes issue 105.
2014-05-14 23:15:49 -07:00
Jonathan Boulle
b505e6241c Merge pull request #103 from jonboulle/20
feat(*): add more configuration options for update.conf
2014-05-14 13:14:35 -07:00
Jonathan Boulle
e413a97741 feat(update): add more configuration options for update.conf 2014-05-14 13:13:19 -07:00
Jonathan Boulle
41cbec8729 Merge pull request #101 from jonboulle/fleet
feat(*): add basic fleet configuration to cloud-config
2014-05-14 12:28:52 -07:00
Jonathan Boulle
919298e545 feat(fleet): add basic fleet configuration to cloud-config 2014-05-14 12:28:20 -07:00
Jonathan Boulle
ae424b5637 Merge pull request #106 from jonboulle/locksmith_to_update
refactor(init): rename locksmith to update
2014-05-14 11:50:09 -07:00
Jonathan Boulle
e93911344d refactor(init): rename locksmith to update 2014-05-14 11:40:39 -07:00
Jonathan Boulle
32c52d8729 Merge pull request #100 from jonboulle/rework
refactor(*): rework cloudconfig for better extensibility and consistency
2014-05-14 11:39:53 -07:00
Jonathan Boulle
cdee32d245 refactor(systemd): don't allow users to set DropIn=true yet 2014-05-14 11:34:13 -07:00
Jonathan Boulle
31cfad91e3 refactor(*): rework cloudconfig for better extensibility and consistency
This change creates a few simple interfaces for coreos-specific
configuration options and moves things to them wherever possible; so if
an option needs to write a file, or create a unit, it is acted on
exactly the same way as every other file/unit that needs to be written
during the cloud configuration process.
2014-05-14 11:34:07 -07:00
Brian Waldon
e814b37839 Merge pull request #107 from bcwaldon/locksmith-no-etc
fix(coreos-cloudinit): Ensure /etc/coreos exists before writing to it
2014-05-14 10:49:23 -07:00
Brian Waldon
cb4d9e81a4 fix(coreos-cloudinit): Ensure /etc/coreos exists before writing to it 2014-05-14 10:47:18 -07:00
Jonathan Boulle
b87a4628e6 Merge pull request #99 from jonboulle/simple
chore(cloudinit): remove superfluous check
2014-05-12 10:51:51 -07:00
Jonathan Boulle
b22fdd5ac9 Merge pull request #104 from jonboulle/tests
feat(tests): add coverage script
2014-05-12 10:51:38 -07:00
Jonathan Boulle
6939fc2ddc feat(tests): add cover script 2014-05-10 01:42:57 -07:00
Jonathan Boulle
e3117269cb chore(cloudinit): remove superfluous check 2014-05-09 20:32:51 -07:00
Brandon Philips
3bb3a683a4 Merge pull request #98 from philips/remove-oem-from-default
chore(Documentation): move OEM into its own doc
2014-05-08 09:41:42 -07:00
Brandon Philips
e1033c979e chore(Documentation): move OEM into its own doc
People are customizing the OEM needlessly. Just move it into its own
doc.
2014-05-08 09:32:21 -07:00
Jonathan Boulle
9a4d24826f Merge pull request #80 from jonboulle/master
users[i].primary-group option seems invalid
2014-05-07 21:12:45 -07:00
Jonathan Boulle
7bed1307e1 fix(user): user correct primary group flag for useradd 2014-05-07 14:06:51 -07:00
Brandon Philips
47b536532d chore(coreos-cloudinit): version +git 2014-05-06 21:09:40 -07:00
Brandon Philips
7df5cf761e chore(coreos-cloudinit): bump to 0.6.0
The major feature in this release is coreos.update.reboot-strategy
2014-05-06 21:05:42 -07:00
Brandon Philips
799c02865c Merge pull request #96 from philips/locksmith-support
Add locksmith support v2
2014-05-06 21:00:44 -07:00
Brandon Philips
9f38792d43 fix(initialize): use REBOOT_STRATEGY in update.conf
Change from STRATEGY to REBOOT_STRATEGY and update the function names to
reflect that this is a config now.
2014-05-06 20:57:29 -07:00
Alex Polvi
7e4fa423e4 feat(initialize): add locksmith configuration
configure locksmith strategy based on the cloud config.
2014-05-06 20:57:28 -07:00
Brandon Philips
c3f17bd07b feat(system): add MaskUnit to systemd 2014-05-06 17:46:16 -07:00
Brandon Philips
85a473d972 Merge pull request #95 from philips/various-code-cleanups
chore(initialize): code cleanups and gofmt
2014-05-06 16:19:35 -07:00
Brandon Philips
aea5ca5252 chore(initialize): code cleanups and gofmt 2014-05-06 16:13:21 -07:00
Michael Marineau
4e84180ad5 chore(release): Bump version to v0.5.2+git 2014-05-05 14:09:08 -07:00
Michael Marineau
0f1717bf26 chore(release): Bump version to v0.5.2 2014-05-05 14:07:50 -07:00
Michael Marineau
6a9aa60a8d Merge pull request #93 from marineam/reload
Revert "fix(units): Drop automatic daemon-reload"
2014-05-05 14:02:16 -07:00
Michael Marineau
7cacb2e127 Revert "fix(units): Drop automatic daemon-reload"
daemon-reload should be fixed now and the latest CoreOS with locksmith
is causing the etcd unit to get lazy-loaded before all the cloudinit
processes have finished configuring etcd via dropin files. In short,
the luck we were relying on to get by without daemon-reload has
officially run out. Cross your fingers!

This reverts commit 580460ff3f.
2014-05-05 13:16:07 -07:00
Brian Waldon
1f688dcdca Merge pull request #92 from bcwaldon/crlf-test
test(crlf): Add test that parses user-data with carriage returns
2014-05-05 10:50:25 -07:00
Brian Waldon
f6d8190e8f test(crlf): Add test that parses user-data with carriage returns 2014-05-05 10:49:02 -07:00
Brandon Philips
3263816cf5 Merge https://github.com/coreos/template-project 2014-05-05 09:44:59 -07:00
Michael Marineau
96e1cb5a7a Merge pull request #89 from robszumski/doc-write-files
feat(docs): include write_files example
2014-04-29 11:26:44 -07:00
Rob Szumski
cf556d2a81 feat(docs): include write_files example 2014-04-29 11:17:22 -07:00
Jonathan Boulle
62bda8e6cc Merge pull request #88 from robszumski/master
fix(docs): start the example unit
2014-04-29 12:15:44 -06:00
Rob Szumski
0d1d1f77be fix(docs): start the example unit 2014-04-28 10:57:11 -07:00
Michael Marineau
a7e21747fa Merge pull request #87 from marineam/hack
fix(configdrive): Always run after OEM and ec2 metadata.
2014-04-23 14:54:19 -07:00
Michael Marineau
26b54534d6 fix(configdrive): Always run after OEM and ec2 metadata.
A workaround for https://github.com/coreos/coreos-cloudinit/issues/86

Longer term cloudinit needs to be fixed to not corrupt the system when
multiple config sources are being used. We've pretty much gotten this
far without this coming up because most configs don't conflict so badly.
2014-04-23 14:38:54 -07:00
Brian Waldon
8201d75115 chore(release): Bump version to v0.5.1+git 2014-04-22 18:22:35 -07:00
Brian Waldon
1d024af4c1 chore(release): Bump version to v0.5.1 2014-04-22 18:22:24 -07:00
Brian Waldon
09c690cbe7 Merge pull request #85 from bcwaldon/pxe-unit
feat(proc-cmdline): Add proc-cmdline unit
2014-04-22 18:21:51 -07:00
Brian Waldon
49adf19081 feat(proc-cmdline): Add proc-cmdline unit
This unit will always be started, but will only do anything if
a `cloud-config-url=<url>` token is provided in /proc/cmdline.
2014-04-22 17:56:52 -07:00
Brian Waldon
46b046c82e chore(release): Bump version to v0.5.0+git 2014-04-22 16:48:32 -07:00
Brian Waldon
e64b61b312 chore(release): Bump version to v0.5.0 2014-04-22 16:48:21 -07:00
Brian Waldon
d72e10125a Merge pull request #84 from bcwaldon/proc-cmdline
feat(proc-cmdline): Parse /proc/cmdline for cloud-config-url
2014-04-22 16:43:05 -07:00
Brian Waldon
3de3d2c050 feat(proc-cmdline): Parse /proc/cmdline for cloud-config-url
If the --from-proc-cmdline flag is given to coreos-cloudinit, the local
/proc/cmdline file will be parsed for a cloud-config-url
2014-04-22 16:38:01 -07:00
Brian Waldon
2ff0762b0c Merge pull request #83 from robszumski/correct-headers
docs(cloud-config): correct headers
2014-04-21 19:15:50 -07:00
Rob Szumski
d6bacb24bc docs(cloud-config): correct headers 2014-04-21 17:56:35 -07:00
Brian Waldon
926eb4dbb7 Merge pull request #77 from chexxor/master
Update cloud-config.md to include expected file format
2014-04-21 14:27:22 -07:00
Brian Waldon
e7599fea58 Merge pull request #82 from bcwaldon/fix-68
fix(userdata): Strip \r when checking header
2014-04-21 14:26:31 -07:00
Brian Waldon
e98c58c656 fix(userdata): Strip \r when checking header
Fix #68
2014-04-21 13:40:26 -07:00
Alex Berg
ae350a3b34 Update cloud-config.md - use "you" 2014-04-18 11:45:02 -05:00
Alex Berg
c3b53f24cf Update cloud-config.md to use "parameter", not "option" 2014-04-18 11:45:01 -05:00
Alex Berg
8bee85e63d Update cloud-config.md based on feedback 2014-04-18 11:43:54 -05:00
Alex Berg
4c02e99bc8 Update cloud-config.md option descriptions
Re-word a few more things to look more like docs.
2014-04-18 11:43:53 -05:00
Alex Berg
0fb5291dd0 Update cloud-config.md to include expected file format
Clarify root-level keys. Use page structure to indicate expected values.
2014-04-18 11:43:53 -05:00
Brian Waldon
7f55876378 Merge pull request #79 from robszumski/note-config-drive
feat(docs): note config-drive
2014-04-17 09:36:57 -07:00
Brian Waldon
eb51a89f78 Merge pull request #72 from bcwaldon/unit-enable
Address unit enabling issues
2014-04-17 09:32:47 -07:00
Rob Szumski
588ff4c26c feat(docs): note config-drive 2014-04-16 22:35:39 -07:00
Michael Marineau
5472de8821 Merge pull request #78 from robszumski/update-user-group
fix(docs): use better group example
2014-04-16 16:49:50 -07:00
Rob Szumski
e6b632f817 fix(docs): use better group example 2014-04-16 16:48:04 -07:00
Michael Marineau
13a3d892ca Merge pull request #76 from marineam/units2
fix(units): Relax ordering requirements for now.
2014-04-15 15:19:13 -07:00
Brian Waldon
2e237ebead Merge pull request #66 from bcwaldon/doc-encoding
doc(write_files): Explicitly document lack of encoding support
2014-04-15 10:06:14 -07:00
Brian Waldon
61bb63b6e6 feat(unit): Allow units to be enabled even if contents not provided 2014-04-15 09:00:53 -07:00
Brian Waldon
476761cf62 refactor(unit): Separate UnitDestination from PlaceUnit 2014-04-15 09:00:53 -07:00
Brian Waldon
5981e12ac0 feat(unit): Allow user to control enabling units
Fix #69 - A user may provide an `enable` attribute of a unit in their
cloud config document. If true, coreos-cloudinit will instruct systemd
to enable the associated unit. If false, the unit will not be enabled.

Fix #71 - The default enable behavior has been changed from on to off.
2014-04-15 09:00:52 -07:00
Michael Marineau
78d8be8427 fix(units): Relax ordering requirements for now.
The current cloudinit implementation blocks when starting units which
causes it to deadlock the boot process if a system cloud config starts a
user cloud config because the user configs want to run after system is
done. Until cloudinit switches to non-blocking calls user configs will
go back to just depending on coreos-setup-environment.service.
2014-04-14 21:39:40 -04:00
Michael Marineau
10d73930d9 Merge pull request #62 from marineam/units
add(units): Generic config drive and other systemd units.
2014-04-11 13:26:59 -07:00
Brandon Philips
ea12c0bfd1 Merge pull request #67 from robszumski/remove_disco
fix(docs): remove real discovery token
2014-04-09 21:56:26 -07:00
Rob Szumski
6540d12d25 fix(docs): remove real discovery token 2014-04-09 21:55:19 -07:00
Michael Marineau
c438a42587 feat(units): Generic config drive and other systemd units. 2014-04-09 19:10:07 -07:00
Brian Waldon
19f8fe49af doc(write_files): Explicitly document lack of encoding support 2014-04-08 08:34:39 -07:00
Michael Marineau
58b091061e Merge pull request #57 from marineam/passwd
fix(user): Use '*' as default password field rather than '!'
2014-04-07 14:13:25 -07:00
Brian Waldon
8a7df360ac Merge pull request #65 from bcwaldon/hosts-newline
fix(manage_etc_hosts): Append newline to /etc/hosts
2014-04-07 11:09:27 -07:00
Brian Waldon
ba7cf90315 fix(manage_etc_hosts): Append newline to /etc/hosts 2014-04-07 11:01:17 -07:00
Brian Waldon
8841740a2b doc(oem): remove quotes from oem doc 2014-04-07 10:58:13 -07:00
Brian Waldon
dfe1255ac3 chore(release): Bump version to v0.4.0+git 2014-04-07 10:23:58 -07:00
Brian Waldon
0fddd1735d chore(release): Bump version to v0.4.0 2014-04-07 10:23:28 -07:00
Brandon Philips
f779a3f7f5 Merge pull request #64 from philips/no-quotes-on-oem-id-or-version
fix(initialize): don't quote version or ID
2014-04-07 10:17:29 -07:00
Brandon Philips
7015338aef fix(initialize): don't quote version or ID
The update_engine parsing and XML generation code is very naive. Instead
of trying to implement a correct parser and generater in C++ just
generate a file that doesn't have quote's around fields that we know
won't have spaces.
2014-04-07 09:56:57 -07:00
Jonathan Boulle
e01a1f70c3 Merge pull request #2 from jonboulle/master
Clean up CONTRIBUTING.md and other bits of template-project
2014-04-04 10:41:40 -07:00
Jonathan Boulle
2e4ea503b0 chore(contributing): clean up CONTRIBUTING.md and split out DCO 2014-04-04 10:40:37 -07:00
Brian Waldon
34aa147ebe Merge pull request #58 from gabrtv/manage_etc_hosts
feat(etc-hosts) add support for manage_etc_hosts: localhost
2014-04-02 23:11:03 -07:00
Gabriel Monroy
4d02e1da8e feat(etc-hosts) add support for manage_etc_hosts: localhost
This feature is based on https://github.com/number5/cloud-init/blob/master/doc/examples/cloud-config.txt#L447:L482
2014-04-01 16:02:12 -06:00
Michael Marineau
5ef3e1f32b fix(user): Use '*' as default password field rather than '!'
When using openssh without pam it checks for a ! prefix in the password
field, locking the account entirely if found. The other common lock
character, *, is allowed by ssh to login via ssh keys so use it instead.
2014-03-31 22:20:02 -07:00
polvi
23d02363ee Merge pull request #56 from cbmd/patch-1
Fixed indentation for users creation example
2014-03-28 08:40:34 -07:00
Vadym Okun
3c4fe9e260 Fixed indentation for users creation example 2014-03-28 13:23:58 +02:00
Brian Waldon
a594e053f5 chore(doc): clean up formatting 2014-03-27 20:19:42 -07:00
Brian Waldon
f3ba47ac89 Merge pull request #48 from calavera/key_import_url
feat(ssh-import): Add ssh-import-url user attribute.
2014-03-27 20:16:10 -07:00
David Calavera
7d814396b7 feat(ssh-import): Add ssh-import-url user attribute. 2014-03-28 09:39:47 +08:00
Brian Waldon
47ca113385 chore(release): Bump version to v0.3.2+git 2014-03-27 18:14:24 -07:00
Brian Waldon
639c693153 chore(release): Bump version to v0.3.2 2014-03-27 18:14:16 -07:00
Brian Waldon
b4027077ff Merge pull request #55 from bcwaldon/drop-reload
fix(units): Drop automatic daemon-reload
2014-03-27 18:12:22 -07:00
Brian Waldon
580460ff3f fix(units): Drop automatic daemon-reload 2014-03-27 17:30:05 -07:00
Brian Waldon
b246ec0397 chore(release): Bump version to v0.3.1+git 2014-03-25 20:06:19 -07:00
Brian Waldon
4977c774d8 chore(release): Bump version to v0.3.1 2014-03-25 20:06:07 -07:00
Brian Waldon
661bae11fc Merge pull request #53 from bcwaldon/fix-reload
Fix systemd daemon-reload
2014-03-25 20:04:24 -07:00
Brian Waldon
58ae898948 fix(systemd): Update usage of dbus.Reload 2014-03-25 19:37:05 -07:00
Brian Waldon
f5f9a0a6a9 bump(github.com/coreos/go-systemd/dbus): 4fbc5060a317b142e6c7bfbedb65596d5f0ab99b 2014-03-25 19:37:05 -07:00
Brian Waldon
477ae29135 fix(systemd): Fail if daemon-reload returns error 2014-03-25 18:50:48 -07:00
Brian Waldon
0203d4a9f3 chore(release): Bump version to v0.3.0+git 2014-03-24 18:03:45 -07:00
Brian Waldon
e68134d884 chore(release): Bump version to v0.3.0 2014-03-24 18:03:34 -07:00
Brian Waldon
2ad33487d7 Merge pull request #51 from bcwaldon/default-command
fix(unit): Default unit command to NOP
2014-03-24 16:29:58 -07:00
Brian Waldon
b778fe6f41 fix(unit): Default unit command to NOP 2014-03-24 14:12:59 -07:00
Brian Waldon
3d7bda9f6b Merge pull request #49 from bcwaldon/oem-release
oem-release
2014-03-24 13:43:10 -07:00
Brian Waldon
3d01211937 feat(coreos.oem): Write coreos.oem fields to /etc/oem-release 2014-03-24 13:42:35 -07:00
Brian Waldon
61808c2002 chore(release): Bump version to v0.2.2+git 2014-03-21 14:43:16 -07:00
Brian Waldon
35655809ff chore(release): Bump version to v0.2.2 2014-03-21 14:43:05 -07:00
Brian Waldon
81e4f1f896 Merge pull request #46 from bcwaldon/doc-field-sub
doc(fields): Document field substitution
2014-03-21 14:41:53 -07:00
Brian Waldon
e0b65066ab doc(fields): Document field substitution 2014-03-21 14:36:12 -07:00
Brian Waldon
8e0f0998df Merge pull request #44 from bcwaldon/ip-sub
feat($ip): Substitute $[public|private]_ipv4 in whole user-data
2014-03-21 13:58:24 -07:00
Brian Waldon
ddd035aaa7 feat($ip): Substitute $[public|private]_ipv4 in whole user-data 2014-03-21 11:01:16 -07:00
Brian Waldon
568714cadb Merge pull request #43 from robszumski/master
fix(docs): remove extra quotation mark
2014-03-20 11:37:15 -07:00
Rob Szumski
9c94b3fe21 fix(docs): remove extra quotation mark 2014-03-20 11:26:17 -07:00
Brian Waldon
267617ed1f Merge pull request #42 from bcwaldon/doc
doc(user-data): Move user-data doc to README.md
2014-03-20 10:50:40 -07:00
Brian Waldon
bc37171a2e doc(user-data): Move user-data doc to README.md 2014-03-20 09:13:32 -07:00
Brian Waldon
490152bd16 chore(release): Bump version to v0.2.1+git 2014-03-19 19:08:00 -07:00
Brian Waldon
9ade6673ba chore(release): Bump version to v0.2.1 2014-03-19 19:07:01 -07:00
Brian Waldon
67043681cd fix(doc): Add missing backticks 2014-03-19 17:16:28 -07:00
Brian Waldon
b89ddae983 Merge pull request #40 from bcwaldon/unit-command
Implement unit.command
2014-03-19 16:06:01 -07:00
Brian Waldon
88a6e77449 feat(unit.command): Add command field to units 2014-03-19 15:56:29 -07:00
Brian Waldon
09c473a6cb fix(fleet): Drop coreos.fleet from cloud-config 2014-03-19 14:43:25 -07:00
Brian Waldon
48f733f448 Merge pull request #39 from bcwaldon/update-etcd-docs
Replace lost etcd docs
2014-03-19 14:30:01 -07:00
Brian Waldon
aeac9f987d doc(etcd): Update etcd docs 2014-03-19 14:26:23 -07:00
Brian Waldon
9757705ae8 chore(release): Bump version to v0.2.0+git 2014-03-19 08:57:49 -07:00
Brian Waldon
2c328f3829 chore(release): Bump version to v0.2.0 2014-03-19 08:57:30 -07:00
Brian Waldon
907131496b Merge pull request #21 from robszumski/master
refactor(docs): rearrange order and add full example
2014-03-19 08:56:31 -07:00
Brian Waldon
b7bd997a3e Merge pull request #35 from bcwaldon/github-keys
feat(github): add ssh-import-github user attribute
2014-03-19 08:55:22 -07:00
Brian Waldon
6f5acf53cb feat(github): add coreos-ssh-import-github user attribute 2014-03-19 08:54:45 -07:00
polvi
c2faaa503b Merge pull request #37 from polvi/ignore-failure
feat(ignore-failure): add ignore failure flag, fixes #36
2014-03-18 20:50:00 -07:00
Alex Polvi
f98ec17f3d feat(ignore-failure): add ignore failure flag, fixes #36 2014-03-18 20:47:20 -07:00
Rob Szumski
c8dd424f89 refactor(docs): rearrange order and add full example 2014-03-18 14:57:19 -07:00
Brian Waldon
06cf75b660 Merge pull request #34 from bcwaldon/disco-url
fix(etcd): Transform DISCOVERY_URL to DISCOVERY
2014-03-18 13:08:46 -07:00
Brian Waldon
01542ecec7 fix(etcd): Transform DISCOVERY_URL to DISCOVERY 2014-03-18 13:04:07 -07:00
Brian Waldon
818bcd4b59 Merge pull request #32 from bcwaldon/etcd-name
feat(etcd): Default etcd name to /etc/machine-id
2014-03-18 11:26:13 -07:00
Brian Waldon
dcd82e6c50 fix(system): Use os.Hostname 2014-03-18 11:04:33 -07:00
Brian Waldon
9818565c7d feat(etcd): Fall back to hostname if no machine-id 2014-03-18 10:58:47 -07:00
Brian Waldon
f5765e4dde feat(etcd): Default etcd name to /etc/machine-id 2014-03-18 10:58:47 -07:00
Brian Waldon
61ffbd41c9 Merge pull request #33 from bcwaldon/etcd-dropin-run
fix(etcd): Place etcd.service drop-in in /run
2014-03-18 10:58:34 -07:00
Brian Waldon
cfa17ca2d2 fix(etcd): Place etcd.service drop-in in /run 2014-03-18 10:36:34 -07:00
Brian Waldon
c57464c845 Merge pull request #31 from bcwaldon/refactor
Refactor package layout
2014-03-18 10:33:39 -07:00
Brian Waldon
d2dabee0c6 refactor(*): Break apart packages 2014-03-18 09:14:11 -07:00
Brian Waldon
5185fe48da Merge pull request #29 from bcwaldon/permissions
Fix permissions-related bugs
2014-03-17 17:22:43 -07:00
Brian Waldon
d397906b7f fix(write_files): Create directories with mode 0755
Fix #28
2014-03-17 17:09:32 -07:00
Brian Waldon
fdc2e68497 feat(write_files): Set default permissions to 0644
Fix #26
2014-03-17 17:08:50 -07:00
Brian Waldon
3df9c40520 Merge pull request #25 from bcwaldon/etcd-env-file
Write environment file from coreos.etcd options
2014-03-17 16:27:34 -07:00
Brian Waldon
137949f5ad feat(etcd): Write etcd systemd snippet 2014-03-17 16:27:15 -07:00
Brian Waldon
0841173dfc Merge pull request #24 from philips/fixup-minor-readme-nits
fix(README): cleanup the README a bit
2014-03-17 10:56:18 -07:00
Brandon Philips
0a83ef5e23 fix(README): cleanup the README a bit
Improves the README a bit by using more specific and enabling language.
2014-03-16 17:13:15 -07:00
Brian Waldon
29b558caec Merge pull request #19 from bcwaldon/doc
doc(config): Fill in missing config options
2014-03-14 12:08:34 -07:00
Brian Waldon
a153ed02a7 doc(config): Fill in missing config options 2014-03-14 12:05:26 -07:00
Brian Waldon
b2eafb0efb Merge pull request #18 from brianredbeard/cloud-config-hashing
docs: Additional information on generating password hashes
2014-03-14 11:47:45 -07:00
Brian 'Redbeard' Harrington
98a19e6fc6 Moar hashing stuff
This time with less conflicts and a new file!
2014-03-14 11:42:17 -07:00
Brian Waldon
f0b5b2906b chore(release): Bump version to v0.1.2+git 2014-03-14 10:37:16 -07:00
Brandon Philips
c7aef5fdf2 Merge pull request #1 from bcwaldon/fix-case
fix(CONTRIBUTING.md): Fix title case
2014-02-05 15:52:24 -08:00
Brian Waldon
c4605160c5 fix(CONTRIBUTING.md): Fix title case 2014-02-05 15:51:24 -08:00
Brandon Philips
054de85da2 feat(*): initial commit 2014-01-19 12:25:11 -08:00
71 changed files with 3903 additions and 812 deletions

3
.gitignore vendored
View File

@@ -1,3 +1,4 @@
*.swp
bin/
pkg/
coverage/
gopath/

8
.travis.yml Normal file
View File

@@ -0,0 +1,8 @@
language: go
go: 1.2
install:
- go get code.google.com/p/go.tools/cmd/cover
script:
- ./test

87
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,87 @@
# How to Contribute
CoreOS projects are [Apache 2.0 licensed](LICENSE) and accept contributions via
GitHub pull requests. This document outlines some of the conventions on
development workflow, commit message formatting, contact points and other
resources to make it easier to get your contribution accepted.
# Certificate of Origin
By contributing to this project you agree to the Developer Certificate of
Origin (DCO). This document was created by the Linux Kernel community and is a
simple statement that you, as a contributor, have the legal right to make the
contribution. See the [DCO](DCO) file for details.
# Email and Chat
The project currently uses the general CoreOS email list and IRC channel:
- Email: [coreos-dev](https://groups.google.com/forum/#!forum/coreos-dev)
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) IRC channel on freenode.org
## Getting Started
- Fork the repository on GitHub
- Read the [README](README.md) for build and test instructions
- Play with the project, submit bugs, submit patches!
## Contribution Flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where you want to base your work (usually master).
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- Make sure the tests pass, and add any new tests as appropriate.
- Submit a pull request to the original repository.
Thanks for your contributions!
### Format of the Commit Message
We follow a rough convention for commit messages borrowed from AngularJS. This
is an example of a commit:
```
feat(scripts/test-cluster): add a cluster test command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
```
The format can be described more formally as follows:
```
<type>(<scope>): <subject>
<BLANK LINE>
<body>
<BLANK LINE>
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
#### Subject Line
The subject line contains a succinct description of the change.
#### Allowed `<type>`s
- *feat* (feature)
- *fix* (bug fix)
- *docs* (documentation)
- *style* (formatting, missing semi colons, …)
- *refactor*
- *test* (when adding missing tests)
- *chore* (maintain)
#### Allowed `<scope>`s
Scopes can anything specifying the place of the commit change in the code base -
for example, "api", "store", etc.
For more details on the commit format, see the [AngularJS commit style
guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).

36
DCO Normal file
View File

@@ -0,0 +1,36 @@
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

View File

@@ -0,0 +1,37 @@
## OEM configuration
The `coreos.oem.*` parameters follow the [os-release spec][os-release], but have been repurposed as a way for coreos-cloudinit to know about the OEM partition on this machine. Customizing this section is only needed when generating a new OEM of CoreOS from the SDK. The fields include:
- **id**: Lowercase string identifying the OEM
- **name**: Human-friendly string representing the OEM
- **version-id**: Lowercase string identifying the version of the OEM
- **home-url**: Link to the homepage of the provider or OEM
- **bug-report-url**: Link to a place to file bug reports about this OEM
coreos-cloudinit renders these fields to `/etc/oem-release`.
If no **id** field is provided, coreos-cloudinit will ignore this section.
For example, the following cloud-config document...
```
#cloud-config
coreos:
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
```
...would be rendered to the following `/etc/oem-release`:
```
ID=rackspace
NAME="Rackspace Cloud Servers"
VERSION_ID=168.0.0
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
```
[os-release]: http://www.freedesktop.org/software/systemd/man/os-release.html

View File

@@ -1,98 +1,135 @@
# Customize CoreOS with Cloud-Config
# Using Cloud-Config
CoreOS allows you to configure machine parameters, launch systemd units on startup and more. Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS.
CoreOS allows you to declaratively customize various OS-level items, such as network configuration, user accounts, and systemd units. This document describes the full list of items we can configure. The `coreos-cloudinit` program uses these files as it configures the OS after startup or during runtime.
## Configuration File
The file used by this system initialization program is called a "cloud-config" file. It is inspired by the [cloud-init][cloud-init] project's [cloud-config][cloud-config] file. which is "the defacto multi-distribution package that handles early initialization of a cloud instance" ([cloud-init docs][cloud-init-docs]). Because the cloud-init project includes tools which aren't used by CoreOS, only the relevant subset of its configuration items will be implemented in our cloud-config file. In addition to those, we added a few CoreOS-specific items, such as etcd configuration, OEM definition, and systemd units.
We've designed our implementation to allow the same cloud-config file to work across all of our supported platforms.
[cloud-init]: https://launchpad.net/cloud-init
[cloud-init-docs]: http://cloudinit.readthedocs.org/en/latest/index.html
[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
## Supported cloud-config Parameters
### File Format
### ssh_authorized_keys
The cloud-config file uses the [YAML][yaml] file format, which uses whitespace and new-lines to delimit lists, associative arrays, and values.
Provided public SSH keys will be authorized for the `core` user.
A cloud-config file should contain an associative array which has zero or more of the following keys:
The keys will be named "coreos-cloudinit" by default.
Override this with the `--ssh-key-name` flag when calling `coreos-cloudinit`.
- `coreos`
- `ssh_authorized_keys`
- `hostname`
- `users`
- `write_files`
- `manage_etc_hosts`
#### users
The expected values for these keys are defined in the rest of this document.
Add or modify users with the `users` directive by providing a list of user objects, each consisting of the following fields.
Each field is optional and of type string unless otherwise noted.
All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
[yaml]: https://en.wikipedia.org/wiki/YAML
- **name**: Required. Login name of user
- **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name>
- **no-create-home**: Boolean. Skip home directory createion.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups
- **no-user-group**: Boolean. Skip default group creation.
- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
- **system**: Create the user as a system user. No home directory will be created.
- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
### Providing Cloud-Config with Config-Drive
The following fields are not yet implemented:
CoreOS tries to conform to each platform's native method to provide user data. Each cloud provider tends to be unique, but this complexity has been abstracted by CoreOS. You can view each platform's instructions on their documentation pages. The most universal way to provide cloud-config is [via config-drive](https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/config-drive.md), which attaches a read-only device to the machine, that contains your cloud-config file.
- **inactive**: Deactivate the user upon creation
- **lock-passwd**: Boolean. Disable password login for user
- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
- **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad.
## Configuration Parameters
##### Generating a password hash
### coreos
You can generate a safe hash via:
#### etcd
mkpasswd --method=SHA-512 --rounds=4096
The `coreos.etcd.*` parameters will be translated to a partial systemd unit acting as an etcd configuration file.
We can use the templating feature of coreos-cloudinit to automate etcd configuration with the `$private_ipv4` and `$public_ipv4` fields. For example, the following cloud-config document...
Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed.
## Custom cloud-config Parameters
### coreos.etcd.discovery_url
The value of `coreos.etcd.discovery_url` will be used to discover the instance's etcd peers using the [etcd discovery protocol][disco-proto]. Usage of the [public discovery service][disco-service] is encouraged.
[disco-proto]: https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md
[disco-service]: http://discovery.etcd.io
### coreos.units
Arbitrary systemd units may be provided in the `coreos.units` attribute.
`coreos.units` is a list of objects with the following fields:
- **name**: string representing unit's name
- **runtime**: boolean indicating whether or not to persist the unit across reboots. This is analagous to the `--runtime` flag to `systemd enable`.
- **content**: plaintext string representing entire unit file
See docker example below.
## user-data Script
Simply set your user-data to a script where the first line is a shebang:
```
#!/bin/bash
echo 'Hello, world!'
```
## Examples
### Inject an SSH key, bootstrap etcd, and start fleet
```
#cloud-config
coreos:
etcd:
discovery_url: https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877
fleet:
autostart: yes
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
etcd:
name: node001
# generate a new token for each unique cluster from https://discovery.etcd.io/new
discovery: https://discovery.etcd.io/<token>
# multi-region and multi-cloud deployments need to use $public_ipv4
addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001
```
### Start a docker container on boot
...will generate a systemd unit drop-in like this:
```
[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/<token>"
Environment="ETCD_ADDR=203.0.113.29:4001"
Environment="ETCD_PEER_ADDR=192.0.2.13:7001"
```
For more information about the available configuration parameters, see the [etcd documentation][etcd-config].
Note that hyphens in the coreos.etcd.* keys are mapped to underscores.
[etcd-config]: https://github.com/coreos/etcd/blob/master/Documentation/configuration.md
#### fleet
The `coreos.fleet.*` parameters work very similarly to `coreos.etcd.*`, and allow for the configuration of fleet through environment variables. For example, the following cloud-config document...
```
#cloud-config
coreos:
fleet:
public-ip: $public_ipv4
metadata: region=us-west
```
...will generate a systemd unit drop-in like this:
```
[Service]
Environment="FLEET_PUBLIC_IP=203.0.113.29"
Environment="FLEET_METADATA=region=us-west"
```
For more information on fleet configuration, see the [fleet documentation][fleet-config].
[fleet-config]: https://github.com/coreos/fleet/blob/master/Documentation/configuration.md
#### update
The `coreos.update.*` parameters manipulate settings related to how CoreOS instances are updated.
These fields will be written out to and replace `/etc/coreos/update.conf`. If only one of the parameters is given it will only overwrite the given field.
- **reboot-strategy**: One of "reboot", "etcd-lock", "best-effort" or "off" for controlling when reboots are issued after an update is performed.
- _reboot_: Reboot immediately after an update is applied.
- _etcd-lock_: Reboot after first taking a distributed lock in etcd, this guarantees that only one host will reboot concurrently and that the cluster will remain available during the update.
- _best-effort_ - If etcd is running, "etcd-lock", otherwise simply "reboot".
- _off_ - Disable rebooting after updates are applied (not recommended).
- **server**: is the omaha endpoint URL which will be queried for updates.
- **group**: signifies the channel which should be used for automatic updates. This value defaults to the version of the image initially downloaded. (one of "master", "alpha", "beta", "stable")
```
#cloud-config
coreos:
update:
reboot-strategy: etcd-lock
```
#### units
The `coreos.units.*` parameters define a list of arbitrary systemd units to start. Each item is an object with the following fields:
- **name**: String representing unit's name. Required.
- **runtime**: Boolean indicating whether or not to persist the unit across reboots. This is analogous to the `--runtime` argument to `systemd enable`. Default value is false.
- **enable**: Boolean indicating whether or not to handle the [Install] section of the unit file. This is similar to running `systemctl enable <name>`. Default value is false.
- **content**: Plaintext string representing entire unit file. If no value is provided, the unit is assumed to exist already.
- **command**: Command to execute on unit: start, stop, reload, restart, try-restart, reload-or-restart, reload-or-try-restart. Default value is restart.
- **mask**: Whether to mask the unit file by symlinking it to `/dev/null` (analogous to `systemctl mask <name>`). Note that unlike `systemctl mask`, **this will destructively remove any existing unit file** located at `/etc/systemd/system/<unit>`, to ensure that the mask succeeds. Default value is false.
**NOTE:** The command field is ignored for all network, netdev, and link units. The systemd-networkd.service unit will be restarted in their place.
##### Examples
Write a unit to disk, automatically starting it.
```
#cloud-config
@@ -100,6 +137,7 @@ ssh_authorized_keys:
coreos:
units:
- name: docker-redis.service
command: start
content: |
[Unit]
Description=Redis container
@@ -115,17 +153,172 @@ coreos:
WantedBy=local.target
```
### Add a user
Start the built-in `etcd` and `fleet` services:
```
#cloud-config
coreos:
units:
- name: etcd.service
command: start
- name: fleet.service
command: start
```
### ssh_authorized_keys
The `ssh_authorized_keys` parameter adds public SSH keys which will be authorized for the `core` user.
The keys will be named "coreos-cloudinit" by default.
Override this by using the `--ssh-key-name` flag when calling `coreos-cloudinit`.
```
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
```
### hostname
The `hostname` parameter defines the system's hostname.
This is the local part of a fully-qualified domain name (i.e. `foo` in `foo.example.com`).
```
#cloud-config
hostname: coreos1
```
### users
The `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted.
All but the `passwd` and `ssh-authorized-keys` fields will be ignored if the user already exists.
- **name**: Required. Login name of user
- **gecos**: GECOS comment of user
- **passwd**: Hash of the password to use for this user
- **homedir**: User's home directory. Defaults to /home/<name>
- **no-create-home**: Boolean. Skip home directory creation.
- **primary-group**: Default group for the user. Defaults to a new group created named after the user.
- **groups**: Add user to these additional groups
- **no-user-group**: Boolean. Skip default group creation.
- **ssh-authorized-keys**: List of public SSH keys to authorize for this user
- **coreos-ssh-import-github**: Authorize SSH keys from Github user
- **coreos-ssh-import-url**: Authorize SSH keys imported from a url endpoint.
- **system**: Create the user as a system user. No home directory will be created.
- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases.
The following fields are not yet implemented:
- **inactive**: Deactivate the user upon creation
- **lock-passwd**: Boolean. Disable password login for user
- **sudo**: Entry to add to /etc/sudoers for user. By default, no sudo access is authorized.
- **selinux-user**: Corresponding SELinux user
- **ssh-import-id**: Import SSH keys by ID from Launchpad.
```
#cloud-config
users:
- name: elroy
passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
groups:
- staff
- docker
ssh-authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5yYv6mTlWfOoCYHZmfi3LDKVltj.E8XNKEcwWm...
groups:
- sudo
- docker
ssh-authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0g+ZTxC7weoIJLUafOgrm+h...
```
#### Generating a password hash
If you choose to use a password instead of an SSH key, generating a safe hash is extremely important to the security of your system. Simplified hashes like md5crypt are trivial to crack on modern GPU hardware. Here are a few ways to generate secure hashes:
```
# On Debian/Ubuntu (via the package "whois")
mkpasswd --method=SHA-512 --rounds=4096
# OpenSSL (note: this will only make md5crypt. While better than plantext it should not be considered fully secure)
openssl passwd -1
# Python (change password and salt values)
python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
# Perl (change password and salt values)
perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
```
Using a higher number of rounds will help create more secure passwords, but given enough time, password hashes can be reversed. On most RPM based distributions there is a tool called mkpasswd available in the `expect` package, but this does not handle "rounds" nor advanced hashing algorithms.
#### Retrieving SSH Authorized Keys
##### From a GitHub User
Using the `coreos-ssh-import-github` field, we can import public SSH keys from a GitHub user to use as authorized keys to a server.
```
#cloud-config
users:
- name: elroy
coreos-ssh-import-github: elroy
```
##### From an HTTP Endpoint
We can also pull public SSH keys from any HTTP endpoint which matches [GitHub's API response format](https://developer.github.com/v3/users/keys/#list-public-keys-for-a-user).
For example, if you have an installation of GitHub Enterprise, you can provide a complete URL with an authentication token:
```
#cloud-config
users:
- name: elroy
coreos-ssh-import-url: https://token:<OAUTH-TOKEN>@github-enterprise.example.com/users/elroy/keys
```
You can also specify any URL whose response matches the JSON format for public keys:
```
#cloud-config
users:
- name: elroy
coreos-ssh-import-url: https://example.com/public-keys
```
### write_files
The `write-file` parameter defines a list of files to create on the local filesystem. Each file is represented as an associative array which has the following keys:
- **path**: Absolute location on disk where contents should be written
- **content**: Data to write at the provided `path`
- **permissions**: String representing file permissions in octal notation (i.e. '0644')
- **owner**: User and group that should own the file written to disk. This is equivalent to the `<user>:<group>` argument to `chown <user>:<group> <path>`.
Explicitly not implemented is the **encoding** attribute.
The **content** field must represent exactly what should be written to disk.
```
#cloud-config
write_files:
- path: /etc/fleet/fleet.conf
permissions: 0644
content: |
verbosity=1
metadata="region=us-west,type=ssd"
```
### manage_etc_hosts
The `manage_etc_hosts` parameter configures the contents of the `/etc/hosts` file, which is used for local name resolution.
Currently, the only supported value is "localhost" which will cause your system's hostname
to resolve to "127.0.0.1". This is helpful when the host does not have DNS
infrastructure in place to resolve its own hostname, for example, when using Vagrant.
```
#cloud-config
manage_etc_hosts: localhost
```

View File

@@ -0,0 +1,30 @@
# Distribution via Config Drive
CoreOS supports providing configuration data via [config drive][config-drive]
disk images. Currently only providing a single script or cloud config file is
supported.
[config-drive]: http://docs.openstack.org/user-guide/content/enable_config_drive.html#config_drive_contents
## Contents and Format
The image should be a single FAT or ISO9660 file system with the label
`config-2` and the configuration data should be located at
`openstack/latest/user_data`.
For example, to wrap up a config named `user_data` in a config drive image:
mkdir -p /tmp/new-drive/openstack/latest
cp user_data /tmp/new-drive/openstack/latest/user_data
mkisofs -R -V config-2 -o configdrive.iso /tmp/new-drive
rm -r /tmp/new-drive
## QEMU virtfs
One exception to the above, when using QEMU it is possible to skip creating an
image and use a plain directory containing the same contents:
qemu-system-x86_64 \
-fsdev local,id=conf,security_model=none,readonly,path=/tmp/new-drive \
-device virtio-9p-pci,fsdev=conf,mount_tag=config-2 \
[usual qemu options here...]

202
LICENSE Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

5
NOTICE Normal file
View File

@@ -0,0 +1,5 @@
CoreOS Project
Copyright 2014 CoreOS, Inc
This product includes software developed at CoreOS, Inc.
(http://www.coreos.com/).

View File

@@ -1,9 +1,79 @@
# coreos-cloudinit
# coreos-cloudinit [![Build Status](https://travis-ci.org/coreos/coreos-cloudinit.png?branch=master)](https://travis-ci.org/coreos/coreos-cloudinit)
coreos-cloudinit allows a user to customize CoreOS machines by providing either an executable script or a cloud-config document as instance user-data. See below to learn how to use these features.
coreos-cloudinit enables a user to customize CoreOS machines by providing either a cloud-config document or an executable script through user-data.
## Supported Cloud-Config Features
## Configuration with cloud-config
Only a subset of [cloud-config functionality][cloud-config] is implemented. A set of custom parameters were added to the cloud-config format that are specific to CoreOS, which are [documented here](https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md).
A subset of the [official cloud-config spec][official-cloud-config] is implemented by coreos-cloudinit.
Additionally, several [CoreOS-specific options][custom-cloud-config] have been implemented to support interacting with unit files, bootstrapping etcd clusters, and more.
All supported cloud-config parameters are [documented here][all-cloud-config].
[cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
[official-cloud-config]: http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data
[custom-cloud-config]: https://github.com/coreos/coreos-cloudinit/blob/master/Documentation/cloud-config.md#coreos-parameters
[all-cloud-config]: https://github.com/coreos/coreos-cloudinit/tree/master/Documentation/cloud-config.md
The following is an example cloud-config document:
```
#cloud-config
coreos:
units:
- name: etcd.service
command: start
users:
- name: core
passwd: $1$allJZawX$00S5T756I5PGdQga5qhqv1
write_files:
- path: /etc/resolv.conf
content: |
nameserver 192.0.2.2
nameserver 192.0.2.3
```
## Executing a Script
coreos-cloudinit supports executing user-data as a script instead of parsing it as a cloud-config document.
Make sure the first line of your user-data is a shebang and coreos-cloudinit will attempt to execute it:
```
#!/bin/bash
echo 'Hello, world!'
```
## user-data Field Substitution
coreos-cloudinit will replace the following set of tokens in your user-data with system-generated values.
| Token | Description |
| ------------- | ----------- |
| $public_ipv4 | Public IPv4 address of machine |
| $private_ipv4 | Private IPv4 address of machine |
These values are determined by CoreOS based on the given provider on which your machine is running.
Read more about provider-specific functionality in the [CoreOS OEM documentation][oem-doc].
[oem-doc]: https://coreos.com/docs/sdk-distributors/distributors/notes-for-distributors/
For example, submitting the following user-data...
```
#cloud-config
coreos:
etcd:
addr: $public_ipv4:4001
peer-addr: $private_ipv4:7001
```
...will result in this cloud-config document being executed:
```
#cloud-config
coreos:
etcd:
addr: 203.0.113.29:4001
peer-addr: 192.0.2.13:7001
```

14
build
View File

@@ -1,6 +1,14 @@
#!/bin/bash -e
export GOBIN=${PWD}/bin
export GOPATH=${PWD}
ORG_PATH="github.com/coreos"
REPO_PATH="${ORG_PATH}/coreos-cloudinit"
go build -o bin/coreos-cloudinit github.com/coreos/coreos-cloudinit
if [ ! -h gopath/src/${REPO_PATH} ]; then
mkdir -p gopath/src/${ORG_PATH}
ln -s ../../../.. gopath/src/${REPO_PATH} || exit 255
fi
export GOBIN=${PWD}/bin
export GOPATH=${PWD}/gopath
go build -o bin/coreos-cloudinit ${REPO_PATH}

View File

@@ -1,143 +0,0 @@
package cloudinit
import (
"fmt"
"log"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
)
const DefaultSSHKeyName = "coreos-cloudinit"
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd struct{ Discovery_URL string }
Fleet struct{ Autostart bool }
Units []Unit
}
WriteFiles []WriteFile `yaml:"write_files"`
Hostname string
Users []User
}
func NewCloudConfig(contents []byte) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal(contents, &cfg)
return &cfg, err
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
func ApplyCloudConfig(cfg CloudConfig, sshKeyName string) error {
if cfg.Hostname != "" {
if err := SetHostname(cfg.Hostname); err != nil {
return err
}
log.Printf("Set hostname to %s", cfg.Hostname)
}
if len(cfg.Users) > 0 {
for _, user := range cfg.Users {
if user.Name == "" {
log.Printf("User object has no 'name' field, skipping")
continue
}
if UserExists(&user) {
log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
if user.PasswordHash != "" {
log.Printf("Setting '%s' user's password", user.Name)
if err := SetUserPassword(user.Name, user.PasswordHash); err != nil {
log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
return err
}
}
} else {
log.Printf("Creating user '%s'", user.Name)
if err := CreateUser(&user); err != nil {
log.Printf("Failed creating user '%s': %v", user.Name, err)
return err
}
}
if len(user.SSHAuthorizedKeys) > 0 {
log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
if err := AuthorizeSSHKeys(user.Name, sshKeyName, user.SSHAuthorizedKeys); err != nil {
return err
}
}
}
}
if len(cfg.SSHAuthorizedKeys) > 0 {
err := AuthorizeSSHKeys("core", sshKeyName, cfg.SSHAuthorizedKeys)
if err == nil {
log.Printf("Authorized SSH keys for core user")
} else {
return err
}
}
if len(cfg.WriteFiles) > 0 {
for _, file := range cfg.WriteFiles {
if err := ProcessWriteFile("/", &file); err != nil {
return err
}
log.Printf("Wrote file %s to filesystem", file.Path)
}
}
if cfg.Coreos.Etcd.Discovery_URL != "" {
err := PersistEtcdDiscoveryURL(cfg.Coreos.Etcd.Discovery_URL)
if err == nil {
log.Printf("Consumed etcd discovery url")
} else {
log.Fatalf("Failed to persist etcd discovery url to filesystem: %v", err)
}
}
if len(cfg.Coreos.Units) > 0 {
for _, unit := range cfg.Coreos.Units {
log.Printf("Placing unit %s on filesystem", unit.Name)
dst, err := PlaceUnit("/", &unit)
if err != nil {
return err
}
log.Printf("Placed unit %s at %s", unit.Name, dst)
if unit.Group() != "network" {
log.Printf("Enabling unit file %s", dst)
if err := EnableUnitFile(dst, unit.Runtime); err != nil {
return err
}
log.Printf("Enabled unit %s", unit.Name)
} else {
log.Printf("Skipping enable for network-like unit %s", unit.Name)
}
}
DaemonReload()
StartUnits(cfg.Coreos.Units)
}
if cfg.Coreos.Fleet.Autostart {
err := StartUnitByName("fleet.service")
if err == nil {
log.Printf("Started fleet service.")
} else {
return err
}
}
return nil
}

View File

@@ -1,25 +0,0 @@
package cloudinit
import (
"io/ioutil"
"log"
"os"
"path"
)
const (
etcdDiscoveryPath = "/var/run/etcd/bootstrap.disco"
)
func PersistEtcdDiscoveryURL(url string) error {
dir := path.Dir(etcdDiscoveryPath)
if _, err := os.Stat(dir); err != nil {
log.Printf("Creating directory /var/run/etcd")
err := os.MkdirAll(dir, os.FileMode(0644))
if err != nil {
return err
}
}
return ioutil.WriteFile(etcdDiscoveryPath, []byte(url), os.FileMode(0644))
}

View File

@@ -1,36 +0,0 @@
package cloudinit
import (
"io/ioutil"
"net/http"
)
type metadataService struct {
url string
client http.Client
}
func NewMetadataService(url string) *metadataService {
return &metadataService{url, http.Client{}}
}
func (ms *metadataService) UserData() ([]byte, error) {
resp, err := ms.client.Get(ms.url)
if err != nil {
return []byte{}, err
}
defer resp.Body.Close()
if resp.StatusCode / 100 != 2 {
return []byte{}, nil
}
respBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return respBytes, nil
}

View File

@@ -1,157 +0,0 @@
package cloudinit
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
)
type Unit struct {
Name string
Runtime bool
Content string
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() (group string) {
t := u.Type()
if t == "network" || t == "netdev" || t == "link" {
group = "network"
} else {
group = "system"
}
return
}
type Script []byte
func PlaceUnit(root string, u *Unit) (string, error) {
dir := "etc"
if u.Runtime {
dir = "run"
}
dst := path.Join(root, dir, "systemd", u.Group())
if _, err := os.Stat(dst); os.IsNotExist(err) {
if err := os.MkdirAll(dst, os.FileMode(0755)); err != nil {
return "", err
}
}
dst = path.Join(dst, u.Name)
err := ioutil.WriteFile(dst, []byte(u.Content), os.FileMode(0644))
if err != nil {
return "", err
}
return dst, nil
}
func EnableUnitFile(file string, runtime bool) error {
conn, err := dbus.New()
if err != nil {
return err
}
files := []string{file}
_, _, err = conn.EnableUnitFiles(files, runtime, true)
return err
}
func separateNetworkUnits(units []Unit) ([]Unit, []Unit) {
networkUnits := make([]Unit, 0)
nonNetworkUnits := make([]Unit, 0)
for _, unit := range units {
if unit.Group() == "network" {
networkUnits = append(networkUnits, unit)
} else {
nonNetworkUnits = append(nonNetworkUnits, unit)
}
}
return networkUnits, nonNetworkUnits
}
func StartUnits(units []Unit) error {
networkUnits, nonNetworkUnits := separateNetworkUnits(units)
if len(networkUnits) > 0 {
if err := RestartUnitByName("systemd-networkd.service"); err != nil {
return err
}
}
for _, unit := range nonNetworkUnits {
if err := RestartUnitByName(unit.Name); err != nil {
return err
}
}
return nil
}
func DaemonReload() error {
conn, err := dbus.New()
if err != nil {
return err
}
_, err = conn.Reload()
return err
}
func RestartUnitByName(name string) error {
log.Printf("Restarting unit %s", name)
conn, err := dbus.New()
if err != nil {
return err
}
output, err := conn.RestartUnit(name, "replace")
log.Printf("Restart completed with '%s'", output)
return err
}
func StartUnitByName(name string) error {
conn, err := dbus.New()
if err != nil {
return err
}
_, err = conn.StartUnit(name, "replace")
return err
}
func ExecuteScript(scriptPath string) (string, error) {
props := []dbus.Property{
dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
dbus.PropExecStart([]string{"/bin/bash", scriptPath}, false),
}
base := path.Base(scriptPath)
name := fmt.Sprintf("coreos-cloudinit-%s.service", base)
log.Printf("Creating transient systemd unit '%s'", name)
conn, err := dbus.New()
if err != nil {
return "", err
}
_, err = conn.StartTransientUnit(name, "replace", props...)
return name, err
}
func SetHostname(hostname string) error {
return exec.Command("hostnamectl", "set-hostname", hostname).Run()
}

View File

@@ -1,102 +0,0 @@
package cloudinit
import (
"io/ioutil"
"os"
"path"
"syscall"
"testing"
)
func TestPlaceNetworkUnit(t *testing.T) {
u := Unit{
Name: "50-eth0.network",
Runtime: true,
Content: `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`,
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if _, err := PlaceUnit(dir, &u); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
}
}
func TestPlaceMountUnit(t *testing.T) {
u := Unit{
Name: "media-state.mount",
Runtime: false,
Content: `[Mount]
What=/dev/sdb1
Where=/media/state
`,
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if _, err := PlaceUnit(dir, &u); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
fullPath := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Mount]
What=/dev/sdb1
Where=/media/state
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expect)
}
}

View File

@@ -1,30 +0,0 @@
package cloudinit
import (
"bufio"
"bytes"
"fmt"
"log"
"strings"
)
func ParseUserData(contents []byte) (interface{}, error) {
bytereader := bytes.NewReader(contents)
bufreader := bufio.NewReader(bytereader)
header, _ := bufreader.ReadString('\n')
if strings.HasPrefix(header, "#!") {
log.Printf("Parsing user-data as script")
return Script(contents), nil
} else if header == "#cloud-config\n" {
log.Printf("Parsing user-data as cloud-config")
cfg, err := NewCloudConfig(contents)
if err != nil {
log.Fatal(err.Error())
}
return *cfg, nil
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}
}

View File

@@ -1,66 +0,0 @@
package cloudinit
import (
"fmt"
"io/ioutil"
"os"
"path"
)
func PrepWorkspace(workspace string) error {
// Ensure workspace exists and is a directory
info, err := os.Stat(workspace)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", workspace)
}
} else {
err = os.MkdirAll(workspace, 0755)
if err != nil {
return err
}
}
// Ensure scripts dir in workspace exists and is a directory
scripts := path.Join(workspace, "scripts")
info, err = os.Stat(scripts)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", scripts)
}
} else {
err = os.Mkdir(scripts, 0755)
if err != nil {
return err
}
}
return nil
}
func PersistScriptInWorkspace(script Script, workspace string) (string, error) {
scriptsDir := path.Join(workspace, "scripts")
f, err := ioutil.TempFile(scriptsDir, "")
if err != nil {
return "", err
}
defer f.Close()
f.Chmod(0744)
_, err = f.Write(script)
if err != nil {
return "", err
}
// Ensure script has been written to disk before returning, as the
// next natural thing to do is execute it
f.Sync()
return f.Name(), nil
}
func PersistScriptUnitNameInWorkspace(name string, workspace string) error {
unitPath := path.Join(workspace, "scripts", "unit-name")
return ioutil.WriteFile(unitPath, []byte(name), 0644)
}

View File

@@ -1,46 +0,0 @@
package cloudinit
import (
"errors"
"io/ioutil"
"os"
"os/exec"
"path"
"strconv"
)
type WriteFile struct {
Encoding string
Content string
Owner string
Path string
Permissions string
}
func ProcessWriteFile(base string, wf *WriteFile) error {
fullPath := path.Join(base, wf.Path)
if err := os.MkdirAll(path.Dir(fullPath), os.FileMode(0744)); err != nil {
return err
}
// Parse string representation of file mode as octal
perm, err := strconv.ParseInt(wf.Permissions, 8, 32)
if err != nil {
return errors.New("Unable to parse file permissions as octal integer")
}
if err := ioutil.WriteFile(fullPath, []byte(wf.Content), os.FileMode(perm)); err != nil {
return err
}
if wf.Owner != "" {
// We shell out since we don't have a way to look up unix groups natively
cmd := exec.Command("chown", wf.Owner, fullPath)
if err := cmd.Run(); err != nil {
return err
}
}
return nil
}

View File

@@ -1,35 +1,44 @@
package main
import (
"fmt"
"flag"
"io/ioutil"
"os"
"fmt"
"log"
"os"
"github.com/coreos/coreos-cloudinit/cloudinit"
"github.com/coreos/coreos-cloudinit/datasource"
"github.com/coreos/coreos-cloudinit/initialize"
"github.com/coreos/coreos-cloudinit/system"
)
const version = "0.1.2"
const version = "0.7.3"
func init() {
//Removes timestamp since it is displayed already during booting
log.SetFlags(0)
}
func main() {
var userdata []byte
var err error
var printVersion bool
flag.BoolVar(&printVersion, "version", false, "Print the version and exit")
var ignoreFailure bool
flag.BoolVar(&ignoreFailure, "ignore-failure", false, "Exits with 0 status in the event of malformed input from user-data")
var file string
flag.StringVar(&file, "from-file", "", "Read user-data from provided file")
var url string
flag.StringVar(&url, "from-url", "", "Download user-data from provided url")
var useProcCmdline bool
flag.BoolVar(&useProcCmdline, "from-proc-cmdline", false, fmt.Sprintf("Parse %s for '%s=<url>', using the cloud-config served by an HTTP GET to <url>", datasource.ProcCmdlineLocation, datasource.ProcCmdlineCloudConfigFlag))
var workspace string
flag.StringVar(&workspace, "workspace", "/var/lib/coreos-cloudinit", "Base directory coreos-cloudinit should use to store data")
var sshKeyName string
flag.StringVar(&sshKeyName, "ssh-key-name", cloudinit.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
flag.StringVar(&sshKeyName, "ssh-key-name", initialize.DefaultSSHKeyName, "Add SSH keys to the system with the given name")
flag.Parse()
@@ -38,54 +47,64 @@ func main() {
os.Exit(0)
}
if file != "" && url != "" {
fmt.Println("Provide one of --from-file or --from-url")
os.Exit(1)
}
var ds datasource.Datasource
if file != "" {
log.Printf("Reading user-data from file: %s", file)
userdata, err = ioutil.ReadFile(file)
if err != nil {
log.Fatal(err.Error())
}
ds = datasource.NewLocalFile(file)
} else if url != "" {
log.Printf("Reading user-data from metadata service")
svc := cloudinit.NewMetadataService(url)
userdata, err = svc.UserData()
if err != nil {
log.Fatal(err.Error())
}
ds = datasource.NewMetadataService(url)
} else if useProcCmdline {
ds = datasource.NewProcCmdline()
} else {
fmt.Println("Provide one of --from-file or --from-url")
fmt.Println("Provide one of --from-file, --from-url or --from-proc-cmdline")
os.Exit(1)
}
if len(userdata) == 0 {
log.Printf("Fetching user-data from datasource of type %q", ds.Type())
userdataBytes, err := ds.Fetch()
if err != nil {
log.Printf("Failed fetching user-data from datasource: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
os.Exit(1)
}
}
if len(userdataBytes) == 0 {
log.Printf("No user data to handle, exiting.")
os.Exit(0)
}
parsed, err := cloudinit.ParseUserData(userdata)
env := initialize.NewEnvironment("/", workspace)
userdata := string(userdataBytes)
userdata = env.Apply(userdata)
parsed, err := initialize.ParseUserData(userdata)
if err != nil {
log.Fatalf("Failed parsing user-data: %v", err)
log.Printf("Failed parsing user-data: %v", err)
if ignoreFailure {
os.Exit(0)
} else {
os.Exit(1)
}
}
err = cloudinit.PrepWorkspace(workspace)
err = initialize.PrepWorkspace(env.Workspace())
if err != nil {
log.Fatalf("Failed preparing workspace: %v", err)
}
switch t := parsed.(type) {
case cloudinit.CloudConfig:
err = cloudinit.ApplyCloudConfig(t, sshKeyName)
case cloudinit.Script:
case initialize.CloudConfig:
err = initialize.Apply(t, env)
case system.Script:
var path string
path, err = cloudinit.PersistScriptInWorkspace(t, workspace)
path, err = initialize.PersistScriptInWorkspace(t, env.Workspace())
if err == nil {
var name string
name, err = cloudinit.ExecuteScript(path)
cloudinit.PersistScriptUnitNameInWorkspace(name, workspace)
name, err = system.ExecuteScript(path)
initialize.PersistUnitNameInWorkspace(name, workspace)
}
}

27
cover Executable file
View File

@@ -0,0 +1,27 @@
#!/bin/bash -e
#
# Generate coverage HTML for a package
# e.g. PKG=./initialize ./cover
#
if [ -z "$PKG" ]; then
echo "cover only works with a single package, sorry"
exit 255
fi
COVEROUT="coverage"
if ! [ -d "$COVEROUT" ]; then
mkdir "$COVEROUT"
fi
# strip out slashes and dots
COVERPKG=${PKG//\//}
COVERPKG=${COVERPKG//./}
# generate arg for "go test"
export COVER="-coverprofile ${COVEROUT}/${COVERPKG}.out"
source ./test
go tool cover -html=${COVEROUT}/${COVERPKG}.out

6
datasource/datasource.go Normal file
View File

@@ -0,0 +1,6 @@
package datasource
type Datasource interface {
Fetch() ([]byte, error)
Type() string
}

21
datasource/file.go Normal file
View File

@@ -0,0 +1,21 @@
package datasource
import (
"io/ioutil"
)
type localFile struct {
path string
}
func NewLocalFile(path string) *localFile {
return &localFile{path}
}
func (self *localFile) Fetch() ([]byte, error) {
return ioutil.ReadFile(self.path)
}
func (self *localFile) Type() string {
return "local-file"
}

View File

@@ -0,0 +1,20 @@
package datasource
import "github.com/coreos/coreos-cloudinit/pkg"
type metadataService struct {
url string
}
func NewMetadataService(url string) *metadataService {
return &metadataService{url}
}
func (ms *metadataService) Fetch() ([]byte, error) {
client := pkg.NewHttpClient()
return client.Get(ms.url)
}
func (ms *metadataService) Type() string {
return "metadata-service"
}

View File

@@ -0,0 +1,72 @@
package datasource
import (
"errors"
"io/ioutil"
"log"
"strings"
"github.com/coreos/coreos-cloudinit/pkg"
)
const (
ProcCmdlineLocation = "/proc/cmdline"
ProcCmdlineCloudConfigFlag = "cloud-config-url"
)
type procCmdline struct{
Location string
}
func NewProcCmdline() *procCmdline {
return &procCmdline{Location: ProcCmdlineLocation}
}
func (self *procCmdline) Fetch() ([]byte, error) {
contents, err := ioutil.ReadFile(self.Location)
if err != nil {
return nil, err
}
cmdline := strings.TrimSpace(string(contents))
url, err := findCloudConfigURL(cmdline)
if err != nil {
return nil, err
}
client := pkg.NewHttpClient()
cfg, err := client.Get(url)
if err != nil {
return nil, err
}
return cfg, nil
}
func (self *procCmdline) Type() string {
return "proc-cmdline"
}
func findCloudConfigURL(input string) (url string, err error) {
err = errors.New("cloud-config-url not found")
for _, token := range strings.Split(input, " ") {
parts := strings.SplitN(token, "=", 2)
key := parts[0]
key = strings.Replace(key, "_", "-", -1)
if key != "cloud-config-url" {
continue
}
if len(parts) != 2 {
log.Printf("Found cloud-config-url in /proc/cmdline with no value, ignoring.")
continue
}
url = parts[1]
err = nil
}
return
}

View File

@@ -0,0 +1,88 @@
package datasource
import (
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"os"
"testing"
)
func TestParseCmdlineCloudConfigFound(t *testing.T) {
tests := []struct {
input string
expect string
}{
{
"cloud-config-url=example.com",
"example.com",
},
{
"cloud_config_url=example.com",
"example.com",
},
{
"cloud-config-url cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url= cloud-config-url=example.com",
"example.com",
},
{
"cloud-config-url=one.example.com cloud-config-url=two.example.com",
"two.example.com",
},
{
"foo=bar cloud-config-url=example.com ping=pong",
"example.com",
},
}
for i, tt := range tests {
output, err := findCloudConfigURL(tt.input)
if output != tt.expect {
t.Errorf("Test case %d failed: %s != %s", i, output, tt.expect)
}
if err != nil {
t.Errorf("Test case %d produced error: %v", i, err)
}
}
}
func TestProcCmdlineAndFetchConfig(t *testing.T) {
var (
ProcCmdlineTmpl = "foo=bar cloud-config-url=%s/config\n"
CloudConfigContent = "#cloud-config\n"
)
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" && r.RequestURI == "/config" {
fmt.Fprint(w, CloudConfigContent)
}
}))
defer ts.Close()
file, err := ioutil.TempFile(os.TempDir(), "test_proc_cmdline")
defer os.Remove(file.Name())
if err != nil {
t.Errorf("Test produced error: %v", err)
}
_, err = file.Write([]byte(fmt.Sprintf(ProcCmdlineTmpl, ts.URL)))
if err != nil {
t.Errorf("Test produced error: %v", err)
}
p := NewProcCmdline()
p.Location = file.Name()
cfg, err := p.Fetch()
if err != nil {
t.Errorf("Test produced error: %v", err)
}
if string(cfg) != CloudConfigContent {
t.Errorf("Test failed, response body: %s != %s", cfg, CloudConfigContent)
}
}

290
initialize/config.go Normal file
View File

@@ -0,0 +1,290 @@
package initialize
import (
"errors"
"fmt"
"log"
"path"
"github.com/coreos/coreos-cloudinit/third_party/launchpad.net/goyaml"
"github.com/coreos/coreos-cloudinit/system"
)
// CloudConfigFile represents a CoreOS specific configuration option that can generate
// an associated system.File to be written to disk
type CloudConfigFile interface {
// File should either return (*system.File, error), or (nil, nil) if nothing
// needs to be done for this configuration option.
File(root string) (*system.File, error)
}
// CloudConfigUnit represents a CoreOS specific configuration option that can generate
// an associated system.Unit to be created/enabled appropriately
type CloudConfigUnit interface {
// Unit should either return (*system.Unit, error), or (nil, nil) if nothing
// needs to be done for this configuration option.
Unit(root string) (*system.Unit, error)
}
// CloudConfig encapsulates the entire cloud-config configuration file and maps directly to YAML
type CloudConfig struct {
SSHAuthorizedKeys []string `yaml:"ssh_authorized_keys"`
Coreos struct {
Etcd EtcdEnvironment
Fleet FleetEnvironment
OEM OEMRelease
Update UpdateConfig
Units []system.Unit
}
WriteFiles []system.File `yaml:"write_files"`
Hostname string
Users []system.User
ManageEtcHosts EtcHosts `yaml:"manage_etc_hosts"`
}
type warner func(format string, v ...interface{})
// warnOnUnrecognizedKeys parses the contents of a cloud-config file and calls
// warn(msg, key) for every unrecognized key (i.e. those not present in CloudConfig)
func warnOnUnrecognizedKeys(contents string, warn warner) {
// Generate a map of all understood cloud config options
var cc map[string]interface{}
b, _ := goyaml.Marshal(&CloudConfig{})
goyaml.Unmarshal(b, &cc)
// Now unmarshal the entire provided contents
var c map[string]interface{}
goyaml.Unmarshal([]byte(contents), &c)
// Check that every key in the contents exists in the cloud config
for k, _ := range c {
if _, ok := cc[k]; !ok {
warn("Warning: unrecognized key %q in provided cloud config - ignoring section", k)
}
}
// Check for unrecognized coreos options, if any are set
coreos, ok := c["coreos"]
if ok {
set := coreos.(map[interface{}]interface{})
known := cc["coreos"].(map[interface{}]interface{})
for k, _ := range set {
key := k.(string)
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in coreos section of provided cloud config - ignoring", key)
}
}
}
// Check for any badly-specified users, if any are set
users, ok := c["users"]
if ok {
var known map[string]interface{}
b, _ := goyaml.Marshal(&system.User{})
goyaml.Unmarshal(b, &known)
set := users.([]interface{})
for _, u := range set {
user := u.(map[interface{}]interface{})
for k, _ := range user {
key := k.(string)
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in user section of cloud config - ignoring", key)
}
}
}
}
// Check for any badly-specified files, if any are set
files, ok := c["write_files"]
if ok {
var known map[string]interface{}
b, _ := goyaml.Marshal(&system.File{})
goyaml.Unmarshal(b, &known)
set := files.([]interface{})
for _, f := range set {
file := f.(map[interface{}]interface{})
for k, _ := range file {
key := k.(string)
if _, ok := known[key]; !ok {
warn("Warning: unrecognized key %q in file section of cloud config - ignoring", key)
}
}
}
}
}
// NewCloudConfig instantiates a new CloudConfig from the given contents (a
// string of YAML), returning any error encountered. It will ignore unknown
// fields but log encountering them.
func NewCloudConfig(contents string) (*CloudConfig, error) {
var cfg CloudConfig
err := goyaml.Unmarshal([]byte(contents), &cfg)
if err != nil {
return &cfg, err
}
warnOnUnrecognizedKeys(contents, log.Printf)
return &cfg, nil
}
func (cc CloudConfig) String() string {
bytes, err := goyaml.Marshal(cc)
if err != nil {
return ""
}
stringified := string(bytes)
stringified = fmt.Sprintf("#cloud-config\n%s", stringified)
return stringified
}
// Apply renders a CloudConfig to an Environment. This can involve things like
// configuring the hostname, adding new users, writing various configuration
// files to disk, and manipulating systemd services.
func Apply(cfg CloudConfig, env *Environment) error {
if cfg.Hostname != "" {
if err := system.SetHostname(cfg.Hostname); err != nil {
return err
}
log.Printf("Set hostname to %s", cfg.Hostname)
}
for _, user := range cfg.Users {
if user.Name == "" {
log.Printf("User object has no 'name' field, skipping")
continue
}
if system.UserExists(&user) {
log.Printf("User '%s' exists, ignoring creation-time fields", user.Name)
if user.PasswordHash != "" {
log.Printf("Setting '%s' user's password", user.Name)
if err := system.SetUserPassword(user.Name, user.PasswordHash); err != nil {
log.Printf("Failed setting '%s' user's password: %v", user.Name, err)
return err
}
}
} else {
log.Printf("Creating user '%s'", user.Name)
if err := system.CreateUser(&user); err != nil {
log.Printf("Failed creating user '%s': %v", user.Name, err)
return err
}
}
if len(user.SSHAuthorizedKeys) > 0 {
log.Printf("Authorizing %d SSH keys for user '%s'", len(user.SSHAuthorizedKeys), user.Name)
if err := system.AuthorizeSSHKeys(user.Name, env.SSHKeyName(), user.SSHAuthorizedKeys); err != nil {
return err
}
}
if user.SSHImportGithubUser != "" {
log.Printf("Authorizing github user %s SSH keys for CoreOS user '%s'", user.SSHImportGithubUser, user.Name)
if err := SSHImportGithubUser(user.Name, user.SSHImportGithubUser); err != nil {
return err
}
}
if user.SSHImportURL != "" {
log.Printf("Authorizing SSH keys for CoreOS user '%s' from '%s'", user.Name, user.SSHImportURL)
if err := SSHImportKeysFromURL(user.Name, user.SSHImportURL); err != nil {
return err
}
}
}
if len(cfg.SSHAuthorizedKeys) > 0 {
err := system.AuthorizeSSHKeys("core", env.SSHKeyName(), cfg.SSHAuthorizedKeys)
if err == nil {
log.Printf("Authorized SSH keys for core user")
} else {
return err
}
}
for _, ccf := range []CloudConfigFile{cfg.Coreos.OEM, cfg.Coreos.Update, cfg.ManageEtcHosts} {
f, err := ccf.File(env.Root())
if err != nil {
return err
}
if f != nil {
cfg.WriteFiles = append(cfg.WriteFiles, *f)
}
}
for _, ccu := range []CloudConfigUnit{cfg.Coreos.Etcd, cfg.Coreos.Fleet, cfg.Coreos.Update} {
u, err := ccu.Unit(env.Root())
if err != nil {
return err
}
if u != nil {
cfg.Coreos.Units = append(cfg.Coreos.Units, *u)
}
}
for _, file := range cfg.WriteFiles {
file.Path = path.Join(env.Root(), file.Path)
if err := system.WriteFile(&file); err != nil {
return err
}
log.Printf("Wrote file %s to filesystem", file.Path)
}
commands := make(map[string]string, 0)
reload := false
for _, unit := range cfg.Coreos.Units {
dst := system.UnitDestination(&unit, env.Root())
if unit.Content != "" {
log.Printf("Writing unit %s to filesystem at path %s", unit.Name, dst)
if err := system.PlaceUnit(&unit, dst); err != nil {
return err
}
log.Printf("Placed unit %s at %s", unit.Name, dst)
reload = true
}
if unit.Mask {
log.Printf("Masking unit file %s", unit.Name)
if err := system.MaskUnit(unit.Name, env.Root()); err != nil {
return err
}
}
if unit.Enable {
if unit.Group() != "network" {
log.Printf("Enabling unit file %s", unit.Name)
if err := system.EnableUnitFile(unit.Name, unit.Runtime); err != nil {
return err
}
log.Printf("Enabled unit %s", unit.Name)
} else {
log.Printf("Skipping enable for network-like unit %s", unit.Name)
}
}
if unit.Group() == "network" {
commands["systemd-networkd.service"] = "restart"
} else if unit.Command != "" {
commands[unit.Name] = unit.Command
}
}
if reload {
if err := system.DaemonReload(); err != nil {
return errors.New(fmt.Sprintf("failed systemd daemon-reload: %v", err))
}
}
for unit, command := range commands {
log.Printf("Calling unit command '%s %s'", command, unit)
res, err := system.RunUnitCommand(command, unit)
if err != nil {
return err
}
log.Printf("Result of '%s %s': %s", command, unit, res)
}
return nil
}

View File

@@ -1,13 +1,78 @@
package cloudinit
package initialize
import (
"fmt"
"strings"
"testing"
)
func TestCloudConfigUnknownKeys(t *testing.T) {
contents := `
coreos:
etcd:
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
coreos_unknown:
foo: "bar"
section_unknown:
dunno:
something
bare_unknown:
bar
write_files:
- content: fun
path: /var/party
file_unknown: nofun
users:
- name: fry
passwd: somehash
user_unknown: philip
hostname:
foo
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("error instantiating CloudConfig with unknown keys: %v", err)
}
if cfg.Hostname != "foo" {
t.Fatalf("hostname not correctly set when invalid keys are present")
}
if len(cfg.Coreos.Etcd) < 1 {
t.Fatalf("etcd section not correctly set when invalid keys are present")
}
if len(cfg.WriteFiles) < 1 || cfg.WriteFiles[0].Content != "fun" || cfg.WriteFiles[0].Path != "/var/party" {
t.Fatalf("write_files section not correctly set when invalid keys are present")
}
if len(cfg.Users) < 1 || cfg.Users[0].Name != "fry" || cfg.Users[0].PasswordHash != "somehash" {
t.Fatalf("users section not correctly set when invalid keys are present")
}
var warnings string
catchWarn := func(f string, v ...interface{}) {
warnings += fmt.Sprintf(f, v...)
}
warnOnUnrecognizedKeys(contents, catchWarn)
if !strings.Contains(warnings, "coreos_unknown") {
t.Errorf("warnings did not catch unrecognized coreos option coreos_unknown")
}
if !strings.Contains(warnings, "bare_unknown") {
t.Errorf("warnings did not catch unrecognized key bare_unknown")
}
if !strings.Contains(warnings, "section_unknown") {
t.Errorf("warnings did not catch unrecognized key section_unknown")
}
if !strings.Contains(warnings, "user_unknown") {
t.Errorf("warnings did not catch unrecognized user key user_unknown")
}
if !strings.Contains(warnings, "file_unknown") {
t.Errorf("warnings did not catch unrecognized file key file_unknown")
}
}
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfigEmpty(t *testing.T) {
cfg, err := NewCloudConfig([]byte{})
cfg, err := NewCloudConfig("")
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
}
@@ -17,14 +82,6 @@ func TestCloudConfigEmpty(t *testing.T) {
t.Error("Parsed incorrect number of SSH keys")
}
if cfg.Coreos.Etcd.Discovery_URL != "" {
t.Error("Parsed incorrect value of discovery url")
}
if cfg.Coreos.Fleet.Autostart {
t.Error("Expected AutostartFleet not to be defined")
}
if len(cfg.WriteFiles) != 0 {
t.Error("Expected zero WriteFiles")
}
@@ -36,12 +93,12 @@ func TestCloudConfigEmpty(t *testing.T) {
// Assert that the parsing of a cloud config file "generally works"
func TestCloudConfig(t *testing.T) {
contents := []byte(`
contents := `
coreos:
etcd:
discovery_url: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
fleet:
autostart: Yes
discovery: "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877"
update:
reboot-strategy: reboot
units:
- name: 50-eth0.network
runtime: yes
@@ -55,6 +112,12 @@ coreos:
Address=10.209.171.177/19
'
oem:
id: rackspace
name: Rackspace Cloud Servers
version-id: 168.0.0
home-url: https://www.rackspace.com/cloud/servers/
bug-report-url: https://github.com/coreos/coreos-overlay
ssh_authorized_keys:
- foobar
- foobaz
@@ -66,7 +129,7 @@ write_files:
permissions: '0644'
owner: root:dogepack
hostname: trontastic
`)
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
@@ -81,14 +144,6 @@ hostname: trontastic
t.Error("Expected first SSH key to be 'foobaz'")
}
if cfg.Coreos.Etcd.Discovery_URL != "https://discovery.etcd.io/827c73219eeb2fa5530027c37bf18877" {
t.Error("Failed to parse etcd discovery url")
}
if !cfg.Coreos.Fleet.Autostart {
t.Error("Expected AutostartFleet to be true")
}
if len(cfg.WriteFiles) != 1 {
t.Error("Failed to parse correct number of write_files")
} else {
@@ -99,8 +154,8 @@ hostname: trontastic
if wf.Encoding != "" {
t.Errorf("WriteFile has incorrect encoding %s", wf.Encoding)
}
if wf.Permissions != "0644" {
t.Errorf("WriteFile has incorrect permissions %s", wf.Permissions)
if perm, _ := wf.Permissions(); perm != 0644 {
t.Errorf("WriteFile has incorrect permissions %s", perm)
}
if wf.Path != "/etc/dogepack.conf" {
t.Errorf("WriteFile has incorrect path %s", wf.Path)
@@ -134,20 +189,27 @@ Address=10.209.171.177/19
}
}
if cfg.Coreos.OEM.ID != "rackspace" {
t.Errorf("Failed parsing coreos.oem. Expected ID 'rackspace', got %q.", cfg.Coreos.OEM.ID)
}
if cfg.Hostname != "trontastic" {
t.Errorf("Failed to parse hostname")
}
if cfg.Coreos.Update["reboot-strategy"] != "reboot" {
t.Errorf("Failed to parse locksmith strategy")
}
}
// Assert that our interface conversion doesn't panic
func TestCloudConfigKeysNotList(t *testing.T) {
contents := []byte(`
contents := `
ssh_authorized_keys:
- foo: bar
`)
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error :%v", err)
t.Fatalf("Encountered unexpected error: %v", err)
}
keys := cfg.SSHAuthorizedKeys
@@ -157,7 +219,7 @@ ssh_authorized_keys:
}
func TestCloudConfigSerializationHeader(t *testing.T) {
cfg, _ := NewCloudConfig([]byte{})
cfg, _ := NewCloudConfig("")
contents := cfg.String()
header := strings.SplitN(contents, "\n", 2)[0]
if header != "#cloud-config" {
@@ -165,8 +227,28 @@ func TestCloudConfigSerializationHeader(t *testing.T) {
}
}
// TestDropInIgnored asserts that users are unable to set DropIn=True on units
func TestDropInIgnored(t *testing.T) {
contents := `
coreos:
units:
- name: test
dropin: true
`
cfg, err := NewCloudConfig(contents)
if err != nil || len(cfg.Coreos.Units) != 1 {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Coreos.Units) != 1 || cfg.Coreos.Units[0].Name != "test" {
t.Fatalf("Expected 1 unit, but got %d: %v", len(cfg.Coreos.Units), cfg.Coreos.Units)
}
if cfg.Coreos.Units[0].DropIn {
t.Errorf("dropin option on unit in cloud-config was not ignored!")
}
}
func TestCloudConfigUsers(t *testing.T) {
contents := []byte(`
contents := `
users:
- name: elroy
passwd: somehash
@@ -182,7 +264,7 @@ users:
no-user-group: true
system: y
no-log-init: True
`)
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
@@ -223,7 +305,7 @@ users:
t.Errorf("Failed to parse no-create-home field")
}
if user.PrimaryGroup != "things"{
if user.PrimaryGroup != "things" {
t.Errorf("Failed to parse primary-group field, got %q", user.PrimaryGroup)
}

60
initialize/env.go Normal file
View File

@@ -0,0 +1,60 @@
package initialize
import (
"os"
"path"
"strings"
)
const DefaultSSHKeyName = "coreos-cloudinit"
type Environment struct {
root string
workspace string
sshKeyName string
substitutions map[string]string
}
func NewEnvironment(root, workspace string) *Environment {
substitutions := map[string]string{
"$public_ipv4": os.Getenv("COREOS_PUBLIC_IPV4"),
"$private_ipv4": os.Getenv("COREOS_PRIVATE_IPV4"),
}
return &Environment{root, workspace, DefaultSSHKeyName, substitutions}
}
func (self *Environment) Workspace() string {
return path.Join(self.root, self.workspace)
}
func (self *Environment) Root() string {
return self.root
}
func (self *Environment) SSHKeyName() string {
return self.sshKeyName
}
func (self *Environment) SetSSHKeyName(name string) {
self.sshKeyName = name
}
func (self *Environment) Apply(data string) string {
for key, val := range self.substitutions {
data = strings.Replace(data, key, val, -1)
}
return data
}
// normalizeSvcEnv standardizes the keys of the map (environment variables for a service)
// by replacing any dashes with underscores and ensuring they are entirely upper case.
// For example, "some-env" --> "SOME_ENV"
func normalizeSvcEnv(m map[string]string) map[string]string {
out := make(map[string]string, len(m))
for key, val := range m {
key = strings.ToUpper(key)
key = strings.Replace(key, "-", "_", -1)
out[key] = val
}
return out
}

27
initialize/env_test.go Normal file
View File

@@ -0,0 +1,27 @@
package initialize
import (
"os"
"testing"
)
func TestEnvironmentApply(t *testing.T) {
os.Setenv("COREOS_PUBLIC_IPV4", "192.0.2.3")
os.Setenv("COREOS_PRIVATE_IPV4", "192.0.2.203")
env := NewEnvironment("./", "./")
input := `[Service]
ExecStart=/usr/bin/echo "$public_ipv4"
ExecStop=/usr/bin/echo $private_ipv4
ExecStop=/usr/bin/echo $unknown
`
expected := `[Service]
ExecStart=/usr/bin/echo "192.0.2.3"
ExecStop=/usr/bin/echo 192.0.2.203
ExecStop=/usr/bin/echo $unknown
`
output := env.Apply(input)
if output != expected {
t.Fatalf("Environment incorrectly applied.\nOutput:\n%s\nExpected:\n%s", output, expected)
}
}

54
initialize/etcd.go Normal file
View File

@@ -0,0 +1,54 @@
package initialize
import (
"errors"
"fmt"
"github.com/coreos/coreos-cloudinit/system"
)
type EtcdEnvironment map[string]string
func (ee EtcdEnvironment) String() (out string) {
norm := normalizeSvcEnv(ee)
if val, ok := norm["DISCOVERY_URL"]; ok {
delete(norm, "DISCOVERY_URL")
if _, ok := norm["DISCOVERY"]; !ok {
norm["DISCOVERY"] = val
}
}
out += "[Service]\n"
for key, val := range norm {
out += fmt.Sprintf("Environment=\"ETCD_%s=%s\"\n", key, val)
}
return
}
// Unit creates a Unit file drop-in for etcd, using any configured
// options and adding a default MachineID if unset.
func (ee EtcdEnvironment) Unit(root string) (*system.Unit, error) {
if ee == nil {
return nil, nil
}
if _, ok := ee["name"]; !ok {
if machineID := system.MachineID(root); machineID != "" {
ee["name"] = machineID
} else if hostname, err := system.Hostname(); err == nil {
ee["name"] = hostname
} else {
return nil, errors.New("Unable to determine default etcd name")
}
}
return &system.Unit{
Name: "etcd.service",
Runtime: true,
DropIn: true,
Content: ee.String(),
}, nil
}

166
initialize/etcd_test.go Normal file
View File

@@ -0,0 +1,166 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestEtcdEnvironment(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryURLTranslated(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "http://disco.example.com/foobar"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentDiscoveryOverridesDiscoveryURL(t *testing.T) {
cfg := make(EtcdEnvironment, 0)
cfg["discovery_url"] = "ping"
cfg["discovery"] = "pong"
cfg["peer-bind-addr"] = "127.0.0.1:7002"
env := cfg.String()
expect := `[Service]
Environment="ETCD_DISCOVERY=pong"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestEtcdEnvironmentWrittenToDisk(t *testing.T) {
ec := EtcdEnvironment{
"name": "node001",
"discovery": "http://disco.example.com/foobar",
"peer-bind-addr": "127.0.0.1:7002",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
u, err := ec.Unit(dir)
if err != nil {
t.Fatalf("Generating etcd unit failed: %v", err)
}
if u == nil {
t.Fatalf("Returned nil etcd unit unexpectedly")
}
dst := system.UnitDestination(u, dir)
os.Stderr.WriteString("writing to " + dir + "\n")
if err := system.PlaceUnit(u, dst); err != nil {
t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node001"
Environment="ETCD_DISCOVERY=http://disco.example.com/foobar"
Environment="ETCD_PEER_BIND_ADDR=127.0.0.1:7002"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentWrittenToDiskDefaultToMachineID(t *testing.T) {
ee := EtcdEnvironment{}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
err = ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007"), os.FileMode(0444))
if err != nil {
t.Fatalf("Failed writing out /etc/machine-id: %v", err)
}
u, err := ee.Unit(dir)
if err != nil {
t.Fatalf("Generating etcd unit failed: %v", err)
}
if u == nil {
t.Fatalf("Returned nil etcd unit unexpectedly")
}
dst := system.UnitDestination(u, dir)
os.Stderr.WriteString("writing to " + dir + "\n")
if err := system.PlaceUnit(u, dst); err != nil {
t.Fatalf("Writing of EtcdEnvironment failed: %v", err)
}
fullPath := path.Join(dir, "run", "systemd", "system", "etcd.service.d", "20-cloudinit.conf")
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `[Service]
Environment="ETCD_NAME=node007"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}
func TestEtcdEnvironmentWhenNil(t *testing.T) {
// EtcdEnvironment will be a nil map if it wasn't in the yaml
var ee EtcdEnvironment
if ee != nil {
t.Fatalf("EtcdEnvironment is not nil")
}
u, err := ee.Unit("")
if u != nil || err != nil {
t.Fatalf("Unit returned a non-nil value for nil input")
}
}

34
initialize/fleet.go Normal file
View File

@@ -0,0 +1,34 @@
package initialize
import (
"fmt"
"github.com/coreos/coreos-cloudinit/system"
)
type FleetEnvironment map[string]string
func (fe FleetEnvironment) String() (out string) {
norm := normalizeSvcEnv(fe)
out += "[Service]\n"
for key, val := range norm {
out += fmt.Sprintf("Environment=\"FLEET_%s=%s\"\n", key, val)
}
return
}
// Unit generates a Unit file drop-in for fleet, if any fleet options were
// configured in cloud-config
func (fe FleetEnvironment) Unit(root string) (*system.Unit, error) {
if len(fe) < 1 {
return nil, nil
}
return &system.Unit{
Name: "fleet.service",
Runtime: true,
DropIn: true,
Content: fe.String(),
}, nil
}

42
initialize/fleet_test.go Normal file
View File

@@ -0,0 +1,42 @@
package initialize
import "testing"
func TestFleetEnvironment(t *testing.T) {
cfg := make(FleetEnvironment, 0)
cfg["public-ip"] = "12.34.56.78"
env := cfg.String()
expect := `[Service]
Environment="FLEET_PUBLIC_IP=12.34.56.78"
`
if env != expect {
t.Errorf("Generated environment:\n%s\nExpected environment:\n%s", env, expect)
}
}
func TestFleetUnit(t *testing.T) {
cfg := make(FleetEnvironment, 0)
u, err := cfg.Unit("/")
if u != nil {
t.Errorf("unexpectedly generated unit with empty FleetEnvironment")
}
cfg["public-ip"] = "12.34.56.78"
u, err = cfg.Unit("/")
if err != nil {
t.Errorf("error generating fleet unit: %v", err)
}
if u == nil {
t.Fatalf("unexpectedly got nil unit generating fleet unit!")
}
if !u.Runtime {
t.Errorf("bad Runtime for generated fleet unit!")
}
if !u.DropIn {
t.Errorf("bad DropIn for generated fleet unit!")
}
}

18
initialize/github.go Normal file
View File

@@ -0,0 +1,18 @@
package initialize
import (
"fmt"
"github.com/coreos/coreos-cloudinit/system"
)
func SSHImportGithubUser(system_user string, github_user string) error {
url := fmt.Sprintf("https://api.github.com/users/%s/keys", github_user)
keys, err := fetchUserKeys(url)
if err != nil {
return err
}
key_name := fmt.Sprintf("github-%s", github_user)
return system.AuthorizeSSHKeys(system_user, key_name, keys)
}

32
initialize/github_test.go Normal file
View File

@@ -0,0 +1,32 @@
package initialize
import (
"testing"
)
func TestCloudConfigUsersGithubUser(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-github: bcwaldon
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportGithubUser != "bcwaldon" {
t.Errorf("github user is %q, expected 'bcwaldon'", user.SSHImportGithubUser)
}
}

View File

@@ -0,0 +1,46 @@
package initialize
import (
"errors"
"fmt"
"os"
"path"
"github.com/coreos/coreos-cloudinit/system"
)
const DefaultIpv4Address = "127.0.0.1"
type EtcHosts string
func (eh EtcHosts) generateEtcHosts() (out string, err error) {
if eh != "localhost" {
return "", errors.New("Invalid option to manage_etc_hosts")
}
// use the operating system hostname
hostname, err := os.Hostname()
if err != nil {
return "", err
}
return fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname), nil
}
func (eh EtcHosts) File(root string) (*system.File, error) {
if eh == "" {
return nil, nil
}
etcHosts, err := eh.generateEtcHosts()
if err != nil {
return nil, err
}
return &system.File{
Path: path.Join("etc", "hosts"),
RawFilePermissions: "0644",
Content: etcHosts,
}, nil
}

View File

@@ -0,0 +1,85 @@
package initialize
import (
"fmt"
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestCloudConfigManageEtcHosts(t *testing.T) {
contents := `
manage_etc_hosts: localhost
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
manageEtcHosts := cfg.ManageEtcHosts
if manageEtcHosts != "localhost" {
t.Errorf("ManageEtcHosts value is %q, expected 'localhost'", manageEtcHosts)
}
}
func TestManageEtcHostsInvalidValue(t *testing.T) {
eh := EtcHosts("invalid")
if f, err := eh.File(""); err == nil || f != nil {
t.Fatalf("EtcHosts File succeeded with invalid value!")
}
}
func TestEtcHostsWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
eh := EtcHosts("localhost")
f, err := eh.File(dir)
if err != nil {
t.Fatalf("Error calling File on EtcHosts: %v", err)
}
if f == nil {
t.Fatalf("manageEtcHosts returned nil file unexpectedly")
}
f.Path = path.Join(dir, f.Path)
if err := system.WriteFile(f); err != nil {
t.Fatalf("Error writing EtcHosts: %v", err)
}
fullPath := path.Join(dir, "etc", "hosts")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("Unable to read OS hostname: %v", err)
}
expect := fmt.Sprintf("%s %s\n", DefaultIpv4Address, hostname)
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

41
initialize/oem.go Normal file
View File

@@ -0,0 +1,41 @@
package initialize
import (
"fmt"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
type OEMRelease struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
VersionID string `yaml:"version-id"`
HomeURL string `yaml:"home-url"`
BugReportURL string `yaml:"bug-report-url"`
}
func (oem OEMRelease) String() string {
fields := []string{
fmt.Sprintf("ID=%s", oem.ID),
fmt.Sprintf("VERSION_ID=%s", oem.VersionID),
fmt.Sprintf("NAME=%q", oem.Name),
fmt.Sprintf("HOME_URL=%q", oem.HomeURL),
fmt.Sprintf("BUG_REPORT_URL=%q", oem.BugReportURL),
}
return strings.Join(fields, "\n") + "\n"
}
func (oem OEMRelease) File(root string) (*system.File, error) {
if oem.ID == "" {
return nil, nil
}
return &system.File{
Path: path.Join("etc", "oem-release"),
RawFilePermissions: "0644",
Content: oem.String(),
}, nil
}

64
initialize/oem_test.go Normal file
View File

@@ -0,0 +1,64 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
func TestOEMReleaseWrittenToDisk(t *testing.T) {
oem := OEMRelease{
ID: "rackspace",
Name: "Rackspace Cloud Servers",
VersionID: "168.0.0",
HomeURL: "https://www.rackspace.com/cloud/servers/",
BugReportURL: "https://github.com/coreos/coreos-overlay",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
f, err := oem.File(dir)
if err != nil {
t.Fatalf("Processing of OEMRelease failed: %v", err)
}
if f == nil {
t.Fatalf("OEMRelease returned nil file unexpectedly")
}
f.Path = path.Join(dir, f.Path)
if err := system.WriteFile(f); err != nil {
t.Fatalf("Writing of OEMRelease failed: %v", err)
}
fullPath := path.Join(dir, "etc", "oem-release")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expect := `ID=rackspace
VERSION_ID=168.0.0
NAME="Rackspace Cloud Servers"
HOME_URL="https://www.rackspace.com/cloud/servers/"
BUG_REPORT_URL="https://github.com/coreos/coreos-overlay"
`
if string(contents) != expect {
t.Fatalf("File has incorrect contents")
}
}

43
initialize/ssh_keys.go Normal file
View File

@@ -0,0 +1,43 @@
package initialize
import (
"encoding/json"
"fmt"
"github.com/coreos/coreos-cloudinit/pkg"
"github.com/coreos/coreos-cloudinit/system"
)
type UserKey struct {
ID int `json:"id,omitempty"`
Key string `json:"key"`
}
func SSHImportKeysFromURL(system_user string, url string) error {
keys, err := fetchUserKeys(url)
if err != nil {
return err
}
key_name := fmt.Sprintf("coreos-cloudinit-%s", system_user)
return system.AuthorizeSSHKeys(system_user, key_name, keys)
}
func fetchUserKeys(url string) ([]string, error) {
client := pkg.NewHttpClient()
data, err := client.Get(url)
if err != nil {
return nil, err
}
var userKeys []UserKey
err = json.Unmarshal(data, &userKeys)
if err != nil {
return nil, err
}
keys := make([]string, 0)
for _, key := range userKeys {
keys = append(keys, key.Key)
}
return keys, err
}

View File

@@ -0,0 +1,69 @@
package initialize
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
)
func TestCloudConfigUsersUrlMarshal(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gh_res := `
[
{
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
},
{
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBANxpzIbTzKTeBRaOIdUxwwGwvDasTfU/PonhbNIuhYjc+xFGvBRTumox2F+luVAKKs4WdvA4nJXaY1OFi6DZftk5Bp4E2JaSzp8ulAzHsMexDdv6LGHGEJj/qdHAL1vHk2K89PpwRFSRZI8XRBLjvkr4ZgBKLG5ZILXPJEPP2j3lAAAAFQCtxoTnV8wy0c4grcGrQ+1sCsD7WQAAAIAqZsW2GviMe1RQrbZT0xAZmI64XRPrnLsoLxycHWlS7r6uUln2c6Ae2MB/YF0d4Kd1XZii9GHj7rrypqEo7MW8uSabhu70nmu1J8m2O3Dsr+4oJLeat9vwPsJV92IKO0jQwjKnAOHOiB9JKGeCw+NfXfogbti9/q38Q6XcS+SI5wAAAIEA1803Y2h+tOOpZXAsNIwl9mRfExWzLQ3L7knwJdznQu/6SW1H/1oyoYLebuk187Qj2UFI5qQ6AZNc49DvohWx0Cg6ABcyubNyoaCjZKWIdxVnItHWNbLe//+tyTu0I2eQwJOORsEPK5gMpf599C7wXQ//DzZOWbTWiHEX52gCTmk="
},
{
"id": 5224438,
"key": "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
}
]
`
fmt.Fprintln(w, gh_res)
}))
defer ts.Close()
keys, err := fetchUserKeys(ts.URL)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
expected := "ssh-dss AAAAB3NzaC1kc3MAAACBAIHAu822ggSkIHrJYvhmBceOSVjuflfQm8RbMMDNVe9relQfuPbN+nxGGTCKzPLebeOcX+Wwi77TPXWwK3BZMglfXxhABlFPsuMb63Tqp94pBYsJdx/iFj9iGo6pKoM1k8ubOcqsUnq+BR9895zRbE7MjdwkGo67+QhCEwvkwAnNAAAAFQCuddVqXLCubzqnWmeHLQE+2GFfHwAAAIBnlXW5h15ndVuwi0htF4oodVSB1KwnTWcuBK+aE1zRs76yvRb0Ws+oifumThDwB/Tec6FQuAfRKfy6piChZqsu5KvL98I+2t5yyi1td+kMvdTnVL2lW44etDKseOcozmknCOmh4Dqvhl/2MwrDAhlPaN08EEq9h3w3mXtNLWH64QAAAIBAzDOKr17llngaKIdDXh+LtXKh87+zfjlTA36/9r2uF2kYE5uApDtu9sPCkt7+YBQt7R8prADPckwAiXwVdk0xijIOpLDBmoydQJJRQ+zTMxvpQmUr/1kUOv0zb+lB657CgvN0vVTmP2swPeMvgntt3C4vw7Ab+O+MS9peOAJbbQ=="
if keys[0] != expected {
t.Fatalf("expected %s, got %s", expected, keys[0])
}
expected = "ssh-dss AAAAB3NzaC1kc3MAAACBAPKRWdKhzGZuLAJL6M1eM51hWViMqNBC2C6lm2OqGRYLuIf1GJ391widUuSf4wQqnkR22Q9PCmAZ19XCf11wBRMnuw9I/Z3Bt5bXfc+dzFBCmHYGJ6wNSv++H9jxyMb+usmsenWOFZGNO2jN0wrJ4ay8Yt0bwtRU+VCXpuRLszMzAAAAFQDZUIuPjcfK5HLgnwZ/J3lvtvlUjQAAAIEApIkAwLuCQV5j3U6DmI/Y6oELqSUR2purFm8jo8jePFfe1t+ghikgD254/JXlhDCVgY0NLXcak+coJfGCTT23quJ7I5xdpTn/OZO2Q6Woum/bijFC/UWwQbLz0R2nU3DoHv5v6XHQZxuIG4Fsxa91S+vWjZFtI7RuYlBCZA//ANMAAACBAJO0FojzkX6IeaWLqrgu9GTkFwGFazZ+LPH5JOWPoPn1hQKuR32Uf6qNcBZcIjY7SF0P7HF5rLQd6zKZzHqqQQ92MV555NEwjsnJglYU8CaaZsfYooaGPgA1YN7RhTSAuDmUW5Hyfj5BH4NTtrzrvJxIhDoQLf31Fasjw00r4R0O"
if keys[2] != expected {
t.Fatalf("expected %s, got %s", expected, keys[2])
}
}
func TestCloudConfigUsersSSHImportURL(t *testing.T) {
contents := `
users:
- name: elroy
coreos-ssh-import-url: https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys
`
cfg, err := NewCloudConfig(contents)
if err != nil {
t.Fatalf("Encountered unexpected error: %v", err)
}
if len(cfg.Users) != 1 {
t.Fatalf("Parsed %d users, expected 1", cfg.Users)
}
user := cfg.Users[0]
if user.Name != "elroy" {
t.Errorf("User name is %q, expected 'elroy'", user.Name)
}
if user.SSHImportURL != "https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys" {
t.Errorf("ssh import url is %q, expected 'https://token:x-auth-token@github.enterprise.com/api/v3/polvi/keys'", user.SSHImportURL)
}
}

149
initialize/update.go Normal file
View File

@@ -0,0 +1,149 @@
package initialize
import (
"bufio"
"errors"
"fmt"
"os"
"path"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
const (
locksmithUnit = "locksmithd.service"
)
// updateOption represents a configurable update option, which, if set, will be
// written into update.conf, replacing any existing value for the option
type updateOption struct {
key string // key used to configure this option in cloud-config
valid []string // valid values for the option
prefix string // prefix for the option in the update.conf file
value string // used to store the new value in update.conf (including prefix)
seen bool // whether the option has been seen in any existing update.conf
}
// updateOptions defines the update options understood by cloud-config.
// The keys represent the string used in cloud-config to configure the option.
var updateOptions = []*updateOption{
&updateOption{
key: "reboot-strategy",
prefix: "REBOOT_STRATEGY=",
valid: []string{"best-effort", "etcd-lock", "reboot", "off"},
},
&updateOption{
key: "group",
prefix: "GROUP=",
valid: []string{"master", "beta", "alpha", "stable"},
},
&updateOption{
key: "server",
prefix: "SERVER=",
},
}
// isValid checks whether a supplied value is valid for this option
func (uo updateOption) isValid(val string) bool {
if len(uo.valid) == 0 {
return true
}
for _, v := range uo.valid {
if val == v {
return true
}
}
return false
}
type UpdateConfig map[string]string
// File generates an `/etc/coreos/update.conf` file (if any update
// configuration options are set in cloud-config) by either rewriting the
// existing file on disk, or starting from `/usr/share/coreos/update.conf`
func (uc UpdateConfig) File(root string) (*system.File, error) {
if len(uc) < 1 {
return nil, nil
}
var out string
// Generate the list of possible substitutions to be performed based on the options that are configured
subs := make([]*updateOption, 0)
for _, uo := range updateOptions {
val, ok := uc[uo.key]
if !ok {
continue
}
if !uo.isValid(val) {
return nil, errors.New(fmt.Sprintf("invalid value %v for option %v (valid options: %v)", val, uo.key, uo.valid))
}
uo.value = uo.prefix + val
subs = append(subs, uo)
}
etcUpdate := path.Join(root, "etc", "coreos", "update.conf")
usrUpdate := path.Join(root, "usr", "share", "coreos", "update.conf")
conf, err := os.Open(etcUpdate)
if os.IsNotExist(err) {
conf, err = os.Open(usrUpdate)
}
if err != nil {
return nil, err
}
scanner := bufio.NewScanner(conf)
for scanner.Scan() {
line := scanner.Text()
for _, s := range subs {
if strings.HasPrefix(line, s.prefix) {
line = s.value
s.seen = true
break
}
}
out += line
out += "\n"
if err := scanner.Err(); err != nil {
return nil, err
}
}
for _, s := range subs {
if !s.seen {
out += s.value
out += "\n"
}
}
return &system.File{
Path: path.Join("etc", "coreos", "update.conf"),
RawFilePermissions: "0644",
Content: out,
}, nil
}
// GetUnit generates a locksmith system.Unit, if reboot-strategy was set in
// cloud-config, for the cloud-init initializer to act on appropriately
func (uc UpdateConfig) Unit(root string) (*system.Unit, error) {
strategy, ok := uc["reboot-strategy"]
if !ok {
return nil, nil
}
u := &system.Unit{
Name: locksmithUnit,
Command: "restart",
Mask: false,
}
if strategy == "off" {
u.Command = "stop"
u.Mask = true
}
return u, nil
}

217
initialize/update_test.go Normal file
View File

@@ -0,0 +1,217 @@
package initialize
import (
"io/ioutil"
"os"
"path"
"sort"
"strings"
"testing"
"github.com/coreos/coreos-cloudinit/system"
)
const (
base = `SERVER=https://example.com
GROUP=thegroupc`
configured = base + `
REBOOT_STRATEGY=awesome
`
expected = base + `
REBOOT_STRATEGY=etcd-lock
`
)
func setupFixtures(dir string) {
os.MkdirAll(path.Join(dir, "usr", "share", "coreos"), 0755)
os.MkdirAll(path.Join(dir, "run", "systemd", "system"), 0755)
ioutil.WriteFile(path.Join(dir, "usr", "share", "coreos", "update.conf"), []byte(base), 0644)
}
func TestEmptyUpdateConfig(t *testing.T) {
uc := &UpdateConfig{}
f, err := uc.File("")
if err != nil {
t.Error("unexpected error getting file from empty UpdateConfig")
}
if f != nil {
t.Errorf("getting file from empty UpdateConfig should have returned nil, got %v", f)
}
u, err := uc.Unit("")
if err != nil {
t.Error("unexpected error getting unit from empty UpdateConfig")
}
if u != nil {
t.Errorf("getting unit from empty UpdateConfig should have returned nil, got %v", u)
}
}
func TestInvalidUpdateOptions(t *testing.T) {
uon := &updateOption{
key: "numbers",
prefix: "numero_",
valid: []string{"one", "two"},
}
uoa := &updateOption{
key: "any_will_do",
prefix: "any_",
}
if !uon.isValid("one") {
t.Error("update option did not accept valid option \"one\"")
}
if uon.isValid("three") {
t.Error("update option accepted invalid option \"three\"")
}
for _, s := range []string{"one", "asdf", "foobarbaz"} {
if !uoa.isValid(s) {
t.Errorf("update option with no \"valid\" field did not accept %q", s)
}
}
uc := &UpdateConfig{"reboot-strategy": "wizzlewazzle"}
f, err := uc.File("")
if err == nil {
t.Errorf("File did not give an error on invalid UpdateOption")
}
if f != nil {
t.Errorf("File did not return a nil file on invalid UpdateOption")
}
}
func TestServerGroupOptions(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
u := &UpdateConfig{"group": "master", "server": "http://foo.com"}
want := `
GROUP=master
SERVER=http://foo.com`
f, err := u.File(dir)
if err != nil {
t.Errorf("unexpected error getting file from UpdateConfig: %v", err)
} else if f == nil {
t.Error("unexpectedly got empty file from UpdateConfig")
} else {
out := strings.Split(f.Content, "\n")
sort.Strings(out)
got := strings.Join(out, "\n")
if got != want {
t.Errorf("File has incorrect contents, got %v, want %v", got, want)
}
}
}
func TestRebootStrategies(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
strategies := []struct {
name string
line string
uMask bool
uCommand string
}{
{"best-effort", "REBOOT_STRATEGY=best-effort", false, "restart"},
{"etcd-lock", "REBOOT_STRATEGY=etcd-lock", false, "restart"},
{"reboot", "REBOOT_STRATEGY=reboot", false, "restart"},
{"off", "REBOOT_STRATEGY=off", true, "stop"},
}
for _, s := range strategies {
uc := &UpdateConfig{"reboot-strategy": s.name}
f, err := uc.File(dir)
if err != nil {
t.Errorf("update failed to generate file for reboot-strategy=%v: %v", s.name, err)
} else if f == nil {
t.Errorf("generated empty file for reboot-strategy=%v", s.name)
} else {
seen := false
for _, line := range strings.Split(f.Content, "\n") {
if line == s.line {
seen = true
break
}
}
if !seen {
t.Errorf("couldn't find expected line %v for reboot-strategy=%v", s.line)
}
}
u, err := uc.Unit(dir)
if err != nil {
t.Errorf("failed to generate unit for reboot-strategy=%v!", s.name)
} else if u == nil {
t.Errorf("generated empty unit for reboot-strategy=%v", s.name)
} else {
if u.Name != locksmithUnit {
t.Errorf("unit generated for reboot strategy=%v had bad name: %v", s.name, u.Name)
}
if u.Mask != s.uMask {
t.Errorf("unit generated for reboot strategy=%v had bad mask: %t", s.name, u.Mask)
}
if u.Command != s.uCommand {
t.Errorf("unit generated for reboot strategy=%v had bad command: %v", s.name, u.Command)
}
}
}
}
func TestUpdateConfWrittenToDisk(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
setupFixtures(dir)
for i := 0; i < 2; i++ {
if i == 1 {
err = ioutil.WriteFile(path.Join(dir, "etc", "coreos", "update.conf"), []byte(configured), 0644)
if err != nil {
t.Fatal(err)
}
}
uc := &UpdateConfig{"reboot-strategy": "etcd-lock"}
f, err := uc.File(dir)
if err != nil {
t.Fatalf("Processing UpdateConfig failed: %v", err)
} else if f == nil {
t.Fatal("Unexpectedly got nil updateconfig file")
}
f.Path = path.Join(dir, f.Path)
if err := system.WriteFile(f); err != nil {
t.Fatalf("Error writing update config: %v", err)
}
fullPath := path.Join(dir, "etc", "coreos", "update.conf")
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(fullPath)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
if string(contents) != expected {
t.Fatalf("File has incorrect contents, got %v, wanted %v", string(contents), expected)
}
}
}

33
initialize/user_data.go Normal file
View File

@@ -0,0 +1,33 @@
package initialize
import (
"fmt"
"log"
"strings"
"github.com/coreos/coreos-cloudinit/system"
)
func ParseUserData(contents string) (interface{}, error) {
header := strings.SplitN(contents, "\n", 2)[0]
// Explicitly trim the header so we can handle user-data from
// non-unix operating systems. The rest of the file is parsed
// by goyaml, which correctly handles CRLF.
header = strings.TrimSpace(header)
if strings.HasPrefix(header, "#!") {
log.Printf("Parsing user-data as script")
return system.Script(contents), nil
} else if header == "#cloud-config" {
log.Printf("Parsing user-data as cloud-config")
cfg, err := NewCloudConfig(contents)
if err != nil {
log.Fatal(err.Error())
}
return *cfg, nil
} else {
return nil, fmt.Errorf("Unrecognized user-data header: %s", header)
}
}

View File

@@ -0,0 +1,49 @@
package initialize
import (
"testing"
)
func TestParseHeaderCRLF(t *testing.T) {
configs := []string{
"#cloud-config\nfoo: bar",
"#cloud-config\r\nfoo: bar",
}
for i, config := range configs {
_, err := ParseUserData(config)
if err != nil {
t.Errorf("Failed parsing config %d: %v", i, err)
}
}
scripts := []string{
"#!bin/bash\necho foo",
"#!bin/bash\r\necho foo",
}
for i, script := range scripts {
_, err := ParseUserData(script)
if err != nil {
t.Errorf("Failed parsing script %d: %v", i, err)
}
}
}
func TestParseConfigCRLF(t *testing.T) {
contents := "#cloud-config\r\nhostname: foo\r\nssh_authorized_keys:\r\n - foobar\r\n"
ud, err := ParseUserData(contents)
if err != nil {
t.Fatalf("Failed parsing config: %v", err)
}
cfg := ud.(CloudConfig)
if cfg.Hostname != "foo" {
t.Error("Failed parsing hostname from config")
}
if len(cfg.SSHAuthorizedKeys) != 1 {
t.Error("Parsed incorrect number of SSH keys")
}
}

48
initialize/workspace.go Normal file
View File

@@ -0,0 +1,48 @@
package initialize
import (
"io/ioutil"
"path"
"github.com/coreos/coreos-cloudinit/system"
)
func PrepWorkspace(workspace string) error {
if err := system.EnsureDirectoryExists(workspace); err != nil {
return err
}
scripts := path.Join(workspace, "scripts")
if err := system.EnsureDirectoryExists(scripts); err != nil {
return err
}
return nil
}
func PersistScriptInWorkspace(script system.Script, workspace string) (string, error) {
scriptsPath := path.Join(workspace, "scripts")
tmp, err := ioutil.TempFile(scriptsPath, "")
if err != nil {
return "", err
}
tmp.Close()
file := system.File{
Path: tmp.Name(),
RawFilePermissions: "0744",
Content: string(script),
}
err = system.WriteFile(&file)
return file.Path, err
}
func PersistUnitNameInWorkspace(name string, workspace string) error {
file := system.File{
Path: path.Join(workspace, "scripts", "unit-name"),
RawFilePermissions: "0644",
Content: name,
}
return system.WriteFile(&file)
}

123
pkg/http_client.go Normal file
View File

@@ -0,0 +1,123 @@
package pkg
import (
"crypto/tls"
"errors"
"fmt"
"io/ioutil"
"log"
"net"
"net/http"
neturl "net/url"
"strings"
"time"
)
const (
HTTP_2xx = 2
HTTP_4xx = 4
)
type HttpClient struct {
// Maximum exp backoff duration. Defaults to 5 seconds
MaxBackoff time.Duration
// Maximum number of connection retries. Defaults to 15
MaxRetries int
// HTTP client timeout, this is suggested to be low since exponential
// backoff will kick off too. Defaults to 2 seconds
Timeout time.Duration
// Whether or not to skip TLS verification. Defaults to false
SkipTLS bool
}
func NewHttpClient() *HttpClient {
return &HttpClient{
MaxBackoff: time.Second * 5,
MaxRetries: 15,
Timeout: time.Duration(2) * time.Second,
SkipTLS: false,
}
}
func expBackoff(interval, max time.Duration) time.Duration {
interval = interval * 2
if interval > max {
interval = max
}
return interval
}
// Fetches a given URL with support for exponential backoff and maximum retries
func (h *HttpClient) Get(rawurl string) ([]byte, error) {
if rawurl == "" {
return nil, errors.New("URL is empty. Skipping.")
}
url, err := neturl.Parse(rawurl)
if err != nil {
return nil, err
}
// Unfortunately, url.Parse is too generic to throw errors if a URL does not
// have a valid HTTP scheme. So, we have to do this extra validation
if !strings.HasPrefix(url.Scheme, "http") {
return nil, fmt.Errorf("URL %s does not have a valid HTTP scheme. Skipping.", rawurl)
}
dataURL := url.String()
// We need to create our own client in order to add timeout support.
// TODO(c4milo) Replace it once Go 1.3 is officially used by CoreOS
// More info: https://code.google.com/p/go/source/detail?r=ada6f2d5f99f
transport := &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: h.SkipTLS,
},
Dial: func(network, addr string) (net.Conn, error) {
deadline := time.Now().Add(h.Timeout)
c, err := net.DialTimeout(network, addr, h.Timeout)
if err != nil {
return nil, err
}
c.SetDeadline(deadline)
return c, nil
},
}
client := &http.Client{
Transport: transport,
}
duration := 50 * time.Millisecond
for retry := 1; retry <= h.MaxRetries; retry++ {
log.Printf("Fetching data from %s. Attempt #%d", dataURL, retry)
resp, err := client.Get(dataURL)
if err == nil {
defer resp.Body.Close()
status := resp.StatusCode / 100
if status == HTTP_2xx {
return ioutil.ReadAll(resp.Body)
}
if status == HTTP_4xx {
return nil, fmt.Errorf("Not found. HTTP status code: %d", resp.StatusCode)
}
log.Printf("Server error. HTTP status code: %d", resp.StatusCode)
} else {
log.Printf("Unable to fetch data: %s", err.Error())
}
duration = expBackoff(duration, h.MaxBackoff)
log.Printf("Sleeping for %v...", duration)
time.Sleep(duration)
}
return nil, fmt.Errorf("Unable to fetch data. Maximum retries reached: %d", h.MaxRetries)
}

140
pkg/http_client_test.go Normal file
View File

@@ -0,0 +1,140 @@
package pkg
import (
"fmt"
"io"
"math"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestExpBackoff(t *testing.T) {
duration := time.Millisecond
max := time.Hour
for i := 0; i < math.MaxUint16; i++ {
duration = expBackoff(duration, max)
if duration < 0 {
t.Fatalf("duration too small: %v %v", duration, i)
}
if duration > max {
t.Fatalf("duration too large: %v %v", duration, i)
}
}
}
// Test exponential backoff and that it continues retrying if a 5xx response is
// received
func TestGetURLExpBackOff(t *testing.T) {
var expBackoffTests = []struct {
count int
body string
}{
{0, "number of attempts: 0"},
{1, "number of attempts: 1"},
{2, "number of attempts: 2"},
}
client := NewHttpClient()
for i, tt := range expBackoffTests {
mux := http.NewServeMux()
count := 0
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if count == tt.count {
io.WriteString(w, fmt.Sprintf("number of attempts: %d", count))
return
}
count++
http.Error(w, "", 500)
})
ts := httptest.NewServer(mux)
defer ts.Close()
data, err := client.Get(ts.URL)
if err != nil {
t.Errorf("Test case %d produced error: %v", i, err)
}
if count != tt.count {
t.Errorf("Test case %d failed: %d != %d", i, count, tt.count)
}
if string(data) != tt.body {
t.Errorf("Test case %d failed: %s != %s", i, tt.body, data)
}
}
}
// Test that it stops retrying if a 4xx response comes back
func TestGetURL4xx(t *testing.T) {
client := NewHttpClient()
retries := 0
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
retries++
http.Error(w, "", 404)
}))
defer ts.Close()
_, err := client.Get(ts.URL)
if err == nil {
t.Errorf("Incorrect result\ngot: %s\nwant: %s", err.Error(), "Not found. HTTP status code: 404")
}
if retries > 1 {
t.Errorf("Number of retries:\n%d\nExpected number of retries:\n%s", retries, 1)
}
}
// Test that it fetches and returns user-data just fine
func TestGetURL2xx(t *testing.T) {
var cloudcfg = `
#cloud-config
coreos:
oem:
id: test
name: CoreOS.box for Test
version-id: %VERSION_ID%+%BUILD_ID%
home-url: https://github.com/coreos/coreos-cloudinit
bug-report-url: https://github.com/coreos/coreos-cloudinit
update:
reboot-strategy: best-effort
`
client := NewHttpClient()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, cloudcfg)
}))
defer ts.Close()
data, err := client.Get(ts.URL)
if err != nil {
t.Errorf("Incorrect result\ngot: %v\nwant: %v", err, nil)
}
if string(data) != cloudcfg {
t.Errorf("Incorrect result\ngot: %s\nwant: %s", string(data), cloudcfg)
}
}
// Test attempt to fetching using malformed URL
func TestGetMalformedURL(t *testing.T) {
client := NewHttpClient()
var tests = []struct {
url string
want string
}{
{"boo", "URL boo does not have a valid HTTP scheme. Skipping."},
{"mailto://boo", "URL mailto://boo does not have a valid HTTP scheme. Skipping."},
{"ftp://boo", "URL ftp://boo does not have a valid HTTP scheme. Skipping."},
{"", "URL is empty. Skipping."},
}
for _, test := range tests {
_, err := client.Get(test.url)
if err == nil || err.Error() != test.want {
t.Errorf("Incorrect result\ngot: %v\nwant: %v", err, test.want)
}
}
}

View File

@@ -1 +0,0 @@
../../../

76
system/file.go Normal file
View File

@@ -0,0 +1,76 @@
package system
import (
"errors"
"fmt"
"io/ioutil"
"os"
"os/exec"
"path"
"strconv"
)
type File struct {
Encoding string
Content string
Owner string
Path string
RawFilePermissions string `yaml:"permissions"`
}
func (f *File) Permissions() (os.FileMode, error) {
if f.RawFilePermissions == "" {
return os.FileMode(0644), nil
}
// Parse string representation of file mode as octal
perm, err := strconv.ParseInt(f.RawFilePermissions, 8, 32)
if err != nil {
return 0, errors.New("Unable to parse file permissions as octal integer")
}
return os.FileMode(perm), nil
}
func WriteFile(f *File) error {
if f.Encoding != "" {
return fmt.Errorf("Unable to write file with encoding %s", f.Encoding)
}
if err := os.MkdirAll(path.Dir(f.Path), os.FileMode(0755)); err != nil {
return err
}
perm, err := f.Permissions()
if err != nil {
return err
}
if err := ioutil.WriteFile(f.Path, []byte(f.Content), perm); err != nil {
return err
}
if f.Owner != "" {
// We shell out since we don't have a way to look up unix groups natively
cmd := exec.Command("chown", f.Owner, f.Path)
if err := cmd.Run(); err != nil {
return err
}
}
return nil
}
func EnsureDirectoryExists(dir string) error {
info, err := os.Stat(dir)
if err == nil {
if !info.IsDir() {
return fmt.Errorf("%s is not a directory", dir)
}
} else {
err = os.MkdirAll(dir, 0755)
if err != nil {
return err
}
}
return nil
}

View File

@@ -1,4 +1,4 @@
package cloudinit
package system
import (
"io/ioutil"
@@ -9,22 +9,23 @@ import (
)
func TestWriteFileUnencodedContent(t *testing.T) {
wf := WriteFile{
Path: "/tmp/foo",
Content: "bar",
Permissions: "0644",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := ProcessWriteFile(dir, &wf); err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
fullPath := path.Join(dir, "tmp", "foo")
wf := File{
Path: fullPath,
Content: "bar",
RawFilePermissions: "0644",
}
fullPath := path.Join(dir, "tmp", "foo")
if err := WriteFile(&wf); err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
}
fi, err := os.Stat(fullPath)
if err != nil {
@@ -46,36 +47,65 @@ func TestWriteFileUnencodedContent(t *testing.T) {
}
func TestWriteFileInvalidPermission(t *testing.T) {
wf := WriteFile{
Path: "/tmp/foo",
Content: "bar",
Permissions: "pants",
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := ProcessWriteFile(dir, &wf); err == nil {
wf := File{
Path: path.Join(dir, "tmp", "foo"),
Content: "bar",
RawFilePermissions: "pants",
}
if err := WriteFile(&wf); err == nil {
t.Fatalf("Expected error to be raised when writing file with invalid permission")
}
}
func TestWriteFileEncodedContent(t *testing.T) {
wf := WriteFile{
Path: "/tmp/foo",
Content: "",
Encoding: "base64",
}
func TestWriteFilePermissions(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
if err := ProcessWriteFile(dir, &wf); err == nil {
fullPath := path.Join(dir, "tmp", "foo")
wf := File{
Path: fullPath,
RawFilePermissions: "0755",
}
if err := WriteFile(&wf); err != nil {
t.Fatalf("Processing of WriteFile failed: %v", err)
}
fi, err := os.Stat(fullPath)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0755) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
}
func TestWriteFileEncodedContent(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer syscall.Rmdir(dir)
wf := File{
Path: path.Join(dir, "tmp", "foo"),
Content: "",
Encoding: "base64",
}
if err := WriteFile(&wf); err == nil {
t.Fatalf("Expected error to be raised when writing file with encoding")
}
}

View File

@@ -1,4 +1,4 @@
package cloudinit
package system
import (
"fmt"

196
system/systemd.go Normal file
View File

@@ -0,0 +1,196 @@
package system
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"github.com/coreos/coreos-cloudinit/third_party/github.com/coreos/go-systemd/dbus"
)
// fakeMachineID is placed on non-usr CoreOS images and should
// never be used as a true MachineID
const fakeMachineID = "42000000000000000000000000000042"
// Name for drop-in service configuration files created by cloudconfig
const cloudConfigDropIn = "20-cloudinit.conf"
type Unit struct {
Name string
Mask bool
Enable bool
Runtime bool
Content string
Command string
// For drop-in units, a cloudinit.conf is generated.
// This is currently unbound in YAML (and hence unsettable in cloud-config files)
// until the correct behaviour for multiple drop-in units is determined.
DropIn bool `yaml:"-"`
}
func (u *Unit) Type() string {
ext := filepath.Ext(u.Name)
return strings.TrimLeft(ext, ".")
}
func (u *Unit) Group() (group string) {
t := u.Type()
if t == "network" || t == "netdev" || t == "link" {
group = "network"
} else {
group = "system"
}
return
}
type Script []byte
// UnitDestination builds the appropriate absolute file path for
// the given Unit. The root argument indicates the effective base
// directory of the system (similar to a chroot).
func UnitDestination(u *Unit, root string) string {
dir := "etc"
if u.Runtime {
dir = "run"
}
if u.DropIn {
return path.Join(root, dir, "systemd", u.Group(), fmt.Sprintf("%s.d", u.Name), cloudConfigDropIn)
} else {
return path.Join(root, dir, "systemd", u.Group(), u.Name)
}
}
// PlaceUnit writes a unit file at the provided destination, creating
// parent directories as necessary.
func PlaceUnit(u *Unit, dst string) error {
dir := filepath.Dir(dst)
if _, err := os.Stat(dir); os.IsNotExist(err) {
if err := os.MkdirAll(dir, os.FileMode(0755)); err != nil {
return err
}
}
file := File{
Path: dst,
Content: u.Content,
RawFilePermissions: "0644",
}
err := WriteFile(&file)
if err != nil {
return err
}
return nil
}
func EnableUnitFile(unit string, runtime bool) error {
conn, err := dbus.New()
if err != nil {
return err
}
units := []string{unit}
_, _, err = conn.EnableUnitFiles(units, runtime, true)
return err
}
func RunUnitCommand(command, unit string) (string, error) {
conn, err := dbus.New()
if err != nil {
return "", err
}
var fn func(string, string) (string, error)
switch command {
case "start":
fn = conn.StartUnit
case "stop":
fn = conn.StopUnit
case "restart":
fn = conn.RestartUnit
case "reload":
fn = conn.ReloadUnit
case "try-restart":
fn = conn.TryRestartUnit
case "reload-or-restart":
fn = conn.ReloadOrRestartUnit
case "reload-or-try-restart":
fn = conn.ReloadOrTryRestartUnit
default:
return "", fmt.Errorf("Unsupported systemd command %q", command)
}
return fn(unit, "replace")
}
func DaemonReload() error {
conn, err := dbus.New()
if err != nil {
return err
}
return conn.Reload()
}
func ExecuteScript(scriptPath string) (string, error) {
props := []dbus.Property{
dbus.PropDescription("Unit generated and executed by coreos-cloudinit on behalf of user"),
dbus.PropExecStart([]string{"/bin/bash", scriptPath}, false),
}
base := path.Base(scriptPath)
name := fmt.Sprintf("coreos-cloudinit-%s.service", base)
log.Printf("Creating transient systemd unit '%s'", name)
conn, err := dbus.New()
if err != nil {
return "", err
}
_, err = conn.StartTransientUnit(name, "replace", props...)
return name, err
}
func SetHostname(hostname string) error {
return exec.Command("hostnamectl", "set-hostname", hostname).Run()
}
func Hostname() (string, error) {
return os.Hostname()
}
func MachineID(root string) string {
contents, _ := ioutil.ReadFile(path.Join(root, "etc", "machine-id"))
id := strings.TrimSpace(string(contents))
if id == fakeMachineID {
id = ""
}
return id
}
// MaskUnit masks a Unit by the given name by symlinking its unit file (in
// /etc/systemd/system) to /dev/null, analogous to `systemctl mask`
// N.B.: Unlike `systemctl mask`, this function will *remove any existing unit
// file* in /etc/systemd/system, to ensure that the mask will succeed.
func MaskUnit(unit string, root string) error {
masked := path.Join(root, "etc", "systemd", "system", unit)
if _, err := os.Stat(masked); os.IsNotExist(err) {
if err := os.MkdirAll(path.Dir(masked), os.FileMode(0755)); err != nil {
return err
}
} else if err := os.Remove(masked); err != nil {
return err
}
return os.Symlink("/dev/null", masked)
}

186
system/systemd_test.go Normal file
View File

@@ -0,0 +1,186 @@
package system
import (
"io/ioutil"
"os"
"path"
"testing"
)
func TestPlaceNetworkUnit(t *testing.T) {
u := Unit{
Name: "50-eth0.network",
Runtime: true,
Content: `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`,
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
dst := UnitDestination(&u, dir)
expectDst := path.Join(dir, "run", "systemd", "network", "50-eth0.network")
if dst != expectDst {
t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
}
if err := PlaceUnit(&u, dst); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
fi, err := os.Stat(dst)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(dst)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expectContents := `[Match]
Name=eth47
[Network]
Address=10.209.171.177/19
`
if string(contents) != expectContents {
t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expectContents)
}
}
func TestUnitDestination(t *testing.T) {
dir := "/some/dir"
name := "foobar.service"
u := Unit{
Name: name,
DropIn: false,
}
dst := UnitDestination(&u, dir)
expectDst := path.Join(dir, "etc", "systemd", "system", "foobar.service")
if dst != expectDst {
t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
}
u.DropIn = true
dst = UnitDestination(&u, dir)
expectDst = path.Join(dir, "etc", "systemd", "system", "foobar.service.d", cloudConfigDropIn)
if dst != expectDst {
t.Errorf("UnitDestination returned %s, expected %s", dst, expectDst)
}
}
func TestPlaceMountUnit(t *testing.T) {
u := Unit{
Name: "media-state.mount",
Runtime: false,
Content: `[Mount]
What=/dev/sdb1
Where=/media/state
`,
}
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
dst := UnitDestination(&u, dir)
expectDst := path.Join(dir, "etc", "systemd", "system", "media-state.mount")
if dst != expectDst {
t.Fatalf("UnitDestination returned %s, expected %s", dst, expectDst)
}
if err := PlaceUnit(&u, dst); err != nil {
t.Fatalf("PlaceUnit failed: %v", err)
}
fi, err := os.Stat(dst)
if err != nil {
t.Fatalf("Unable to stat file: %v", err)
}
if fi.Mode() != os.FileMode(0644) {
t.Errorf("File has incorrect mode: %v", fi.Mode())
}
contents, err := ioutil.ReadFile(dst)
if err != nil {
t.Fatalf("Unable to read expected file: %v", err)
}
expectContents := `[Mount]
What=/dev/sdb1
Where=/media/state
`
if string(contents) != expectContents {
t.Fatalf("File has incorrect contents '%s'.\nExpected '%s'", string(contents), expectContents)
}
}
func TestMachineID(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
os.Mkdir(path.Join(dir, "etc"), os.FileMode(0755))
ioutil.WriteFile(path.Join(dir, "etc", "machine-id"), []byte("node007\n"), os.FileMode(0444))
if MachineID(dir) != "node007" {
t.Fatalf("File has incorrect contents")
}
}
func TestMaskUnit(t *testing.T) {
dir, err := ioutil.TempDir(os.TempDir(), "coreos-cloudinit-")
if err != nil {
t.Fatalf("Unable to create tempdir: %v", err)
}
defer os.RemoveAll(dir)
// Ensure mask works with units that do not currently exist
if err := MaskUnit("foo.service", dir); err != nil {
t.Fatalf("Unable to mask new unit: %v", err)
}
fooPath := path.Join(dir, "etc", "systemd", "system", "foo.service")
fooTgt, err := os.Readlink(fooPath)
if err != nil {
t.Fatalf("Unable to read link", err)
}
if fooTgt != "/dev/null" {
t.Fatalf("unit not masked, got unit target", fooTgt)
}
// Ensure mask works with unit files that already exist
barPath := path.Join(dir, "etc", "systemd", "system", "bar.service")
if _, err := os.Create(barPath); err != nil {
t.Fatalf("Error creating new unit file: %v", err)
}
if err := MaskUnit("bar.service", dir); err != nil {
t.Fatalf("Unable to mask existing unit: %v", err)
}
barTgt, err := os.Readlink(barPath)
if err != nil {
t.Fatalf("Unable to read link", err)
}
if barTgt != "/dev/null" {
t.Fatalf("unit not masked, got unit target", barTgt)
}
}

View File

@@ -1,4 +1,4 @@
package cloudinit
package system
import (
"fmt"
@@ -9,17 +9,19 @@ import (
)
type User struct {
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
Name string `yaml:"name"`
PasswordHash string `yaml:"passwd"`
SSHAuthorizedKeys []string `yaml:"ssh-authorized-keys"`
SSHImportGithubUser string `yaml:"coreos-ssh-import-github"`
SSHImportURL string `yaml:"coreos-ssh-import-url"`
GECOS string `yaml:"gecos"`
Homedir string `yaml:"homedir"`
NoCreateHome bool `yaml:"no-create-home"`
PrimaryGroup string `yaml:"primary-group"`
Groups []string `yaml:"groups"`
NoUserGroup bool `yaml:"no-user-group"`
System bool `yaml:"system"`
NoLogInit bool `yaml:"no-log-init"`
}
func UserExists(u *User) bool {
@@ -32,6 +34,8 @@ func CreateUser(u *User) error {
if u.PasswordHash != "" {
args = append(args, "--password", u.PasswordHash)
} else {
args = append(args, "--password", "*")
}
if u.GECOS != "" {
@@ -49,7 +53,7 @@ func CreateUser(u *User) error {
}
if u.PrimaryGroup != "" {
args = append(args, "--primary-group", u.PrimaryGroup)
args = append(args, "--gid", u.PrimaryGroup)
}
if len(u.Groups) > 0 {

37
test
View File

@@ -1,8 +1,37 @@
#!/bin/bash -e
#
# Run all coreos-cloudinit tests
# ./test
# ./test -v
#
# Run tests for one package
# PKG=initialize ./test
#
echo "Building bin/coreos-cloudinit"
. build
# Invoke ./cover for HTML output
COVER=${COVER:-"-cover"}
source ./build
declare -a TESTPKGS=(initialize system datasource pkg)
if [ -z "$PKG" ]; then
GOFMTPATH="$TESTPKGS coreos-cloudinit.go"
# prepend repo path to each package
TESTPKGS=${TESTPKGS[@]/#/${REPO_PATH}/}
else
GOFMTPATH="$TESTPKGS"
# strip out slashes and dots from PKG=./foo/
TESTPKGS=${PKG//\//}
TESTPKGS=${TESTPKGS//./}
TESTPKGS=${TESTPKGS/#/${REPO_PATH}/}
fi
echo "Running tests..."
go test -i github.com/coreos/coreos-cloudinit/cloudinit
go test -v github.com/coreos/coreos-cloudinit/cloudinit
go test -i ${TESTPKGS}
go test ${COVER} $@ ${TESTPKGS}
echo "Checking gofmt..."
fmtRes=$(gofmt -l $GOFMTPATH)
echo "Success"

View File

@@ -18,6 +18,8 @@ limitations under the License.
package dbus
import (
"os"
"strconv"
"strings"
"sync"
@@ -73,7 +75,12 @@ func (c *Conn) initConnection() error {
return err
}
err = c.sysconn.Auth(nil)
// Only use EXTERNAL method, and hardcode the uid (not username)
// to avoid a username lookup (which requires a dynamically linked
// libc)
methods := []dbus.Auth{dbus.AuthExternal(strconv.Itoa(os.Getuid()))}
err = c.sysconn.Auth(methods)
if err != nil {
c.sysconn.Close()
return err

View File

@@ -35,6 +35,7 @@ func (c *Conn) jobComplete(signal *dbus.Signal) {
out, ok := c.jobListener.jobs[job]
if ok {
out <- result
delete(c.jobListener.jobs, job)
}
c.jobListener.Unlock()
}
@@ -137,8 +138,8 @@ func (c *Conn) KillUnit(name string, signal int32) {
c.sysobj.Call("org.freedesktop.systemd1.Manager.KillUnit", 0, name, "all", signal).Store()
}
// GetUnitProperties takes the unit name and returns all of its dbus object properties.
func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
// getProperties takes the unit name and returns all of its dbus object properties, for the given dbus interface
func (c *Conn) getProperties(unit string, dbusInterface string) (map[string]interface{}, error) {
var err error
var props map[string]dbus.Variant
@@ -148,7 +149,7 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
}
obj := c.sysconn.Object("org.freedesktop.systemd1", path)
err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, "org.freedesktop.systemd1.Unit").Store(&props)
err = obj.Call("org.freedesktop.DBus.Properties.GetAll", 0, dbusInterface).Store(&props)
if err != nil {
return nil, err
}
@@ -161,6 +162,55 @@ func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
return out, nil
}
// GetUnitProperties takes the unit name and returns all of its dbus object properties.
func (c *Conn) GetUnitProperties(unit string) (map[string]interface{}, error) {
return c.getProperties(unit, "org.freedesktop.systemd1.Unit")
}
func (c *Conn) getProperty(unit string, dbusInterface string, propertyName string) (*Property, error) {
var err error
var prop dbus.Variant
path := ObjectPath("/org/freedesktop/systemd1/unit/" + unit)
if !path.IsValid() {
return nil, errors.New("invalid unit name: " + unit)
}
obj := c.sysconn.Object("org.freedesktop.systemd1", path)
err = obj.Call("org.freedesktop.DBus.Properties.Get", 0, dbusInterface, propertyName).Store(&prop)
if err != nil {
return nil, err
}
return &Property{Name: propertyName, Value: prop}, nil
}
func (c *Conn) GetUnitProperty(unit string, propertyName string) (*Property, error) {
return c.getProperty(unit, "org.freedesktop.systemd1.Unit", propertyName)
}
// GetUnitTypeProperties returns the extra properties for a unit, specific to the unit type.
// Valid values for unitType: Service, Socket, Target, Device, Mount, Automount, Snapshot, Timer, Swap, Path, Slice, Scope
// return "dbus.Error: Unknown interface" if the unitType is not the correct type of the unit
func (c *Conn) GetUnitTypeProperties(unit string, unitType string) (map[string]interface{}, error) {
return c.getProperties(unit, "org.freedesktop.systemd1."+unitType)
}
// SetUnitProperties() may be used to modify certain unit properties at runtime.
// Not all properties may be changed at runtime, but many resource management
// settings (primarily those in systemd.cgroup(5)) may. The changes are applied
// instantly, and stored on disk for future boots, unless runtime is true, in which
// case the settings only apply until the next reboot. name is the name of the unit
// to modify. properties are the settings to set, encoded as an array of property
// name and value pairs.
func (c *Conn) SetUnitProperties(name string, runtime bool, properties ...Property) error {
return c.sysobj.Call("SetUnitProperties", 0, name, runtime, properties).Store()
}
func (c *Conn) GetUnitTypeProperty(unit string, unitType string, propertyName string) (*Property, error) {
return c.getProperty(unit, "org.freedesktop.systemd1." + unitType, propertyName)
}
// ListUnits returns an array with all currently loaded units. Note that
// units may be known by multiple names at the same time, and hence there might
// be more unit names loaded than actual units behind them.
@@ -253,8 +303,52 @@ type EnableUnitFileChange struct {
Destination string // Destination of the symlink
}
// DisableUnitFiles() may be used to disable one or more units in the system (by
// removing symlinks to them from /etc or /run).
//
// It takes a list of unit files to disable (either just file names or full
// absolute paths if the unit files are residing outside the usual unit
// search paths), and one boolean: whether the unit was enabled for runtime
// only (true, /run), or persistently (false, /etc).
//
// This call returns an array with the changes made. The changes list
// consists of structures with three strings: the type of the change (one of
// symlink or unlink), the file name of the symlink and the destination of the
// symlink.
func (c *Conn) DisableUnitFiles(files []string, runtime bool) ([]DisableUnitFileChange, error) {
result := make([][]interface{}, 0)
err := c.sysobj.Call("DisableUnitFiles", 0, files, runtime).Store(&result)
if err != nil {
return nil, err
}
resultInterface := make([]interface{}, len(result))
for i := range result {
resultInterface[i] = result[i]
}
changes := make([]DisableUnitFileChange, len(result))
changesInterface := make([]interface{}, len(changes))
for i := range changes {
changesInterface[i] = &changes[i]
}
err = dbus.Store(resultInterface, changesInterface...)
if err != nil {
return nil, err
}
return changes, nil
}
type DisableUnitFileChange struct {
Type string // Type of the change (one of symlink or unlink)
Filename string // File name of the symlink
Destination string // Destination of the symlink
}
// Reload instructs systemd to scan for and reload unit files. This is
// equivalent to a 'systemctl daemon-reload'.
func (c *Conn) Reload() (string, error) {
return c.runJob("org.freedesktop.systemd1.Manager.Reload")
func (c *Conn) Reload() error {
return c.sysobj.Call("org.freedesktop.systemd1.Manager.Reload", 0).Store()
}

View File

@@ -18,9 +18,11 @@ package dbus
import (
"fmt"
"github.com/coreos/coreos-cloudinit/third_party/github.com/guelfey/go.dbus"
"math/rand"
"os"
"path/filepath"
"reflect"
"testing"
)
@@ -50,13 +52,16 @@ func setupUnit(target string, conn *Conn, t *testing.T) {
fixture := []string{abs}
install, changes, err := conn.EnableUnitFiles(fixture, true, true)
if err != nil {
t.Fatal(err)
}
if install != false {
t.Fatal("Install was true")
}
if len(changes) < 1 {
t.Fatal("Expected one change, got %v", changes)
t.Fatalf("Expected one change, got %v", changes)
}
if changes[0].Filename != targetRun {
@@ -118,6 +123,37 @@ func TestStartStopUnit(t *testing.T) {
}
}
// Enables a unit and then immediately tears it down
func TestEnableDisableUnit(t *testing.T) {
target := "enable-disable.service"
conn := setupConn(t)
setupUnit(target, conn, t)
abs, err := filepath.Abs("../fixtures/" + target)
if err != nil {
t.Fatal(err)
}
path := filepath.Join("/run/systemd/system/", target)
// 2. Disable the unit
changes, err := conn.DisableUnitFiles([]string{abs}, true)
if err != nil {
t.Fatal(err)
}
if len(changes) != 1 {
t.Fatalf("Changes should include the path, %v", changes)
}
if changes[0].Filename != path {
t.Fatalf("Change should include correct filename, %+v", changes[0])
}
if changes[0].Destination != "" {
t.Fatalf("Change destination should be empty, %+v", changes[0])
}
}
// TestGetUnitProperties reads the `-.mount` which should exist on all systemd
// systems and ensures that one of its properties is valid.
func TestGetUnitProperties(t *testing.T) {
@@ -139,6 +175,20 @@ func TestGetUnitProperties(t *testing.T) {
if names[0] != "system.slice" {
t.Fatal("unexpected wants for /")
}
prop, err := conn.GetUnitProperty(unit, "Wants")
if err != nil {
t.Fatal(err)
}
if prop.Name != "Wants" {
t.Fatal("unexpected property name")
}
val := prop.Value.Value().([]string)
if !reflect.DeepEqual(val, names) {
t.Fatal("unexpected property value")
}
}
// TestGetUnitPropertiesRejectsInvalidName attempts to get the properties for a
@@ -150,10 +200,37 @@ func TestGetUnitPropertiesRejectsInvalidName(t *testing.T) {
unit := "//invalid#$^/"
_, err := conn.GetUnitProperties(unit)
if err == nil {
t.Fatal("Expected an error, got nil")
}
_, err = conn.GetUnitProperty(unit, "Wants")
if err == nil {
t.Fatal("Expected an error, got nil")
}
}
// TestSetUnitProperties changes a cgroup setting on the `tmp.mount`
// which should exist on all systemd systems and ensures that the
// property was set.
func TestSetUnitProperties(t *testing.T) {
conn := setupConn(t)
unit := "tmp.mount"
if err := conn.SetUnitProperties(unit, true, Property{"CPUShares", dbus.MakeVariant(uint64(1023))}); err != nil {
t.Fatal(err)
}
info, err := conn.GetUnitTypeProperties(unit, "Mount")
if err != nil {
t.Fatal(err)
}
value := info["CPUShares"].(uint64)
if value != 1023 {
t.Fatal("CPUShares of unit is not 1023, %s", value)
}
}
// Ensure that basic transient unit starting and stopping works.
@@ -211,3 +288,27 @@ func TestStartStopTransientUnit(t *testing.T) {
t.Fatalf("Test unit found in list, should be stopped")
}
}
func TestConnJobListener(t *testing.T) {
target := "start-stop.service"
conn := setupConn(t)
setupUnit(target, conn, t)
jobSize := len(conn.jobListener.jobs)
_, err := conn.StartUnit(target, "replace")
if err != nil {
t.Fatal(err)
}
_, err = conn.StopUnit(target, "replace")
if err != nil {
t.Fatal(err)
}
currentJobSize := len(conn.jobListener.jobs)
if jobSize != currentJobSize {
t.Fatal("JobListener jobs leaked")
}
}

View File

@@ -209,3 +209,12 @@ func PropPropagatesReloadTo(units ...string) Property {
func PropRequiresMountsFor(units ...string) Property {
return propDependency("RequiresMountsFor", units)
}
// PropSlice sets the Slice unit property. See
// http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#Slice=
func PropSlice(slice string) Property {
return Property{
Name: "Slice",
Value: dbus.MakeVariant(slice),
}
}

View File

@@ -0,0 +1,11 @@
# Automatically trigger configdrive mounting.
ACTION!="add|change", GOTO="coreos_configdrive_end"
# A normal config drive. Block device formatted with iso9660 or fat
SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="iso9660|vfat", ENV{ID_FS_LABEL}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-block.service"
# Addtionally support virtfs from QEMU
SUBSYSTEM=="virtio", DRIVER=="9pnet_virtio", ATTR{mount_tag}=="config-2", TAG+="systemd", ENV{SYSTEMD_WANTS}+="configdrive-virtfs.service"
LABEL="coreos_configdrive_end"

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Mount config drive
Conflicts=configdrive-virtfs.service umount.target
ConditionPathIsMountPoint=!/media/configdrive
# Only mount config drive block devices automatically in virtual machines
ConditionVirtualization=vm
[Service]
Type=oneshot
RemainAfterExit=no
ExecStart=/bin/mount -t auto -o ro,x-mount.mkdir LABEL=config-2 /media/configdrive

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Mount config drive from virtfs
Conflicts=configdrive-block.service umount.target
ConditionPathIsMountPoint=!/media/configdrive
ConditionVirtualization=vm
# Support old style setup for now
Wants=addon-run@media-configdrive.service addon-config@media-configdrive.service
Before=addon-run@media-configdrive.service addon-config@media-configdrive.service
[Service]
Type=oneshot
RemainAfterExit=no
ExecStart=/bin/mount -t 9p -o trans=virtio,version=9p2000.L,x-mount.mkdir config-2 /media/configdrive

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Load cloud-config from %f
Requires=dbus.service
After=dbus.service
Before=system-config.target
ConditionFileNotEmpty=%f
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

View File

@@ -0,0 +1,10 @@
[Unit]
Description=Load system-provided cloud configs
# Generate /etc/environment
Requires=coreos-setup-environment.service
After=coreos-setup-environment.service
# Load OEM cloud-config.yml
Requires=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service
After=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Load cloud-config from url defined in /proc/cmdline
Requires=coreos-setup-environment.service
After=coreos-setup-environment.service
Before=user-config.target
ConditionKernelCommandLine=cloud-config-url
[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=-/etc/environment
ExecStart=/usr/bin/coreos-cloudinit --from-proc-cmdline

View File

@@ -0,0 +1,5 @@
[Unit]
Description=Watch for a cloud-config at %f
[Path]
PathExists=%f

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Load cloud-config from %f
Requires=coreos-setup-environment.service
After=coreos-setup-environment.service
Before=user-config.target
ConditionFileNotEmpty=%f
[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=-/etc/environment
ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

View File

@@ -0,0 +1,24 @@
[Unit]
Description=Load cloud-config from %f
Requires=coreos-setup-environment.service
After=coreos-setup-environment.service
Before=user-config.target
ConditionFileNotEmpty=%f
# HACK: work around ordering between config drive and ec2 metadata It is
# possible for OpenStack style systems to provide both the metadata service
# and config drive, to prevent the two from stomping on eachother force
# this to run after OEM and after metadata (if it exsts). I'm doing this
# here instead of in the ec2 service because the ec2 unit is not written
# to disk until the OEM cloud config is evaluated and I want to make sure
# systemd knows about the ordering as early as possible.
# coreos-cloudinit could implement a simple lock but that cannot be used
# until after the systemd dbus calls are made non-blocking.
After=system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service
After=ec2-cloudinit.service
[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=-/etc/environment
ExecStart=/usr/bin/coreos-cloudinit --from-file=%f

13
units/user-config.target Normal file
View File

@@ -0,0 +1,13 @@
[Unit]
Description=Load user-provided cloud configs
Requires=system-config.target
After=system-config.target
# Watch for configs at a couple common paths
Requires=user-cloudinit@media-configdrive-openstack-latest-user_data.path
After=user-cloudinit@media-configdrive-openstack-latest-user_data.path
Requires=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
After=user-cloudinit@var-lib-coreos\x2dinstall-user_data.path
Requires=user-cloudinit-proc-cmdline.service
After=user-cloudinit-proc-cmdline.service