This replaces the old clustercfg script with a brand spanking new mostly-equivalent Go reimplementation. But it's not exactly the same, here are the differences: 1. No cluster deployment logic anymore - we expect everyone to use ops/ machine at this point. 2. All certs/keys are Ed25519 and do not expire by default - but support for short-lived certificates is there, and is actually more generic and reusable. Currently it's only used for admincreds. 3. Speaking of admincreds: the new admincreds automatically figure out your username. 4. admincreds also doesn't shell out to kubectl anymore, and doesn't override your default context. The generated creds can live peacefully alongside your normal prodaccess creds. 5. gencerts (the new nodestrap without deployment support) now automatically generates certs for all nodes, based on local Nix modules in ops/. 6. No secretstore support. This will be changed once we rebuild secretstore in Go. For now users are expected to manually run secretstore sync on cluster/secrets. Change-Id: Ida935f44e04fd933df125905eee10121ac078495 Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1498 Reviewed-by: q3k <q3k@hackerspace.pl> |
||
---|---|---|
.. | ||
ceph | ||
monitoring | ||
sso/kube | ||
README.md | ||
exports.nix | ||
machines.nix | ||
provision.nix |
README.md
Operations
Deploying NixOS machines
Machine configurations are in ops/machines.nix
.
Wrapper script to show all available machines and provision a single machine:
$ $(nix-build -A ops.provision)
Available machines:
- bc01n01.hswaw.net
- bc01n02.hswaw.net
- dcr01s22.hswaw.net
- dcr01s24.hswaw.net
- edge01.waw.bgp.wtf
$ $(nix-build -A ops.provision) edge01.waw.bgp.wtf
This can be slow, as it evaluates/builds all machines' configs. If you just want to deploy one machine and possible iterate faster:
$ $(nix-build -A 'ops.machines."edge01.waw.bgp.wtf".config.passthru.hscloud.provision')
Remote Builders (cross-compiling)
If you're attempting to deploy a machine which has a system architecture other than your host machine (eg. are deploying an Aarch64 Raspberry Pi4 from an Intel machine), you'll need to use a remote builder which has that target architecture.
Any machine of that target architecture running Nix(OS) will do, even the machine you're deploing. But we also have some dedicated build machines:
Name | Architecture | CPUs | RAM |
---|---|---|---|
larrythebuilder.q3k.org | AArch64 | 4 | 24GiB |
To use a machine $name
as a remote builder:
-
Make sure you have access to the machine.
ssh $username@$name
should work. If not, file a CR to get your key added to the machine and ask someone to review and deploy it. The machines' key confiurations are in hscloud. -
Check
nix store ping --store ssh-ng://$username@$name
. It should work. -
On NixOS, configure builders in your system configuration.nix and rebuild, eg.:
nix.buildMachines = [
{
system = "aarch64-linux";
sshUser = "root";
sshKey = "/home/q3k/.ssh/id_ed25519";
maxJobs = 4;
hostName = "larrythebuilder.q3k.org";
}
];
nix.distributedBuilds = true;
- On non-NixOS, configure builders in your nix.conf, eg.
builders = ssh://$username@$name aarch64-linux
in your system/user nix.conf. Your nix-daemon should also specify that the local user is trusted.
We should automate this some day.