Public pull ACL in the middle had priority over our more specific rules
- moving these to the top fixes common registry namespace ACLs.
Change-Id: Ia6f05cef09c0db4eb71155d2c0e2d9944b81f903
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1522
Reviewed-by: q3k <q3k@hackerspace.pl>
This replaces the old clustercfg script with a brand spanking new
mostly-equivalent Go reimplementation. But it's not exactly the same,
here are the differences:
1. No cluster deployment logic anymore - we expect everyone to use ops/
machine at this point.
2. All certs/keys are Ed25519 and do not expire by default - but
support for short-lived certificates is there, and is actually more
generic and reusable. Currently it's only used for admincreds.
3. Speaking of admincreds: the new admincreds automatically figure out
your username.
4. admincreds also doesn't shell out to kubectl anymore, and doesn't
override your default context. The generated creds can live
peacefully alongside your normal prodaccess creds.
5. gencerts (the new nodestrap without deployment support) now
automatically generates certs for all nodes, based on local Nix
modules in ops/.
6. No secretstore support. This will be changed once we rebuild
secretstore in Go. For now users are expected to manually run
secretstore sync on cluster/secrets.
Change-Id: Ida935f44e04fd933df125905eee10121ac078495
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1498
Reviewed-by: q3k <q3k@hackerspace.pl>
This completes the migration away from the old CA/cert infrastructure.
The tool which was used to generate all these certs will come next. It's
effectively a reimplementation of clustercfg in Go.
We also removed the unused kube-serviceaccounts cert, which was
generated by the old tooling for no good reason (we only need a key for
service accounts, not an actual cert...).
Change-Id: Ied9e5d8fc90c64a6b4b9fdd20c33981410c884b4
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1501
Reviewed-by: q3k <q3k@hackerspace.pl>
This finishes the regeneration of all cluster CAs/certs to be never
expiring ED25519 certs.
We still have leftovers of the old Kube CA (and it's still being
accepted in Kubernetes components). Cleaning that up is the next step.
Change-Id: I883f94fd8cef3e3b5feefdf56ee106e462bb04a9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1500
Reviewed-by: q3k <q3k@hackerspace.pl>
This is already deployed, and it allows Kubernetes components
(temporary) freedom to use the old or new CA cert.
Change-Id: I8ac7f773a333c30fa22902b8edc327c0c700a482
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1490
Reviewed-by: q3k <q3k@hackerspace.pl>
This gets rid of cfssl for the kubernetes bits of prodvider, instead
using plain crypto/x509. This also allows to support our new fancy
ED25519 CA.
Change-Id: If677b3f4523014f56ea802b87499d1c0eb6d92e9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1489
Reviewed-by: q3k <q3k@hackerspace.pl>
Done:
1. etcd peer CA & certs
2. etcd client CA & certs
3. kube CA (currently all components set to accept both new and old CA,
new CA called ca-kube-new)
4. kube apiserver
5. kubelet & kube-proxy
6. prodvider intermediate
TODO:
1. kubernetes controller-manager & kubernetes scheduler
2. kubefront CA
3. admitomatic?
4. undo bundle on kube CA components to fully transition away from old
CA
Change-Id: If529eeaed9a6a2063bed23c9d81c57b36b9a0115
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1487
Reviewed-by: q3k <q3k@hackerspace.pl>
This will happen at next boot via early microcode - no risk to currently
running processes.
Change-Id: I88553fa9a1350ebb80aaf978e29e8f1156783a2c
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1469
Reviewed-by: q3k <q3k@hackerspace.pl>
This will be our postgres pet machine.
Change-Id: Ifff6648394ca6407fb5b5daa853f4abc42541703
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1467
Reviewed-by: q3k <q3k@hackerspace.pl>
After installing HBJ11s and spreading out the mons we're going full
Rook.
Change-Id: Ia00cbe953548f06cf27343371fc67890619c8262
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1466
Reviewed-by: q3k <q3k@hackerspace.pl>
This bumps it on bc01n01, but nowhere else yet.
We have to vendor some more kubelet bits unfortunately.
Change-Id: Ifb169dd9c2c19d60f88d946d065d4446141601b1
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1465
Reviewed-by: implr <implr@hackerspace.pl>
the spark one has been an abandoned experiment from years ago, and
I could use a personal one right now
Change-Id: I78a706c3371d441b2f8460fd796d0cfd9a198cc6
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1464
Reviewed-by: q3k <q3k@hackerspace.pl>
This is needed for running some memory-intensive workloads, like
ElasticSearch/OpenSearch.
Change-Id: I7b00ec5faca73ec69bdbf1ca41c025d7efeae55c
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1443
Reviewed-by: implr <implr@hackerspace.pl>
This was never used and only caused scary warnings during OSDs reboots
due to lack of availability.
Change-Id: I14eacd88855bc56e06f2a61cc2d914d985330852
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1423
Reviewed-by: implr <implr@hackerspace.pl>
Leaving the CRD definitions as YAML, extracted without modifications
from the original install file - this should make upgrades simpler.
Change-Id: I7211d2711e2af014b36dd887a951abb9e1032eb9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1179
Reviewed-by: q3k <q3k@hackerspace.pl>
This unforks benji back into upstream. The old fork didn't support a new
authentication method on Ceph, and we don't have multiple clusters
anymore (so we don't need the functionality of the fork).
Change-Id: Ie79313b2321ca2e22ad2874b75a71385af95105f
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1321
Reviewed-by: informatic <informatic@hackerspace.pl>
This is a chonky refactor that get rids of the previous cluster-centric
defs-* plain nix file setup.
Now, nodes are configured individually in plain nixos modules, and are
provided a view of all other nodes in the 'machines' attribute. Cluster
logic is moved into modules which inspect this array to find other nodes
within the same cluster.
Kubernetes options are not fully clusterified yet (ie., they are still
hardcode to only provide the 'k0' cluster) but that can be fixed later.
The Ceph machinery is a good example of how that can be done.
The new NixOS configs are zero-diff against prod. While this is done
mostly by keeping the logic, we had to keep a few newly discovered
'bugs' around by adding some temporary options which keeps things as they
are. These will be removed in a future CL, then introducing a diff (but
no functional changes, hopefully).
We also remove the nix eval from clustercfg as it was not used anymore
(basically since we refactored certs at some point).
Change-Id: Id79772a96249b0e6344046f96f9c2cb481c4e1f4
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1322
Reviewed-by: informatic <informatic@hackerspace.pl>
Reminded by a power failure on bc01n0{1,2}, we migrate away from at
least one of them into another server.
We also fix up the startup join parameter to not include the node itself
(which is not necessary, but a nice thing to have nonetheless).
Since bc01n01 was the initial node of the cluster, we also disable the
init job for k0 (which we don't care about anyway).
Change-Id: I3406471c0f9542e9d802d39138e400b5a5e74794
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1176
Reviewed-by: q3k <q3k@hackerspace.pl>
This removes the need to source env.{sh,fish} when working with hscloud.
This is done by:
1. Implementing a Go library to reliably detect the location of the
active hscloud checkout. That in turn is enabled by
BUILD_WORKSPACE_DIRECTORY being now a thing in Bazel.
2. Creating a tool `hscloud`, with a command `hscloud workspace` that
returns the workspace path.
3. Wrapping this tool to be accessible from Python and Bash.
4. Bumping all users of hscloud_root to use either the Go library or
one of the two implemented wrappers.
We also drive-by replace tools/install.sh to be a proper sh_binary, and
make it yell at people if it isn't being ran as `bazel run
//tools:install`.
Finally, we also drive-by delete cluster/tools/nixops.sh which was never used.
Change-Id: I7873714319bfc38bbb930b05baa605c5aa36470a
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1169
Reviewed-by: informatic <informatic@hackerspace.pl>
With this we can use Ceph's multi-site support to easily migrate to our
new k0 Ceph cluster.
This migration was done by using radosgw-admin to rename the existing
realm/zonegroup to the new names (hscloud and eu), and then reworking
the jsonnet so that the Rook operator would effectively do nothing.
It sounds weird that creating a bunch of CRs like
Object{Realm,ZoneGroup,Zone} realm would be a no-op for the operator,
but that's how Rook works - a CephObjectStore generally creates
everything that the above CRs would create too, but implicitly. Adding
the extra CRs just allows specifying extra settings, like names.
(it wasn't fully a no-op, as the rgw daemon is parametrized by
realm/zonegroup/zone names, so that had to be restarted)
We also make the radosgw serve under object.ceph-eu.hswaw.net, which
allows us to right away start using a zonegroup URL instead of the
zone-only URL.
Change-Id: I4dca55a705edb3bd28e54f50982c85720a17b877
This enables radosgw wherever osds are. This should be fast and works
for us because we have little osd hosts.
Change-Id: I4ed014d2790d6c02a2ba8e775aaa1846032dee1e
informatic
q3k
implr
patryk