1
0
Fork 0
Commit Graph

1008 Commits (20c6bcb7305d4b85c5fd6dfc72c04c68b772d15f)

Author SHA1 Message Date
q3k 434393bc84 Merge "app/matrix: parametrize configuration" 2020-11-03 23:10:28 +00:00
q3k 062d199da9 Merge "matrix: refactor synapse into library" 2020-11-03 23:10:21 +00:00
q3k 52e796606f app/matrix: add matrix.0x3c.pl
Change-Id: I1f1d41fd80c08cefd274141ca598526dd466ad24
2020-11-04 00:09:03 +01:00
q3k 21a9616583 app/matrix: finish parametrization, fix running as non-root
This should allow us to run matrix on our default podsecuritypolicy.

Change-Id: I054f527ebab2f499e7a6595e618281a58c82e283
2020-11-04 00:09:03 +01:00
q3k ace32c0360 app/matrix: parametrize configuration
This adds cfg.cas.enable, and parametrizes homeserver.yaml from jsonnet
configuration.

Change-Id: I37a9b27a7080248cdd70282b897bbf6d3f9ae5f6
2020-11-03 22:04:08 +01:00
q3k 60076c70f8 matrix: refactor synapse into library
This is in prepration for bringing up a Matrix server for hsp.sh.

Verified to cause no diff on prod.

Change-Id: Ied2de210692e3ddfdb1d3f37b12893b214c34b0b
2020-11-03 21:36:49 +01:00
q3k 9dbc263d8e Merge "third_party/go: pay off some technical debt and hide the bodies" 2020-11-03 20:16:57 +00:00
q3k cdfbacb178 Merge "third_party/go/kubernetes: fix version reported in kubectl" 2020-11-03 20:16:26 +00:00
q3k 3836b0382a Merge "third_party/go: bump kubernetes to 1.19.3" 2020-11-03 20:16:18 +00:00
q3k 301435f3c3 third_party/go: pay off some technical debt and hide the bodies
This is an amalgamation of a handful of small changes to Go deps.
Notably:

 - we remove our opencensus-proto fork, use upstream, use exclude=src to
   fix the build
 - unvendorify some deps
 - bump io_rules_go to fix WKT resolution

Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.

Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
2020-11-03 21:15:41 +01:00
q3k e23717d520 third_party/go/kubernetes: fix version reported in kubectl
Change-Id: I5a45158004280c8ce6ba04c18af93e8dbbc300e4
2020-11-03 21:15:41 +01:00
q3k c3f36e9bf1 third_party/go: bump kubernetes to 1.19.3
Change-Id: Id9245765936997088e94135fde409ff4c1539bba
2020-11-03 21:15:41 +01:00
q3k 376f58720e go/statusz: do not crash on failure to resolve uid
This can happen if a binary is built statically and fails to load
libgcc/nsd libraries at runtime.

Change-Id: Ia76645471b83a6cc75fe6552e70e6a251b50129c
2020-10-30 14:05:07 +01:00
q3k 99ce53c79a third_party: remove uWSGI
It's not being used outside of personal/q3k for now, and it's really
fucking up the build system.

Change-Id: Ie8f3e59e40e8be8ef3ec32118a591da2274e398c
2020-10-29 01:43:37 +01:00
q3k b1de757249 laserproxy: add nix build
Change-Id: If93f4ba69afa028fed9098663a523f46d6134f7c
2020-10-29 00:43:43 +01:00
q3k bfe9bb0e3a k0: add woju's personal s3 user
Change-Id: I8ed5bb5428594b74460f1b89185d684cb6c26268
2020-10-27 20:50:50 +01:00
q3k 491542589b tools/gostatic: init
This adds Bazel/hscloud integration to gostatic, via gostatic_tarball.

A sample is provided in //tools/gostatic/example, it can be built using:

    bazel build //tools/gostatic/example

The resulting tarball can then be extracted and viewed in a web
browser.

Change-Id: Idf8d4a8e0ee3a5ae07f7449a25909478c2d8b105
2020-10-26 12:08:33 +01:00
q3k 94a1af8714 hackdoc: add table css, make it colorful
Change-Id: Idab1f911c10832ef4cfcf7073f77577d1b8673ff
2020-10-24 20:20:18 +02:00
q3k 79b506bcc2 third_party/go: unbreak build
This was missed in gerrit/486. Whoops, we should CI sooner than later.

Change-Id: Ic70b742c75d52dd615d4e2f946233783d156cead
2020-10-24 17:36:25 +02:00
q3k b4c3f342e4 third_party/go: add gostatic
To test:

    bazel run '@com_github_piranha_gostatic//:gostatic'

Change-Id: Ie846429df0d1f1914f2734735591edebd5d29094
2020-10-24 17:30:44 +02:00
q3k e401735fdd Merge "bgpwtf: add static v6 routes via bird" 2020-10-16 17:09:18 +00:00
q3k d9a6365f8b bgpwtf: add static v6 routes via bird
A customer was missing a static v6 route via their router. Since we
don't want to add them to networking.interfaces.routes.* (as this
restarts the whole scripted network stack in NixOS), we add them to
bird. This requires implementing hscloud.routing.static.

Change-Id: I0a205ed1e1f17a86de43aaf72ab6c2694a069112
2020-10-16 19:07:52 +02:00
q3k 78753aa275 Merge "k0: bump to 1.16.5" 2020-10-10 20:40:56 +00:00
q3k b014a95e0a Merge "k0: expose controller-manager and scheduler metrics" 2020-10-10 20:40:35 +00:00
q3k bfe2fe6455 Merge "clustercfg: show diff before switching to new configuration" 2020-10-10 20:40:31 +00:00
q3k e77f7717d4 k0: bump to 1.16.5
Change-Id: I548808ce4e0deb0513a1e00963f383d84b9d920c
2020-10-10 22:39:50 +02:00
informatic cf47f08481 app/covid-formity: enable redis password
This has already been deployed in production

Change-Id: I9c603a4985332d422d8875ecf6f8dca157f32f22
2020-10-10 18:40:45 +00:00
informatic 7e3447f3ff Merge "kube/redis: implement optional cfg.password option" 2020-10-10 18:40:37 +00:00
q3k d9e32f19f6 Merge "kube/upstream: bump to 1.14.4" 2020-10-10 18:24:48 +00:00
informatic 89a1ee90cd kube/redis: implement optional cfg.password option
If set, this enables internal redis authentication scheme. Supports
secretRefs, as well as values passed directly.

Change-Id: Ie902b8d79fdc4aa83ad8ad123e79f0bc80c1251f
2020-10-10 19:44:14 +02:00
informatic 018d219dc9 Merge changes Ie974e7e8,I0bda7f6e
* changes:
  app/covid-formity: add kurjerzy integration
  app/covid-formity: image update, add /qr1, /manual, /video redirect
2020-10-10 17:13:53 +00:00
q3k 1257389d3d k0: expose controller-manager and scheduler metrics
We want to be able to scrape controller-manager and scheduler metrics
into Prometheus. For that, each of them needs to:

 1) listen on a secure port
 2) have authn enabled

With this, any k8s user with the right permissions (and a bearer token
or TLS certificate) can come in and access metrics over a node's public
IP address. Access without a certificate/token gets thrown into the
system:anonymous user, which as no access to any API.

Change-Id: I267680f92f748ba63b6762e6aaba3c417446e50b
2020-10-10 16:00:15 +00:00
q3k 36224c617a clustercfg: show diff before switching to new configuration
This is mildly hacky, but lets us be more informed before we switch to a
new configuration.

Change-Id: I008f3f698db702f1e0992bd41a8d1050449d59b5
2020-10-10 16:00:11 +00:00
q3k a4a5a66f88 Merge "nix: provide a python2 toolchain" 2020-10-10 15:59:58 +00:00
q3k 7d311e9602 ops/monitoring: pull in grafonnet-7.0
Change-Id: Ie036ef767419418876a18255a5ad378f5cfa1535
2020-10-10 15:59:45 +00:00
q3k 3af7da1988 third_party/licenses: create, import Apache-2.0
Change-Id: I3f1a9ede192e70244c8d51bd58e9232a186a203f
2020-10-10 15:59:29 +00:00
q3k c824405e2e Update COPYING
Change-Id: I22661254b16840bcea7b352d51171a232fa7041a
2020-10-10 15:59:10 +00:00
q3k 531cacf14a Merge "WORKSPACE: use nix for python/go if available" 2020-10-07 12:56:38 +00:00
q3k eb09c6a347 speedtest: fix mimetype on served JS
Change-Id: Ifcb1d4f8a58a5e6120f31373b2a8c0e307e414be
2020-10-06 15:29:08 +00:00
q3k 363bf4f341 monitoring: global: implement
This creates a basic Global instance, running Victoria Metrics on k0.

Change-Id: Ib03003213d79b41cc54efe40cd2c4837f652c0f4
2020-10-06 14:28:27 +00:00
q3k 27885a9979 nix: provide a python2 toolchain
This allows us to use rules_docker from NixOS. However, the built
binaries are broken because of the Docker base image not being NixOS
based.

Change-Id: I29b93f1bae1575b04f97265c67497081d11a1910
2020-10-03 16:41:39 +00:00
q3k 2e001e5046 k0: bump to 1.15.4
This notably fixes the annoying loopback issues that prevented hosts
from accessing externalip services with externalTrafficPolicy: local
from nodes that weren't running the service.

Which means, hopefuly, no more registry pull failures when
nginx-ingress gets misplaced!

Change-Id: Id4923fd0fce2e28c31a1e65518b0e984165ca9ec
2020-10-03 16:32:38 +00:00
q3k 2a223705fd cluster: bump certs
This has been deployed to k0 nodes.

Current state of cluster certificates:

cluster/certs/ca-etcd.crt
            Not After : Apr  4 17:59:00 2024 GMT
cluster/certs/ca-etcdpeer.crt
            Not After : Apr  4 17:59:00 2024 GMT
cluster/certs/ca-kube.crt
            Not After : Apr  4 17:59:00 2024 GMT
cluster/certs/ca-kubefront.crt
            Not After : Apr  4 17:59:00 2024 GMT
cluster/certs/ca-kube-prodvider.cert
            Not After : Sep  1 21:30:00 2021 GMT
cluster/certs/etcd-bc01n01.hswaw.net.cert
            Not After : Mar 28 15:53:00 2021 GMT
cluster/certs/etcd-bc01n02.hswaw.net.cert
            Not After : Mar 28 16:45:00 2021 GMT
cluster/certs/etcd-bc01n03.hswaw.net.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/etcd-calico.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/etcd-dcr01s22.hswaw.net.cert
            Not After : Oct  3 15:33:00 2021 GMT
cluster/certs/etcd-dcr01s24.hswaw.net.cert
            Not After : Oct  3 15:38:00 2021 GMT
cluster/certs/etcd-kube.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/etcdpeer-bc01n01.hswaw.net.cert
            Not After : Mar 28 15:53:00 2021 GMT
cluster/certs/etcdpeer-bc01n02.hswaw.net.cert
            Not After : Mar 28 16:45:00 2021 GMT
cluster/certs/etcdpeer-bc01n03.hswaw.net.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/etcdpeer-dcr01s22.hswaw.net.cert
            Not After : Oct  3 15:33:00 2021 GMT
cluster/certs/etcdpeer-dcr01s24.hswaw.net.cert
            Not After : Oct  3 15:38:00 2021 GMT
cluster/certs/etcd-root.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kube-apiserver.cert
            Not After : Oct  3 15:26:00 2021 GMT
cluster/certs/kube-controllermanager.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kubefront-apiserver.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kube-kubelet-bc01n01.hswaw.net.cert
            Not After : Mar 28 15:53:00 2021 GMT
cluster/certs/kube-kubelet-bc01n02.hswaw.net.cert
            Not After : Mar 28 16:45:00 2021 GMT
cluster/certs/kube-kubelet-bc01n03.hswaw.net.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kube-kubelet-dcr01s22.hswaw.net.cert
            Not After : Oct  3 15:33:00 2021 GMT
cluster/certs/kube-kubelet-dcr01s24.hswaw.net.cert
            Not After : Oct  3 15:38:00 2021 GMT
cluster/certs/kube-proxy.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kube-scheduler.cert
            Not After : Mar 28 15:15:00 2021 GMT
cluster/certs/kube-serviceaccounts.cert
            Not After : Mar 28 15:15:00 2021 GMT

Change-Id: I94030ce78c10f7e9a0c0257d55145ef629195314
2020-10-03 16:32:32 +00:00
q3k 194b1c8e62 WORKSPACE: use nix for python/go if available
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.

There are two reasons behind this:

 - on NixOS, it's painful or at least very difficult to run hscloud out
   of the box. Especially with rules_go, that download a blob from the
   Internet to get a Go toolchain, it just fails outright. This solves
   this and allows hscloud to be used on NixOS.

 - on non-NixOS platforms that still might have access to Nix this
   allows to somewhat hermeticize the build. Notably, Python now comes
   from nixpkgs, and is fabricobbled in a way that makes pip3_import
   use Nix system dependencies for ncurses and libpq.

This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo
~amd64.

Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
2020-10-03 18:31:38 +02:00
q3k 6abe4fa771 bgpwtf/machines: init edge01.waw
This configures our WAW edge router using NixOS. This replaces our
previous Ubuntu installation.

Change-Id: Ibd72bde66ec413164401da407c5b268ad83fd3af
2020-10-03 14:57:38 +00:00
q3k 2efb698d22 *: add default.nix/readTree
This makes all Nix files addressable from root by file path.

For instance, if a file is located in //foo/bar:baz.nix containing:

    { pkgs, ... }:

    pkgs.stdenv.mkDerivation {
      pname = "foo";
      # ...
    }

You can then do:

    nix-build -A foo.bar.baz

All nix files loaded this way must be a function taking a 'config'
attrset - see nix/readTree.nix for more information. Currently the
config attrset contains the following fields:

 - hscloud: the root of the hscloud repository itself, which allows
            for traversal via readTree (eg. hscloud.foo.bar.baz)
 - pkgs: nixpkgs
 - pkgsSrc: nixpkgs souce/channel, useful to load NixOS modules.
 - lib, stdenv: lib and stdenv from pkgs.

Change-Id: Ieaacdcabceec18dd6c670d346928bff08b66cf79
2020-10-03 14:57:34 +00:00
q3k fbe234bdb2 cluster: rename module-* into modules/*
Change-Id: I65e06f3e9cec2ba0071259eb755eddbbd1025b97
2020-10-03 14:57:30 +00:00
q3k c7de7e562f cluster: do not export metallb routes to mesh peers
This prevents metallb routes being announced from all peers to our ToR,
thereby preventing issues with traffic hitting services with
externalTrafficPolicy: local.

There still is the from-host loopback issue, but that will be fixed by
upgrading to kube 1.15.

Change-Id: Ifc9964b46840aee82d99f0b6550188550e46fe04
2020-10-03 14:56:52 +00:00
q3k f0acf16564 prodvider: use SANs in service certificates
This fixes compatibility with prodaccess tools built with Go 1.15, which
introduced 'X.509 CommonName deprecation' [1].

[1] - https://golang.org/doc/go1.15#commonname

Change-Id: I228cde3e5651a3e36f527783f2ccb4a2f6b7a8e3
2020-10-03 14:56:35 +00:00
q3k 44628f2b9e Merge "k0.hswaw.net: pass metallb through Calico" 2020-10-02 22:54:57 +00:00