There's a lot of ugly hacks here, but this has been the state of prod
for months now, so we should reflect that.
Also, this bumps a bunch of workspace deps.
Change-Id: I744e0d3aff27036cfed73416cf442c7d62444a8b
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1473
Reviewed-by: q3k <q3k@hackerspace.pl>
Building jq portably is annoying, and the way we were doing it (which we
iirc stole from some google project?) sucked. Let's use a Go jq clone
instead.
This is an alternative for 1535. jq is currently used only in one
script, which could really be replaced by a Go program, but let's keep
it simple for now.
Change-Id: Ie25dffadd545df143490f510e9b75a74adf81492
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1540
Reviewed-by: palid <palid@hackerspace.pl>
Those are far from the latest versions still, but this change should be
non-breaking.
Change-Id: Ieeb9d6b301184f46677d821fe8276391346d6285
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1459
Reviewed-by: q3k <q3k@hackerspace.pl>
This commit aims to increase the speed of hscloud rebuild process
by optimizing the behavior of Go dependency fetching routines.
Gazelle v0.25.0 introduced a new dependency resolution mode
where it does not reach out for external dependencies; instead,
it operates solely on what is contained inside the workspace.
Because static dependency resolution mode is now the default
behavior in go_repository() rules, we are also updating the
contents of //third_party/go/repositories.bzl.
Also, I changed some of the bigger Go dependencies to be downloaded
by a tarball fetch in order to speed up the rebuild process.
Other changes:
* Bump nixpkgs to a fresh snapshot
* Upgrade to Bazel v5
Change-Id: Icfe752411b3128bcd5b25fa28bb76bec45ae2f71
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1441
Reviewed-by: q3k <q3k@hackerspace.pl>
You can test this using:
bazel run '@io_filippo_age//cmd/age'
The same target can now be used in data dependencies for secretstore
(you'll need to hardcode the runfile path, or use some
Bazel-runfile-resolving library for Python).
This required adding a few dependencies to
third_party/go/repositories.bzl, but also moving golang.org/x/crypto
from that file into WORKSPACE, before gazelle_deps gets loaded (as the
version requested by gazelle_deps is too old). We also moved shlex that
shouldn't have been in WORKSPACE into third_party/go/repositories.bzl.
Otherwise, this was just a few small deps - bumped golang.org/x/crypto,
new golang.org/x/term, new filippo.io/edwards25519. Hooray low
dependency code.
Change-Id: I0e684d88efffde13a3b4e253860aabcb35a3c94d
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1158
Reviewed-by: patryk <patryk@hackerspace.pl>
Looks like .ml DNS servers are currently down, and this repository
import path is deprecated anyway. Really, we should bump Kubernetes...
Change-Id: I3e0c834a49ccf1111b9412371489bae5f80ff6ab
This will let us populate upcoming events server-side on the website (or
serve this data in a format that can be more easily consumed by JS).
Change-Id: I0f6b5bf9831f4d07acebb4eb77a7d88b63fe8e46
This fixes resolution errors with newer pip versions (where
flask-oauthlib wants to pull in oauthlib==2 while requests-oauthlib
wants to pull in oauthlib==3).
We remove flask-oauthlib fully, as it's abandoned. Our copy of
flask-spaceauth (//hswaw/lib/flask_spaceauth) depends on it, but we will
update it to use authlib instead.
Change-Id: I51a9336015ddc0cad7457844d186bd8408371ab3
This adds github.com/minio/minio-go, a library that can be used to
access S3-like storage, eg. our own radosgw. It's significantly lighter
than the entire Go AWS SDK, and seems to also be more idiomatic than it.
Change-Id: I1e18c7665b58480fb72e789692aa7f37816cd28f
This is an attempt to see how well we do without rules_nixpkgs.
rules_nixpkgs has the following problems:
- complicates our build system significantly (generated external
repository indirection for picking local/nix python and go)
- creates builds that cannot run on production (as they are tainted by
/nix/store libraries)
- is not a full solution to the bazel hermeticity problem anyway, and
we'll have to tackle that some other way (eg. by introducing proper
C++ cross-compilation toolchains and building everything from C,
including Python and Go)
Instead of rules_nixpkgs, we ship a shell.nix file, so NixOS users can
just:
jane@hacker:~/hscloud $ nix-shell
hscloud-build-chrootenv:jane@hacker:~/hscloud$ prodaccess
This shell.nix is in a way nicer, as it immediately gives you all tools
needed to access production straight away.
Change-Id: Ieceb5ae0fb4d32e87301e5c99416379cedc900c5
This is an amalgamation of a handful of small changes to Go deps.
Notably:
- we remove our opencensus-proto fork, use upstream, use exclude=src to
fix the build
- unvendorify some deps
- bump io_rules_go to fix WKT resolution
Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.
Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
This allows us to use rules_docker from NixOS. However, the built
binaries are broken because of the Docker base image not being NixOS
based.
Change-Id: I29b93f1bae1575b04f97265c67497081d11a1910
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.
There are two reasons behind this:
- on NixOS, it's painful or at least very difficult to run hscloud out
of the box. Especially with rules_go, that download a blob from the
Internet to get a Go toolchain, it just fails outright. This solves
this and allows hscloud to be used on NixOS.
- on non-NixOS platforms that still might have access to Nix this
allows to somewhat hermeticize the build. Notably, Python now comes
from nixpkgs, and is fabricobbled in a way that makes pip3_import
use Nix system dependencies for ncurses and libpq.
This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo
~amd64.
Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
instead of Python packages
As usual with Python sadness, the @pydeps wheels are built on the bazel
host, so stuffing them inside a container_image (or py_image) will cause
new and unexpected kinds of misery.
Change-Id: Id4e4d53741cf2da367f01aa15c21c133c5cf0dba
This removes some old skylib/zlib deps, moves all Go repositories to a
separate file, and in general does a small cleanup pass on external
repository magic.
Change-Id: Ic9700ee4c40cdb2e5a68e4fc7c6b3e386a109f8a
This turns the existing script into a proper sh_binary, and injects
dependencies (kubectl and jq) as deps into it.
This change also pulls in BUILDfiles for jq, and a dep (oniguruma) into
//third_party, and adds buildable external repositories for them.
The jq/oniguruma BUILDfiles are lifted from
https://github.com/attilaolah/bazel-tools/.
Change-Id: If2e548bd60a8fd34e4f3be767ae59c6b2f2286d9
We turn the existing experimental BUILD file into a nicely abstracted
starlark rule generator, for generating both external repositories and
container build rules from a single source of truth.
We also add 0.18.22 (which we already pushed via :push_latest).
Change-Id: I521d6e5cb9447eaf6f237671b7ef07d621cd9c77
For us this manifests when doing
kubecfg update cluster/kube/cluster.libsonnet
To be precise: when hitting the Ceph/Rook CRD definition.
This is a weird bug. I've seen it manifest earlier on NixOS, but I am
now also seeing it on Gentoo. I've thought that it was because of Go API
breakage, but I've quickly tried to specify older toolchain versions,
but that didn't seem to help? :/
Regardless, I've applied a patch by rnb [1] that seems to fix this. I
also have a suspicion that updating to a newer k8s version might just
fix this, that's why I'm not not too concerned about this for now.
[1] - a32521024f
Change-Id: Id66e3c0bd56e84d785e1baeca86373aa2d0eb6f9
