This is a mega-change, but attempting to split this up further is
probably not worth the effort.
Summary:
1. Bump up bazel, rules_go, and others.
2. Switch to new go target naming (bye bye go_default_library)
3. Move go deps to go.mod/go.sum, use make gazelle generate from that
4. Bump up Python deps a bit
And also whatever was required to actually get things to work - loads of
small useless changes.
Tested to work on NixOS and Ubuntu 20.04:
$ bazel build //...
$ bazel test //...
Change-Id: I8364bdaa1406b9ae4d0385a6b607f3e7989f98a9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1583
Reviewed-by: q3k <q3k@hackerspace.pl>
There's a lot of ugly hacks here, but this has been the state of prod
for months now, so we should reflect that.
Also, this bumps a bunch of workspace deps.
Change-Id: I744e0d3aff27036cfed73416cf442c7d62444a8b
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1473
Reviewed-by: q3k <q3k@hackerspace.pl>
Building jq portably is annoying, and the way we were doing it (which we
iirc stole from some google project?) sucked. Let's use a Go jq clone
instead.
This is an alternative for 1535. jq is currently used only in one
script, which could really be replaced by a Go program, but let's keep
it simple for now.
Change-Id: Ie25dffadd545df143490f510e9b75a74adf81492
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1540
Reviewed-by: palid <palid@hackerspace.pl>
Those are far from the latest versions still, but this change should be
non-breaking.
Change-Id: Ieeb9d6b301184f46677d821fe8276391346d6285
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1459
Reviewed-by: q3k <q3k@hackerspace.pl>
This commit aims to increase the speed of hscloud rebuild process
by optimizing the behavior of Go dependency fetching routines.
Gazelle v0.25.0 introduced a new dependency resolution mode
where it does not reach out for external dependencies; instead,
it operates solely on what is contained inside the workspace.
Because static dependency resolution mode is now the default
behavior in go_repository() rules, we are also updating the
contents of //third_party/go/repositories.bzl.
Also, I changed some of the bigger Go dependencies to be downloaded
by a tarball fetch in order to speed up the rebuild process.
Other changes:
* Bump nixpkgs to a fresh snapshot
* Upgrade to Bazel v5
Change-Id: Icfe752411b3128bcd5b25fa28bb76bec45ae2f71
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1441
Reviewed-by: q3k <q3k@hackerspace.pl>
You can test this using:
bazel run '@io_filippo_age//cmd/age'
The same target can now be used in data dependencies for secretstore
(you'll need to hardcode the runfile path, or use some
Bazel-runfile-resolving library for Python).
This required adding a few dependencies to
third_party/go/repositories.bzl, but also moving golang.org/x/crypto
from that file into WORKSPACE, before gazelle_deps gets loaded (as the
version requested by gazelle_deps is too old). We also moved shlex that
shouldn't have been in WORKSPACE into third_party/go/repositories.bzl.
Otherwise, this was just a few small deps - bumped golang.org/x/crypto,
new golang.org/x/term, new filippo.io/edwards25519. Hooray low
dependency code.
Change-Id: I0e684d88efffde13a3b4e253860aabcb35a3c94d
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1158
Reviewed-by: patryk <patryk@hackerspace.pl>
Looks like .ml DNS servers are currently down, and this repository
import path is deprecated anyway. Really, we should bump Kubernetes...
Change-Id: I3e0c834a49ccf1111b9412371489bae5f80ff6ab
This will let us populate upcoming events server-side on the website (or
serve this data in a format that can be more easily consumed by JS).
Change-Id: I0f6b5bf9831f4d07acebb4eb77a7d88b63fe8e46
This fixes resolution errors with newer pip versions (where
flask-oauthlib wants to pull in oauthlib==2 while requests-oauthlib
wants to pull in oauthlib==3).
We remove flask-oauthlib fully, as it's abandoned. Our copy of
flask-spaceauth (//hswaw/lib/flask_spaceauth) depends on it, but we will
update it to use authlib instead.
Change-Id: I51a9336015ddc0cad7457844d186bd8408371ab3
This adds github.com/minio/minio-go, a library that can be used to
access S3-like storage, eg. our own radosgw. It's significantly lighter
than the entire Go AWS SDK, and seems to also be more idiomatic than it.
Change-Id: I1e18c7665b58480fb72e789692aa7f37816cd28f
This is an attempt to see how well we do without rules_nixpkgs.
rules_nixpkgs has the following problems:
- complicates our build system significantly (generated external
repository indirection for picking local/nix python and go)
- creates builds that cannot run on production (as they are tainted by
/nix/store libraries)
- is not a full solution to the bazel hermeticity problem anyway, and
we'll have to tackle that some other way (eg. by introducing proper
C++ cross-compilation toolchains and building everything from C,
including Python and Go)
Instead of rules_nixpkgs, we ship a shell.nix file, so NixOS users can
just:
jane@hacker:~/hscloud $ nix-shell
hscloud-build-chrootenv:jane@hacker:~/hscloud$ prodaccess
This shell.nix is in a way nicer, as it immediately gives you all tools
needed to access production straight away.
Change-Id: Ieceb5ae0fb4d32e87301e5c99416379cedc900c5
This is an amalgamation of a handful of small changes to Go deps.
Notably:
- we remove our opencensus-proto fork, use upstream, use exclude=src to
fix the build
- unvendorify some deps
- bump io_rules_go to fix WKT resolution
Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.
Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
This allows us to use rules_docker from NixOS. However, the built
binaries are broken because of the Docker base image not being NixOS
based.
Change-Id: I29b93f1bae1575b04f97265c67497081d11a1910
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.
There are two reasons behind this:
- on NixOS, it's painful or at least very difficult to run hscloud out
of the box. Especially with rules_go, that download a blob from the
Internet to get a Go toolchain, it just fails outright. This solves
this and allows hscloud to be used on NixOS.
- on non-NixOS platforms that still might have access to Nix this
allows to somewhat hermeticize the build. Notably, Python now comes
from nixpkgs, and is fabricobbled in a way that makes pip3_import
use Nix system dependencies for ncurses and libpq.
This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo
~amd64.
Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
instead of Python packages
As usual with Python sadness, the @pydeps wheels are built on the bazel
host, so stuffing them inside a container_image (or py_image) will cause
new and unexpected kinds of misery.
Change-Id: Id4e4d53741cf2da367f01aa15c21c133c5cf0dba
This removes some old skylib/zlib deps, moves all Go repositories to a
separate file, and in general does a small cleanup pass on external
repository magic.
Change-Id: Ic9700ee4c40cdb2e5a68e4fc7c6b3e386a109f8a
This turns the existing script into a proper sh_binary, and injects
dependencies (kubectl and jq) as deps into it.
This change also pulls in BUILDfiles for jq, and a dep (oniguruma) into
//third_party, and adds buildable external repositories for them.
The jq/oniguruma BUILDfiles are lifted from
https://github.com/attilaolah/bazel-tools/.
Change-Id: If2e548bd60a8fd34e4f3be767ae59c6b2f2286d9
We turn the existing experimental BUILD file into a nicely abstracted
starlark rule generator, for generating both external repositories and
container build rules from a single source of truth.
We also add 0.18.22 (which we already pushed via :push_latest).
Change-Id: I521d6e5cb9447eaf6f237671b7ef07d621cd9c77
