This instead grabs a prebuild viz.js 2.x from a GH release.
Ideally we would use some more generic JS dep mechanism here (there's
some Good (tm) ones for Bazel now!), but this will do for now.
Change-Id: I58e9f67534acd2e3d08d93dc5f9a989dbbbbe3d1
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1721
Reviewed-by: radex <radex@hackerspace.pl>
This involved messing with both of our source-built plugins (owners and
oauth).
The main issue seems to have been the desync between Jackson as
requested by different plugins. Jackson is split into multiple Maven
packages, and they all have to be the same version to work together. The
oauth plugin was requesting only part of it, and these parts were
incompatible with the parts that the owners plugin requested.
In addition, we have to make the owners plugin include more bits of
Jackson.
Without these changes, we would get runtime
`java.lang.NoClassDefFoundError: com/fasterxml/jackson/...` errors,
which were a symptom of Jackson either not being included fully into the
plugin's JAR, or a mixup between Jackson component/package versions.
While we're at it, we remove the broken theming attempt.
Change-Id: I26531818a395de2a8bb6054d2583881fd1d5b806
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1642
Reviewed-by: q3k <q3k@hackerspace.pl>
This was used by q3k's minecraft plugins, and it was in the process of
being patched to better resolve snapshot versions as used in the Spigot
artifact server.
I have since decided this is not worth it, and yote the plugins and thus
we don't need rules_jvm_external at all.
Change-Id: I1a02354ec5e706c5e44501512149fe9a197ddb7c
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1644
Reviewed-by: q3k <q3k@hackerspace.pl>
This is a mega-change, but attempting to split this up further is
probably not worth the effort.
Summary:
1. Bump up bazel, rules_go, and others.
2. Switch to new go target naming (bye bye go_default_library)
3. Move go deps to go.mod/go.sum, use make gazelle generate from that
4. Bump up Python deps a bit
And also whatever was required to actually get things to work - loads of
small useless changes.
Tested to work on NixOS and Ubuntu 20.04:
$ bazel build //...
$ bazel test //...
Change-Id: I8364bdaa1406b9ae4d0385a6b607f3e7989f98a9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1583
Reviewed-by: q3k <q3k@hackerspace.pl>
There's a lot of ugly hacks here, but this has been the state of prod
for months now, so we should reflect that.
Also, this bumps a bunch of workspace deps.
Change-Id: I744e0d3aff27036cfed73416cf442c7d62444a8b
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1473
Reviewed-by: q3k <q3k@hackerspace.pl>
Building jq portably is annoying, and the way we were doing it (which we
iirc stole from some google project?) sucked. Let's use a Go jq clone
instead.
This is an alternative for 1535. jq is currently used only in one
script, which could really be replaced by a Go program, but let's keep
it simple for now.
Change-Id: Ie25dffadd545df143490f510e9b75a74adf81492
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1540
Reviewed-by: palid <palid@hackerspace.pl>
Those are far from the latest versions still, but this change should be
non-breaking.
Change-Id: Ieeb9d6b301184f46677d821fe8276391346d6285
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1459
Reviewed-by: q3k <q3k@hackerspace.pl>
This commit aims to increase the speed of hscloud rebuild process
by optimizing the behavior of Go dependency fetching routines.
Gazelle v0.25.0 introduced a new dependency resolution mode
where it does not reach out for external dependencies; instead,
it operates solely on what is contained inside the workspace.
Because static dependency resolution mode is now the default
behavior in go_repository() rules, we are also updating the
contents of //third_party/go/repositories.bzl.
Also, I changed some of the bigger Go dependencies to be downloaded
by a tarball fetch in order to speed up the rebuild process.
Other changes:
* Bump nixpkgs to a fresh snapshot
* Upgrade to Bazel v5
Change-Id: Icfe752411b3128bcd5b25fa28bb76bec45ae2f71
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1441
Reviewed-by: q3k <q3k@hackerspace.pl>
You can test this using:
bazel run '@io_filippo_age//cmd/age'
The same target can now be used in data dependencies for secretstore
(you'll need to hardcode the runfile path, or use some
Bazel-runfile-resolving library for Python).
This required adding a few dependencies to
third_party/go/repositories.bzl, but also moving golang.org/x/crypto
from that file into WORKSPACE, before gazelle_deps gets loaded (as the
version requested by gazelle_deps is too old). We also moved shlex that
shouldn't have been in WORKSPACE into third_party/go/repositories.bzl.
Otherwise, this was just a few small deps - bumped golang.org/x/crypto,
new golang.org/x/term, new filippo.io/edwards25519. Hooray low
dependency code.
Change-Id: I0e684d88efffde13a3b4e253860aabcb35a3c94d
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1158
Reviewed-by: patryk <patryk@hackerspace.pl>
This switches over to rules_python's new pip_parse remote dependency
fetching, which significantly reduces Python hell in Bazel.
Now each Python dependency gets its own external repository, which means
we don't have to build psycopg on every hscloud checkout!
Change-Id: Icc3b39197fae1046648d9a483876f5de5bd415d0
Instead of manually packaging leaflet.js into the Git repository, this
uses an http_archive to download it on demand, and augments the static
serving code to accept different regexes as paths so that the
http_archive's contents can be served directly.
Change-Id: Icb8d624fea855fb748f107471133ac8adb5f2776
Gerrit 3.3.1 seems to have introduced a bug which makes the reviewers
column in the dashboard entry: https://bugs.chromium.org/p/gerrit/issues/detail?id=13899
This adds an override of gerrit.war to our Docker containers. The .war
is pulled over HTTP. It has been manually built by q3k from a source
checkout. The details on how this was done are in the WORKSPACE
http_file archive.
Once 3.3.3 lands we should get rid of it.
Change-Id: I8b64103cb87d8b185ff35165695a18cb19fea523
This is an attempt to see how well we do without rules_nixpkgs.
rules_nixpkgs has the following problems:
- complicates our build system significantly (generated external
repository indirection for picking local/nix python and go)
- creates builds that cannot run on production (as they are tainted by
/nix/store libraries)
- is not a full solution to the bazel hermeticity problem anyway, and
we'll have to tackle that some other way (eg. by introducing proper
C++ cross-compilation toolchains and building everything from C,
including Python and Go)
Instead of rules_nixpkgs, we ship a shell.nix file, so NixOS users can
just:
jane@hacker:~/hscloud $ nix-shell
hscloud-build-chrootenv:jane@hacker:~/hscloud$ prodaccess
This shell.nix is in a way nicer, as it immediately gives you all tools
needed to access production straight away.
Change-Id: Ieceb5ae0fb4d32e87301e5c99416379cedc900c5
per bazel error message:
DEBUG: Rule X indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = Y
Change-Id: I1c29609197d776536b7bc0336858047d7494d795
The new Go release fixes the following security issues:
CVE-2020-28362, CVE-2020-28367, CVE-2020-28366
See https://groups.google.com/g/golang-announce/c/NpBGTTmKzpM for more
detail.
The next step will be to re-build all our production Go binaries.
Change-Id: I97e4678a2e06a9559e66c0c0db008f0b6dfd81a2
This is an amalgamation of a handful of small changes to Go deps.
Notably:
- we remove our opencensus-proto fork, use upstream, use exclude=src to
fix the build
- unvendorify some deps
- bump io_rules_go to fix WKT resolution
Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.
Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.
There are two reasons behind this:
- on NixOS, it's painful or at least very difficult to run hscloud out
of the box. Especially with rules_go, that download a blob from the
Internet to get a Go toolchain, it just fails outright. This solves
this and allows hscloud to be used on NixOS.
- on non-NixOS platforms that still might have access to Nix this
allows to somewhat hermeticize the build. Notably, Python now comes
from nixpkgs, and is fabricobbled in a way that makes pip3_import
use Nix system dependencies for ncurses and libpq.
This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo
~amd64.
Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
instead of Python packages
As usual with Python sadness, the @pydeps wheels are built on the bazel
host, so stuffing them inside a container_image (or py_image) will cause
new and unexpected kinds of misery.
Change-Id: Id4e4d53741cf2da367f01aa15c21c133c5cf0dba
per bazel warning
DEBUG: Rule 'com_apt_itude_rules_pip' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1564255337 -0400"
Change-Id: I6564e8325aa31bbd156ffdf85854f3f5459bd4df
This removes some old skylib/zlib deps, moves all Go repositories to a
separate file, and in general does a small cleanup pass on external
repository magic.
Change-Id: Ic9700ee4c40cdb2e5a68e4fc7c6b3e386a109f8a
This turns the existing script into a proper sh_binary, and injects
dependencies (kubectl and jq) as deps into it.
This change also pulls in BUILDfiles for jq, and a dep (oniguruma) into
//third_party, and adds buildable external repositories for them.
The jq/oniguruma BUILDfiles are lifted from
https://github.com/attilaolah/bazel-tools/.
Change-Id: If2e548bd60a8fd34e4f3be767ae59c6b2f2286d9
We're not using them for anything. Initially they were going to be used
for nixops, but nixops is not very good, so let's just drop them.
We still have a Nix dependency for clustercfg.py when provisioning
nodes, but rules_nix/nixpkgs in WORKSPACE were unrelated to that.
Change-Id: I28c249507d1be9c5dbbd1ee764deccd9ab038549
Also drive-by modify WORKSPACE to add required deps.
Also drive-by update deps in WORKSPACE.
Also drive-by remove old stackb/proto library from WORKSPACE (only used
in cccampix, which is dead, and stackb/proto should be replaceable by
the main grpc lib by this point).
Change-Id: I7ac7fe2237e859dc1c45bf41a016174ed8e9ee71
We turn the existing experimental BUILD file into a nicely abstracted
starlark rule generator, for generating both external repositories and
container build rules from a single source of truth.
We also add 0.18.22 (which we already pushed via :push_latest).
Change-Id: I521d6e5cb9447eaf6f237671b7ef07d621cd9c77
q3k
implr
patryk
radex
Norbert Szulc