Fork 0
Commit Graph

98 Commits (master)

Author SHA1 Message Date
q3k 49431e8909 gerrit/bazlets: use fork for better fetching
Change-Id: Id0c51b2e1591bef0c3d597cbcae64b373a2aa17f
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1744
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-10-27 19:42:30 +00:00
q3k fe94c9b649 dc/topo: unvendor viz.js
This instead grabs a prebuild viz.js 2.x from a GH release.

Ideally we would use some more generic JS dep mechanism here (there's
some Good (tm) ones for Bazel now!), but this will do for now.

Change-Id: I58e9f67534acd2e3d08d93dc5f9a989dbbbbe3d1
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1721
Reviewed-by: radex <radex@hackerspace.pl>
2023-10-10 15:25:34 +00:00
q3k 8dcca254ce bazlets: note down version origin
Change-Id: I3fcead7676ec41ae9905c29098f36737668cf475
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1693
Reviewed-by: implr <implr@hackerspace.pl>
2023-10-09 20:28:23 +00:00
q3k 2ceb69f30b gerrit: bump to 3.7.5
This involved messing with both of our source-built plugins (owners and

The main issue seems to have been the desync between Jackson as
requested by different plugins. Jackson is split into multiple Maven
packages, and they all have to be the same version to work together. The
oauth plugin was requesting only part of it, and these parts were
incompatible with the parts that the owners plugin requested.

In addition, we have to make the owners plugin include more bits of

Without these changes, we would get runtime
`java.lang.NoClassDefFoundError: com/fasterxml/jackson/...` errors,
which were a symptom of Jackson either not being included fully into the
plugin's JAR, or a mixup between Jackson component/package versions.

While we're at it, we remove the broken theming attempt.

Change-Id: I26531818a395de2a8bb6054d2583881fd1d5b806
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1642
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-10-09 19:11:02 +00:00
q3k 95b8c57415 rules_jvm_external: remove leftovers
This was used by q3k's minecraft plugins, and it was in the process of
being patched to better resolve snapshot versions as used in the Spigot
artifact server.

I have since decided this is not worth it, and yote the plugins and thus
we don't need rules_jvm_external at all.

Change-Id: I1a02354ec5e706c5e44501512149fe9a197ddb7c
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1644
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-10-08 12:38:42 +00:00
q3k 97b5cd7b58 go: re-do the entire thing
This is a mega-change, but attempting to split this up further is
probably not worth the effort.


1. Bump up bazel, rules_go, and others.
2. Switch to new go target naming (bye bye go_default_library)
3. Move go deps to go.mod/go.sum, use make gazelle generate from that
4. Bump up Python deps a bit

And also whatever was required to actually get things to work - loads of
small useless changes.

Tested to work on NixOS and Ubuntu 20.04:

   $ bazel build //...
   $ bazel test //...

Change-Id: I8364bdaa1406b9ae4d0385a6b607f3e7989f98a9
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1583
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-09-22 21:50:19 +00:00
implr f5b1a215f4 app/mailman-web: create
There's a lot of ugly hacks here, but this has been the state of prod
for months now, so we should reflect that.
Also, this bumps a bunch of workspace deps.

Change-Id: I744e0d3aff27036cfed73416cf442c7d62444a8b
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1473
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-09-10 21:11:37 +00:00
q3k 0d3e609013 bazlets: use python3
Change-Id: Idf8ec4b70eed991874a0bcdcced132b9f6da3f83
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1584
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-09-01 17:17:24 +00:00
q3k 7631880620 *: remove java/minecraft
Change-Id: Id2b1e69dcad240d7ef8a80b844531ef862e27dd2
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1582
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-09-01 16:50:48 +00:00
q3k 8100a2de97 third_party: replace jq with gojq
Building jq portably is annoying, and the way we were doing it (which we
iirc stole from some google project?) sucked. Let's use a Go jq clone

This is an alternative for 1535. jq is currently used only in one
script, which could really be replaced by a Go program, but let's keep
it simple for now.

Change-Id: Ie25dffadd545df143490f510e9b75a74adf81492
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1540
Reviewed-by: palid <palid@hackerspace.pl>
2023-07-24 14:47:54 +00:00
implr f21ca388ba WORKSPACE: rules_python->0.13; switch to hermetic interpreter
Change-Id: I0145f9db6a71fa9080b166dd75ff2c1b93e2b241
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1462
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-02-19 17:12:35 +00:00
implr 9851b38608 WORKSPACE,third_party/py: bump rules_{python,docker} and some py packages
Those are far from the latest versions still, but this change should be

Change-Id: Ieeb9d6b301184f46677d821fe8276391346d6285
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1459
Reviewed-by: q3k <q3k@hackerspace.pl>
2023-01-29 01:40:25 +00:00
patryk 28742b8106 Make golang deps fetching go faster
This commit aims to increase the speed of hscloud rebuild process
by optimizing the behavior of Go dependency fetching routines.

Gazelle v0.25.0 introduced a new dependency resolution mode
where it does not reach out for external dependencies; instead,
it operates solely on what is contained inside the workspace.

Because static dependency resolution mode is now the default
behavior in go_repository() rules, we are also updating the
contents of //third_party/go/repositories.bzl.

Also, I changed some of the bigger Go dependencies to be downloaded
by a tarball fetch in order to speed up the rebuild process.

Other changes:
  * Bump nixpkgs to a fresh snapshot
  * Upgrade to Bazel v5

Change-Id: Icfe752411b3128bcd5b25fa28bb76bec45ae2f71
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1441
Reviewed-by: q3k <q3k@hackerspace.pl>
2022-11-27 16:27:53 +00:00
q3k a63c315f7d prodimage: bump to focal
Change-Id: I502ef4bbc593dae4db900eb10dc4ad93daad3985
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1382
Reviewed-by: q3k <q3k@hackerspace.pl>
2022-08-30 17:30:50 +00:00
implr 4ea5cdb0eb WORKSPACE: bump protobuf
Change-Id: I7dcf4abc0a4717fd9d6d4ea43cdcfac11fdff359
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1325
Reviewed-by: q3k <q3k@hackerspace.pl>
2022-07-05 22:51:04 +00:00
implr e69e98da47 third_party/py: update rules_python, use pip-compile for requirements
Change-Id: If8309e8e3a4b58142f7479005a9eb4cbb1043cdb
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1324
Reviewed-by: q3k <q3k@hackerspace.pl>
2022-07-05 21:27:31 +00:00
q3k 19c8b60a42 hswaw/site: mirror google fonts
More privacy more better.

Change-Id: I2186a3ee47f72e4a8c3e52a45c15727da0a6a9c4
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1251
Reviewed-by: q3k <q3k@hackerspace.pl>
2022-02-01 09:38:54 +00:00
implr 12f176c1eb calico 3.14 -> 1.15
Change-Id: I9eceaf26017e483235b97c8d08717d2750fabe25
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/995
Reviewed-by: q3k <q3k@hackerspace.pl>
2021-11-20 22:12:52 +00:00
q3k 3e2a5a5957 third_party/go: add filippo.io/age
You can test this using:

   bazel run '@io_filippo_age//cmd/age'

The same target can now be used in data dependencies for secretstore
(you'll need to hardcode the runfile path, or use some
Bazel-runfile-resolving library for Python).

This required adding a few dependencies to
third_party/go/repositories.bzl, but also moving golang.org/x/crypto
from that file into WORKSPACE, before gazelle_deps gets loaded (as the
version requested by gazelle_deps is too old). We also moved shlex that
shouldn't have been in WORKSPACE into third_party/go/repositories.bzl.

Otherwise, this was just a few small deps - bumped golang.org/x/crypto,
new golang.org/x/term, new filippo.io/edwards25519. Hooray low
dependency code.

Change-Id: I0e684d88efffde13a3b4e253860aabcb35a3c94d
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1158
Reviewed-by: patryk <patryk@hackerspace.pl>
2021-10-07 20:18:25 +00:00
q3k d01f9e5fa2 WORKSPACE,third_party/go: reformat
Change-Id: If263013bd9a544696ee2530688f7f7d4ded49a92
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1159
Reviewed-by: patryk <patryk@hackerspace.pl>
2021-10-07 20:17:12 +00:00
q3k 3943744814 WORKSPACE: reformat, add novnc
Change-Id: I0162f3a704967cac4c20ec23f962a9be5c210490
Reviewed-on: https://gerrit.hackerspace.pl/c/hscloud/+/1155
Reviewed-by: q3k <q3k@hackerspace.pl>
2021-10-07 18:50:27 +00:00
radex 38203d2dbe *: update for M1 support
preliminary pass to build site on an M1 Mac

Change-Id: I89e6ac5874bbb8db92040ec98717fc0ed3ee4455
2021-08-30 18:58:54 +02:00
q3k d0b76e62b9 WORKSPACE: remove duplicate library
Change-Id: Ia165c1a44ffb557f37e5a61d372d945016190e08
2021-08-30 18:46:23 +02:00
q3k 50e987cb68 third_party/go: move some repositories from WORKSPACE
Change-Id: Icc23c741512d148c9b3dfb19c52b205d4c0150bd
2021-07-11 17:17:33 +00:00
q3k 4e534cc03c WORKSPACE: use pip_parse
This switches over to rules_python's new pip_parse remote dependency
fetching, which significantly reduces Python hell in Bazel.

Now each Python dependency gets its own external repository, which means
we don't have to build psycopg on every hscloud checkout!

Change-Id: Icc3b39197fae1046648d9a483876f5de5bd415d0
2021-07-10 13:41:51 +00:00
implr 999a8f53a2 WORKSPACE: update rules_python
fixes https://github.com/bazelbuild/rules_python/issues/489

Change-Id: I60a1577e168376b23a8daac5dc4d976713a4eaeb
2021-06-29 20:20:09 +02:00
q3k 1f0623064f hswaw/site: load leaflet from NPM package
Instead of manually packaging leaflet.js into the Git repository, this
uses an http_archive to download it on demand, and augments the static
serving code to accept different regexes as paths so that the
http_archive's contents can be served directly.

Change-Id: Icb8d624fea855fb748f107471133ac8adb5f2776
2021-06-01 22:25:23 +00:00
Norbert Szulc a6e1b8dd1d Bump version of "io_bazel_rules_docker"
This presumably enables build on bazel 4.0.0 outside of nix install

Change-Id: I5acab20a20bc0fb63f20208a86c573529ce65c75
2021-05-12 14:17:31 +02:00
q3k 146c99e58e devtools/gerrit: backport reviewiers column fix to 3.3.2
Gerrit 3.3.1 seems to have introduced a bug which makes the reviewers
column in the dashboard entry: https://bugs.chromium.org/p/gerrit/issues/detail?id=13899

This adds an override of gerrit.war to our Docker containers. The .war
is pulled over HTTP. It has been manually built by q3k from a source
checkout. The details on how this was done are in the WORKSPACE
http_file archive.

Once 3.3.3 lands we should get rid of it.

Change-Id: I8b64103cb87d8b185ff35165695a18cb19fea523
2021-03-18 19:51:33 +00:00
q3k dd692217c1 gerrit: bump to 3.3.2
Deployed to prod.

Change-Id: Iac1fdee4ec22e6a6d92ff31ab5535a43cfbcffb1
2021-03-11 23:31:28 +00:00
q3k 7849e8a2af WORKSPACE: bump rules_docker
This fixes a bazel_tools incompatibility with Bazel 3.x.

Change-Id: I73a5beafe03b1d8e68a1fee794961146a2f10c52
2021-03-11 23:30:52 +00:00
q3k 4b613303b1 RFC: *: move away from rules_nixpkgs
This is an attempt to see how well we do without rules_nixpkgs.

rules_nixpkgs has the following problems:

 - complicates our build system significantly (generated external
   repository indirection for picking local/nix python and go)
 - creates builds that cannot run on production (as they are tainted by
   /nix/store libraries)
 - is not a full solution to the bazel hermeticity problem anyway, and
   we'll have to tackle that some other way (eg. by introducing proper
   C++ cross-compilation toolchains and building everything from C,
   including Python and Go)

Instead of rules_nixpkgs, we ship a shell.nix file, so NixOS users can

  jane@hacker:~/hscloud $ nix-shell
  hscloud-build-chrootenv:jane@hacker:~/hscloud$ prodaccess

This shell.nix is in a way nicer, as it immediately gives you all tools
needed to access production straight away.

Change-Id: Ieceb5ae0fb4d32e87301e5c99416379cedc900c5
2021-02-15 22:11:35 +01:00
q3k 55cc9ab177 third_party: bump minecraft deps
Change-Id: Ib03669eef2f535ecf6711618b960f3058337fda5
2021-02-15 22:11:00 +01:00
implr 0e2057fba9 make WORKSPACE rules reproducible
per bazel error message:
DEBUG: Rule X indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = Y

Change-Id: I1c29609197d776536b7bc0336858047d7494d795
2020-12-28 21:43:33 +01:00
q3k faa326a37d WORKSPACE: update for new gerrit
Forgot to commit in https://gerrit.hackerspace.pl/c/hscloud/+/581 .

Change-Id: I9605b07079e4d1a9c916e6106034f3dba98964c2
2020-12-17 22:33:32 +00:00
q3k 1572e52c19 wow: init
This is a shitty MMORPG server. Private. Do not touch.

Change-Id: Iddfce069f5895632d305a73fcaa2d963e25dc600
2020-12-03 23:21:37 +01:00
q3k 42b21ecd84 Go: bump to 1.15.5
The new Go release fixes the following security issues:

CVE-2020-28362, CVE-2020-28367, CVE-2020-28366

See https://groups.google.com/g/golang-announce/c/NpBGTTmKzpM for more

The next step will be to re-build all our production Go binaries.

Change-Id: I97e4678a2e06a9559e66c0c0db008f0b6dfd81a2
2020-11-27 10:22:17 +01:00
q3k 301435f3c3 third_party/go: pay off some technical debt and hide the bodies
This is an amalgamation of a handful of small changes to Go deps.

 - we remove our opencensus-proto fork, use upstream, use exclude=src to
   fix the build
 - unvendorify some deps
 - bump io_rules_go to fix WKT resolution

Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.

Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
2020-11-03 21:15:41 +01:00
q3k 194b1c8e62 WORKSPACE: use nix for python/go if available
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.

There are two reasons behind this:

 - on NixOS, it's painful or at least very difficult to run hscloud out
   of the box. Especially with rules_go, that download a blob from the
   Internet to get a Go toolchain, it just fails outright. This solves
   this and allows hscloud to be used on NixOS.

 - on non-NixOS platforms that still might have access to Nix this
   allows to somewhat hermeticize the build. Notably, Python now comes
   from nixpkgs, and is fabricobbled in a way that makes pip3_import
   use Nix system dependencies for ncurses and libpq.

This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo

Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
2020-10-03 18:31:38 +02:00
q3k 1e1a4ddfc8 BUILD: fix gazelle repo file marker
Change-Id: I2ba10be7f4af784782d684b662529f926c348232
2020-08-01 12:10:19 +00:00
implr cae27ecd99 Replace rules_pip with rules_python; use bazel built upstream grpc
instead of Python packages

As usual with Python sadness, the @pydeps wheels are built on the bazel
host, so stuffing them inside a container_image (or py_image) will cause
new and unexpected kinds of misery.

Change-Id: Id4e4d53741cf2da367f01aa15c21c133c5cf0dba
2020-07-08 18:55:34 +02:00
implr 7418a5a963 Add shallow_since to WORKSPACE git_repository rules
per bazel warning
DEBUG: Rule 'com_apt_itude_rules_pip' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1564255337 -0400"

Change-Id: I6564e8325aa31bbd156ffdf85854f3f5459bd4df
2020-07-01 05:43:28 +02:00
q3k 0c3b9a570e WORKSPACE: cleanup
This removes some old skylib/zlib deps, moves all Go repositories to a
separate file, and in general does a small cleanup pass on external
repository magic.

Change-Id: Ic9700ee4c40cdb2e5a68e4fc7c6b3e386a109f8a
2020-06-25 19:54:25 +02:00
q3k 0037edaa5b cluster/tools/rook-s3cmd-config: build using bazel
This turns the existing script into a proper sh_binary, and injects
dependencies (kubectl and jq) as deps into it.

This change also pulls in BUILDfiles for jq, and a dep (oniguruma) into
//third_party, and adds buildable external repositories for them.

The jq/oniguruma BUILDfiles are lifted from

Change-Id: If2e548bd60a8fd34e4f3be767ae59c6b2f2286d9
2020-06-13 22:46:41 +02:00
q3k 66a26a8f02 WORKSPACE: remove nixpkgs/rules_nix
We're not using them for anything. Initially they were going to be used
for nixops, but nixops is not very good, so let's just drop them.

We still have a Nix dependency for clustercfg.py when provisioning
nodes, but rules_nix/nixpkgs in WORKSPACE were unrelated to that.

Change-Id: I28c249507d1be9c5dbbd1ee764deccd9ab038549
2020-06-07 02:22:14 +02:00
patryk 30f9d03106 WORKSPACE: Shuffle things around to fix tools build on macOS
Change-Id: I281c1209620e9f3a017718fda1401d1ead9c52a4
2020-06-05 23:37:34 +02:00
q3k 0ae0728ea8 Merge "third_party/factorio: init" 2020-06-04 21:45:55 +00:00
q3k de3d3fa641 bgpwtf/invoice: hide invoicee VAT number when not given
Also re-add go-bindata to WORKSPACE, something nuked it.

Change-Id: I723ebee7f843d0135a3e1121e2e93ae5fe56bc4b
2020-05-31 00:12:11 +02:00
q3k 32f8a58236 personal/q3k: add minecraft plugins
Also drive-by modify WORKSPACE to add required deps.

Also drive-by update deps in WORKSPACE.

Also drive-by remove old stackb/proto library from WORKSPACE (only used
in cccampix, which is dead, and stackb/proto should be replaceable by
the main grpc lib by this point).

Change-Id: I7ac7fe2237e859dc1c45bf41a016174ed8e9ee71
2020-05-17 23:06:42 +00:00
q3k 480505768d third_party/factorio: init
We turn the existing experimental BUILD file into a nicely abstracted
starlark rule generator, for generating both external repositories and
container build rules from a single source of truth.

We also add 0.18.22 (which we already pushed via :push_latest).

Change-Id: I521d6e5cb9447eaf6f237671b7ef07d621cd9c77
2020-05-18 00:46:31 +02:00