This adds github.com/minio/minio-go, a library that can be used to
access S3-like storage, eg. our own radosgw. It's significantly lighter
than the entire Go AWS SDK, and seems to also be more idiomatic than it.
Change-Id: I1e18c7665b58480fb72e789692aa7f37816cd28f
This is an attempt to see how well we do without rules_nixpkgs.
rules_nixpkgs has the following problems:
- complicates our build system significantly (generated external
repository indirection for picking local/nix python and go)
- creates builds that cannot run on production (as they are tainted by
/nix/store libraries)
- is not a full solution to the bazel hermeticity problem anyway, and
we'll have to tackle that some other way (eg. by introducing proper
C++ cross-compilation toolchains and building everything from C,
including Python and Go)
Instead of rules_nixpkgs, we ship a shell.nix file, so NixOS users can
just:
jane@hacker:~/hscloud $ nix-shell
hscloud-build-chrootenv:jane@hacker:~/hscloud$ prodaccess
This shell.nix is in a way nicer, as it immediately gives you all tools
needed to access production straight away.
Change-Id: Ieceb5ae0fb4d32e87301e5c99416379cedc900c5
This is an amalgamation of a handful of small changes to Go deps.
Notably:
- we remove our opencensus-proto fork, use upstream, use exclude=src to
fix the build
- unvendorify some deps
- bump io_rules_go to fix WKT resolution
Notably, we now do not have the 'protoc-gen-go' error when running
kubecfg/kubectl anymore.
Change-Id: I34fb9e78b2b12e4543142183d601d01987076f32
This allows us to use rules_docker from NixOS. However, the built
binaries are broken because of the Docker base image not being NixOS
based.
Change-Id: I29b93f1bae1575b04f97265c67497081d11a1910
This introduces Nix, the package manager, and nixpkgs, the package
collection, into hscloud's bazel build machinery.
There are two reasons behind this:
- on NixOS, it's painful or at least very difficult to run hscloud out
of the box. Especially with rules_go, that download a blob from the
Internet to get a Go toolchain, it just fails outright. This solves
this and allows hscloud to be used on NixOS.
- on non-NixOS platforms that still might have access to Nix this
allows to somewhat hermeticize the build. Notably, Python now comes
from nixpkgs, and is fabricobbled in a way that makes pip3_import
use Nix system dependencies for ncurses and libpq.
This has been tested to run ci_presubmit on NixOS 20.09pre and Gentoo
~amd64.
Change-Id: Ic16e4827cb52a05aea0df0eed84d80c5e9ae0e07
instead of Python packages
As usual with Python sadness, the @pydeps wheels are built on the bazel
host, so stuffing them inside a container_image (or py_image) will cause
new and unexpected kinds of misery.
Change-Id: Id4e4d53741cf2da367f01aa15c21c133c5cf0dba
This removes some old skylib/zlib deps, moves all Go repositories to a
separate file, and in general does a small cleanup pass on external
repository magic.
Change-Id: Ic9700ee4c40cdb2e5a68e4fc7c6b3e386a109f8a
This turns the existing script into a proper sh_binary, and injects
dependencies (kubectl and jq) as deps into it.
This change also pulls in BUILDfiles for jq, and a dep (oniguruma) into
//third_party, and adds buildable external repositories for them.
The jq/oniguruma BUILDfiles are lifted from
https://github.com/attilaolah/bazel-tools/.
Change-Id: If2e548bd60a8fd34e4f3be767ae59c6b2f2286d9
We turn the existing experimental BUILD file into a nicely abstracted
starlark rule generator, for generating both external repositories and
container build rules from a single source of truth.
We also add 0.18.22 (which we already pushed via :push_latest).
Change-Id: I521d6e5cb9447eaf6f237671b7ef07d621cd9c77
For us this manifests when doing
kubecfg update cluster/kube/cluster.libsonnet
To be precise: when hitting the Ceph/Rook CRD definition.
This is a weird bug. I've seen it manifest earlier on NixOS, but I am
now also seeing it on Gentoo. I've thought that it was because of Go API
breakage, but I've quickly tried to specify older toolchain versions,
but that didn't seem to help? :/
Regardless, I've applied a patch by rnb [1] that seems to fix this. I
also have a suspicion that updating to a newer k8s version might just
fix this, that's why I'm not not too concerned about this for now.
[1] - a32521024f
Change-Id: Id66e3c0bd56e84d785e1baeca86373aa2d0eb6f9
rules_pip has a new version [1] of their rule system, incompatible with the
version we used, that fixes a bunch of issues, notably:
- explicit tagging of repositories for PY2/PY3/PY23 support
- removal of dependency on host pip (in exchange for having to vendor
wheels)
- higher quality tooling for locking
We update to the newer version of pip_rules, rename the external
repository to pydeps and move requirements.txt, the lockfile and the
newly vendored wheels to third_party/, where they belong.
[1] - https://github.com/apt-itude/rules_pip/issues/16
Change-Id: I1065ee2fc410e52fca2be89fcbdd4cc5a4755d55