It reached the stage of being crapped out so much that the OSDs spurious
IOPS killed the performance of disks colocated on the same M610 RAID
controllers. This made etcd _very_ slow, to the point of churning
through re-elections due to timeouts.
etcd/apiserver latencies, observe the difference at ~15:38:
https://object.ceph-waw3.hswaw.net/q3k-personal/4fbe8d4cfc8193cad307d487371b4e44358b931a7494aa88aff50b13fae9983c.png
I moved gerrit/* and matrix/appservice-irc-freenode PVCs to ceph-waw3 by
hand. The rest were non-critical so I removed them, they can be
recovered from benji backups if needed.
Change-Id: Iffbe87aefc06d8324a82b958a579143b7dd9914c
More as-builts. This has already been bumped. Had to coax ceph-waw2 to
upgrade despite the fact that it's horribly broken.
Change-Id: Ia762f5d7d88d6420c2fc25cf199037cbccde0cb3
This is after the monster^Wrook outage of the week two weeks ago caused
by bc01n03 dying.
Plan is to migrate ceph-waw3 to be external, yeet ceph-waw2, and extend
crdb-waw1 to another node.
Change-Id: I133af3b1171fea383b45bf06c51e48a5c40341e4
This call will return a stream of repeated Invoices, in order to submit
monthly audit summaries to accounting, including PDFs and JPK_V7 codes
(ie. GTU and SP codes).
Change-Id: Id9da2952a6358c5c2c737eee08c473c1fbcfbe7d
Also drive-by fix two proto issues:
- rename gtu_codes to gtu_code (following convention)
- move denormalized Item.due_date field past denormalized comment.
Change-Id: Ibfe0a21aadc0a5d4e2f784b182e530b9603aae62
per bazel error message:
DEBUG: Rule X indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = Y
Change-Id: I1c29609197d776536b7bc0336858047d7494d795
This disables DHCP on all k0 nodes. This change has been tentatively
deployed to bc01n01 (which is cordoned off in kube), and I will deploy
it to the rest of k0 machines once merged.
Change-Id: I96253a9d0acedb4512c877c64174992ffdb43d58
The new Go release fixes the following security issues:
CVE-2020-28362, CVE-2020-28367, CVE-2020-28366
See https://groups.google.com/g/golang-announce/c/NpBGTTmKzpM for more
detail.
The next step will be to re-build all our production Go binaries.
Change-Id: I97e4678a2e06a9559e66c0c0db008f0b6dfd81a2
These are codes defined in JPK_V7, ie. the control file submitted to
Polish tax authorities for all invoices created in a given month. We
need to now tag our invoices with the appropriate codes when submitting
them to our accountants, which will pass them on over to the tax
authoritiex in a JPK_V7M file.
This only adds the relevant enums and adds them to proto messages, but
does not actually implement any control/model logic.
Change-Id: I695d05b55143a248dab9346151da3c572c7665d4
These tests are broken as they depend on some test data that we
currently don't have in hscloud. They should be fixed ASAP.
Change-Id: I2571c2958cb84e145a7e3a44171685ecf43cf499
This forks bitnami/kubecfg into kartongips. The rationale is that we
want to implement hscloud-specific functionality that wouldn't really be
upstreamable into kubecfg (like secret support, mulit-cluster support).
We forked off from github.com/q3k/kubecfg at commit b6817a94492c561ed61a44eeea2d92dcf2e6b8c0.
Change-Id: If5ba513905e0a86f971576fe7061a471c1d8b398