Environments
At this point, our configuration is already flexible and concise, but not really reusable. Let’s take a look at Tanka’s third buzzword as well: Environments.
These days, the same piece of software is usually deployed many times inside a
single organization. This could be dev
, testing
and prod
environments, but
also regions (europe
, us
, asia
) or individual customers (foo-corp
,
bar-gmbh
, baz-inc
).
Most of the application however is exactly the same across those environments …
usually only configuration, scaling or small details are different after all.
YAML (and thus kubectl
) provides us only one solution here: Duplicating the
directory, changing the details, maintaining both. But what if you have 32
environments? Correct! Then you have to maintain 32 directories of YAML. And we can all
imagine the nightmare of these files drifting apart from each other.
But again, Jsonnet can be the solution: By extracting the actual objects into a library, you can import them in as many environments as you need!
Creating a library
A library is nothing special, just a folder of .libsonnet
files somewhere in the import paths:
Path | Description |
---|---|
/lib | Custom, user-created libraries only for this project. |
/vendor | External libraries installed using Jsonnet-bundler |
So for our purpose /lib
fits best, as we are only creating it for our current
project. Let’s set one up:
mkdir lib/prom-grafana # a folder for our prom-grafana librarycd lib/prom-grafana
# inside lib/prom-grafana:touch prom-grafana.libsonnet # library file that will be importedtouch config.libsonnet # _config and images
For documentation purposes it is handy to have a separate file for parameters and used images:
{ // +:: is important (we don't want to override the // _config object, just add to it) _config+:: { // define a namespace for this library promgrafana: { grafana: { port: 3000, name: "grafana", }, prometheus: { port: 9090, name: "prometheus" } } },
// again, make sure to use +:: _images+:: { promgrafana: { grafana: "grafana/grafana", prometheus: "prom/prometheus", } }}
local k = import "ksonnet-util/kausal.libsonnet";
(import "./config.libsonnet") +{ local deployment = k.apps.v1.deployment, local container = k.core.v1.container, local port = k.core.v1.containerPort, local service = k.core.v1.service,
// alias our params, too long to type every time local c = $._config.promgrafana,
promgrafana: { prometheus: { deployment: deployment.new( name=c.prometheus.name, replicas=1, containers=[ container.new(c.prometheus.name, $._images.promgrafana.prometheus) + container.withPorts([port.new("api", c.prometheus.port)]), ], ), service: k.util.serviceFor(self.deployment), },
grafana: { deployment: deployment.new( name=c.grafana.name, replicas=1, containers=[ container.new(c.grafana.name, $._images.promgrafana.grafana) + container.withPorts([port.new("ui", c.grafana.port)]), ], ), service: k.util.serviceFor(self.deployment) + service.mixin.spec.withType("NodePort"), }, }}
Dev and Prod
So far we have only used the environments/default
environment. Let’s create some real ones:
tk env add environments/prom-grafana/dev --namespace=prom-grafana-dev # one for dev ...tk env add environments/prom-grafana/prod --namespace=prom-grafana-prod # and one for prod
All that’s left now is importing the library and configuring it. For dev
, the defaults defined in /lib/prom-grafana/config.libsonnet
should be sufficient, so we do not override anything:
import "prom-grafana/prom-grafana.libsonnet"
For prod
however, it is a bad idea to rely on latest
for the images .. let’s
add some proper tags:
(import "prom-grafana/prom-grafana.libsonnet") +{ // again, we only want to patch, not replace, thus +:: _images+:: { // we update this one entirely, so we can replace this one (:) promgrafana: { prometheus: "prom/prometheus:v2.14", grafana: "grafana/grafana:6.5.2" } }}
Patching
The above works well for libraries we control ourselves, but what when another
team wrote the library, it was installed using jb
from GitHub or you can’t
change it easily?
Here comes the already familiar +:
(or +::
) syntax into play. It allows to
partially override values of an object. Let’s say we wanted to add some labels to the Prometheus Deployment
, but our _config
params don’t allow us to. We can still do this in our main.jsonnet
:
(import "prom-grafana/prom-grafana.libsonnet") +{ promgrafana+: { prometheus+: { deployment+: { metadata+: { labels+: { foo: "bar" } } } } }}
By using the +:
operator all the time and only foo: "bar"
uses “:
”, we only
override the value of "foo"
, while leaving the rest of the object like it was.
Let’s check it worked:
tk show environments/prom-grafana/patched -t deployment/prometheus
apiVersion: apps/v1kind: Deploymentmetadata: labels: foo: bar # <- There it is! name: prometheus namespace: defaultspec: minReadySeconds: 10 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: name: prometheus template: metadata: labels: name: prometheus spec: containers: - image: prom/prometheus imagePullPolicy: IfNotPresent name: prometheus ports: - containerPort: 9090 name: api