Configuring iocaine, serialized

Overview

iocaine can be configured in a number of dialects: TOML, JSON, YAML, and KDL. The configuration reference covers the KDL format, on these pages, we’ll deal with the rest.

All configuration formats allow expressing everything iocaine supports. Which one you choose, is up to you. KDL lends itself best when you’re writing the configuration directly. TOML, JSON, and YAML are more suitable when you’re configuring iocaine by other means, such as generating its configuration from Nix expressions. Or when all your other configuration is using a particular format, and you’d like iocaine to do the same.

You’re free to pick your poison! And if you chose TOML, JSON, or YAML, this document will attempt to describe their structure.

Perhaps the best way to do that, is to show you a fancy little trick:

# iocaine show config -f yaml
initial-seed: ''
state-directory: ''
instance-id: null
server:
  http:default:
    bind: 127.0.0.1:42069
    unix-socket-access: null
    mode: http
    initial-seed: null
    use:
      metrics: null
      handler-from: default
handler:
  default:
    path: null
    language: roto
    compiler: null
    config: null

And here we have the structure of the default configuration, as a glorious soup of YAML. You can print it as JSON or TOML (and KDL) too. But lets explore the structure! We’ll look at examples at the end of this document.

Syntax

Unlike the KDL format, TOML, JSON, and YAML all map almost 1:1 to the configuration structure iocaine uses internally. They are all meant to be accurate representations of that, and the internal structure is designed to be easy for iocaine to work with. It’s machine-optimized, in other words.

There are three top-level objects: initial-seed, server, and handler. Lets look at them in detail.

initial-seed

This is described in the configuration reference, and works exactly the same here. There’s even initial-seed-file, too!

instance-id & state-directory

  Important

Added in iocaine 3.1.0.

This is also described in the configuration reference, and works exactly the same here.

server

The server object declares the servers iocaine will launch, all three types of them: HTTP servers, HAProxy SPOA servers, and Prometheus servers. However, because this is almost a direct mapping to internals, the server types are not namespaced here, and every server needs to have a unique name.

If we glance back at the overview, we’ll see that what used to be this in KDL:

http-server "default" {  }

Turns into the following YAML:

server:
  http:default:
    mode: http

Because the KDL format uses different stanzas for declaring the various server types, it can - and does - namespace them, to avoid conflicts. With YAML, JSON, and TOML, such conflicts would be much more obvious, and thus, the namespacing is unnecessary.

Server properties

Each and every server has a required property: mode. Its value can be either one of http, haproxy-spoa, or prometheus. They all have a similarly required property: bind. This bind is very similar to the one described in the reference, except that the unix-socket-access property is not attached to it, but is another property of the particular server.

The rest of the available properties depend on the server type (or mode).

http and haproxy-spoa properties

Similar to the KDL format, http and haproxy-spoa servers have the exact same properties:

initial-seed

Similar to the top-level initial-seed, but bound to the particular server. If unset, uses the global default.

use

Describes connections the server has: the metrics it uses (the name of a prometheus server), and the template its handler will be instantiated from (the name of a handler, set with the handler-from property).

The metrics property is optional, handler-from is required.

server:
  http:default:
    mode: http
    use:
      handler-from: default

prometheus properties

Servers of the prometheus type have different properties: persist-path and persist-interval, both function exactly the same as in the KDL case.

This is the single case where the mapping between the configuration and iocaine’s internal structures differ: persist-interval is a human-readable string in this case, too. The reason for that is that this makes it nicer to configure the interval when configuring iocaine through other means, such as Nix expressions: we don’t have to translate it into seconds!

handler

The handler object declares handler templates, much like declare-handler does for KDL. Instead of declaring each handler at the top level, they’re collected within this handler object. Otherwise it’s very similar.

The path, language, and compiler properties are all properties of a named handler, and work the same way as in KDL.

However, the configuration that will be passed along to the handler instance live under a config key. There is no required structure there: whatever can be represented in JSON, TOML, or YAML, can go there, the format is entirely up to the handler. The contents of the config property will be serialized and passed on to the handler, all iocaine cares about is that it is syntactially valid, it does no interpretation on its own.

Examples

To show how this all works, we’ll start with a KDL configuration, and will look at how the same thing is represented in each of the other formats. We’ll try to exercise all aspects of the configuration system.

initial-seed-file "/run/current-system/boot.json"

http-server default {
  bind "sd-listen:iocaine-http-main"
  use metrics=default handler-from=default
  initial-seed "trans rights are human rights"
}

haproxy-spoa-server default {
  bind "/run/iocaine/haproxy-spoa.sock" unix-socket-access="group"
  use metrics=default handler-from=default
}

prometheus-server default {
  bind "127.0.0.1:42042"
  persist-path "/var/lib/iocaine/metrics.json"
  persist-interval "10min"
}

declare-handler default {
  ai-robots-txt-path "/var/lib/iocaine/ai.robots.txt-robots.json"
  sources {
    training-corpus "/var/lib/iocaine/corpus/1984.txt" \
                    "/var/lib/iocaine/corpus/brave-new-world.txt"
    wordlists "/var/lib/iocaine/corpus/words.txt"
  }
}

YAML

initial-seed-file: /run/current-system/boot.json
server:
  http:default:
    bind: sd-listen:iocaine-http-main
    mode: http
    initial-seed: "trans rights are human rights"
    use:
      metrics: prometheus:default
      handler-from: default
  prometheus:default:
    bind: 127.0.0.1:42042
    mode: prometheus
    persist-path: /var/lib/iocaine/metrics.json
    persist-interval: 10min
  haproxy-spoa:default:
    bind: /run/iocaine/haproxy-spoa.sock
    unix-socket-access: group
    mode: haproxy-spoa
    use:
      metrics: prometheus:default
      handler-from: default
handler:
  default:
    language: roto
    config:
      ai-robots-txt-path: /var/lib/iocaine/ai.robots.txt-robots.json
      sources:
        training-corpus:
        - /var/lib/iocaine/corpus/1984.txt
        - /var/lib/iocaine/corpus/brave-new-world.txt
        wordlists: /var/lib/iocaine/corpus/words.txt

JSON

{
  "initial-seed-file": "/run/current-system/boot.json",
  "server": {
    "haproxy-spoa:default": {
      "bind": "/run/iocaine/haproxy-spoa.sock",
      "unix-socket-access": "group",
      "mode": "haproxy-spoa",
      "use": {
        "metrics": "prometheus:default",
        "handler-from": "default"
      }
    },
    "http:default": {
      "bind": "sd-listen:iocaine-http-main",
      "mode": "http",
      "initial-seed": "trans rights are human rights",
      "use": {
        "metrics": "prometheus:default",
        "handler-from": "default"
      }
    },
    "prometheus:default": {
      "bind": "127.0.0.1:42042",
      "mode": "prometheus",
      "persist-path": "/var/lib/iocaine/metrics.json",
      "persist-interval": "10min"
    }
  },
  "handler": {
    "default": {
      "language": "roto",
      "config": {
        "ai-robots-txt-path": "/var/lib/iocaine/ai.robots.txt-robots.json",
        "sources": {
          "training-corpus": [
            "/var/lib/iocaine/corpus/1984.txt",
            "/var/lib/iocaine/corpus/brave-new-world.txt"
          ],
          "wordlists": "/var/lib/iocaine/corpus/words.txt"
        }
      }
    }
  }
}

TOML

initial-seed-file = "/run/current-system/boot.json"

[server."haproxy-spoa:default"]
bind = "/run/iocaine/haproxy-spoa.sock"
unix-socket-access = "group"
mode = "haproxy-spoa"

[server."haproxy-spoa:default".use]
metrics = "prometheus:default"
handler-from = "default"

[server."http:default"]
bind = "sd-listen:iocaine-http-main"
mode = "http"
initial-seed = "trans rights are human rights"

[server."http:default".use]
metrics = "prometheus:default"
handler-from = "default"

[server."prometheus:default"]
bind = "127.0.0.1:42042"
mode = "prometheus"
persist-path = "/var/lib/iocaine/metrics.json"
persist-interval = "10min"

[handler.default]
language = "roto"

[handler.default.config]
ai-robots-txt-path = "/var/lib/iocaine/ai.robots.txt-robots.json"

[handler.default.config.sources]
training-corpus = ["/var/lib/iocaine/corpus/1984.txt", "/var/lib/iocaine/corpus/brave-new-world.txt"]
wordlists = "/var/lib/iocaine/corpus/words.txt"