Agent check-in

[runleveldev]

Currently, the API server (manager/create-a-container) knows almost nothing about site agents (pull-config clients). Agents are essentially stateless, just querying the API server for configs per the rules of pull-config. What I'd like to do instead is have each agent "check-in" with the manager. This will be on the same schedule as pull-config, but will send additional data, including system info (hostname, ip addresses, etc) and last sync status (success/failure).

Instead of relying on ETags, the API server should respond to the check-in with the services that need updated. This should reduce the overall request load as well and enable caching in a future update.

This will then allow admins to see what agents are active for a site, when they last checked in, their current health. It will also allow the API server to make intelligent decisions about other containers based on the agent status.

[runleveldev]

Implementation details:

1. This is a full rewrite of the pull-config architecture
2. Remove all of the /nginx and /dnsmasq-* routes
3. Add POST /api/v1/agents which accepts a JSON object of details about the agent. Returns a JSON object with information about that site's containers and services.

{
  "siteId": 1,
  "hostname": "agent",
  "currentTime": 177123456,  // current server time epoch utc normalized
  "ipv4Address": 172.17.0.2,
  "services": {
    "nginx": {
      "state": "running",  // systemd state: active, inactive, failed, etc.
      "lastApply": "success"  // success or failure. failed if, for example, nginx -t failed and we rolled back
    },
    ...
}

4. Add GET /api/v1/agents which returns the current status of all agents to admins
5. Remove the pull-config subfolder, create a new "agent" subfolder. This will be another NodeJS (Typescript preferably) project.
6. The new agent is a oneshot script launched every 30s by a SystemD timer.
7. When launched the agent reads its config from the environment, like pull-config currently does, and POSTs to the /api/v1/agents endpoint. It reads the response, renders the configs (templates moved from create-a-container), tests and applies them, POSTs to the endpoint again. The Agent should record the ETAG and send it in the If-None-Match header in the request. It repeats until it gets a 304 Not Modified meaning there's been no changes to the site's config at which point it exits until the SystemD timer wakes it up again.
8. The /agents path on the Web client should show the current state of all agents. The database will record them into a new Agents model storing the information.

[cmyers-mieweb]

Pull request created for review: #407