Reference: R. Wolski, C. Krintz, F. Bakir, G. George, and W-T. Lin, CSPOT: Portable, Multi-scale Functions-as-a-Service for IoT, ACM Symposium on Edge Computing (SEC), Nov 2019, https://sites.cs.ucsb.edu/~ckrintz/papers/cspot19.pdf
CSPOT now comes with its own installer (which uses CSPOT). To get the latest CSPOT binaries, run the following command.
mkdir -p cspot cd cspot rm -rf update-cspot-distribution.sh (curl -fsSL https://raw.githubusercontent.com/MAYHEM-Lab/cspot/master/dist/update-cspot-distribution.sh | /bin/bash)
The installer will put the release binary distribution in the cspot
subdirectory. Not that the installer will install the install script as well so
future updates need only run:
cd cspot ./update-cspot-distribution.sh release
to get the latest release binaries or
cd cspot ./update-cspot-distribution.sh daily
to get current repo head (that compiles) as compiled binaries.
Note that this script uses CSPOT capabilities (see below) and, thus, will create a .cspot subdirectory in the home directory of the user running it, and it will add a public read-only capability to the capabilities.yaml file it will also create.
Also — CSPOT is statically linked so the binaries might look a little bit larger than expected.
Previous releases of CSPOT build from source using various bash scripts. CSPOT was originally written to install on cloud instances. Thus, the installation scripts (written in bash) run as root and were not "gentle" with respect to updating and installing any needed dependencies. This mode of installation is still supported but static linking will require MUSL. If you are installing CSPOT on a server, and you want to go this route, please read the installation script for Ubuntu 24.04 before you do anything. MUSL may have a problem coexisting with the standard C compiler. The script assumes that the image does not have the C language toolchain installed and it will install MUSL. Please be careful with this option. It is not easy to remediate if it goes awry.
The following tutorial assumes that the path to the CSPOT binary directory is
/home/ubuntu/cspot
Among CSPOT code base are a Linux command-line client toolset called senspot that consists of three client programs
senspot-init senspot-put senspot-get
These programs are designed to be called form the shell to create a CSPOT WOOF (a persistent log in CSPOT is called a WOOF which stands for "Wide-area Object of Functions"), to append values to the end of a WOOF, and to fetch values from a WOOF.
To test a CSPOT installation with senspot, one needs to first start a CSPOT namespace server, run senspot-init to create an initial test WOOF, run sesnpot-put to append an initial value to that WOOF, and then run senspot-get to fetch the value back. Note that senspot-init must be run in the directory that serves as the namespace with the namespace server running and (for security reasons) it only works locally. Both senspot-put and senspot-get take URNs with IP addresses specifying the host IP addresses.
The following example assumes that cspot has been installed on two hosts with the IP addresses
192.168.10.2 192.168.10.3
and that ports 50000 to 60000 are open for ingress and egress between both hosts.
To test, using senspot, the cspot installation on these two hosts, log into 192.168.10.2 as the ubuntu user and type the following
cd cspot ./woofc-namespace-platform >& namespace.log &
This command sequence will start the namespace server in /home/ubuntu/cspot where the binaries have been deposited by the make installer. If there is an error it will be printed to namespace.log. Note that woofc-namespace-platform will need read and write permission to the cspot directory. If the installation was done as root, change the ownership of this directory to ubuntu:ubuntu.
Next, in that same directory, initialize a test WOOF using senspot-init
./senspot-init -W testwoof -s 100
This command will create a CSPOT WOOF that is capable of storing the last 100 entries that have been appended to it as a log. The -s command sets this history size.
The next step is to test senspot locally. To append a first entry to testwoof type
echo "3.1415" | ./senspot-put -W woof://192.168.10.2/home/ubuntu/cspot/testwoof -T d
the senspot-put command takes a full URN for the WOOF which is woof://IPaddress/path-to-namespace/woofname with the -W argument. It reads a string from standard in and converts it to a data type specified by the -T parameter (-d indicates a double precision value in this case). If this command completes without printing an error message, then the value 3.1415 will have been appended to testwooof in the namespace /home/ubuntu/cspot.
To read the value back, type
./senspot-get -W woof://192.168.10.2/home/ubuntu/cspot/testwoof
This command will read the most recent value appended to the WOOF and print it along with a timestamp and a sequence number.
In this example, the output should be
3.141500 time: 1738353021.8613910675 192.168.10.2 seq_no: 1
where the time value is Linux epoch.
In order to tear down a running namespace that was created with
woofc-namespace-platform, use the following command.
pkill -x woofc-containerThis command will terminate the woofc-container process, which will also tear
down the woofc-forker-helper processes that were spawned by running
woofc-namespace-platform.
The next step is to test senspot between machines, running it on each referencing a namespace on the other. In the next example, 192.168.10.2 will be the client and 192.168.10.3 will be the server.
Log into 192.168.10.3 as the ubuntu user and type
cd cspot ./woofc-namespace-platform >& namespace.log & ./senspot-init -W testwoof
which will start a namespace server at /home/ubuntu/cspot and then create testwoof as a WOOF in it.
From 192.168.10.2 in the directory /home/ubuntu/cspot run
echo "2.71828" | ./senspot-put woof://192.168.10.3/home/ubuntu/cspot/testwoof -T d ./senspot-get woof://192.168.10.3/home/ubuntu/cspot/testwoof
If this sequence is a success, the output should be something like
2.718280 time: 1738353221.8613910675 192.168.10.2 seq_no: 1
Next, test the reverse direction. On the host 192.168.10.3, from the directory /home/ubuntu/cspot/build/bin run the following commands
echo "1234" | ./senspot-put woof://192.168.10.2/home/ubuntu/cspot/testwoof -T d ./senspot-get woof://192.168.10.2/home/ubuntu/cspot/testwoof
If this test is successful, the output will be something like
1234.000000 time: 1738431242.2993268967 192.168.10.3 seq_no: 2
indicating that the append from 192.168.10.3 to /home/ubuntu/cspot/testwoof on 192.168.10.2 has been completed.
If senspot-put fails, it is usually for one of four reasons
-
woofc-namespace-platform on the server host is not running or has thrown an error
-
senspot-init has not been run in the namespace where a woofc-namespace-platform was started
-
the IP address or the path to the namespace specified in senspot-put do not match the IP address and/or path where woofc-namespace-platform is running
-
ports 50000 to 60000 are not open for TCP ingress on the server host or egress on the client host or both
The senspot toolchain includes the ability to automatically trigger a forwarding operation when data is appended to a WOOF. The command
./senspot-forward-init -W local-src-woof-name -F woof://target-ip/namespace-path/dst-woof-name
in the source namespace which must also contain the handler code in senspot_forward.
In this example, to automatically forward data from /home/ubuntu/cspot/testwoof on 192.168.10.2 to /home/ubuntu/cspot/testwoof on 192.169.10.3, cd into /home/ubuntu/cspot on 192.168.10.2 and run the command
./senspot-forward-init -W testwoof -F woof://192.168.10.3/home/ubuntu/cspot/testwoof
This command will initialize forwarding state to keep track of what data needs to be forwarded in the event the network partitions between the source and destination namespaces.
To trigger forwarding when data is appended to the source, add the parameters "-H senspot_forward" to the senspot-put command. Thus, in this example, running the command
echo "8888" | ./senspot-put -W woof://192.168.10.2/home/ubuntu/cspot/testwoof -T d -H senspot_forward ./senspot-get -W woof://192.168.10.3/home/ubuntu/cspot/testwoof
should show "8888" as the payload fetched from 192.168.10.3 after the put to 192.168.10.2. Note that the sequence numbers are not kept consistent between the source WOOF and the destination WOOF by senspot_forward. It does preserve the initial timestamp and, if the network partitions, it will forward all of the data that has yet to be forwarded (in order) with the original timestamps so that the data (but not the sequence numbers) are consistent between source and destination after the partition resolves.
Note that if the destination is protected by a capability, the source namespace needs to contain a capabilities.yaml (for the forwarder to use) — handlers do not have access to the .cspot directory of the user running woofc-namespace-platform (see section on capabilities below).
The senspot toolchain includes the ability to store and retrieve files from WOOFs that are sufficiently large enough to hold them. The senspot application toolchain includes three commands for using CSPOT as a versioned file store
senspot-file-init -W localwoofname -s size -M transfer-block-size (in KB) senspot-file-send -f localfilename -W woofname <-V> senspot-file-recv -f localfilename -W woofname <-v version-number> <-V>
where the -V parameter (which enables verbose mode) is optional. The version number parameter will specify a previous version to retrieve. If it is omitted, the command fetches the latest version.
The commands
senspot-file-recv -W woofname -l senspot-file-recv -W woofname -L
print the latest version number and all version numbers stored in the specified WOOF respectively.
The command senspot-file-init creates a local WOOF with the number of elements specified after the -s parameter, each one of which is the transfer-block-size (in KB) given by the -M parameter. The WOOF must be large enough to hold at least one copy of the file. That is the the product "size * transfer-block-size" must be greater than or equal to the file size.
The model is that each file is stored in its own WOOF but that (depending on the total size specified in senspot-file-init) each time a file is sent to the WOOF, it will be given a new version number and appended (subject to space availability).
For example, the following commands store a file created on 192.168.10.2 in a WOOF on 192.168.10.3 in the cspot namespace in the home directory of the ubuntu user.
On 192.168.10.3 in /home/ubuntu/cspot, run the command
senspot-file-init -W testfilewoof -M 4 -s 10
which creates the destination WOOF and sets the element size to 4K bytes. The WOOF contains 10 elements.
On the host 192.168.10.2, running the commands
echo "Mary had a little lamb." > testfile.txt senspot-file-send -f testfile.txt -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof
stores the file in the destination WOOF.
Running the commands
senspot-file-recv -f testfile2.txt -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof
fetches the file at the end of the woof testfilewoof. In this example, testfile.txt and testfile2.txt should contain the same text.
The following commands create a new version of the file and append it.
echo "Whose fleece was white as snow." >> testfile.txt senspot-file-send -f testfile.txt -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof
The following command will then print the version numbers for both versions
senspot-file-recv -L -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof
The output should should resemble the following.
version 2:2 at 2, created: 2026-07-07 11:06:50 (1783447610) size: 56 start_seqno: 2 version 1:1 at 1, created: 2026-07-07 11:06:40 (1783447600) size: 24 start_seqno: 1
Note that the senspot-file tools create version numbers that have two components: a major and minor version number (separated by colons in the output). It is possible for two files to have the same major version number (because the system does not maintain a shared version number than can be updated atomically amongst distributed clients) but the minor version number (which is a WOOF sequence number) is guaranteed to be unique (although not incrementing by 1 for each file creation).
The following command fetches version 1:1 of the file and puts it in testfile3.txt
senspot-file-recv -f testfile3.txt -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof -v 1
Note that if the major version number is unique, only the -v parameter is necessary. The -m parameter specifies a particular minor number. Thus the following command is equivalent to the previous one
senspot-file-recv -f testfile3.txt -W woof://192.168.10.3/home/ubuntu/cspot/testfilewoof -v 1 -m 1
Files are stored in WOOFs in elements that are the size of the transfer size specified by the -M parameter in the senspot-file-init command. Thus, in this example, each file version is stored in a 4 kilobyte block and up to 10 versions of the file (all smaller than 4 kilobytes each) can be stored. If a version is larger than 4 kilobytes it could also be stored but the number of versions would be fewer than 10. Because the senspot-init command specified 10 elements of 4 KB each, the total space allocated for all versions in the WOOF is 40 KB. Thus, in this example, no version could be larger than 40 KB. === CSPOT Message Authentication using CAPLets
Reference: Bakir, Fatih, Chandra Krintz, and Rich Wolski. "Caplets: Resource aware, capability-based access control for IoT." 2021 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 2021. https://sites.cs.ucsb.edu/~ckrintz/papers/caplets21.pdf
CSPOT can use a version of CAPLets capabilities for message authentication. Doing so requires the creation of a principal capability (stored in a woof with a .CAP extent) in the CSPOT namespace either for each woof that is to be protected or for the namespace as a whole. When the CSPOT namespace platform detects a capability woof, it will check the message accessing that woof to make sure it has been signed using a capability of sufficient privilege. If the capability is not present (e.g. an older version of CSPOT is attempting access) or if the signature does not match, the access is denied. If capability woofs are not present in the namespace for a woof or for the namespace itself, no authentication is performed (for backward compatibility).
Note that the messages are signed but not encrypted. That is, each message is authenticated but the channel is not private. If the capability used to sign each message is treated as a shared secret, then an eavesdropping attacker can replay an observed message but cannot alter it. Further, because CSPOT woofs are logs, replayed messages can be deduplicated (although this deduplication support currently must be application level).
Privileges are ordered. The currently supported privileges (from least to greatest privilege are)
-
READ (0x00000001)
-
WRITE (0x00000002)
-
EXECUTE (0x00000004)
-
CREATE (0x00000008)
-
DESTROY (0x00000010)
-
POLICY_ATTACH (0x00000020)
Any capability can be attenuated to a lower level of privilege by the capability’s holder and CSPOT uses the least privileged capability to sign a CSPOT message. Currently, all messages require at least READ privilege, the PUT operation requires WRITE privileges and PUT with a concomitant handler invocation requires EXECUTE permissions. Remote CREATE, DESTROY, and policy implementation are not yet supported. Also, in the current release, message authentication is only checked when a CSPOT operation is remote (i.e. uses the network communication path and not the local fast path).
It is possible to create capabilities for individual WOOFs (i.e. CSPOT logs — see below) or for an entire namespace (again — see CSPOT abstractions below). This example shows how to create a read-only capability for a CSPOT namespace and also a separate read-write capability. Any process that has access to the read-only capability will be able to read (call WooFGet) on any WOOF in the namespace that is not further protected by a capability. Any process holding the read-write capability will be able to either read (WooFGet) or write (WooFPut) but will not be able to trigger a CSPOT handler as a result of a Put.
The first step is to create a principle capability in the namespace. This capability grants all access so it should be protected using the most restrictive file systems permissions (e.g. as an ssh private key would be). The trust model assumes that a process with access permissions to the directory containing the namespace has access to all of its contents, including the principle capabilities.
To create a principle capability for the namespace, log into the machine hosting the namespace and change into the namespace directory. Ensure that woofc-namespace-platform is running for that namespace and that the user login has write permissions to the namespace directory.
Run the command
./woofc-init-principle -N
If the command is successful, it will create (silently) a file called CSPOT.CAP (which is a WOOF) that contains the principle capability. Note that this file is analogous to an ssh private key. If it is stolen, the possessor has full access rights to everything in the namespace.
All accesses to WOOFs in this namespace will need to include a capability in the CSPOT API calls that will be checked on each access to ensure that it is a legal attenuation of this principle capability.
The CSPOT API uses a keyring (formatted as a YAML file) located in the .cspot subdirectory of the user ID’s home directory (e.g. similar to the .ssh directory used to hold ssh keys) or the current working directory. The keyring file must be called "capabilities.yaml" and it must have all group and world permissions disabled.
To create a read-only attenuation for the namespace, run the command
./woofc-print-cap -N -h 192.168.10.2 -p 1
where the IP address occurring after the -h parameter is the externally visible IP address for the namespace. The "-p 1" says that the only permission enabled is READ since the capability bit pattern for READ is READ 0x00000001.
The output of this command is YAML text that needs to be put into the capabilities.yaml keyring file of any user running a process that exercises the CSPOT API and contacts this namespace and that is granted read-only access to the WOOFs in the namespace.
To create a read-write attenuation, in the namespace directory, run the command
./woofc-print-cap -N -h 192.168.10.2 -p 3
which will print a YAML-formatted capability with READ and WRITE permissions enabled, but no other permissions enabled. Because permissions are ordered, only the bit associated with the "highest" permission need be set to convey all lower permissions. Thus, "-p 3" (READ/WRITE/EXECUTE) is equivalent to "-p 2" (WRITE).
Note that the procedure for individual WOOFs is the same, but the -N flag is replaced with -W local-woof-name. The principle capability will be placed in local-woof-name.CAP and woofc-print-cap will print YAML for capabilities that are specific to that WOOF (and not the name space).
Finally, if there is both a namespace principle capability and a principle capability for a specific WOOF in the namespace, the access is granted if the presented capability is a legal attenuation of either (i.e. not the most restrictive).
One the namespace principle capability has been created, woofc-namespace-platform will check all remote CSPOT API requests to ensure that they carry the minimum necessary access rights for all WOOFs in the namespace. If there is no namespace principle capability, but there is a capability for a specific WOOF or WOOFs, then it will check access rights on API calls for those WOOFs.
Older released versions of CSPOT did not include capability support. To allow older CSPOT clients to work with the current platform, if there are no principle capabilities, woofc-namespace-platform will allow all accesses without rights checking.
To enable a client to have READ access to the namespace, the output of woof-print-cap -N -h 192.168.10.2 -p 1 needs to be present in the file
~/.cspot/capabilities.yaml
exactly as it prints and the permissions on the .cspot directory and on the capabilities.yaml file must be read/write for the owner only. In particular, YAML parser checks the indentation when it determines whether the capability that is present in the file is valid.
In this example, when ever an API call accesses a WOOF in the namespace on 192.168.10.2, it will scan the capabilities.yaml file and send a request with READ permissions. If the request can be completed with READ permissions, the API call will succeed. If not, it will fail because woofc-namespace-platform will deny the request due to insufficient permissions.
Note that woof-print-cap must be run locally within the namespace (there is no provision in the current release for creating capabilities remotely). Thus only the "owner" of the namespace (a user on the machine hosting the namespace with write access to the directory in which it has been created) is entitled to generate capabilities for and within that namespace.
In the original CAPlets formulation, capabilities are sent in messages so that the server need not keep a record of them (i.e. on a resource restricted device). The current CSPOT release API does not send capabilities in messages (and instead stores them either in the namespace or on the client in the capabilities.yaml keychain file). Instead, it signs the message payload with the capability (possibly attenuating it down to the minimum access level) on the client side and the server derives a capability (from the principle) with minimum access rights and checks the signature with this locally generated capability.
In this way, capabilities can be treated like secret keys since they are never transmitted over the network. However, an application can elect to send capabilities and use CAPlets attenuation to implement distributed access control.
The current release includes support for using MQTT as a network transport protocol instead of the default ZeroMQ messaging system. This feature should be considered experimental. However, the ZeroMQ message transport requires that the client be able to establish a TCP connection to the machine running the namespace on a port between 50000 and 60000. As a result, the server cannot be behind a firewall that blocks TCP connections to this port range.
If it is possible to configure an MQTT broker two which the client(s) and namespace machine can all connect, the MQTT transport permit the namespace to be behind a firewall that allows outgoing TCP connections, but blocks incoming ones.
The current CSPOT release uses the mosquitto Linux utility to implement a socket emulation layer that is configured along side the ZermoMQ transport.
Both the client and the server require a YAML file that describes how to contact an MQTT broker. The current implementation (which should not be used in secure production) uses a user name and password (which are transmitted clear test and visible in a ps) to authenticate to the broker.
In the following examples, the broker is running on IP address 192.168.10.4 to which both the client and server can make a TCP connection on port 1883 (the standard MQTT port for a non SSL connection). The broker has been configured to authenticate the user "CSPOT-mqtt-user" with the password "CSPOT-mqtt-password".
On the client side, the CSPOT client API will look for the file $HOME/.cspot/mqtt-proxy.yaml. The .cspot directory and the file must be rw for the owner only.
On the server side, the file mqtt-proxy.yaml must be found in the same directory where woof-namespace-platform is started and it must have permissions rw for the owner only.
The format of the mqtt-proxy.yaml file is
proxy: namespace: name-of-namespace host-ip: local-host-ip broker-ip: broker-ip user: mqtt-user-name pw: mqtt-user-pw
Thus, in this example, if the client runs as the ubuntu user on 192.168.10.2, the file /home/ubuntu/.cspot/mqtt-proxy.yaml should contain
proxy: namespace: test-namespace host-ip: 192.168.10.2 broker-ip: 192.168.10.4 user: CSPOT-mqtt-user pw: CSPOT-mqtt-password
Similarly, this file must be in the directory where woofc-namespace-platform is started.
Note that the implementation uses the mosquitto client tools (which must be installed) and that an abortive termination may leave one or more instances running.
Once configured, the client tools will recognize the prefix "mqtt://" in place of "woof://" in any URI. For example, the client in this example could issue the following commands
echo "1234" | senspot-put -W woof://192.168.10.3/home/ubuntu/cspot/testwoof -T i senspot-get -W mqtt://192.168.10.3/home/ubuntu/cspot/testwoof
where woofc-namespace-platform is running in the directory /home/ubuntu/cspot and that directory contains the mqtt-proxy.yaml file and get back
1234 time: 1738353021.8613910675 192.168.10.2 seq_no: 1
In this example, senspot-put uses ZeroMQ and senspot-get uses MQTT. The message transport will uses the URI prefix to determine which transport to use for each message.
The contents of the
https://github.com/MAYHEM-Lab/cspot/examples/cspot-example-app-with-cmake
subfolder of the main repo contain an simple CSPOT example (that uses a handler)
and the build environment necessary to compile it using cmake. To build and
run it (using the CSPOT binary distributions and the daily binaries, copy the
contents of this directory a directory that will act as a namespace. If CSPOT
is cloned in /home/ubuntu/cspot, then the following commands
mkdir -p test-namespace cd test-namespace cp -r /home/ubuntu/cspot/examples/cspot-example-app-with-cmake/* . ./build-cspot-example-app.sh
will download and install the necessary binaries, build the app contained in the cspot-app-example subdirectory, and run the app which prints the dispatch and round-trip times for local handler invocations.
CSPOT is acronym (slightly reordered) for Serverless Platform of Things in C. It is an empirical coding experiment that amalgamates ``Serverless'' computing (i.e. Functions as a Service – FaaS) and distributed computing principles targeting Internet of Things (IoT) applications.
The goal of CSPOT is to explore the interplay between application programming abstractions, runtime systems and operating systems in a tiered cloud setting. The design presupposes that IoT sensors and actuators will communicate with computing elements at the ``edge'' of the network (e.g. edge clouds that can implement public cloud services near where data is gathered and actuation occurs). Edge computing, in turn, may need to employ resources at a regional level (e.g. a private cloud) or more globally (e.g. in a public cloud).
CSPOT attempts to layer a platform across these three tiers that supports a common set of low-level abstractions for programmers to use to construct applications. The programming model that CSPOT supports is one akin to ``Functions as a Service'' in which control-flow is event driven. Thus a CSPOT program consists of events triggered by functions applied to a distributed set of storage objects that are implemented as part of a single, common storage abstraction.
The initial implementation and its test applications use C as the programming language. However, functions execute within Linux containers making it possible to employ a mixed-language programming approach.
The current implementation supports three primary programming abstractions: * Wide-area Objects of Functions (WOOFs) – Append-only storage objects capable of persisting data in fixed-sized elements * Handlers – Functions that may be triggered by the platform when data is appended to a WOOF * Namespaces – Collections of WOOFs that share a common name prefix
The intention is to begin with a minimal set of abstractions and API semantics and to expand as needed. Performance optimization, in particular, may drive modification and or expansion of the API semantics.
Each namespace defines a flat space for storing WOOFs and handler code. Each namespace is located on some host within the system. While yet to be implemented, the intention is to implement access control between namespaces. Initially, each namespace corresponds to a top level directory on a Linux host that contains related state (WOOFs and handler code).
A WOOF is an append-only sequence of fix-sized memory regions (called elements) managed as a circular buffer. The size of each element in a WOOF as well as the history size (the maximal number of the most recent appends to the WOOF) are specified when the WOOF is created and cannot be changed. CSPOT does not interpret the content of each memory element. However it does assign a unique 64-bit sequence number to each element when the element has been successfully added to a WOOF.
Note that the historical capacity of a WOOF is programmer-determined. When the circular buffer wraps, the oldest data in the WOOF is simply overwritten. However, the sequence number space for each WOOF does not wrap.
The current WOOF C-language API consists of the following API calls
int WooFCreate(char *woof_name, unsigned long element_size, unsigned long history_size)
-
creates a WooF with fixed-sized elements and specified history size
-
woof_name is the local or fully qualified name of the WOOF to create
-
element_size refers to the number of bytes in a memory region
-
history_size refers to the number of elements (not bytes) in the WOOF history
-
returns < 0 on failure
unsigned long WooFPut(char *woof_name, char *handler_name, void *element)
-
woof_name is the local or fully qualified name of an existing WOOF
-
handler_name is the name of a file in the WOOF’s namespace that contains the handler code or NULL. When handler_name is NULL, no handler will be triggered after the append of the element.
-
element is an in parameter that points to an memory region to be appended to the WOOF
-
the call returns the sequence number of the element or a representation of -1 on failure
int WooFGet(char woof_name, void element, unsigned long long seq_no)
-
woof_name is the local or fully qualified name of an existing WOOF
-
element is an in parameter that points to an memory region to be set to the contents of the element from the WOOF (i.e. an out parameter)
-
seq_no is the sequence number, from the WOOF, of the element to be retrieved (sequence number zero is not a valid sequence number and, thus, when specified in a call WooFGet() returns the element having the largest sequence number stored in the WOOF). If the sequence number is invalid (i.e. out of the range of sequence numbers in the WOOF) an error is returned.
void WooFInit()
-
allows a Linux process external to CSPOT to make called to WooFPut()
-
reads its parameters from environment variables that the calling process must set
This API definition is, more or less, stable. There is an internal API for implementing ``fast-path'' WOOF accesses, but it is not maintained in the current release and is definitely subject to change.
There are several features of the API that, perhaps, require some scrutiny.
First, this is the complete API (a WooFRemove() call will be included in a future release). A well-formed CSPOT program uses WOOFs as its only data structures and WooFCreate(), WooFPut(), and WooFGet() are the only operations supported for those data structures.
Secondly, only a call to WooFPut() causes a computation to be initiated. That is, CSPOT requires that program state be appended to a WOOF as a prerequisite to executing a computation. As a result, the elements stored in a program’s set of WOOFs represent the full program state in the event of failure and the program can be resumed from that state. Parsing the program state so that the program can be resumed is not currently automated.
Thirdly, handlers are concurrent and may execute out of order with respect to their invocation. Synchronization occurs when a sequence number is assigned to an element when it is appended to a WOOF. That is, a call to WooFPut() will append the element and return a sequence number as a transaction. Note that there are no primitives for synchronizing handlers beyond this transaction.
Lastly, WooFInit() is included as an optimization that allows CSPOT client applications ``join'' a namespace. By default, each WOOF is addressed by a URN and when the API code parses the WOOF name, if the name is fully qualified, the request will generate a network request and response. As a local optimization, it is possible to address WOOFs by path name, but to do so, the process must initialize the namespace state. WooFInit() is a primitive that implements this initialization.
WOOF names are either interpreted locally, with respect to the namespace
of the handler that is referring to them or fully qualified as a URI
beginning with the string woof://''. A name must be unique within each
namespace. If the prefix of the name string is woof://'' the remainder
of the string is interpreted by the current implementation as an
absolute path to the WOOF on the host where it is located. If not, it is
interpreted relative to the namespace path for the referring handler.
Additionally, each namespace must contain binary files carrying the handlers that can be executed on WOOFs within the namespace. The handler names and the WOOF names must not conflict.
Each WOOF handler must have the following function signature as its top-level entry point
int HandlerName(WOOF *woof, unsigned long seq_no, void *element)
When the CSPOT runtime system invokes the handler, it will pass an opaque handle for the WOOF, the sequence number of the element that the handler is to handle, and a pointer to the element. The handler should return a value >= on success and < 0 on failure. Handlers should not persist state other than by calling WooFPut() on one or more WOOFs (possibly creating them when needed).
Each WOOF is implemented as a memory-mapped file within a namespace. Handlers run within a Docker container associated with the namespace that contains them. Thus, the CSPOT platform creates a container per name space maps all WOOFs referred to in an API call into the address space of the handler making the call. Thus, it is necessary to start a platform component for each namespace. Currently each namespace platform must be started manually using the commands
woofc-namespace-platform
optionally with a -N command specifying a path for the namespace. Note that woofc-namespace-platform prints its output to stdout and (sometimes) to standard error.
The namespace platform must be executing before any puts to a namespace activate. That is, the platform is intended to function as a long running daemon that services the namespace for all applications that access WOOFs contained within it.
The namespace platform creates an internal append-only log for the namespace that the runtime uses to trigger handlers. A threaded process running within the container monitors the tail of the namespace log. When a call to WooFPut() specifies a handler, the code will append a TRIGGER record to the log indicating that a handler must be triggered. Threads within the dispatch process claim TRIGGER records exclusively (and append their claims to the log) and, once claimed, trigger the handler specified in the record.
Each container is also run with the ``-i'' option. As a result, if a handler writes to standard out or standard error, the resulting output will appear on the tty associated with the shell that launched the platform. That is, the platform aggregates the standard out and standard error file descriptors from all handlers executing in the namespace it is managing.
Because the handler is actually executing in a separate process within a namespace container, the process must execute bootstrap code to map the WOOF and pass the sequence number to the handler. As a result, the handler code must be wrapped in a C main() routine that is part of CSPOT. This main() routine is contained in the file woofc-shepherd.c.
Additionally, it is possible to issue CSPOT API calls from outside of a namespace so that CSPOT programs can communicate with external users and programs.
A call to WooFPut() or WooFGet() that specifies a fully-qualified URN will generate network message (using link::https://zeromq.org[ZeroMQ]) when the call is from an application component that is external to the namespace, or when CSPOT determines that a handler is referencing a WOOF in another namespace. It is possible to use a Linux path name to reference a WOOF, but an external process must make a call to WooFInit() before doing so to initialize the runtime environment. Handlers, however, inherit the environment in which they are to execute and, thus, need not call WooFInit().
There is a lot left to do.
The minimalist initial API uses WooFPut() as the primary API abstraction for moving state between application components. This emphasis is intended to promote the use of append-only semantics in a FaaS context. For IoT, doing so will (may) make it possible to program distributed IoT applications in a FaaS style.
However, it introduces an asymmetry between writing and reading program state that may make application programming more difficult. Specifically, all reads must be namespace local (requiring a WooFOpen() to obtain in internal WOOF handle). Logically, no asymmetry is mandated. Thus it will be important to understand whether building it into the API is useful or confusing.
The API design also influences the performance of the system. In particular, mapping a WOOF into the memory space of a process running in a container is a performance-expensive operation under the current implementation supported by Linux. Thus, it is useful, as a programmer-controlled optimization, to allow the mapping to be reused. Because WooFPut() takes a WOOF name, it must first map the WOOF, then do the put, and then unmap the WOOF (there are optimization possibilities here, to be sure). To make make multiple puts to the same WOOF more efficient, the API currently includes WooFAppend() which takes a handle returned from WooFOpen() (in the same way WooFRead() does) to a WOOF in the local namespace. Indeed, WooFPut() uses WooFAppend() internally. Its implementation looks something like
unsigned long WooFPut(char *woof_name, char *handler_name, void *element)
{
if(woof_name is a local WOOF) {
woof = WooFOpen(woof_name);
seq_no = WooFAppend(woof, handler_name, element);
WooFFree(woof);
} else {
seq_no = send a put request to the put proxy for the WOOF's namespace
}
return(seq_no);
}
I/O creates another related question that the project must investigate. In particular, it is possible for a process outside of a namespace to make a call to WooFPut() to introduce data but without an analogous WooFGet() call, there is no way to get data back out of a namespace. Thus the put/get API that, ultimately, is part of the prototype is richer than the minimalist API:
-
unsigned long WooFPut(char woof_name, char handler_name, void *element)
-
woof_name is the local or fully qualified name of an existing WOOF
-
handler_name is the name of a file in the WOOF’s namespace that contains the handler code or NULL. When handler_name is NULL, no handler will be triggered after the append of the element.
-
element is an in parameter that points to an memory region to be appended to the WOOF
-
the call returns the sequence number of the element or a representation of -1 on failure
-
can be called from either wishing a handler or from a process outside of a namespace
-
-
int WooFGet(char woof_name, void element, unsigned long seq_no)
-
woof_name is the local or fully qualified name of an existing WOOF
-
element is an out parameter pointing to memory that will be filled in by the specified WOOF element
-
seq_no is the sequence number of the element to be returned through the element pointer
-
returns < 0 if the call fails to successfully return the element
-
WOOF can either be in the local namespace or a remote namespace
-
-
WOOF * WooFOpen(char *woof_name)
-
woof_name is the local or fully qualified name of an existing WOOF
-
returns an opaque handle to an in-memory data structure referring to the WOOF or NULL on failure
-
if the WOOF is not in the local namespace, the call fails
-
-
int WooFAppend(WOOF woof, char handler_name, void *element)
-
woof is an opaque handle returned from a call to WooFOpen()
-
handler_name is the name of a file in the WOOF’s namespace that contains the handler code or NULL. When handler_name is NULL, no handler will be triggered after the append of the element.
-
element is an in parameter that points to an memory region to be appended to the WOOF
-
the call returns the sequence number of the element or a representation of -1 on failure
-
the WOOF must be in the local namespace
-
-
int WooFRead(WOOF woof, void element, unsigned long seq_no)
-
woof is an opaque handle returned from a call to WooFOpen()
-
element is an out parameter pointing to memory that will be filled in by the specified WOOF element
-
seq_no is the sequence number of the element to be returned through the element pointer
-
returns < 0 if the call fails to successfully return the element
-
-
void WooFFree(WOOF *woof)
-
releases the in-memory data structure created by a call to WooFOpen()
-
There are two possibilities for the API, long-term. The first is that WooFPut() and WooFGet() are symmetric meaning that they can both be called from within a handler or outside of a namespace. From an API design perspective, this option is attractive but it promotes the use of WOOFs as random access memories from a read perspective. The second option is that WooFGet() which turns out to be necessary in some forms – see below) is restricted to be executed only outside of a handler.
The current CSPOT implementation does not restrict WooFGet() – it is symmetric with respect to WooFPut(). However, the applications will not use it to implement cross-namespace random access memory in an attempt to determine if it should be restricted.
WooFGet() turns out to be necessary in order to get application state out of the application. That is, without WooFGet() the final output of an application must reside inside a namespace (as a file – not a WOOF). To get access to this state, then, the application user must have read access to the Linux directory which implements the namespace on the machine where the output is stored. Thus, it is necessary to implement an API primitive to extract application state from the various namespaces it uses (which is WooFGet() in the current API). As mentioned above, there is a question regarding whether WooFGet() should be a full-fledged CSPOT API call (symmetric with respect to WooFPut()) or not.
One glaring omission from the current API is a lack of a way to destroy an existing WOOF. That’s not strictly true in the sense that WooFCreate() resets an existing WOOF if it already exists, thereby overwriting its original contents. However, there is currently no way to remove a WOOF permanently from a namespace.
Because WOOFs can grow and shrink (by being ``recreated'' with different sizes) the argument for a destroy API call is one regarding WOOF name conflicts within a namespace. That is, one wishes to remove a WOOF from the namespace because the name conflicts with another name. However, allowing the name to reused by a subsequent call to WooFCreate() simply delays the conflict resolution until the create. That is, removing a name really only needs to happen when another create wants to use the name.
This delayed binding of name conflict resolution is possible as long as the access control permissions are not associated with the WOOF name. If they are, then a WooFCreate() cannot resolve a name conflict since the caller may not have permission to ``take over'' the name (and thereby delete the WOOF’s contents).
It is possible to use something similar to user-group-world but then the namespace cannot be flat. That is, each user would need to be able to carve out a subtree within the namespace.
Another possibility is that namespaces carry access controls, but all WOOFs within a namespace are viewed to be part of the same trust domain. From the perspective of using messaging as an an authentication mechanism (e.g. CURVE in ZeroMQ), this option makes the most sense, but it then creates the possibility of a proliferation of namespaces.
The project must resolve this issue when determining the security model. At present, there are no authentication mechanisms or access controls implemented.