Computation Payload Manifest

Computation Payload Manifest allows a requestor to define an application package Payload and allows to set constraints on the computations performed on a provider node.

A manifest can be configured in yapapi.

The provider node operator controls what computations can be performed by:

• [Importing certificates] used to sign app authors' certificates into the provider's keystore (which allows the provider agent to verify manifest signatures)
• Adding domain patterns to Provider's domain whitelist (which makes the manifest [signature] optional).

The manifest can be configured as a yapapi.payload.vm.manifest function parameter together with an optional manifest signature and the [App author's certificate].

Computation Payload Manifest must follow a specific JSON Schema (Documentation)

Manifests can be verified using jsonschema library:

wget https://github.com/golemfactory/yagna-docs/blob/master/requestor-tutorials/vm-runtime/computation-payload-manifest.schema.json
pip install jsonschema
jsonschema --instance manifest.json computation-payload-manifest.schema.json

Computation Payload Manifest must contain at least one Payload object.

Payload definition allows to define GVMI images used by Application and supported architecture.

Simple Computation Payload Manifest with Payload definition:

{
  "version": "0.1.0",
  "createdAt": "2022-09-15T00:00:00.000000Z",
  "expiresAt": "2100-01-01T00:00:00.000000Z",
  "payload": [
    {
      "platform": {
        "arch": "x86_64",
        "os": "linux"
      },
      "urls": [
        "http://girepo.dev.golem.network:8000/docker-golem-hello-world-latest-779758b432.gvmi"
      ],
      "hash": "sha3:d646d7b93083d817846c2ae5c62c72ca0507782385a2e29291a3d376"
    }
  ]
}

Computation Payload Manifest can contain Computation Manifests object.

With a Computation Manifest object, requestor constrains themself to a certain set of allowed actions, to be negotiated with and approved by a provider.

Requestors' actions will be verified against the Manifest during computation.

Supported Computation Manifest constraints:

• compManifest.script

  Defines a set of allowed ExeScript commands and applies constraints to their arguments.

  • compManifest.script.commands

    compManifest.script.commands : List[Script] specifies a curated list of commands in a form of:

    • UTF-8 encoded JSON strings

      Command context (e.g. env) or argument matching mode needs to be specified for a command.

      Example:

      [
        "{
          \"run\": {
            \"args\": \"/bin/date -R\",
            \"env\": {
              \"MYVAR\": \"42\",
              \"match\": \"strict\"
            }
          }
        }"
      ]

    • UTF-8 encoded strings

      No command context or matching mode needs to be specified.

      Example:

      ["run /bin/cat /etc/motd", "run /bin/date -R"]

    • Mix of both

    Commands deploy, start and terminate are always allowed. These values become the default if no compManifest.script.commands property has been set, but the compManifest object is present.

  • compManifest.script.match

    compManifest.script.match : String selects a default way of comparing command arguments stated in the manifest and the ones received in the ExeScript, unless stated otherwise in a command JSON object.

    The match property could be one of:

    • strict: byte-to-byte argument equality (default)

    • regex: treat arguments as regular expressions

      Syntax: Perl-compatible regular expressions (UTF-8 Unicode mode), w/o the support for a look around and backreferences (among others).

• compManifest.net

  compManifest.net : Object applies constraints to networking.

  • compManifest.net.inet.out

    compManifest.net.inet.out : Object outgoing requests to the public Internet network constraints.

    • compManifest.net.inet.out.protocols

      compManifest.net.inet.out.protocols : List[String] list of allowed outbound protocols. Currently fixed at ["http", "https"].

    • compManifest.net.inet.out.urls

      compManifest.net.inet.out.urls : List[String] list of allowed external URLs that outbound requests can be sent to. E.g. ["https://api.some-public-service.com", "https://some-other-service.com/api/resource"]

{
  "version": "0.1.0",
  "createdAt": "2022-09-15T00:00:00.000000Z",
  "expiresAt": "2100-01-01T00:00:00.000000Z",
  "payload": [
    {
      "urls": [
        "http://yacn2.dev.golem.network:8000/docker-golem-script-curl-latest-d75268e752.gvmi"
      ],
      "hash": "sha3:e5f5ddfd649525dbe25d93d9ed51d1bdd0849933d9a5720adb4b5810"
    }
  ],
  "compManifest": {
    "version": "0.1.0",
    "script": {
      "commands": ["run curl.*", "transfer .*/output.txt"],
      "match": "regex"
    },
    "net": {
      "inet": {
        "out": {
          "protocols": ["https"],
          "urls": ["https://api.some-public-service.com"]
        }
      }
    }
  }
}

App author's certificate gets sent in a demand together with a Computation Payload Manifest and its signature. The certificate is used to verify the signature. In order to verify the signature, a Provider first needs to verify the incoming App author's certificate. To do so, it has to have a certificate that's used to sign the App author's certificate [imported] into its keystore (together with every intermediate certificate in the chain).

Signature allows a Provider to verify the content of an incoming Computation Payload Manifest.

It can be generated using the openssl tool and a private key related to App author's certificate signed by a certificate that's trusted by Providers.

A signature needs to be generated from the content of the Computation Payload Manifest encoded in base64:

openssl dgst -sha256 -sign author.key -out manifest.json.base64.sign.sha256 manifest.json.base64
# both Signature and App Author Certificate need to be sent in base64 encoded form
base64 manifest.json.base64.sign.sha256 --wrap=0 > manifest.json.base64.sign.sha256.base64
base64 author.crt.pem --wrap=0 > author.crt.pem.base64

A basic example showing the generation of a self-signed root CA certificate to then sign the App author's certificate, and afterwards importing a generated root CA certificate into the Provider's keystore.

Create openssl-ca.conf for CA certificate

[ req ]
distinguished_name  = req_dn
x509_extensions     = v3_ext

[ req_dn ]
organizationName  = Organization Name (company name)
commonName        = Common Name (your name, or app name)
emailAddress      = Email Address (support email address)

[ v3_ext ]
basicConstraints = CA:true

[ ca ]
default_ca      = CA_default

[ CA_default ]
database  = index.txt
serial    = serial.txt
policy    = policy_default

[ policy_default ]
organizationName  = match
commonName        = supplied
emailAddress      = supplied

Then prepare referenced in config files:

touch index.txt index.txt.attr
echo '1000' > serial.txt

Then generate the CA certificate and key pair:

openssl req -new -newkey rsa:2048 -days 360 -nodes -x509 -sha256 -keyout ca.key.pem -out ca.crt.pem -config openssl-ca.conf

Create openssl.conf for the App author's certificate:

[ req ]
distinguished_name  = req_dn
x509_extensions     = v3_ext

[ req_dn ]
organizationName  = Organization Name (company name)
commonName        = Common Name (your name, or app name)
emailAddress      = Email Address (support email address)

[ v3_ext ]
basicConstraints = CA:true

Then generate App author's certificate Signing Request (use same organizationName):

openssl req -new -newkey rsa:2048 -sha256 -keyout author.key.pem -out author.csr.pem -config openssl.conf

Finally, generate App author's certificate using CSR and CA certificate:

openssl x509 -req -in author.csr.pem -CA ca.crt.pem -CAkey ca.key.pem -CAcreateserial -out author.crt.pem

To set the rule to accept payloads with unrestricted outbound access signed by authors who holds certs from a trusted ca, use a the following command:

ya-provider rule set outbound audited-payload import-cert ca.crt.pem --mode all