HTTP API

About

This documentation is part of Pod.

HTTP API is the interface that Pod provides to store and access user data. This document explains the data types that Pod can store, and current API provided for that.

API Authentication & Credentials

Accessing your Pod requires authentication. This is typically the owner_key and auth keys in the request, see below.

Owner key

The owner_key tells the Pod who you claim to be. It is the full client’s public ED25519 key (see also wikipedia). It should be encoded as 64-character hex string.
Pods configured with “ANY” as their owner will accept requests from any owner_key. For debugging, you can fake owner_key by providing any 64 hex characters, e.g. all zeroes.
Note that for now, we don’t really use any cryptography for the owner_key, but in the future we probably will.

Auth json

The auth value is used by Pod to validate that this is you, and to temporarily decrypt parts of your data required to serve your request. This key is immediately lost by Pod when the HTTP request is closed.

There are two types of auth keys,

  • a plugin key (that plugins get from Pod and will use to authorize in Pod),
  • and a client key (that Memri clients will use, e.g. a phone or desktop app).

Auth json of “client” type

If you’re a memri client (mobile, desktop), you keep the secret locally, in an encrypted/protected form.

Your auth key should be of the form:

{
  "type": "ClientAuth",
  "databaseKey": "*********"
}

The databaseKey is a 64-character hex string to be used for validation and encryption. You can use any cryptographically secure PRNG to generate the key. It should be generated once and stored securely, e.g. in a hardware-backed keystore, or secure enclave, etc. The key will internally be used with sqlcipher.

For testing (and for testing only!) you can use any 64 hex characters, e.g. all zeroes.

Auth json of “plugin” type

Plugins receive authentication information when they are started by the Pod.

Their authentication key should be:

{
  "type": "PluginAuth",
  "data": $POD_AUTH_JSON
}

Where $POD_AUTH_JSON is the JSON set by Pod, see the link above again.

As a plugin, you only have access to a subset of Items and their properties, as configured by the user.

For testing/debugging, if you run your own Pod locally, you can use client auth (below) by just making up a fake user and a fake database key.

Schema API

Schema defines what types can be stored in Pod. You can read more about the Schema itself here.

In order to make changes to the schema, insert items of a specific structure in the Pod. The following properties are expected:

{
  "type": "ItemPropertySchema", /* exactly this, and nothing else */
  "itemType": "YourType", /* e.g. "Person" */
  "propertyName": "yourProperty", /* e.g. "age" */
  "valueType": "Integer", /* one of valid Schema types, see Schema docs*/
}

Note that you cannot both change the Schema and refer to the new Schema in one bulk request, so if you want changes to the Schema to happen first, split updates to the Schema into a separate request. (This constraint might be lifted in the future.)

If the new Schema item conflicts with already existing Schema, a failure will be returned.

If the new Schema item duplicates already existing Schema, the new item will be silently ignored and not inserted into the database.

If the new Schema item creates (valid) properties that have not yet been defined, the new Schema will be added permanently.

⚠️ UNSTABLE: We might require more properties to be defined here in the future, e.g. to what Plugin does the Schema addition belong to.

Items API

GET /version

Get version of the Pod: the git commit and cargo version that it was built from.

POST /v4/$owner_key/get_item

{
  "auth": $auth_json,
  "payload": "$id"
}

Get a single item by its id.

Returns an empty array if an item is not found, or an array with 1 item if item exists.

POST /v4/$owner_key/create_item

{
  "auth": $auth_json,
  "payload": { "type": "...", ... }
}

Create a single item.

  • id if set, will be taken as new item’s id; otherwise, a new id will be generated by Pod. It’s recommended not to set this property unless you’ll also need to reference this item for e.g. edges to/from this item. If you need to create a random unique id, it’s recommended to use 32 random hex characters. Example of generating that in shell: hexdump --no-squeezing -e '/1 "%x"' -n 16 /dev/urandom. If you want to incorporate an external id, you can use e.g. "myPluginName-12342" where 12342 is an example of an external id.
  • type sets the type of the item, cannot ever be changed
  • dateCreated if not present, will be set by the backend
  • dateModified if not present, will be set by the backend
  • dateServerModified will be created by the backend
  • deleted and all other properties will be added to the item’s properties

Returns an error if such id already exist in the DB. Returns an error if the new item doesn’t conform to the Schema. Returns id of the created item if the operation is successful.

POST /v4/$owner_key/update_item

{
  "auth": $auth_json,
  "payload": { "id": "$id", ... }
}

Update a single item.

  • id from the input json will be taken as the item’s id
  • type from the input json will be ignored
  • dateCreated from the input json will be ignored
  • dateModified if not present, will be updated by the backend
  • dateServerModified will be created by the backend
  • properties with a value of null will be erased from the item
  • any other properties will be updated

Returns an empty object if the operation is successful.

POST /v4/$owner_key/get_edges

{
  "auth": $auth_json,
  "payload": {
    "item": "$id",
    "direction": "Outgoing", // Either "Outgoing" (default) or "Incoming" edges
    "expandItems": true // Whether to expand the target/source items for each edge
  }
}

Get all edges for a single item.

Example output:

[
  {
    "name": "friend",
    "item": {
      "id": "00000000",
      // all other fields if "expandItems" is requested
    }
  },
  {
    "name": "friend",
    "item": {
      "id": ".........."
    }
  },
  ...
]

Returns an array empty array if either the element does not exist or if it has no outgoing edges.

⚠ WARNING: this endpoint is unstable, and it might be deprecated and removed in next releases of Pod.️

POST /v4/$owner_key/create_edge

{
  "auth": $auth_json,
  "payload": {
    "_source": "$source_id", /* Source item id */
    "_target": "$target_id", /* Target item id */
    "_name": "$edge_name",  /* Text name. For example: "entry" (in a list), "friend" (for a Person), etc */
    "_self": "$self_id", /* Optional field to specify the "self" id to link to, see below */
    // Edge properties will be supported in the future
  }
}

Create a single edge from an already existing item to another already existing item. (Reminder: edges are always directed.)

If the _self field is specified in the request, the edge will be wired to an already existing item. For Plugins this functionality is currently undocumented and advised not to be used. For Clients, see “edges” table definition.

An error will be returned if the edge already exists. An error will be returned if source item or target item do not exist. Returns id of the created edge if the operation is successful (for now, nothing is possible to do with the id, but in the future there will be more options).

Returns an error if such id already exist in the DB. Returns an error if the new item doesn’t conform to the Schema. Returns id of the created item if the operation is successful.

POST /v4/$owner_key/delete_item

{
  "auth": $auth_json,
  "payload": "$id"
}

Mark an item as deleted:

  • Set deleted flag to true
  • Update dateModified (server’s time is taken)
  • Update dateServerModified

POST /v4/$owner_key/search

{
  "auth": $auth_json,
  "payload": {
    "id": "1d22ba3458147f73b8aa3d68744e3f2f", // ID filter
    "type": "Label", // type filter
    "dateServerModified>=": 1234567890, // date filter
    "dateServerModified<": 1234567890,
    "deleted": false, // deleted filter
    "_sortOrder": "Asc", // sort by server modification date, either "Asc" (by default) or "Desc"
    "_limit": 100, // minimum number of items to return, see below
    "_returnProperties": ["propertyA", "propertyB", ...],
    "[[edges]]": {}, // include all forward edges in the response
    "~[[edges]]": {}, // include all backward edges in the response
  }
}

Search items by their properties, all fields above are optional.

The endpoint will return an array of items which have all the properties above.

As a first step of the 2021-03 Pod rewrite, only the above properties are supported. In the future, any item properties will be available.

If _limit is specified, response will include the first _limit number of items from the database, plus also all other items with exactly the same dateServerModified as the last one.

If _returnProperties is specified, then only the specified properties and the relevant edges information will be returned.

The properties of "[[edges]]" and "~[[edges]]" are “magic constants” for now. This will change in the future when we’ll continuously expand our API to make it more intuitive and easier to use, especially around edges and recursive data fetching. For now only literally those two properties are supported, and all edges are returned without filtering.

POST /v4/$owner_key/bulk

{
  "auth": $auth_json,
  "payload": {
    "createItems": [
      { "id": "something-12345", "type": "Person", /* ... */ },
      {}, // same structure as create_item endpoint above
      // ...
    ],
    "updateItems": [
      { "id": "something-67899", /* ... */ },
      {}, // same structure as update_item endpoint above
      // ...
    ],
    "deleteItems": [ "$id", "$id", "$id", /* ... */ ],
    "createEdges": [ 
      {}, // same structure as create_edge endpoint above
      {},
      // ...
    ],
    "search": [
      { /* same structure as in /search API */ }
    ]
  }
}

Perform a bulk of operations in one request. The endpoint is “atomic”, meaning that either all the operations succeed, or the database won’t be changed at all.

Returns an empty object if the operation is successful.

Plugins API

Plugins help getting data into your Pod and enriching it. Plugins must be authorized and started by the user.

In the future we plan to have configurable permissions and automatic rules to start plugins. For now, plugins are started when items of a particular types are inserted into Pod.

Items of the following structure need to be inserted to Pod to start a plugin:

{
  "type": "PluginRun", /* exactly this, and nothing else */
  "containerImage": "your_docker_image:optional_version", /* any locally available docker container if you run Pod locally */
  "targetItemId": "the item id this plugin needs to run against, NOT this item's id",
  /* Any other optional fields that your plugin might need... */
}

See Plugins on how plugins are started exactly.

⚠️ UNSTABLE: We might require more properties for Plugins to start in the future, e.g. permission limitation.

Email API

Email API is a temporary functionality of Pod that allows it to send emails.

POST /v4/$owner_key/send_email

To send an item to a particular user via the Pod, make the following API request:

{
  "auth": $auth_json,
  "payload": {
    "to": "address@example.com",
    "subject": "Example subject",
    "body": "Example message body."
  }
}

This will make Pod send that single email and return a success if the email was accepted by the configured SMTP server.

No retries will be made if SMTP connection did not succeed.

No changes will be made to the database.

⚠️ UNSTABLE: the API is unstable and might be removed without prior notice.

File API

POST /v4/$owner_key/upload_file/$databaseKey/$sha256hashOfTheFile

RAW-FILE-BINARY

Upload a file into Pod and verify it’s sha256. owner_key, database key and sha256 are all hex-encoded.

If a file with a given sha256 has already being uploaded, the request will fail. If the provided sha256 doesn’t match the hash of the contents, the request will fail. If no item with this sha256 exists in the database, Pod wouldn’t be able to store cryptographic information about the file, and the request will also fail.

If sha256 matches, the file has not yet been uploaded to Pod and if an item with such sha256 already exists in DB, Pod will accept the file and store it. The properties nonce and key will be updated for this item.

POST /v4/$owner_key/upload_file_b/$auth/$sha256hashOfTheFile

RAW-FILE-BINARY

The auth should be a URL-encoded Authorization JSON (see above).

Otherwise, this API endpoint behaves exactly like upload_file above.

POST /v4/$owner_key/get_file

{
  "auth": $auth_json,
  "payload": {
    "sha256": "$sha256"
  }
}

Get a file by its sha256 hash. If the file does not yet exist in Pod, a 404 NOT FOUND error will be returned.