Readme

Pod is the open-source backend for Memri project.

It’s written in Rust and provides an HTTP interface for use by the clients.

See documentation on:

• How to run Pod (this document)
• Pod HTTP API
• Writing Plugins
• Running Plugins
• Triggering Plugins
• Plugins Web API
• Security
• What is a Shared Server
• How are data types defined in Schema
• Schema synchronization between clients/plugins and the Pod
• Performance Measurement
• Running local Kubernetes setup

Rust must be at least in version 1.64.0. However we encourage to use the latest version available.

There are 3 main ways to run Pod: using pre-built docker images to just run it, building it in docker, and building it locally/natively.

In all of the approaches, you will also need docker version 20.10.0 (2020-12-08) if you want to run Plugins in containers.

This is the fastest way to get Pod running on your system, however it only works for Pod versions that have already been built on our server. To run branch “dev” on commit “de929382”:

POD_VERSION="dev-de929382" docker-compose --file examples/using-prebuilt-docker.yml up

This is the least involved way to build locally. To build&run Pod inside docker:

docker-compose up --build

This is the fastest way to compile Pod from source, for example, if you’re making any changes in Pod and want to test it. It will also work on any OS and CPU architecture.

./examples/run_development.sh

It’s possible to compile POD faster by switching the linker executable from default GCC LD to Clang's LLD or mold.

1. Install alternative linker on your machine
2. Create a .cargo directory in the pod or any in it’s parent dirs (can be also in user home directory).
3. Inside .cargo create a config file with following content:

   [build]
   incremental = true
   # use mold or lld linker, at your preference
   #rustflags = ["-Clink-args=-fuse-ld=lld"]
   rustflags = ["-Clink-args=-fuse-ld=mold"]
   

POD project is divided into library and binary crates. Library contains whole business logic, whereas binary uses the lib and sets up webserver. Current webserver in use is Warp. The Warp design (use templated filters to build endpoints) makes compilation insanely long, hence it was moved to the bin crate that is meant to not change often (contrary to the lib), and (re)build process happens occasionally.

To build only library and face fast compilation times, call cargo build -p libpod

You can read about various components of the server:

• Memri project: blog.memri.io
• SQLite: sqlite.org
• Sqlcipher: zetetic.net/sqlcipher
• Rusqlite database driver: github.com/rusqlite/rusqlite
• Warp HTTP engine: github.com/seanmonstar/warp
• Rust language: rust-lang.org

Run unit + integration tests by:

cargo test

Run only integration tests by:

RUST_LOG=info cargo test --test "*" -- --nocapture

Some tests can be really slow, and are disabled by default, if you want to execute them, add feature include_slow_tests, like that:

RUST_LOG=info cargo test --test "*" --features=include_slow_tests -- --nocapture

There is also possibility to run CI tasks locally, simply run the following script: tools/local_ci.sh

It formats the code, calls cargo check, cargo audit, runs tests

Pod uses SQLite database as its storage mechanism.

When running Pod, a file named data/db/*.db3 will be created. You can use the following command to browse the database locally:

sqlcipher -cmd "PRAGMA key = \"x'yourDatabaseKey'\";" data/db/your_database_file

For example, .schema will display the current database schema.

If you want to fill the database with some example data, execute res/example_data.sql inside the database.