## What is buildbox-run-oci?

`buildbox-run-oci` is a buildbox-runner implementation that attempts to perform build executions within a container.
It supports sandboxing by spawning a container using [`runc`](https://github.com/opencontainers/runc) for build environment isolation.

## Buildbox-Run-OCI and Container Images

Container images are accepted through the `container-image` platform property, which requires the argument to be an
image ID in a format including its unique sha256 hash, preventing caching errors due to image changes in
the container registry.

`Buildbox-run-oci` pulls the image from its associated container registry with the image ID, and then exports the image file system.
This tar file is extracted, and the unique digest of the file system is calculated and uploaded to CAS. Next, like other
runner implementations, `buildbox-run-oci` merges the file system digest with the other digests of the build, and
finally performs the build execution using `runc`.

## Command line arguments
```
usage: ./buildbox-run-oci [OPTIONS]
    --action=PATH               Path to read input Action from
    --action-result=PATH        Path to write output ActionResult to
    --log-level=LEVEL           (default: info) Log verbosity: trace/debug/info/warning/error
    --verbose                   Set log level to debug
    --log-directory=DIR         Write logs to this directory with filenames:
                                buildbox-run-userchroot.<hostname>.<user name>.log.<severity level>.<date>.<time>.<pid>
    --disable-localcas          Do not use LocalCAS protocol methods
    --workspace-path=PATH       Location on disk which runner will use as root when executing jobs
    --stdout-file=FILE          File to redirect the command's stdout to
    --stderr-file=FILE          File to redirect the command's stderr to
    --no-logs-capture           Do not capture and upload the contents written to stdout and stderr
    --capabilities              Print capabilities supported by this runner
    --validate-parameters       Only check whether all the required parameters are being passed and that no
                                unknown options are given. Exits with a status code containing the result (0 if successful).
    --remote=URL                URL for CAS service
    --instance=NAME             Name of the CAS instance
    --server-cert=PATH          Public server certificate for TLS (PEM-encoded)
    --client-key=PATH           Private client key for TLS (PEM-encoded)
    --client-cert=PATH          Public client certificate for TLS (PEM-encoded)
    --access-token=PATH         Access Token for authentication (e.g. JWT, OAuth access token, etc), will be included as an HTTP Authorization bearer token.
    --token-reload-interval=MINUTES Time to wait before refreshing access token from disk again. The following suffixes can be optionally specified: M (minutes), H (hours). Value defaults to minutes if suffix not specified.
    --googleapi-auth            Use GoogleAPIAuth when this flag is set.
    --retry-limit=INT           Number of times to retry on grpc errors
    --retry-on-code             Which gRPC status code(s) to retry on
    --retry-delay=MILLISECONDS  How long to wait before the first grpc retry

    --memory-limit=UINT64       Default memory limit for the container in bytes.
    --cpu-time=UINT64           Default maximum CPU Time for the container in seconds.
```

## Platform Properties and Resource Control
[See properties.md](update-documentation/properties.md?ref_type=heads)
Currently, `buildbox-run-oci` supports platform properties for memory size, CPU time, UID, and GID.

## Testing

### Unit Tests (as of 8/18/23)
Unit tests can be found in `test/oci.t.cpp` and are called in the CI pipeline. They currently test getBundleDigest(),
prepareFilesystem(), fetchDockerImage(), and extractFileSystemFromImage().

### Integration Test 1:

**Purpose:**
This integration test verifies that `buildbox-run-oci` can be invoked correctly by `buildbox-worker` from BuildGrid.

**Details:**
Starts a BuildGrid instance, creates the worker, and invokes buildbox-run-oci.

**Running the test:**
```bash
docker-compose up --build
# If the above test finishes with exit code 0, the test has passed. End the test and shutdown the containers by
# doing Ctrl+C and running:
docker-compose down
```

### Integration Test 2 (executed in the CI pipeline):

**Purpose:**
This integration test checks whether `buildbox-run-oci` correctly pulls and extracts the file system of a given
container image, and then if it successfully uploads the file system's digest to `casd`.

**Details:**
The digests for the action and input have already been created by the script `create_action.py`. The input used is a
simple text file, `hello.txt`. The action arguments contain a platform property that specifies a minimal container
image (as a sha256 digest). The test first spawn a BuildGrid instance (skips creating the worker) and runs `casd` and
`buildbox-run-oci`, giving the action digest as an argument.

**Running this test:**
```bash
# To re-run the Python script (creates an Action digest based on the arguments in the script and an input digest
# for the file system within the mock-data/inputs directory):
./mock-data/create.sh

# To run the integration test that runs the BuildGrid instance and calls buildbox-run-oci:
./compose-runner-tests/docker_setup.sh

# If the above test finishes with exit code 0, the test has passed
```

### Integration Test 3 (executed in the CI pipeline):

**Purpose:**
This integration test checks whether `buildbox-run-oci` successfully handles and passes resource limits
to runc when executing a given container image.

**Details:**
The digests for the action and input have already been created by the script `create_action2.py`. The input used is a
simple text file, `hello.txt`. The action arguments contain a platform property that specifies a minimal container
image (as a sha256 digest), as well as platform properties for the memory-limit and cpu-time. The test first spawn a
BuildGrid instance (skips creating the worker) and runs `casd` and `buildbox-run-oci`, giving the action digest as an
argument.

**Running this test:**
```bash
# To re-run the Python script (creates an Action digest based on the arguments in the script and an input digest
# for the file system within the mock-data/inputs directory):
./mock-data/create2.sh

# To run the integration test that runs the BuildGrid instance and calls buildbox-run-oci:
./compose-resource-tests/docker_setup_resource.sh

# If the above test finishes with exit code 0, the test has passed
```

## Benchmarking
This package has roughly the same performance as `buildbox-run-userchroot`. Performance was measured by `buildbox-run-userchroot` for 5 times in the respective docker-compose environments and taking the averages of the build times.
```
buildbox-run-userchroot     44.64s
buildbox-run-oci            42.47s
```

## Future Work

### 1. Additional Resource Control (Short-term)
There are many options that may be useful for the user to be able to specify for execution.
More research should be done to determine which `runc` arguments should be supported.
Examples: additionalGIDs, rlimits, umask, etc.

For a complete list of options that `runc` accepts, see [runtime-spec](https://github.com/opencontainers/runtime-spec/blob/main/config.md#posix-process)

### 2. Generic Integration Testing Setup
Currently, the integration tests in `buildbox-run-oci` each have their own set of Dockerfiles, .sh executables, Python scripts, etc.
As more integration tests are added in the future, this testing setup will become very redundant.
The integration testing setup should be refactored to minimize these files, and to create an interface that allows
new tests to be easily added in the future.

### 3. Remove Buildbox-Run-OCI's CASD Dependence (Short-term)
As of right now, multiple parts of `buildbox-run-oci` require CASD to be running, since it uses CASD-specific
functions and objects, such as fetchTree() and LocalCasdStagedDirectory. Although making use of CASD is a useful
optimization, `buildbox-run-oci` should be able to function independently of a CASD instance, and instead use the remote
CAS alone in this scenario. So, `buildbox-run-oci` should be refactored to remove the dependence on CASD.

### 4. Integrate the Remote Asset Server (Long-term)
Integrating the Remote Asset Server and shifting some of the functionality of `buildbox-run-oci` to it is a long-term goal.

The Remote Asset Server implements the Remote Asset API (REAPI) in order to associate semantic names with
CAS assets (as opposed to only having access to the assets through their unique digests), so integrating the Remote Asset
Server would improve user-friendliness of specifying these assets. In addition, the Remote Asset Server could
potentially take on some of the logic of `buildbox-run-oci`, such as doing the work of pulling a container image from the
registry and computing the file system's digest. In this way, the Remote Asset Server would also provide better access to
retrieving file system digests, since it would  be able to be directly accessed from any part of the system, possibly even
being called from the client.
