What is Charliecloud?
---------------------

Charliecloud is a lightweight, fully unprivileged, open source container
implementation for HPC applications. It can handle the entire container
workflow, including building images, pushing/pulling to registries, dealing
with accelerators like GPUs.

Charliecloud uses Linux user namespaces to run containers with no privileged
operations or daemons and minimal configuration changes on center resources.
This simple approach avoids most security risks while maintaining access to
the performance and functionality supercomputers already offer.

Charliecloud takes a unique approach to containerization while remaining
compatible with the broader container ecosystem where it matters. Notable
distinctions include:

  * A *fully* unprivileged end-to-end workflow with no root, no sudo, and no
    setuid/setcap helper programs, and it’s been that way since the beginning.
    Fully unprivileged run and build were both industry firsts.

  * No security boundaries within Charliecloud; we instead rely on existing,
    well-tested UNIX principles enforced by the Linux kernel and userland.

  * Clear focus on containerizing HPC applications, without technical or
    usability overhead of other use cases.

  * Root emulation to enable unprivileged container build based on a simple,
    novel, zero-consistency seccomp system call filter.

  * A layer-free build cache based on Git rather than union-mount (overlay)
    filesystems and tarballs.

  * Dockerfiles are parsed with a formal grammar.

  * SquashFS image mounts are managed internally, linking directly with
    libsquashfuse.

  * Small code base with minimal dependencies.

  * Fully independent from other container implementations, built
    using a thoughtful approach that balances UNIX traditions with technical
    novelty.

  * A spirit of openness and public service. For example, all of
    Charliecloud’s key innovations have been described in open-access papers.

The “Philosophy of Charliecloud”
--------------------------------

We develop Charliecloud under what we call “the philosophy of Charliecloud”,
which is a set of three principles to guide our work, namely:

  #. Be *transparent, not opaque*. For example, in Charliecloud, containers
     are regular files and directories rather than a novel storage format. We
     aim to be compatible with standard UNIX tools and conventions. We assume
     the user is intelligent but unfamiliar with Charliecloud internals, and
     we are honest and straightforward with them about what is going on. We
     try to be explicit, rather than implicit or magical (too much “magic”
     leads to over-helping and confusion).

  #. Be *simple, not complex*. In Charliecloud, containers are just user
     processes. We use the minimum namespaces and implement only the right
     features; Charliecloud is far smaller in lines of code than competitors.
     We use technology familiar to HPC people and minimize dependencies. We
     embrace UNIX: “make each program do one thing well”. (UNIX traditions and
     concepts have staying power for a reason.) In particular, we don’t
     re-invent, re-build, and/or re-optimize what HPC has already done.

  #. Be *unprivileged*. Writing secure code is difficult and risky, so we
     don’t do it. Charliecloud does not maintain a security boundary. Instead,
     we trust the kernel and let it enforce security, just like HPC has done
     for years. Thus, Charliecloud stays fully unprivileged, with no root, no
     sudo, and no setuid or setcap helper programs.

How do I learn more?
--------------------

* Documentation: https://hpc.github.io/charliecloud

* GitLab repository: https://gitlab.com/charliecloud/charliecloud

* Mailing lists: https://lists.charliecloud.io

* We wrote an article for USENIX's magazine *;login:* that explains in more
  detail the motivation for Charliecloud and the technology upon which it is
  based: https://www.usenix.org/publications/login/fall2017/priedhorsky

* For technical papers about Charliecloud, refer to the *Technical
  publications* section below.

Who is responsible?
-------------------

Contributors:

* Richard Berger <rberger@lanl.gov>
* Lucas Caudill <lcaudill@lanl.gov>
* Rusty Davis <rustyd@lanl.gov>
* Hunter Easterday <heasterday@lanl.gov>
* Oliver Freyermuth <o.freyermuth@googlemail.com>
* Shane Goff <rgoff@lanl.gov>
* Michael Jennings <mej@lanl.gov>
* Christoph Junghans <junghans@lanl.gov>
* Dave Love <dave.love@manchester.ac.uk>
* Jordan Ogas <jogas@lanl.gov>
* Kevin Pelzel <kpelzel@lanl.gov>
* Megan Phinney <mphinney@lanl.gov>
* Reid Priedhorsky <reidpr@lanl.gov>, co-founder and project lead
* Tim Randles <trandles@lanl.gov>, co-founder
* Benjamin "The Storm" Stormer <bstormer@lanl.gov>
* Meisam Tabriz <meisam.tabriz@mpp.mpg.de>
* Matthew Vernon <mv3@sanger.ac.uk>
* Peter Wienemann <wienemann@physik.uni-bonn.de>
* Lowell Wofford <lowell@lanl.gov>

How can I participate?
----------------------

Use our GitLab page: https://gitlab.com/charliecloud

Bug reports and feature requests should be filed as “Issues”. Questions,
comments, support requests, and everything else should use our “Discussions”.
Don't worry if you put something in the wrong place; we’ll be more than happy
to help regardless.

We also have mailing lists for users and developers:
https://lists.charliecloud.io

Patches are much appreciated on the software itself as well as documentation.
Optionally, please include in your first patch a credit for yourself in the
list above.

We are friendly and welcoming of diversity on all dimensions.

Technical publications
----------------------

If Charliecloud helped your research, or it was useful to you in any other
context where bibliographic citations are appropriate, please cite the
following open-access paper:

  Reid Priedhorsky and Tim Randles. “Charliecloud: Unprivileged containers for
  user-defined software stacks in HPC”, 2017. In *Proc. Supercomputing*.
  DOI: `10.1145/3126908.3126925 <https://doi.org/10.1145/3126908.3126925>`_.

*Note:* This paper contains out-of-date number for the size of Charliecloud’s
code. Please instead use the current number in the FAQ.

Other publications:

* We compared the performance of three HPC-specific container technologies
  against bare metal and found no concerns about performance degradation.

    Alfred Torrez, Tim Randles, and Reid Priedhorsky. “HPC container runtimes
    have minimal or no performance impact”, 2019. In *Proc. CANOPIE HPC
    Workshop @ SC*. DOI: `10.1109/CANOPIE-HPC49598.2019.00010
    <https://ieeexplore.ieee.org/document/8950978>`_.

* We demonstrated how low-privilege containers solve increasing demand for
  software flexibility in HPC centers.

    Reid Priedhorsky, R. Shane Canon, Timothy Randles, and Andrew J. Younge.
    “Minimizing privilege for building HPC containers”, 2021. In *Proc.
    Supercomputing*. DOI: `10.1145/3458817.3476187
    <https://doi.org/10.1145/3458817.3476187>`_.

* Charliecloud’s build cache performs competitively with the standard
  many-layered union filesystem approach and has structural advantages
  including a better diff format, lower cache overhead, and better file
  de-duplication.

    Reid Priedhorsky, Jordan Ogas, Claude H. (Rusty) Davis IV, Z. Noah
    Hounshel, Ashlyn Lee, Benjamin Stormer, and R. Shane Goff. "Charliecloud’s
    layer-free, Git-based container build cache", 2023. In *Proc. CANOPIE HPC
    Workshop @ SC*. DOI: `10.1145/3624062.3624585
    <https://doi.org/10.1145/3624062.3624585>`_.

* We introduced a new root emulation approach that turns privileged operations
  requested by package managers into no-ops. Surprisingly, package managers
  don’t notice and the build works fine.

    Reid Priedhorsky, Michael Jennings, Megan Phinney. “Zero-consistency root
    emulation for unprivileged container image build”, 2024. In *Proc. CANOPIE
    HPC Workshop @ SC*. DOI: `10.1109/SCW63240.2024.00023
    <https://doi.org/10.1109/SCW63240.2024.00023>`_.

License and intellectual property
---------------------------------

Charliecloud is open source software; you can redistribute it and/or modify it
under the terms of the Apache License, Version 2.0. A copy is included in file
LICENSE. You may not use this software except in compliance with the license.

Copyrights and patents are retained by contributors. No copyright assignment
is required to contribute to Charliecloud. Please use the Git history to
determine copyright for any specific contribution.

Notable copyright holders include:

* Triad National Security, LLC (contributions by LANL employees)


..  LocalWords:  USENIX's CNA Meisam figshare
