Update documentation to explain split caches

2 files changed, 59 insertions, 0 deletions

diff --git a/doc/source/format_project.rst b/doc/source/format_project.rst
index d33a4974f..c4988527a 100644
--- a/doc/source/format_project.rst
+++ b/doc/source/format_project.rst
@@ -225,6 +225,44 @@ and keys, please see: :ref:`Key pair for the server <server_authentication>`.
    new server API. As a result newer buildstream clients won't work with older
    servers.
 
+.. _project_essentials_split_artifacts:
+
+Split cache servers
+~~~~~~~~~~~~~~~~~~~
+
+Should you need to configure an artifact cache to work with a CAS
+server that does not support BuildStream's artifact format, you can
+"split" that cache and run an artifacts-only server separately. The
+format for that is as such:
+
+.. code:: yaml
+
+  #
+  # Artifacts
+  #
+  artifacts:
+    # A remote cache from which to download prebuilt artifacts
+    - url: https://storage.foo.com:11001
+      server-cert: server.crt
+      # "storage" remotes store the artifact contents only - this can
+      # be a normal CAS implementation such as those provided by
+      # Buildbarn or Bazel Buildfarm
+      type: storage
+    - url: https://index.foo.com:11001
+      server-cert: server.crt
+      # "index" remotes store only artifact metadata. This is
+      # currently only provided by the bst-artifact-server and BuildGrid
+      type: index
+    # A remote cache from which to upload/download built/prebuilt artifacts
+    - url: https://foo.com:11002
+      push: true
+      server-cert: server.crt
+      client-cert: client.crt
+      client-key: client.key
+      # Caches that support both can omit the type, or set it to "both" -
+      # currently, also only supported by bst-artifact-server and BuildGrid
+      type: both
+
 .. _project_source_cache:
 
 Source cache server
@@ -247,6 +285,11 @@ Exactly the same as artifact servers, source cache servers can be specified.
       client-cert: client.crt
       client-key: client.key
 
+.. note::
+
+   Source caches also support "splitting" like :ref:`artifact servers
+   <project_essentials_split_artifacts>`.
+
 .. _project_remote_execution:
 
 Remote execution
diff --git a/doc/source/using_configuring_cache_server.rst b/doc/source/using_configuring_cache_server.rst
index 856046f35..d31a6661c 100644
--- a/doc/source/using_configuring_cache_server.rst
+++ b/doc/source/using_configuring_cache_server.rst
@@ -159,6 +159,22 @@ Instance with push and requiring client authentication:
 
     bst-artifact-server --port 11002 --server-key server.key --server-cert server.crt --client-certs authorized.crt --enable-push /home/artifacts/artifacts
 
+.. note::
+
+   BuildStream's artifact cache is an extension of `Google's Remote
+   Execution CAS server
+   <https://github.com/bazelbuild/remote-apis/>`_.
+
+   Sometimes, when using Remote Execution, it is useful to run
+   BuildStream with just a basic CAS server, without using the
+   artifact extensions, but BuildStream still needs to store these to
+   work correctly.
+
+   For this scenario, you can add the `--index-only` flag to the above
+   commands, and configure BuildStream to store artifact metadata and
+   files in a separate caches (e.g. bst-artifact-server and Buildbarn)
+   using :ref:`"types" <project_essentials_split_artifacts>`.
+
 Managing the cache with systemd
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~