doc: Add documentation for source mirroring

3 files changed, 85 insertions, 0 deletions

diff --git a/buildstream/source.py b/buildstream/source.py
index f4b81ee7f..9866c99e9 100644
--- a/buildstream/source.py
+++ b/buildstream/source.py
@@ -65,6 +65,33 @@ these methods are mandatory to implement.
 
   **Optional**: If left unimplemented, this will default to calling
   :func:`Source.stage() <buildstream.source.Source.stage>`
+
+* :func:`Source.get_source_fetchers() <buildstream.source.Source.get_source_fetchers>`
+
+  Get the objects that are used for fetching.
+
+  **Optional**: This only needs to be implemented for sources that need to
+  download from multiple URLs while fetching (e.g. a git repo and its
+  submodules). For details on how to define a SourceFetcher, see
+  :ref:`SourceFetcher <core_source_fetcher>`.
+
+
+.. _core_source_fetcher:
+
+SourceFetcher - Object for fetching individual URLs
+===================================================
+
+
+Abstract Methods
+----------------
+SourceFetchers expose the following abstract methods. Unless explicitly
+mentioned, these methods are mandatory to implement.
+
+* :func:`SourceFetcher.fetch() <buildstream.source.SourceFetcher.fetch>`
+
+  Fetches the URL associated with this SourceFetcher, optionally taking an
+  alias override.
+
 """
 
 import os
diff --git a/doc/source/format_project.rst b/doc/source/format_project.rst
index 36de41a55..cf0970f77 100644
--- a/doc/source/format_project.rst
+++ b/doc/source/format_project.rst
@@ -198,6 +198,43 @@ You can also specify a list of caches here; earlier entries in the list
 will have higher priority than later ones.
 
 
+.. _project_essentials_mirrors:
+
+Mirrors
+~~~~~~~
+A list of mirrors can be defined that couple a location to a mapping of aliases to a
+list of URIs, e.g.
+
+.. code:: yaml
+
+  mirrors:
+  - name: middle-earth
+    aliases:
+      foo:
+      - http://www.middle-earth.com/foo/1
+      - http://www.middle-earth.com/foo/2
+      bar:
+      - http://www.middle-earth.com/bar/1
+      - http://www.middle-earth.com/bar/2
+  - name: oz
+    aliases:
+      foo:
+      - http://www.oz.com/foo
+      bar:
+      - http://www.oz.com/bar
+
+The order that the mirrors (and the URIs therein) are consulted is in the order
+they are defined when fetching, and in reverse-order when tracking.
+
+A default mirror to consult first can be defined via
+:ref:`user config <config_default_mirror>`, or the command-line argument
+:ref:`--default-mirror <invoking_bst>`.
+
+.. note::
+
+   The ``mirrors`` field is available since :ref:`format version 11 <project_format_version>`
+
+
 .. _project_plugins:
 
 External plugins
diff --git a/doc/source/using_config.rst b/doc/source/using_config.rst
index d02aaf3d4..c707cd04b 100644
--- a/doc/source/using_config.rst
+++ b/doc/source/using_config.rst
@@ -89,6 +89,27 @@ modifying some low level component.
    the ``--strict`` and ``--no-strict`` command line options.
 
 
+.. _config_default_mirror:
+
+Default Mirror
+~~~~~~~~~~~~~~
+When using :ref:`mirrors <project_essentials_mirrors>`, a default mirror can
+be defined to be fetched first.
+The default mirror is defined by its name, e.g.
+
+.. code:: yaml
+
+  projects:
+    project-name:
+      default-mirror: oz
+
+
+.. note::
+
+   It is possible to override this at invocation time using the
+   ``--default-mirror`` command-line option.
+
+
 Default configuration
 ---------------------
 The default BuildStream configuration is specified here for reference: