summaryrefslogtreecommitdiff
path: root/doc/source/main_install.rst
blob: c715b818f3b85cf82bb277b41965e3e6c56d20c3 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
Installing
==========

Until BuildStream is available in your distro, you may need to install
it yourself from source. The necessary steps are:

* :ref:`Install dependencies<install-dependencies>`
* :ref:`Install BuildBox<install-buildbox>`
* :ref:`Install BuildStream<install-buildstream>` (from a git checkout, or from PyPi)
* :ref:`Update PATH<post-install>`

Alternatively, BuildStream can be run in :ref:`a container<install-container>`.


.. _install-dependencies:

Installing Dependencies
-----------------------

Before installing BuildStream from source, it is necessary to first install
the system dependencies. Below are some linux distribution specific instructions
for installing these dependencies.

BuildStream requires the following base system requirements:

- python3 >= 3.6
- bubblewrap >= 0.1.2
- fuse2
- pip
- :ref:`buildbox-casd<install-buildbox>`

BuildStream also depends on the host tools for the :mod:`Source <buildstream.source>` plugins.
Refer to the respective :ref:`source plugin <plugins_sources>` documentation for host tool
requirements of specific plugins.

The default plugins with extra host dependencies are:

- bzr
- deb
- git
- ostree
- patch
- pip
- tar


Arch Linux
~~~~~~~~~~
Install the dependencies with::


    sudo pacman -S python fuse2 bubblewrap python-pip


For the default plugins::


    sudo pacman -S bzr git lzip patch ostree python-gobject

The package *python-arpy* is required by the deb source plugin. This is not
obtainable via ``pacman``, you must get
`python-arpy from AUR <https://aur.archlinux.org/packages/python-arpy/>`_

To install::


    wget https://aur.archlinux.org/cgit/aur.git/snapshot/python-arpy.tar.gz
    tar -xvf python-arpy.tar.gz
    cd python-arpy
    makepkg -si


Debian
~~~~~~
Install the dependencies with::


    sudo apt-get install \
        python3 fuse bubblewrap \
        python3-pip python3-dev

For the default plugins:

Stretch
+++++++
With stretch, you first need to ensure that you have the backports repository
setup as described `here <https://backports.debian.org/Instructions/>`_

By adding the following line to your sources.list::


    deb http://ftp.debian.org/debian stretch-backports main

And then running::


    sudo apt update

At this point you should be able to get the system requirements for the default plugins with::


    sudo apt install \
        bzr git lzip patch python3-arpy python3-gi
    sudo apt install -t stretch-backports \
        gir1.2-ostree-1.0 ostree

Buster or Sid
++++++++++++++++
For debian unstable or testing, only the following line should be enough
to get the system requirements for the default plugins installed::


    sudo apt-get install \
        lzip git bzr patch python3-arpy gir1.2-ostree-1.0 ostree python3-gi


Fedora
~~~~~~
For recent fedora systems, the following line should get you the system
requirements you need::


    dnf install -y \
        python3 fuse bubblewrap \
        python3-pip python3-devel

For the default plugins::


    dnf install -y \
        bzr git lzip patch ostree python3-gobject
    pip3 install --user arpy


Ubuntu
~~~~~~

Ubuntu 18.04 LTS or later
+++++++++++++++++++++++++
Install the dependencies with::


    sudo apt install \
        python3 fuse bubblewrap \
        python3-pip python3-dev

For the default plugins::


    sudo apt install \
        bzr git lzip patch python3-arpy gir1.2-ostree-1.0 ostree python3-gi

Ubuntu 16.04 LTS
++++++++++++++++
On Ubuntu 16.04, `bubblewrap <https://github.com/projectatomic/bubblewrap/>`_
or `ostree <https://github.com/ostreedev/ostree>`_ are not available in the official repositories)
You will need to install them in whichever way you see fit. Refer the the upstream documentation
for advice on this.


.. _install-buildbox:

Installing BuildBox
-------------------

BuildStream master now depends on buildbox-casd to manage the local CAS cache
and communicate with CAS servers. buildbox-run is used for sandboxing. BuildBox
components are still in development and there are no stable releases yet.
Thus, they're not available yet in Linux distros and they have to be manually
installed.

These components can be installed from binaries, or built from source.

Install binaries
~~~~~~~~~~~~~~~~
| Linux x86-64 users can download statically linked binaries: `buildbox-x86_64-linux.tar.xz <https://buildbox-casd-binaries.nyc3.cdn.digitaloceanspaces.com/buildbox-x86_64-linux-0.0.9-983edf11.tar.xz>`_
| ``sha256sum 983edf113e945ab723a9ddfd4c00dd855db0175c81373dbc0ad809045e5801a3``

The tarball contains the binaries ``buildbox-casd``, ``buildbox-fuse``,
``buildbox-run-bubblewrap`` and the symlink ``buildbox-run``, which should be
extracted into a directory in ``PATH``, e.g., ``~/.local/bin``.


Build from source
~~~~~~~~~~~~~~~~~

Each of the 4 buildbox components can be installed separately from their
respective git repositiories, and each respository has individual install
instructions. Make sure that you're installing the correct version of
each component.

| **Buildbox-common:** See the installation section in:
| https://gitlab.com/BuildGrid/buildbox/buildbox-common/-/blob/0.0.9/README.rst
| (Be sure to install from the 0.0.9 tag.)

| **Buildbox-casd:** See the installation section in:
| https://gitlab.com/BuildGrid/buildbox/buildbox-casd/-/blob/0.0.9/README.rst \
| (Be sure to install from the 0.0.9 tag.)

| **Buildbox-fuse:** See
| https://gitlab.com/BuildGrid/buildbox/buildbox-fuse/-/blob/0.0.8/INSTALL.rst
| (Be sure to install from the 0.0.8 tag.)

| **Buildbox-run-bublewrap:** See the installation section in:
| https://gitlab.com/BuildGrid/buildbox/buildbox-run-bubblewrap/-/blob/master/README.rst
| (Be sure to install from the 0.0.8 tag.)

Finally, configure buildbox-run-bubblewrap as the default buildbox-run
implementation::

    ln -sv buildbox-run-bubblewrap /usr/local/bin/buildbox-run


.. _install-buildstream:

Installing BuildStream
----------------------

Installing from PyPI
~~~~~~~~~~~~~~~~~~~~
Once you have the base system dependencies, you can install the BuildStream
python package as a regular user.

To install from PyPI, you will additionally require:

 - pip for python3 (only required for setup)
 - Python 3 development libraries and headers


For the latest dev snapshot of BuildStream 2, simply run the following command::

    pip3 install --user --pre BuildStream

This will install latest dev snapshot of BuildStream and its pure python
dependencies into your user's homedir in ``~/.local``.

.. note::

   At time of writing, BuildStream 2 is only available as dev snapshots; this
   is why the ``--pre`` option is required.  Running
   ``pip3 install --user BuildStream`` (without the ``--pre`` option)
   will install Buildsteam 1.

You can also install a specific dev snapshot of Buildstream by specifying the
version. eg ``pip3 install --user BuildStream==1.93.2.dev0``.
Available versions can be found on the BuildStream history page `on PyPi 
<https://pypi.org/project/BuildStream/#history>`_.
Note that some of the oldest versions are not available on PyPI.

Keep following the :ref:`instructions below<post-install>` to ensure that the ``bst``
command is in your ``PATH``.

Upgrading from PyPI
+++++++++++++++++++
Once you have already installed BuildStream from PyPI, you can later update
to the latest dev snapshot like so::


    pip3 install --user --upgrade --pre BuildStream



Installing from a git checkout
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To install directly from the `git repository <https://gitlab.com/BuildStream/buildstream.git>`_
using python's ``pip`` package manager, you will additionally require:

- pip for python3 (only required for setup)
- Python 3 development libraries and headers
- git (to checkout BuildStream)

Before installing, please check the existing tags in the git repository
and determine which version you want to install.

Run the following commands::


    git clone https://gitlab.com/BuildStream/buildstream.git
    cd buildstream
    git checkout <desired release tag>
    pip3 install --user .

This will install BuildStream's pure python dependencies into
your user's homedir in ``~/.local`` and will run BuildStream directly
from the git checkout directory.

Keep following the instructions below to ensure that the ``bst``
command is in your ``PATH`` and to enable bash completions for it.


Upgrading from a git checkout
+++++++++++++++++++++++++++++
If you installed BuildStream from a local git checkout using ``-e`` option, all
you need to do to upgrade BuildStream is to update your local git checkout::

    cd /path/to/buildstream
    git pull --rebase

If you did not specify the ``-e`` option at install time, you will
need to cleanly reinstall BuildStream::

    pip3 uninstall buildstream
    cd /path/to/buildstream
    git pull --rebase
    pip3 install --user .

Installing in virtual environments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can consider installing BuildStream in a
`Virtual Environment <https://docs.python.org/3/tutorial/venv.html>`_ if you want
to install multiple versions of BuildStream, or to isolate BuildStream and its
dependencies from other Python packages.

Here is how to install BuildStream stable and development snapshot releases in
virtual environments of their own::


    # Install BuildStream stable in an environment called "venv-bst-stable"
    # (At time of writing, this will be BuildStream 1)
    python3 -m venv venv-bst-stable
    venv-bst-stable/bin/pip install BuildStream

    # Install BuildStream latest development snapshot in an environment
    # called "venv-bst-latest"
    # (At time of writing, this will be Buildstream 2)
    python3 -m venv venv-bst-latest
    venv-bst-latest/bin/pip install --pre BuildStream

To start using BuildStream from the desired environment, you will need to
activate it first. Activating it will automatically add ``bst`` to your ``PATH``
and set up other necessary environment variables::


    # Use BuildStream stable from venv-bst-stable
    source venv-bst-stable/bin/activate
    bst --version

    # Use BuildStream latest from venv-bst-latest
    source venv-bst-latest/bin/activate
    bst --version

    # Once you are done, remember to deactivate the virtual environment
    deactivate

If you do not want to manage your virtual environments manually, you can
consider using `pipx <https://docs.python.org/3/tutorial/venv.html>`_.


.. _post-install:

Post-install setup
------------------

After having installed from source using any of the above methods, some
setup will be required to use BuildStream.



Adjust ``PATH``
~~~~~~~~~~~~~~~
Since BuildStream is now installed under your local user's install directories,
you need to ensure that ``PATH`` is adjusted.

A regular way to do this is to add the following line to the end of your ``~/.bashrc``::

  export PATH="${PATH}:${HOME}/.local/bin"

.. note::

   You will have to restart your terminal in order for these changes to take effect.


.. _install-container:

Buildstream Inside a Container
-------------------------------

If your system cannot provide the base requirements, it is possible to run
BuildStream within a container. This gives you an easy way to get started
using BuildStream on any Unix-like platform where containers are available,
including macOS. 

For details, see the `Buildstream Docker documentation
<https://gitlab.com/BuildStream/buildstream-docker-images/-/blob/master/USING.md>`_