Improve documentation

* Capitalize in the beggining of a sentence
* Capitalize document filenames
* Add missing periods
* Add more markdown formatting where appropriate.
* Add links for URLs and references to documents in the repo
* Remove unnecessary indentation
* Wrap at 80 characters

No changes in the instructions or wording were done is this commit.

1 files changed, 236 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..5d3607b
--- /dev/null
+++ b/README.md
@@ -0,0 +1,236 @@
+GitLab: [![build status](https://gitlab.com/baserock/ybd/badges/master/build.svg)](https://gitlab.com/baserock/ybd/commits/master)
+
+Github: [![Build Status](https://travis-ci.org/devcurmudgeon/ybd.svg?branch=master)](https://travis-ci.org/devcurmudgeon/ybd)
+
+# ybd
+
+ybd is a tool for building integrated software stacks. It does four things:
+
+- parse yaml files which describe integrated collections of software components
+- gather source code (from git repos) for a user-specified target collection
+- build the collection
+- optionally deploy the collection (if the yaml files describe where/how to
+  deploy)
+
+Currently ybd understands the semantics of yaml
+[definitions](http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/)
+from the [Baserock](http://wiki.baserock.org) project.
+
+The total codebase for ybd is only ~ 2200 lines of Python. even so ybd can
+reliably build all systems in Baserock's definitions.git, i.e. all of the FOSS
+components required for Linux appliances up to and including:
+
+- minimal Linux systems
+- self-hosting development systems
+- GENIVI Baseline and GENIVI Demo Platform systems for x86 and ARMv7
+- OpenStack appliances
+- OpenWRT appliances
+
+There are Concourse pipelines using ybd to integrate various examples at
+<concourse.baserock.org> ...
+
+[<img src="http://i.imgur.com/N193G9C.png" height="400" width="600">](http://concourse.baserock.org)
+
+ybd is a simple start-point for building, deploying, learning and experimenting
+with definitions, algorithms and functionality on Baserock projects. With a
+little work it can be used to build other software stacks too.
+
+ybd is under development. things will change :) If you're trying ybd for the
+first time please start with the latest tag, not master.
+
+ybd is designed for Linux, but can be used on other environments (eg MacOS)
+using Vagrant and VirtualBox.
+
+## Quick Start
+
+On Fedora, Ubuntu, Debian (as root... hoping to fix this for non-root soon):
+
+    git clone git://gitlab.com/baserock/ybd && cd ybd
+    # checkout latest tag
+    git checkout `git describe --tags $(git rev-list --tags --max-count=1)`
+    sudo sh ./install_dependencies.sh
+    cd .. && git clone git://git.baserock.org/baserock/baserock/definitions
+    cd definitions
+
+Or using Vagrant with VirtualBox:
+
+    git clone git://gitlab.com/baserock/ybd && cd ybd
+    # checkout latest tag
+    git checkout `git describe --tags $(git rev-list --tags --max-count=1)`
+    vagrant up
+    vagrant ssh
+    sudo -i
+    cd /src/definitions
+
+Once there you can run (as root)
+
+    ../ybd/ybd.py $1 $2
+
+where
+
+- `$1` is relative path to a chunk, stratum, system or cluster definition file
+- `$2` specifies the architecture you're building/deploying for.
+
+Note that all of the current baserock definitions are for native builds, no
+cross-compile. So if you're on a PC/Mac it may be that your only option for
+`$2` is `x86_64`.
+
+Some examples to try:
+
+    # on an x86_64 laptop running linux, to build a build-system...
+    ../ybd/ybd.py systems/build-system-x86_64.morph x86_64
+
+    # on the same laptop, to build gcc (and its dependencies)
+    ../ybd/ybd.py gcc x86_64
+
+    # on the same laptop, to build the genivi stratum (and its dependencies)
+    ../ybd/ybd.py strata/genivi.morph x86_64
+
+    # on a jetson, to build a GENIVI baseline system...
+    ../ybd/ybd.py systems/genivi-baseline-system-armv7lhf-jetson.morph armv7lhf
+
+    # on anything, to build (and deploy?) parts of baserock release...
+    ../ybd/ybd.py clusters/release.morph x86_64
+
+    # in a baserock devel vm (x86_64), to build and deploy a self-upgrade...
+    ../ybd/ybd.py clusters/upgrade-devel.morph x86_64
+
+ybd generates a lot of log output to stdout, which hopefully helps to explain
+what is happening. If you need a permanent log then try
+
+    ../ybd/ybd.py clusters/upgrade-devel.morph x86_64 | tee some-memorable-name.log
+
+More information about ybd's dependencies is in
+[dependencies](docs/Dependencies.md).
+
+
+## Configuration
+
+ybd is designed to be run from the command line and/or as part of an automated
+pipeline. all configuration is taken from conf files and/or environment
+variables, in the following order of precedence:
+
+    YBD_* environment variables         # if found
+    ./ybd.conf                          # if found
+    $path/to/ybd.py/ybd.conf            # if found
+    $path/to/ybd.py/ybd/config/ybd.conf # default, as provided in the ybd repo
+
+This means you can set custom config via env vars, or the definitions top-level
+directory, or the ybd top-level directory, without having to modify the
+supplied default [ybd.conf](ybd/config/ybd.conf).
+
+NOTE we recommend you create your own ybd.conf file. that way you can git merge
+new versions of ybd + definitions, and your custom settings will always take
+precedence, with no possibility of a merge conflict.
+
+To set config via environment variables, each must be prefixed with `YBD_`
+and using `_` instead of `-`. ybd will strip the `YBD_` prefix and
+convert `_` to `-`. For example
+
+    export YBD_artifact_version=1     # artifact-version: 1
+    export YBD_log_verbose=True       # log-verbose: True
+    export YBD_instances=2            # instances: 2
+
+ybd does not support unix-style --flags so far. If enough people complain about
+that, it can be fixed.
+
+For details about the config options themselves, see
+[ybd.conf](ybd/config/ybd.conf).
+
+## Interesting Features
+
+### Run ybd in Parallel
+
+ybd can fork several instances of itself to parallelise its work. There is no
+intelligence in the scheduling - all of the forks just randomise their
+build-order and try to build everything. For building a set of overlapping
+systems in parallel on a many core machine this proves to be quite effective.
+For example on a 36-core AWS c4.8xlarge machine, 4 racing instances of ybd can
+build all of the x86_64 systems in definitions/clusters/ci.morph much faster
+than a single instance.
+
+To try it, just set the `instances` config variable, for example
+
+    # as an environment variable...
+    export YBD_instances=4
+
+    # or in a ybd.conf file...
+    instances: 8
+
+You should probably think about setting `max-jobs` too, taking into account
+your workloads and host machine(s). If `max-jobs` is not set, ybd will default
+it to number-of-cores/instances.
+
+### kbas Cache Server
+
+kbas is a basic server which can be used to allow other users to access
+pre-built artifacts from previous or current runs of ybd. See kbas/kbas.py for
+the code. With minimal configuration it can serve artifacts to instances of ybd
+on other machines, and also receive uploaded artifacts.
+
+To launch kbas, just do
+
+    cd kbas && ./kbas.py
+
+By default ybd is configured to check for artifacts from a kbas server at
+<http://artifacts1.baserock.org:8000/>.
+
+Config for kbas follows the same approach as ybd, defaulting to config in
+`kbas/config/kbas.conf`.
+
+NOTE: the default password is 'insecure' and the uploading is disabled unless
+you change it.
+
+### Concourse Pipelines
+
+[WORK IN PROGRESS] ybd can generate concourse pipelines - see the code at
+ybd/concourse.py
+
+To start a local concourse instance:
+
+    # in your definitions directory
+    vagrant init concourse/lite
+    vagrant up
+    # generate pipeline, run concourse.py (same arguments as ybd.py)
+    python ../ybd/ybd/concourse.py <target> <arch>
+    fly -t local login -c http://192.168.100.4:8080
+    fly -t local set-pipeline -p <target> -c <target>.yml
+
+You can view the local pipelines at <http://192.168.100.4:8080>.
+
+## TODO
+
+In no particular order, here's a list of things that may be interesting to try
+doing to/with ybd:
+- pip install
+- Deterministic (bit-for-bit) build
+- Fix deployment: ybd currently can deploy systems, subsystems (test this?) and
+  upgrades, but it's a mess:
+  - can't select which subsystems to deploy (need to iron out the definitions)
+  - alter the deployment parameters (templating, or commandline?)
+  - deploy other-architecture stuff
+- Run without root privileges on non-Linux environments (eg Mac OS - needs a
+  different isolation method instead of linux-user-chroot).
+- Test assumptions by creating definitions for other OS software sets, eg
+  Aboriginal Linux and non-Linux systems.
+- Test/fix to work on old versions of definitions, and on morphs.git (and
+  separate out the idiosyncrasies, so code is tidy for future changes)
+
+## Project Guidelines
+
+- Contributions are extremely welcome - feedback, ideas, suggestions, bugs,
+  documentation will be appreciated just as much as code.
+- Key criteria for code contributions are:
+  - 'Code LESS: every line creates a work-chain. Code is a liability, not an
+    asset'
+  - think hard before adding dependencies
+  - code should be tested and pass pep8
+
+## License
+
+- License is GPLv2 but other licensing can be considered on request.
+- Most of the copyright is currently Codethink but don't let that put you off.
+  There's no intent to keep this as a Codethink-only project, nor will there be
+  any attempt to get folks to sign a contributor agreement.  contributors
+  retain their own copyright.
+