Offload FrozenJob secrets

The following potential problems were observed with FrozenJob secrets:

1) They may be repetitive: since the FrozenJob contains
lists of playbooks and each playbook record has a copy of all the
secrets which should be used for that playbook, if a job has multiple
playbooks the secrets will be repeated for each job.  Consider a base
job with three playbooks: the base job's secrets will be included
three times.

2) They may be large: secrets in ZK are stored encrypted and suffer the
same size explosion that they do when encrypted into zuul.yaml files.

3) Take #1 and #2 together and we have the possibility of having FrozenJob
objects that are larger than 1MB which is a problem for ZK.

Address all three issues by offloading the secrets to a new ZK node if
they are large (using the existing JobData framework) and de-duplicate
them and refer to them by index.

There is no backwards compatability handling here, so the ZK state needs
to be deleted.

Change-Id: I32133e8dd0e933528381f1187d270142046ff08f

0 files changed, 0 insertions, 0 deletions