diff options
author | trahman1318 <tausif.rahman@mongodb.com> | 2022-01-04 17:47:05 +0000 |
---|---|---|
committer | Evergreen Agent <no-reply@evergreen.mongodb.com> | 2022-01-04 18:42:17 +0000 |
commit | 84727e89407b36263e1d625348e8b26db03d714b (patch) | |
tree | f11108cfdfa7d01f3160904c2d10e6a9856eeb71 | |
parent | da82e5278ca717d3228e4837a3e815c5ced876e0 (diff) | |
download | mongo-84727e89407b36263e1d625348e8b26db03d714b.tar.gz |
SERVER-61400 Add new docker compose for 1 shard configuration
15 files changed, 441 insertions, 53 deletions
diff --git a/.gitignore b/.gitignore index 0146ad41d1b..8a5cc56e361 100644 --- a/.gitignore +++ b/.gitignore @@ -224,7 +224,7 @@ default.profraw /rpm/tmp # antithesis image building assets -/buildscripts/antithesis/database/dist-test -/buildscripts/antithesis/workload/src -/buildscripts/antithesis/workload/mongo +/buildscripts/antithesis/base_images/mongo_binaries/dist-test +/buildscripts/antithesis/base_images/workload/src +/buildscripts/antithesis/base_images/workload/mongo /buildscripts/resmokeconfig/suites/antithesis_*.yml diff --git a/buildscripts/antithesis/Dockerfile b/buildscripts/antithesis/Dockerfile deleted file mode 100644 index 58681a1dee7..00000000000 --- a/buildscripts/antithesis/Dockerfile +++ /dev/null @@ -1,6 +0,0 @@ -FROM scratch -COPY docker-compose.yml / -#COPY .env / -ADD workload /workload -ADD database /database -ADD logs /logs diff --git a/buildscripts/antithesis/README.md b/buildscripts/antithesis/README.md index bfbebb0cbb5..51722d3a2b1 100644 --- a/buildscripts/antithesis/README.md +++ b/buildscripts/antithesis/README.md @@ -1,15 +1,171 @@ -# Antithesis Docker Image Building +# How to Use Antithesis -This directory contains `Dockerfile`s and other resources for constructing -the workload, SUT, and config docker images for use with Antithesis. +## Context +Antithesis is a third party vendor with an environment that can perform network fuzzing. We can +upload images containing `docker-compose.yml` files, which represent various MongoDB topologies, to +the Antithesis Docker registry. Antithesis runs `docker-compose up` from these images to spin up +the corresponding multi-container application in their environment and run a test suite. Network +fuzzing is performed between the containers while the test suite runs & a report is generated by +Antithesis identifying bugs. -- workload: the workload image that runs resmoke -- database: the System Under Test, spawned multiple times to form the - appropriate topology -- config: this directory, contains the above and a docker-compose.yml file for - spawning the system -- logs: the log folders of each database node will be mounted as a subdirectory - in logs. +## Base Images +The `base_images` directory consists of the building blocks for creating a MongoDB test topology. +These images are uploaded to the Antithesis Docker registry weekly during the +`antithesis_image_build` task. For more visibility into how these images are built and uploaded to +the Antithesis Docker registry, please see `evergreen/antithesis_image_build.sh`. + +### mongo_binaries +This image contains the latest `mongo`, `mongos` and `mongod` binaries. It can be used to +start a `mongod` instance, `mongos` instance or execute `mongo` commands. This is the main building +block for creating the System Under Test topology. + +### workload +This image contains the latest `mongo` binary as well as the `resmoke` test runner. The `workload` +container is not part of the actual toplogy. The purpose of a `workload` container is to execute +`mongo` commands to complete the topology setup, and to run a test suite on an existing topology +like so: +```shell +buildscript/resmoke.py run --suite antithesis_concurrency_sharded_with_stepdowns_and_balancer --shellConnString "mongodb://mongos:27017" +``` + +**Every topology must have 1 workload container.** + +Note: During `workload` image build, `buildscripts/antithesis_suite.py` runs, which generates +"antithesis compatible" test suites and prepends them with `antithesis_`. These are the test suites +that can run in antithesis and are available from witihin the `workload` container. + +## Topologies +The `topologies` directory consists of subdirectories representing various mongo test topologies. +Each topology has a `Dockerfile`, a `docker-compose.yml` file and an optional `scripts` directory. + +### Dockerfile +This assembles an image with the necessary files for spinning up the corresponding topology. It +consists of a `docker-compose.yml`, a `logs` directory and an optional `scripts` directory. If this +is structured properly, you should be able to copy the files from this image and run +`docker-compose up` to set up the desired multi-container topology. + +Example from `buildscripts/antithesis/topologies/replica_set/Dockerfile`: +```Dockerfile +FROM scratch +COPY docker-compose.yml / +ADD scripts /scripts +ADD logs /logs +``` + +All topology images are built and uploaded to the Antithesis Docker registry during the +`antithesis_image_build` task in the `evergreen/antithesis_image_build.sh` script. + +Note: These image serve solely as a filesystem containing all necessary files for a topology, +therefore use `FROM scratch`. + +### docker-compose.yml + This describes how to construct the corresponding topology using the + `mongo-binaries` and `workload` images. + +Example from `buildscripts/antithesis/topologies/replica_set/docker-compose.yml`: +```yml +version: '3.0' + +services: + database1: + container_name: database1 + hostname: database1 + image: mongo-binaries:evergreen-latest-master + command: /usr/bin/mongod --bind_ip 0.0.0.0 --replSet RollbackFuzzerTest --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 + volumes: + - ./logs/database1:/var/log/mongodb/ + networks: + antithesis-net: + ipv4_address: 10.20.20.3 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + database2: ... + database3: ... + workload: + container_name: workload + hostname: workload + image: workload:evergreen-latest-master + command: /bin/bash /scripts/workload_init.sh + volumes: + - ./scripts:/scripts/ + depends_on: + - "database1" + - "database2" + - "database3" + networks: + antithesis-net: + ipv4_address: 10.20.20.130 + # The subnet provided here is an example + # An alternative subnet can be used + +networks: + antithesis-net: + driver: bridge + ipam: + config: + - subnet: 10.20.20.0/24 + +``` + +Each container must have a `command`. If it is a single line it can be included directly in the +`docker-compose.yml` -- as in `database1`. If it is more involved, it can be put in a shell script +in the `scripts` directory, included as a volume and `command` can be set to +`/bin/bash /scripts/[script_name].sh` -- as in `workload`. + +When creating `mongod` or `mongos` instances, route the logs like so: +`--logpath /var/log/mongodb/mongodb.log` and utilize `volumes` -- as in `database1`. This enables +us to easily retrieve logs if a bug is detected by Antithesis. + +The `ipv4_address` should be set to `10.20.20.130` or higher if you do not want that container to +be affected by network fuzzing. For instance, you would likely not want the `workload` container +to be affected by network fuzzing -- as shown in the example above. + +Use the `evergreen-latest-master` tag for all images. This is updated automatically in +`evergreen/antithesis_image_build.sh` during the `antithesis_image_build` task -- if needed. + +### scripts +This is an optional directory, which can be used for the reason described above. + +Example from `buildscripts/antithesis/topologies/replica_set/scripts/workload_init.sh`: +```shell +sleep 5s +mongo --host database1 --port 27017 --eval "config={\"_id\" : \"RollbackFuzzerTest\",\"protocolVersion\" : 1,\"members\" : [{\"_id\" : 0,\"host\" : \"database1:27017\"}, {\"_id\" : 1,\"host\" : \"database2:27017\"}, {\"_id\" : 2,\"host\" : \"database3:27017\"} ],\"settings\" : {\"chainingAllowed\" : false,\"electionTimeoutMillis\" : 500, \"heartbeatTimeoutSecs\" : 1, \"catchUpTimeoutMillis\": 700}}; rs.initiate(config)" + +# this cryptic statement keeps the workload container running. +tail -f /dev/null +``` +The `sleep` command can be useful to ensure that other containers have had a chance to start. In +this example, the `workload` container waits 5 seconds while the database containers start. +After that, it initiates the replica set. It is now ready to run a test suite using `resmoke`. +The `tail -f /dev/null` is required for `workload` containers otherwise the container shuts down. + +## How do I create a new topology for Antithesis testing? +To create a new topology for Antithesis testing is easy & requires a few simple steps. +1. Add a new directory in `buildscripts/antithesis/topologies` to represent your desired topology. +You can use existing topologies as an example. +2. Update the `evergreen/antithesis_image_build.sh` file so that your new topology image is +uploaded to the Antithesis Docker registry. +3. Reach out to #server-testing on Slack & provide the new topology image name as well as the + desired test suite to run. +4. Include a member of the STM team on the code review. + +These are the required updates to `evergreen/antithesis_image_build.sh`: +- Add the following command for each of your `mongos` and `mongod` containers in your topology to +create your log directories. +```shell +mkdir -p antithesis/topologies/[topology_name]/logs/[container_name] +``` + +- Build an image for your new topology ending in `-config` +```shell +cd [your_topology_dir] +sed -i s/evergreen-latest-master/$tag/ docker-compose.yml +docker build . -t [your-topology-name]-config:$tag +``` +- Push your new image to the Antithesis Docker registry +```shell +docker tag "[your-topology-name]-config:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/[your-topology-name]-config:$tag" +docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/[your-topology-name]-config:$tag" +``` -Please also see evergreen/antithesis_image_build.sh, which prepares the build -contexts for building these images, and uploads them to the docker registry diff --git a/buildscripts/antithesis/database/Dockerfile b/buildscripts/antithesis/base_images/mongo_binaries/Dockerfile index b3b80d9f525..fe93253eef4 100644 --- a/buildscripts/antithesis/database/Dockerfile +++ b/buildscripts/antithesis/base_images/mongo_binaries/Dockerfile @@ -6,7 +6,9 @@ EXPOSE 27019 # prep the environment RUN mkdir -p /data/db +RUN mkdir -p /data/configdb RUN mkdir -p /var/log/mongodb +RUN mkdir -p /scripts # Install dependencies of MongoDB Server RUN apt-get update @@ -21,5 +23,3 @@ RUN chmod +x /usr/bin/* COPY libvoidstar.so /usr/lib/libvoidstar.so RUN /usr/bin/mongo --version - -CMD /usr/bin/mongod --bind_ip 0.0.0.0 --replSet RollbackFuzzerTest --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 diff --git a/buildscripts/antithesis/workload/Dockerfile b/buildscripts/antithesis/base_images/workload/Dockerfile index bbc3771c99f..58f52bda489 100644 --- a/buildscripts/antithesis/workload/Dockerfile +++ b/buildscripts/antithesis/base_images/workload/Dockerfile @@ -4,6 +4,9 @@ EXPOSE 27017 EXPOSE 27018 EXPOSE 27019 +RUN mkdir -p /scripts +RUN mkdir -p /var/log/resmoke + # tzdata will block waiting for interactive input. Ensure that dpkg gets run # in a non-interactive fashion by preseeding tzdata COPY preseed.txt /tmp/preseed.txt @@ -27,11 +30,8 @@ COPY src /resmoke RUN bash -c "cd /resmoke && python3.9 -m venv python3-venv && . python3-venv/bin/activate && pip install --upgrade pip wheel && pip install -r ./buildscripts/requirements.txt && ./buildscripts/antithesis_suite.py generate-all" -COPY init.sh /resmoke/init.sh COPY mongo /usr/bin RUN chmod +x /usr/bin/mongo COPY libvoidstar.so /usr/lib/libvoidstar.so RUN /usr/bin/mongo --version - -ENTRYPOINT /resmoke/init.sh diff --git a/buildscripts/antithesis/workload/preseed.txt b/buildscripts/antithesis/base_images/workload/preseed.txt index 5b20c6f355c..5b20c6f355c 100644 --- a/buildscripts/antithesis/workload/preseed.txt +++ b/buildscripts/antithesis/base_images/workload/preseed.txt diff --git a/buildscripts/antithesis/topologies/replica_set/Dockerfile b/buildscripts/antithesis/topologies/replica_set/Dockerfile new file mode 100644 index 00000000000..fe3eaf9484e --- /dev/null +++ b/buildscripts/antithesis/topologies/replica_set/Dockerfile @@ -0,0 +1,4 @@ +FROM scratch +COPY docker-compose.yml / +ADD scripts /scripts +ADD logs /logs diff --git a/buildscripts/antithesis/docker-compose.yml b/buildscripts/antithesis/topologies/replica_set/docker-compose.yml index 658f89bef8c..b330315e860 100644 --- a/buildscripts/antithesis/docker-compose.yml +++ b/buildscripts/antithesis/topologies/replica_set/docker-compose.yml @@ -4,8 +4,8 @@ services: database1: container_name: database1 hostname: database1 - build: ./database - image: database:evergreen-latest-master + image: mongo-binaries:evergreen-latest-master + command: /usr/bin/mongod --bind_ip 0.0.0.0 --replSet RollbackFuzzerTest --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 volumes: - ./logs/database1:/var/log/mongodb/ networks: @@ -17,8 +17,8 @@ services: database2: container_name: database2 hostname: database2 - build: ./database - image: database:evergreen-latest-master + image: mongo-binaries:evergreen-latest-master + command: /usr/bin/mongod --bind_ip 0.0.0.0 --replSet RollbackFuzzerTest --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 volumes: - ./logs/database2:/var/log/mongodb/ networks: @@ -30,8 +30,8 @@ services: database3: container_name: database3 hostname: database3 - build: ./database - image: database:evergreen-latest-master + image: mongo-binaries:evergreen-latest-master + command: /usr/bin/mongod --bind_ip 0.0.0.0 --replSet RollbackFuzzerTest --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 volumes: - ./logs/database3:/var/log/mongodb/ networks: @@ -43,8 +43,11 @@ services: workload: container_name: workload hostname: workload - build: ./workload image: workload:evergreen-latest-master + command: /bin/bash /scripts/workload_init.sh + volumes: + - ./logs/workload:/var/log/resmoke/ + - ./scripts:/scripts/ depends_on: - "database1" - "database2" diff --git a/buildscripts/antithesis/workload/init.sh b/buildscripts/antithesis/topologies/replica_set/scripts/workload_init.sh index 52dfe5f75c3..1490a5bf3c8 100755 --- a/buildscripts/antithesis/workload/init.sh +++ b/buildscripts/antithesis/topologies/replica_set/scripts/workload_init.sh @@ -1,7 +1,5 @@ sleep 5s mongo --host database1 --port 27017 --eval "config={\"_id\" : \"RollbackFuzzerTest\",\"protocolVersion\" : 1,\"members\" : [{\"_id\" : 0,\"host\" : \"database1:27017\"}, {\"_id\" : 1,\"host\" : \"database2:27017\"}, {\"_id\" : 2,\"host\" : \"database3:27017\"} ],\"settings\" : {\"chainingAllowed\" : false,\"electionTimeoutMillis\" : 500, \"heartbeatTimeoutSecs\" : 1, \"catchUpTimeoutMillis\": 700}}; rs.initiate(config)" -mongo --host database1 --eval "rs.config()" - # this cryptic statement keeps the workload container running. tail -f /dev/null diff --git a/buildscripts/antithesis/topologies/sharded_cluster/Dockerfile b/buildscripts/antithesis/topologies/sharded_cluster/Dockerfile new file mode 100644 index 00000000000..fe3eaf9484e --- /dev/null +++ b/buildscripts/antithesis/topologies/sharded_cluster/Dockerfile @@ -0,0 +1,4 @@ +FROM scratch +COPY docker-compose.yml / +ADD scripts /scripts +ADD logs /logs diff --git a/buildscripts/antithesis/topologies/sharded_cluster/docker-compose.yml b/buildscripts/antithesis/topologies/sharded_cluster/docker-compose.yml new file mode 100644 index 00000000000..ee25a3ffa98 --- /dev/null +++ b/buildscripts/antithesis/topologies/sharded_cluster/docker-compose.yml @@ -0,0 +1,170 @@ +version: '3.0' + +services: + configsvr1: + container_name: configsvr1 + hostname: configsvr1 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/configsvr1:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --configsvr --replSet ConfigServerReplSet --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 --setParameter reshardingMinimumOperationDurationMillis=30000 --wiredTigerCacheSizeGB 1 --oplogSize 256 + networks: + antithesis-net: + ipv4_address: 10.20.20.6 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + + configsvr2: + container_name: configsvr2 + hostname: configsvr2 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/configsvr2:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --configsvr --replSet ConfigServerReplSet --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 --setParameter reshardingMinimumOperationDurationMillis=30000 --wiredTigerCacheSizeGB 1 --oplogSize 256 + networks: + antithesis-net: + ipv4_address: 10.20.20.7 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + + configsvr3: + container_name: configsvr3 + hostname: configsvr3 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/configsvr3:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --configsvr --replSet ConfigServerReplSet --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 --setParameter reshardingMinimumOperationDurationMillis=30000 --wiredTigerCacheSizeGB 1 --oplogSize 256 + networks: + antithesis-net: + ipv4_address: 10.20.20.8 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + + database1: + container_name: database1 + hostname: database1 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database1:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard1 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.3 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + database2: + container_name: database2 + hostname: database2 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database2:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard1 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.4 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + database3: + container_name: database3 + hostname: database3 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database3:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard1 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.5 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + + database4: + container_name: database4 + hostname: database4 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database4:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard2 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.10 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + database5: + container_name: database5 + hostname: database5 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database5:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard2 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.11 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + # + database6: + container_name: database6 + hostname: database6 + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/database6:/var/log/mongodb/ + command: /usr/bin/mongod --bind_ip 0.0.0.0 --shardsvr --replSet Shard2 --logpath /var/log/mongodb/mongodb.log --oplogSize 256 --setParameter enableTestCommands=1 --setParameter enableElectionHandoff=0 --setParameter roleGraphInvalidationIsFatal=1 --setParameter receiveChunkWaitForRangeDeleterTimeoutMS=90000 --wiredTigerCacheSizeGB 1 + networks: + antithesis-net: + ipv4_address: 10.20.20.12 + # Set the an IPv4 with an address of 10.20.20.130 or higher + # to be ignored by the fault injector + + mongos: + container_name: mongos + hostname: mongos + image: mongo-binaries:evergreen-latest-master + volumes: + - ./logs/mongos:/var/log/mongodb/ + - ./scripts:/scripts/ + command: /bin/bash /scripts/mongos_init.sh + depends_on: + - "configsvr1" + - "configsvr2" + - "configsvr3" + networks: + antithesis-net: + ipv4_address: 10.20.20.9 + # The subnet provided here is an example + # An alternative subnet can be used + workload: + container_name: workload + hostname: workload + image: workload:evergreen-latest-master + volumes: + - ./logs/workload:/var/log/resmoke/ + - ./scripts:/scripts/ + command: /bin/bash /scripts/workload_init.sh + depends_on: + - "database1" + - "database2" + - "database3" + - "database4" + - "database5" + - "database6" + - "configsvr1" + - "configsvr2" + - "configsvr3" + - "mongos" + networks: + antithesis-net: + ipv4_address: 10.20.20.130 + # The subnet provided here is an example + # An alternative subnet can be used +networks: + antithesis-net: + driver: bridge + ipam: + config: + - subnet: 10.20.20.0/24 diff --git a/buildscripts/antithesis/topologies/sharded_cluster/scripts/mongos_init.sh b/buildscripts/antithesis/topologies/sharded_cluster/scripts/mongos_init.sh new file mode 100755 index 00000000000..674cab49843 --- /dev/null +++ b/buildscripts/antithesis/topologies/sharded_cluster/scripts/mongos_init.sh @@ -0,0 +1,7 @@ +sleep 5s +mongo --host configsvr1 --port 27019 --eval "config={\"_id\" : \"ConfigServerReplSet\",\"configsvr\" : true,\"protocolVersion\" : 1,\"members\" : [{\"_id\" : 0,\"host\" : \"configsvr1:27019\"}, {\"_id\" : 1,\"host\" : \"configsvr2:27019\"}, {\"_id\" : 2,\"host\" : \"configsvr3:27019\"} ],\"settings\" : {\"chainingAllowed\" : false,\"electionTimeoutMillis\" : 2000, \"heartbeatTimeoutSecs\" : 1, \"catchUpTimeoutMillis\": 0}}; rs.initiate(config)" + +mongos --bind_ip 0.0.0.0 --configdb ConfigServerReplSet/configsvr1:27019,configsvr2:27019,configsvr3:27019 --logpath /var/log/mongodb/mongodb.log --setParameter enableTestCommands=1 + +# this cryptic statement keeps the container running. +tail -f /dev/null diff --git a/buildscripts/antithesis/topologies/sharded_cluster/scripts/workload_init.sh b/buildscripts/antithesis/topologies/sharded_cluster/scripts/workload_init.sh new file mode 100755 index 00000000000..acaf30ea036 --- /dev/null +++ b/buildscripts/antithesis/topologies/sharded_cluster/scripts/workload_init.sh @@ -0,0 +1,18 @@ +sleep 10s + +mongo --host database1 --port 27018 --eval "config={\"_id\" : \"Shard1\",\"protocolVersion\" : 1,\"members\" : [{\"_id\" : 0,\"host\" : \"database1:27018\"}, {\"_id\" : 1,\"host\" : \"database2:27018\"}, {\"_id\" : 2,\"host\" : \"database3:27018\"} ],\"settings\" : {\"chainingAllowed\" : false,\"electionTimeoutMillis\" : 2000, \"heartbeatTimeoutSecs\" : 1, \"catchUpTimeoutMillis\": 0}}; rs.initiate(config)" + +sleep 5s + +mongo --host database4 --port 27018 --eval "config={\"_id\" : \"Shard2\",\"protocolVersion\" : 1,\"members\" : [{\"_id\" : 0,\"host\" : \"database4:27018\"}, {\"_id\" : 1,\"host\" : \"database5:27018\"}, {\"_id\" : 2,\"host\" : \"database6:27018\"} ],\"settings\" : {\"chainingAllowed\" : false,\"electionTimeoutMillis\" : 2000, \"heartbeatTimeoutSecs\" : 1, \"catchUpTimeoutMillis\": 0}}; rs.initiate(config)" + +sleep 5s + +mongo --host mongos --port 27017 --eval 'sh.addShard("Shard1/database1:27018,database2:27018,database3:27018")' + +sleep 5s + +mongo --host mongos --port 27017 --eval 'sh.addShard("Shard2/database4:27018,database5:27018,database6:27018")' + +# this cryptic statement keeps the workload container running. +tail -f /dev/null diff --git a/buildscripts/antithesis_suite.py b/buildscripts/antithesis_suite.py index 25114fac786..8397a992ee7 100755 --- a/buildscripts/antithesis_suite.py +++ b/buildscripts/antithesis_suite.py @@ -17,6 +17,8 @@ SUITE_BLACKLIST = [ "CheckReplOplogs", "CleanEveryN", "ContinuousStepdown", + "ValidateCollections", + "CheckOrphansDeleted", ] @@ -32,6 +34,12 @@ def _sanitize_hooks(hooks): raise RuntimeError('Unknown structure in hook. File a TIG ticket.') +def _sanitize_test_data(test_data): + if test_data.get("useStepdownPermittedFile", None): + test_data["useStepdownPermittedFile"] = False + return test_data + + _SUITES_PATH = os.path.join("buildscripts", "resmokeconfig", "suites") @@ -70,6 +78,24 @@ def _generate(suite_name: str) -> None: except TypeError: pass + try: + suite["executor"]["config"]["shell_options"]["global_vars"][ + "TestData"] = _sanitize_test_data( + suite["executor"]["config"]["shell_options"]["global_vars"]["TestData"]) + except KeyError: + # pass, don't care + pass + except TypeError: + pass + + try: + suite["executor"]["config"]["shell_options"]["eval"] += "jsTestLog = Function.prototype;" + except KeyError: + # pass, don't care + pass + except TypeError: + pass + out = yaml.dump(suite) with open(os.path.join(_SUITES_PATH, "antithesis_{}.yml".format(suite_name)), "w") as fstream: fstream.write( diff --git a/evergreen/antithesis_image_build.sh b/evergreen/antithesis_image_build.sh index de0d79c99fd..193af6bbc43 100644 --- a/evergreen/antithesis_image_build.sh +++ b/evergreen/antithesis_image_build.sh @@ -20,22 +20,22 @@ ldd src/dist-test/bin/mongo | grep libvoidstar # prepare the image building environment cp -rf src/buildscripts/antithesis/ antithesis # due to gitignore, we can't commit a folder called logs, so make it here -mkdir -p antithesis/logs/database{1,2,3} +mkdir -p antithesis/topologies/replica_set/logs/{database1,database2,database3,workload} +mkdir -p antithesis/topologies/sharded_cluster/logs/{database1,database2,database3,database4,database5,database6,configsvr1,configsvr2,configsvr3,mongos,workload} # copy ... to the build context # resmoke -cp -rf src antithesis/workload/src +cp -rf src antithesis/base_images/workload/src # mongo binary -cp src/dist-test/bin/mongo antithesis/workload +cp src/dist-test/bin/mongo antithesis/base_images/workload # libvoidstar -cp /usr/lib/libvoidstar.so antithesis/workload/ +cp /usr/lib/libvoidstar.so antithesis/base_images/workload # these aren't needed for the workload image, so get rid of them -rm -rf antithesis/workload/src/dist-test +rm -rf antithesis/base_images/workload/src/dist-test # all mongodb binaries -cp -rf src/dist-test antithesis/database -cp /usr/lib/libvoidstar.so antithesis/database/ +cp -rf src/dist-test antithesis/base_images/mongo_binaries +cp /usr/lib/libvoidstar.so antithesis/base_images/mongo_binaries/ -cd antithesis/workload # push images as evergreen-latest-${branch_name}, unless it's a patch tag="evergreen-latest-${branch_name}" if [ "${is_patch}" = "true" ]; then @@ -47,14 +47,19 @@ if [ -n "${antithesis_image_tag}" ]; then tag=$antithesis_image_tag fi +cd antithesis/base_images/mongo_binaries +docker build . -t mongo-binaries:$tag + +cd ../workload docker build . -t workload:$tag -cd ../database -docker build . -t database:$tag -cd .. -# ensure that the embedded image references actually point to the images we're -# pushing here + +cd ../../topologies/replica_set sed -i s/evergreen-latest-master/$tag/ docker-compose.yml -docker build . -t config:$tag +docker build . -t repl-set-config:$tag + +cd ../sharded_cluster +sed -i s/evergreen-latest-master/$tag/ docker-compose.yml +docker build . -t sharded-cluster-config:$tag # login, push, and logout echo "${antithesis_repo_key}" > mongodb.key.json @@ -62,14 +67,17 @@ cat mongodb.key.json | docker login -u _json_key https://us-central1-docker.pkg. rm mongodb.key.json # tag and push to the registry +docker tag "mongo-binaries:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/mongo-binaries:$tag" +docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/mongo-binaries:$tag" + docker tag "workload:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/workload:$tag" docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/workload:$tag" -docker tag "database:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/database:$tag" -docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/database:$tag" +docker tag "repl-set-config:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/repl-set-config:$tag" +docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/repl-set-config:$tag" -docker tag "config:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/config:$tag" -docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/config:$tag" +docker tag "sharded-cluster-config:$tag" "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/sharded-cluster-config:$tag" +docker push "us-central1-docker.pkg.dev/molten-verve-216720/mongodb-repository/sharded-cluster-config:$tag" docker logout https://us-central1-docker.pkg.dev |