summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* WIP 5prototype/fdb-replicator-2Nick Vatamaniuc2019-08-125-671/+281
| | | | | | * streamined doc jobs worker (move halt hadling to the top) * remove invalid tests * updated couch_replicator_docs to take explicit db and doc params
* WIP 4Nick Vatamaniuc2019-08-069-355/+474
| | | | | | | * Replaced VDU with a simpler Erlang module - Avoid conflict handling, updates etc * Don't auto-create _replicator dbs * Doc job acceptor - max acceptors vs max jobs
* FDB Replicator WIPNick Vatamaniuc2019-07-3121-1326/+990
|
* CouchDB map indexes on FDBGarren Smith2019-07-3128-28/+3189
| | | | | | This adds couch_views which builds map indexes and stores them in FDB. Co-authored-by: Paul J. Davis <paul.joseph.davis@gmail.com>
* Fix default key ranges for fold_rangePaul J. Davis2019-07-311-15/+16
| | | | | If a start or end key is not specified we still need to scope the range read to the given `RangePrefix`.
* Expose the is_replicator_db and is_user_db logicPaul J. Davis2019-07-312-12/+31
| | | | | This exposes a single place where we can check for whether a given database or database name is a replicator or users database.
* Make fabric2.hrl publicPaul J. Davis2019-07-311-0/+0
|
* Fix more elixir testsPaul J. Davis2019-07-313-10/+43
|
* Expose ICU ucol_getSortKeyRussell Branca2019-07-313-8/+219
|
* Disable broken couch_att testsPaul J. Davis2019-07-311-187/+187
|
* Reinitialize chttpd_auth_cache on config changePaul J. Davis2019-07-313-12/+56
| | | | | | | | The old test got around this by using couch_httpd_auth cache in its tests which is fairly odd given that we run chttpd_auth_cache in production. This fixes that mistake and upgrades chttpd_auth_cache so that it works in the test scenario of changing the authentication_db configuration.
* Fix formatting of all_docs_test.exsPaul J. Davis2019-07-311-1/+1
|
* Implement `POST /_dbs_info`Paul J. Davis2019-07-311-8/+9
|
* Fix revision tree extensionsPaul J. Davis2019-07-312-8/+20
| | | | | Previously I was forgetting to keep the previous history around which ended up limiting the revision depth to two.
* Fix `COPY` methodPaul J. Davis2019-07-311-1/+1
| | | | Simple function change to `fabric2_db:name/1`
* Fix bulk docs error reportingPaul J. Davis2019-07-313-57/+73
| | | | | | The existing logic around return codes and term formats is labyrinthine. This is the result of much trial and error to get the new logic to behave exactly the same as the previous implementation.
* Implement _all_dbs/_all_docs API parametersPaul J. Davis2019-07-317-251/+512
| | | | | | This adds the mapping of CouchDB start/end keys and so on to the similar yet slightly different concepts in FoundationDB. The handlers for `_all_dbs` and `_all_docs` have been udpated to use this new logic.
* Remove tests for deprecated features.Paul J. Davis2019-07-3115-3148/+0
| | | | | Neither partitioned databases or shard splitting will exist in a FoundationDB layer.
* CouchDB background jobsNick Vatamaniuc2019-07-3120-1/+2909
| | | | | | | RFC: https://github.com/apache/couchdb-documentation/pull/409 Main API is in the `couch_jobs` module. Additional description of internals is in the README.md file.
* Fix exception in cache auth doc updateEric Avdey2019-07-311-1/+1
|
* Fix arity in changes timeout callbackEric Avdey2019-07-312-3/+3
|
* Update get security to use fabric2Eric Avdey2019-07-311-1/+1
|
* Implement `_users` db authenticationPaul J. Davis2019-07-316-26/+212
| | | | | | This changes `chttpd_auth_cache` to use FoundationDB to back the `_users` database including the `before_doc_update` and `after_doc_read` features.
* Database config changes should bump the db versionPaul J. Davis2019-07-311-1/+1
| | | | This was a remnant before we used a version per database.
* Fix validate_doc_update when recreating a documentPaul J. Davis2019-07-311-1/+6
| | | | | | | | This fixes the behavior when validating a document update that is recreating a previously deleted document. Before this fix we were sending a document body with `"_deleted":true` as the existing document. However, CouchDB behavior expects the previous document passed to VDU's to be `null` in this case.
* Allow for previously configured filtersPaul J. Davis2019-07-311-0/+3
| | | | | The older chttpd/fabric split configured filters as one step in the coordinator instead of within each RPC worker.
* Convert attachment info to disk terms correctlyPaul J. Davis2019-07-311-4/+9
| | | | | | | | I was accidentally skipping this step around properly serializing/deserializing attachments. Note to self: If someon specifies attachment headers this will likely break when we attempt to pack the value tuple here.
* Fix revision generation on attachment uploadPaul J. Davis2019-07-312-42/+70
| | | | | | | | | | When uploading an attachment we hadn't yet flushed data to FoundationDB which caused the md5 to be empty. The `new_revid` algorithm then declared that was because it was an old style attachment and thus our new revision would be a random number. This fix just flushes our attachments earlier in the process of updating a document.
* Fix fabric2_txids:terminate/2Paul J. Davis2019-07-311-1/+1
|
* Implement attachment compressionPaul J. Davis2019-07-312-39/+77
| | | | | This still holds all attachment data in RAM which we'll have to revisit at some point.
* Remove debug loggingPaul J. Davis2019-07-312-13/+1
|
* Start switching chttpd HTTP endpoints to fabric2Paul J. Davis2019-07-319-256/+1183
| | | | | | This is not an exhaustive port of the entire chttpd API. However, this is enough to support basic CRUD operations far enough that replication works.
* Update ddoc_cache to use fabric2Paul J. Davis2019-07-312-2/+2
|
* Initial test suite for the fabric2 implementationPaul J. Davis2019-07-3112-0/+2116
| | | | | | | | This provides a good bit of code coverage for the new implementation. We'll want to expand this to include relevant tests from the previous fabric test suite along with reading through the various other tests and ensuring that we cover the API as deeply as is appropriate for this layer.
* Initial fabric2 implementation on FoundationDBPaul J. Davis2019-07-3113-401/+3502
| | | | | | | | | | | | | | | This provides a base implementation of a fabric API backed by FoundationDB. While a lot of functionality is provided there are a number of places that still require work. An incomplete list includes: 1. Document bodies are currently a single key/value 2. Attachments are stored as a range of key/value pairs 3. There is no support for indexing 4. Request size limits are not enforced directly 5. Auth is still backed by a legacy CouchDB database 6. No support for before_doc_update/after_doc_read 7. Various implementation shortcuts need to be expanded for full API support.
* Disable eunit test suite in fabricPaul J. Davis2019-07-3113-2177/+1872
| | | | | | | Most of these tests are for quorum and clustered response handling which will no longer exist with FoundationDB. Eventually we'll want to go through these and pick out anything that is still applicable and ensure that we re-add them to the new test suite.
* Update build system for FoundationDBPaul J. Davis2019-07-316-4/+27
|
* Remove local replication endpoints in CouchDB 3.xNick Vatamaniuc2019-07-3122-456/+136
| | | | | | | | | | | | | | | `local` replication endpoints do something completely unexpected from a user's point of view -- they replicate to and from node local databases on a random node. The only way this worked correctly was if someone used the backend port (:5986) with a single node database. However, that port is getting closed for 3.x release as well, so it makes even less sense to keep this functionality around. For more discussion and voting results see ML list: https://lists.apache.org/thread.html/ddcd9db93cee363db7da571f5cbc7f2bd24b881a34e1ef734d6a0a1c@%3Cdev.couchdb.apache.org%3E The `_replicate` HTTP "hack" was left as is, since it does work more or less, However it is inconsistent with what _replicator docs do so we should probably deprecated it and remove it in 4.x.
* Fix mem3_sync_event_listener EUnit testNick Vatamaniuc2019-07-301-20/+8
| | | | | Fix a race condition in state matching, also parameterize the state field in wait_state.
* Retry EUnit tests on failureNick Vatamaniuc2019-07-291-2/+12
| | | | | | | Whole app is retried 2 extra times if it fails. Added to *nix Makefile only for now. May not be needed for Windows as this is for CI flakiness mostly.
* Merge pull request #2039 from cloudant/exunit-simplifiediilyak2019-07-29205-79/+887
|\ | | | | Exunit simplified
| * Update .travis.ymlILYA Khlopotov2019-07-291-5/+0
| |
| * Unify runners for unit and integration testsILYA Khlopotov2019-07-2912-93/+67
| |
| * Add chained setupsILYA Khlopotov2019-07-2910-0/+685
| |
| * Move eunit tests into test/eunit directoryILYA Khlopotov2019-07-29175-3/+3
| |
| * Minimal ExUnit setupILYA Khlopotov2019-07-2912-3/+157
| |
| * Fix credo complains for dreyfusILYA Khlopotov2019-07-291-2/+2
|/
* Fix EUnit timeouts (#2087)Adam Kocoloski2019-07-286-104/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Proactively increase timeout for PBKDF2 test This test was taking 134s in a recent run, which is uncomfortably close to the threshold. * Extend timeouts for all reshard API tests We're observing timeouts on various tests in this suite so let's keep it consistent and increase timeouts across the board. * Bump default timeout for all mem3_reshard tests A couple of these tests were exceeding the default timeout under normal circumstances, but many of them do a significant amount of work, so for simplicity we set a module-wide timeout and apply it consistently throughout. * Modernize the sync_security test setup/teardown This test actually doesn't do much real work, but I think what was happening is that the setup and teardown time was being charged to the test itself. I've refactored it to use a more modern scaffolding following some of our more recent additions to the test suite, but have left the timeout at the default to test this hypothesis. * Increase timeouts on more heavyweight mem3 tests * Extend timeouts for replication tests
* Fix flaky mem3_sync_event_listener EUnit testNick Vatamaniuc2019-07-281-8/+31
| | | | | | | | Config setting was asynchronous and the waiting function was not waiting for the actual state value to change just that the state function was returning. The fix is to wait for the config value to propagate to the state.
* Increase timeouts on two slow btree testsAdam Kocoloski2019-07-281-2/+7
| | | | | | These two tests are reliably timing out on ARM hardware in Jenkins. They do a lot of individual btree operations so this is not entirely surprising. Appropriate course of action here is to raise the timeout.