summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Decrease the view indexer transaction timedecrease-indexer-transaction-time-limitNick Vatamaniuc2021-03-221-1/+1
| | | | | | | | | | | | | | The indexer transaction time is decreased in order to allow enough time for the client to re-use the same GRV to emit doc bodies. This PR goes along with [1], where emitted doc bodies in a view responses now come from the same database read version as the one used by the indexer. Since the batcher previously used 4.5 seconds as the maximum, that left little time to read any doc bodies. [1]: https://github.com/apache/couchdb/pull/3391 Issue: https://github.com/apache/couchdb/issues/3381
* Fix error_logger reports for OTP >= 21Nick Vatamaniuc2021-03-181-0/+13
| | | | | | | | | | | | | | | | | | | | | Starting with OTP 21 there is a new logging system, and we forgot to add the legacy error logger handler for it. Without it `couch_log` cannot emit gen_server, supervisor and other such system events. Luckily, there is OTP support to enable legacy error_logger behavior and that's what we're doing here. The `add_report_handler/1` call will auto-start the `error_logger` app if needed, and it will also add an `error_logger` handler to the global `logger` system. We also keep the `gen_event:add_sup_handler/3` call, as that will ensure we'll find out when `error_logger` dies so that `couch_log_monitor` can restart everything. Someday(TM) we'll write a proper log event handler for the new logger and have nicely formatted structured logs, but it's better to do that once we don't have to support OTP versions =< 20. Issue: https://github.com/apache/couchdb/pull/3422
* Fix typo causing not saving of configuration changes from chttpd_nodeILYA Khlopotov2021-03-181-1/+1
|
* feat(couchjs): add support for SpiderMonkey 86Jan Lehnardt2021-03-175-4/+821
|
* Fix _changes?filter=_design (#3430)Bessenyei Balázs Donát2021-03-161-1/+8
|
* Set wait_for_built_index=True for 17-multi-type-value-test.pyBessenyei Balázs Donát2021-03-161-1/+1
|
* Ignore unchecked JWT claimsJay Doane2021-03-151-10/+26
| | | | | | | | | | | | | | | | Previously, if a JWT claim was present, it was validated regardless of whether it was required. However, according to the spec [1]: "all claims that are not understood by implementations MUST be ignored" which we interpret to mean that we should not attempt to validate claims we don't require. With this change, only claims listed in required checks are validated. [1] https://tools.ietf.org/html/rfc7519#section-4
* Include necessary dependency in jwtf keystore test setup & teardownJay Doane2021-03-151-2/+2
| | | | | The config application depends on couch_log, so include it when setting up and tearing down tests.
* Fix more couch_jobs flakinessNick Vatamaniuc2021-03-121-0/+11
| | | | | | | | The errors see in #3417 seem to indicate the expiration jobs are interfering with the couch_jobs tests, to prevent that prevent expiration_db job gen_server from starting at all. Fixes #3417
* Fix couch_jobs to be less flakyNick Vatamaniuc2021-03-121-532/+454
| | | | | | | | | | | | | | | | | | | | | | | It turns out fabric is dependent on couch_jobs because of db expiration module. So when couch_jobs was restarted multiple times per test case it could have brought down fabric. However, since couch_jobs needs fabric for transactional stuff it ended up brining couch_jobs app down as well. To fix it: * Switch to explicitly starting/stopping fabric and couch_jobs together * Break appart bad_messages* tests to individually test each type of message as app restarts in the middle of the tests kept killing fabric and intermettently killing couch_jobs a well. * Also make the tests look nicer by re-using ?TDEF_FE macros from `fabric2_test`, this we can avoid the `?_test(begin... end).` pattern. * Remove meck:unload since we don't really meck anything in the module * Don't need to spend time cleaning out database as we don't really create that many dbs (just one) and that one gets cleaned out in its own test.
* Fix and re-enable ChangesAsyncTestBessenyei Balázs Donát2021-03-111-2/+6
|
* Lower the view indexer transaction retry limitNick Vatamaniuc2021-03-041-2/+12
| | | | | | | | Previously, the view indexer used the default retry limit (100) from the `fdb_tx_options` config section. However, since the batching algorithm relies on sensing errors and reacting to them, retrying the batch 100 times before erroring out was not optimal. So value is lowered down to 5 and it's also made configurable.
* Allow applying per-transaction options with fabric2_fdb:transactional/3Nick Vatamaniuc2021-03-033-24/+71
| | | | | | | | | | | | | 1) First, as a cleanup, remove DB `Options` from the `init_db/3 call. We always follow `init_db/3` (sometimes called through the `fabric2_fdb:transactional(DbName, ...)` with a `create(TxDb, Options)` or `open(TxDb, Options)` call, where we overrode `Options` anyway. The only time we didn't follow it up with a `create/2` or `open/2` is when dbs are deleted where `Options` wouldn't matter. 2) Add a new `fabric2_fdb:transactional(DbName|Db, TxOptions, Fun)` call which allows specifying per-transaction TX options in the `TxOptions` arg. The format of `TxOptions` is `#{option_name_as_atom => integer | binary}`
* Relax isolation level when indexer reads from DB (#3393)Adam Kocoloski2021-03-021-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | * Relax isolation level when indexer reads from DB This patch causes the indexing subsystem to use snapshot isolation when reading from the database. This reduces commit conflicts and ensures the index can make progress even in the case of frequently updated docs. In the pathological case, a document updated in a fast loop can cause the indexer to stall out entirely when using serializable reads. Each successful update of the doc will cause the indexer to fail to commit. The indexer will retry with a new GRV but the same target DbSeq. In the meantime, our frequently updated document will have advanced beyond DbSeq and so the indexer will finish without indexing it in that pass. This process can be repeated ad infinitum and the document will never actually show up in a view response. Snapshot reads are safe for this use case precisely because we do have the _changes feed, and we can always be assured that a concurrent doc update will show up again later in the feed. * Bump erlfdb version Needed to pull in fix for snapshot range reads.
* Fix badmatch in couch_views_indexerNick Vatamaniuc2021-03-021-4/+6
| | | | | | | | Previously, when an erlfdb error occured and a recursive call to `update/3` was made, the result of that call was always matched against `{Mrst, State}`. However, in the case when the call had finalized and returned `couch_eval:release_map_context/1` response, the result would be `ok` which would blow with a badmatch error against `{Mrst, State}`.
* Optimize lookup/3ebtree-lookup-optRobert Newson2021-02-271-4/+4
| | | | | A tidier version of https://github.com/apache/couchdb/pull/3384 that saves an unnecessary call to collate.
* Merge pull request #3384 from apache/ebtree-lookup-collate-eqRobert Newson2021-02-261-6/+6
|\ | | | | use collate in lookup
| * use collate in lookupRobert Newson2021-02-261-6/+6
| |
* | Fix ebtree:lookup_multi/3Paul J. Davis2021-02-261-6/+5
| | | | | | | | | | | | | | If one of the provided lookup keys doesn't exist in the ebtree, it can inadvertently prevent a second lookup key from being found if it the first key greater than the missing lookup key is equal to the second lookup key.
* | Add failing cases for ebtree:lookup_multi/3 bugPaul J. Davis2021-02-262-1/+28
| | | | | | | | | | | | These two test cases expose the subtle bug in ebtree:lookup_multi/3 where a key that doesn't exist in the tree can prevent a subsequent lookup key from matching in the same KV node.
* | Fix typoPaul J. Davis2021-02-261-8/+8
|/
* Show process status in active_tasksRobert Newson2021-02-091-1/+10
| | | | | This allows users to verify that compaction processes are suspended outside of any configured strict_window.
* Handle all erlfdb error codes (#3355)Robert Newson2021-02-081-10/+6
|
* Fix PUT of multipart/related attachments support for Transfer-Encoding: ↵Bessenyei Balázs Donát2021-02-033-2/+22
| | | | | | | chunked (#3360) Transfer-Encoding: chunked causes the server to wait indefinitely, then issue a a 500 error when the client finally hangs up, when PUTing a multipart/related document + attachments. This commit fixes that issue by adding proper handling for chunked multipart/related requests.
* fix additional links after branch renaming (master -> main)Ronny2021-01-252-3/+3
|
* add http error for fdb 1031Garren Smith2021-01-181-0/+5
|
* remove {restart_tx, true} from mango _all_docsGarren Smith2021-01-181-2/+1
|
* Update README.mdRobert Newson2021-01-151-6/+0
| | | 1. The caching effort was a bust and has been removed. 2) chunkify can be done externally with a custom persist_fun.
* Allow gzipped requests to _session (#3323)Bessenyei Balázs Donát2021-01-123-3/+14
| | | | | All endpoints but _session support gzip encoding and there's no practical reason for that. This commit enables gzip decoding on compressed requests to _session.
* treat 408 as a retryable error condition (#3303)Robert Newson2020-12-201-18/+18
|
* Add missing default headers to responses (#3279)Bessenyei Balázs Donát2020-11-233-3/+13
| | | | Add missing default headers to responses
* Remove unnecessary line wrapsPaul J. Davis2020-11-171-4/+2
|
* Minimize conflicts while building viewsPaul J. Davis2020-11-172-8/+91
| | | | | | | | | | This flips the view indexer to grab the database update_seq outside of the update transaction. Previously we would cosntantly refresh the db_seq value on every retry of the transactional loop. We use a snapshot to get the update_seq so that we don't trigger spurious read conflicts with any clients that might be updating the database.
* Allow specifying an end_key for fold_changesPaul J. Davis2020-11-172-7/+22
| | | | | | | This is useful so that read conflicts on the changes feed will eventually be resolved. Without an end key specified a reader could end up in an infinite conflict retry loop if there are clients updating documents in the database.
* Allow snapshots through fabric2_fdb:transactionalPaul J. Davis2020-11-172-0/+151
|
* Disable TextIndex test since it is not implemeneted yetILYA Khlopotov2020-11-171-0/+4
|
* `q` and `n` are not needed for FDB based CouchDBILYA Khlopotov2020-11-171-1/+1
|
* Use `req_body` field if presentILYA Khlopotov2020-11-161-2/+5
| | | | | | | When we call `couch_httpd:json_body/1` we can have `req_body` already set. In this case we should return the field as is without any attempt to decompress or decode it. This PR brings the approach we use in `chttpd` into `couch_httpd`.
* Assure that all node ids are the same lengthRobert Newson2020-11-121-1/+1
|
* Avoid deleting UUID keys that start with zerosPaul J. Davis2020-11-121-1/+1
| | | | | Any ebtree that uses chunked key encoding will accidentally wipe out any nodes that have a UUID with more than one leading zero byte.
* Fix flaky testPaul J. Davis2020-11-101-2/+8
| | | | | | | | | Waiting for the timeout option to be set means we could still sneak in and grab the old FDB database handle before fabric2_server updated it in the application environment. This new approach just waits until the handle has been updated by watching the value in the application environment directly.
* Remove use of cachingPaul J. Davis2020-11-101-27/+1
| | | | | Turns out that ebtree caching wasn't quite correct so removing it for now.
* Remove ebtree cachingPaul J. Davis2020-11-101-112/+18
| | | | | | | | The ebtree caching layer does not work correctly in conjunction with FoundationDB transaction retry semantics. If we incorrectly cache nodes that are not actually read from FoundationDB, a retried transaction will rely on incorrectly cached state and corrupt the ebtree persisted in FoundationDB.
* Merge pull request #3244 from apache/aegis_db_infoRobert Newson2020-11-064-1/+29
|\ | | | | Add an "encryption" object to db info
| * Add an "encryption" object to db infoaegis_db_infoRobert Newson2020-11-064-1/+29
| | | | | | | | | | | | The encryption object contains a boolean "enabled" property. Additional properties might be added by the key manager which will appear in the "key_manager" sub-object.
* | Add test for splitting filter batchesPaul J. Davis2020-11-061-0/+55
| |
* | Retry filter_docs sequentially if the patch exceeds couchjs stackRobert Newson2020-11-061-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | A document with lots of conflicts can blow up couchjs if the user calls _changes with a javascript filter and with `style=all_docs` as this option causes up to fetch all the conflicts. All leaf revisions of the document are then passed in a single call to ddoc_prompt, which can fail if there's a lot of them. In that event, we simply try them sequentially and assemble the response from each call. Should be backported to 3.x
* | Allow administrators to optionally encrypt viewsPaul J. Davis2020-11-061-2/+15
|/
* Use snapshot for cache insertionpost-fdb-mergeJay Doane2020-10-261-1/+2
| | | | | | | | | Too many parallel attempts to insert the same keys can result in `{erlfdb_error, 1020}`, which translates to: "Transaction not committed due to conflict with another transaction" This attempts to mitigate the problem by using a snapshot to read the primary key during insertion.
* Fix dialyzer warningsJay Doane2020-10-263-5/+7
| | | | Fix specs to eliminate dialyzer warnings.
View Lorry Controller Status