summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Fetch doc in same transaction as _all_doc rowarchive/all_docs_same_txall_docs_same_txGarren Smith2020-04-171-1/+1
|
* Allow using cached security and revs_limit propertiesNick Vatamaniuc2020-04-163-11/+67
| | | | | | | | | | | | | | By default, transactions are used to check metadata, and possibly reopen the db, to get a current db handle. However, if a `max_age` option is provided and db handle was checked less than `max_age` milliseconds ago, use properties from that cached handle instead. The main use of this feature be in pluggable authorization handlers where it might be necessary to inspect the security doc multiple times for the same request before a final decision is made. `revs_limit/1` was updated as well, mainly for consistency since it is almost identical to `get_security/1`.
* Merge pull request #2783 from cloudant/merge-rebar-configiilyak2020-04-164-3/+16
|\ | | | | Merge keys from rebar.config
| * Merge keys from rebar.configILYA Khlopotov2020-04-144-3/+16
| | | | | | | | | | | | | | | | | | This change allows creation of local `src/couch/rebar.config` and `rebar.config` files to set additional configuration options. This is useful for: - disabling deprecation warnings `{nowarn_deprecated_function, MFAs}` - control debugging in eunit tests - `DEBUG` - `{eunit_compile_opts, [{d, DEBUG, true}]}` - `NODEBUG` - `{eunit_compile_opts, [{d, NODEBUG, true}]}`
* | Clean up old expiry key on update insertJay Doane2020-04-152-2/+44
| | | | | | | | | | | | | | | | | | | | | | | | When an existing key is inserted with different timestamps, the primary key is the same but the primary value is different from the existing one. Currently, this results in a new expiry key being inserted, but the old one is not deleted and lingers until it is removed by the inexorable advance of time via the `remove_expired` server messages. This checks whether there's already primary key for the inserted key, and if so, cleans up the existing expiry key before proceeding with the insert.
* | Refactor expiring cache FDB interfaceJay Doane2020-04-153-25/+38
| | | | | | | | | | | | | | | | | | - Rename `clear_expired_range` to `clear_range_to` - Move `EXPIRING_CACHE` layer prefix into fabric2.hrl - Move primary key setting to just after key & value calculations - Factor out `get_val/2` to lookup a key from FDB and unpack the value - Factor out `prefixes/2` - Factor out `fold_range/5`
* | report changes stats intermittently (#2777)Tony Sun2020-04-154-34/+171
| | | | | | | | | | | | | | | | | | * report changes stats intermittently with boolean market Stats are reported at the end of a request. With changes feeds, sometimes the request can be long or forever. This commit allows stats to be reported intermittently via a configurable time in seconds. The report function can return a boolean whether stats was reported so that a reset may not necessarily be needed.
* | Enable configurable binary chunk sizeJay Doane2020-04-153-4/+16
| | | | | | | | | | | | | | | | | | | | | | | | Currently, the size of binary chunks used for values is fixed at the FDB imposed limit of 100kB, although they recommend using 10KB [1], (also note they subtly change units). This makes that value configurable, allowing e.g. benchmarks to compare performance of runs with varying chunk size. The cost is a ~10µs config lookup penalty each time data needs to be chunked. [1] https://www.foundationdb.org/files/record-layer-paper.pdf
* | Merge pull request #2789 from cloudant/fdb-integrate-emilioiilyak2020-04-157-1/+182
|\ \ | |/ |/| | | | | Integrate emilio - erang linter Merging it on the grounds of CI pass and +1 in the original PR.
| * Integrate emilio - erang linterILYA Khlopotov2020-04-157-1/+182
|/
* Implement couch_views_cleanup_test.erlPaul J. Davis2020-04-101-0/+411
| | | | Add tests for view cleanup.
* Remove failed view jobsPaul J. Davis2020-04-103-8/+22
| | | | | If a client notices that a job has failed we restart it. If a job failed for a different design document id then we resubmit the build request.
* Remove jobs on index cleanupPaul J. Davis2020-04-102-1/+9
|
* Optionally cleanup stale indices automaticallyPaul J. Davis2020-04-102-5/+20
|
* Implement _view_cleanup for FoundationDBPaul J. Davis2020-04-102-2/+2
|
* Implement couch_views:cleanup_indices/2Paul J. Davis2020-04-102-1/+57
|
* Extend fabric2_index callbacks for index cleanupPaul J. Davis2020-04-101-0/+23
| | | | | Each registered index type can now get a signal on when to clean up their indexes.
* Update to use `fabric2_db:get_design_docs/1`Paul J. Davis2020-04-101-17/+1
|
* Move process_db/1 to match the logical progressionPaul J. Davis2020-04-101-10/+10
| | | | | | | Functions are easier to read and process if they're defined in the order that they are referenced. Co-Authored-By: Nick Vatamaniuc <vatamane@apache.org>
* Rename variables to indicate transaction statePaul J. Davis2020-04-101-3/+3
| | | | | | | | Usually we indicate the transaction status of a Db handle by naming it `TxDb`. This updates fabric2_index:build_indices/2 to match that pattern. Co-Authored-By: Nick Vatamaniuc <vatamane@apache.org>
* Fix index updater configuration keysPaul J. Davis2020-04-101-3/+3
| | | | | The defaults commented out in `default.ini` did not match the names used in `fabric2_index.erl`.
* Fix flaky fabric2_index testNick Vatamaniuc2020-04-101-3/+3
| | | | | | Previously in the the test we first set up the callback, then disabled the indexing. By that time, there was a chance that building could have started especially in a slower execution environment.
* Merge pull request #2775 from cloudant/mango-plugin-fixupiilyak2020-04-101-2/+5
|\ | | | | Fix incorrect usage of couch_epi in mango plugin
| * Fix incorrect usage of couch_epi in mango pluginILYA Khlopotov2020-04-101-2/+5
|/ | | | | Previously we used the value returned from couch_epi apply as is. However it returns a list of arguments passed in the same order.
* Make 'make check' run all the passing FDB tests on this branchNick Vatamaniuc2020-04-092-5/+7
| | | | | | | | This will let CI gate PRs hopefully as we now have the necessary FDB server bits installed in Jenkins. Eventually when we can run all the tests revert to using `make check-all-tests` as the default.
* Fix division by zeroPaul J. Davis2020-04-091-0/+2
|
* Fix job removal notificationsNick Vatamaniuc2020-04-092-0/+32
| | | | | | | | | | | Fix the case when a job is removed while there are subscribers waiting for it. Most of the logic was already there except: * Handle the case when when data decoded from subscription results could be `not_found`, in that case we just pass that atom back as is. * Need to notify the watch when jobs are removed or couch_jobs_notifiers would wake up and send notification messages.
* Merge pull request #2767 from cloudant/prototype/fdb-layer-mango-pluginiilyak2020-04-093-10/+57
|\ | | | | Prototype/fdb layer mango plugin
| * Add mango_pluginILYA Khlopotov2020-04-093-10/+57
|/ | | | | | Implement the following extention points: - `before_find(Req) -> {ok, Req}` - `after_find(Req, HttpResp, KVs) -> {ok, KVs}`
* Compress doc bodies and attachmentsNick Vatamaniuc2020-04-074-11/+90
| | | | | | | | | | | | | In CouchDB < 4.x we compressed document bodies by default, so enable it for 4.x as well. Use the basic term_to_binary compression mechanism for: - Document bodies - Local document bodies - Attachments, but only if they have not already been compressed.
* Merge pull request #2666 from apache/db-softdeletionPeng Hui Jiang2020-04-088-43/+847
|\ | | | | soft-deletion for database
| * Support soft-deletion in chttpd leveljiangph2020-04-073-29/+332
| | | | | | | | Co-Authored-By: Paul J. Davis <paul.joseph.davis@gmail.com>
| * Support soft-deletion in fabric leveljiangph2020-04-075-14/+515
|/ | | | | | | | | Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours. Co-Authored-By: Paul J. Davis <paul.joseph.davis@gmail.com>
* Start running chttpd eunit testsNick Vatamaniuc2020-04-0712-215/+92
| | | | | | | | | | | | | | Enable running all chttpd tests. Some fixes needed for this to happen are: * Some tests were not valid (checking shard maps, etc) and were deleted * Some tests were disabled either because functionality is not implemented yet or simply to minimize the diff between 3.x and this branch for when we have to rebase * Some applications used for index querying had to be started explicitly * Mock updated to use new version of modules instead of old ones
* Do not allow editing _security in _user databaseNick Vatamaniuc2020-04-071-1/+1
| | | | | It should only be allowed if explicitly configured. Previously we did not propertly match on the database name and effectively always allowed it.
* Fix POST _all_docs/queries endpointNick Vatamaniuc2020-04-072-4/+5
| | | | | | Call couch_views module instead of the old fabric:query_view also needed to call `view_cb(complete, ...)` when using keys similar to how `all_docs_view/4` does it.
* Return better responses for endpoints which are not implementedNick Vatamaniuc2020-04-073-23/+35
| | | | | | | | | | | | | | | | | | | Endpoints which are removed return a 410 response: - _show - _list - _rewrite Endpoints which will be implemented in CouchDB 4.x eventually now return a 510 response: - _purge - _purge_infos_limit Endpoints which return a 2xx but are a no-op effectively: - _compact - _view_cleanup
* remove defer and load_ddocs from mango_utilsGarren Smith2020-04-062-63/+0
| | | | Clean up unused mango_utils functions.
* update mango tests to work with Mango on FDBGarren Smith2020-04-0614-136/+129
|
* Update mango test creds to same as elixir testsGarren Smith2020-04-063-5/+5
|
* Update Mango query to work with couch_viewsGarren Smith2020-04-069-147/+139
|
* Add mango indexingGarren Smith2020-04-067-19/+187
| | | | | | This uses couch_views_updater to create mango indexes in the doc update along with the couch_views_indexer to update the indexes in the background up to the creation versionstamp.
* Remove quorum statsGarren Smith2020-04-063-13/+1
| | | | Removing quorum stats since they are not relevant with FDB.
* Remove view_cb predicate push downGarren Smith2020-04-061-94/+1
| | | | | Removes the view callback that was performed on the nodes before sending the results back to the co-ordinator.
* remove unneeded r/w parameterGarren Smith2020-04-066-67/+6
|
* remove partition opts from mangoGarren Smith2020-04-0610-200/+10
|
* remove mango native procGarren Smith2020-04-063-375/+0
|
* Add couch_views_encoding max valueGarren Smith2020-04-061-0/+12
| | | | | Adds a max value to use for encoding. This is useful when getting the max range when encoding startkey/endkeys.
* Add couch_views_updater interactive indexerGarren Smith2020-04-069-5/+470
| | | | | | This adds the ability for couch_views to index an index in the docs update transaction. This only happens if a design doc has the field <<"interactive">> = true.
* All couch_view queries to run across transactionsGarren Smith2020-04-061-1/+2
|