summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Parallelize view buildsarchive/prototype/fdb-layer-parallel-view-buildsprototype/fdb-layer-parallel-view-buildsPaul J. Davis2019-10-181-114/+177
|
* Use a shorter name for create_or_open_couchdb_dirNick Vatamaniuc2019-10-153-7/+7
|
* DRY out CouchDB FDB prefix fetchingNick Vatamaniuc2019-10-143-18/+12
| | | | | | It was suggested in another PR's discussion: https://github.com/apache/couchdb/pull/2107#pullrequestreview-274431487
* Fix timeout in couch_viewsNick Vatamaniuc2019-10-031-1/+1
| | | | set_type_timeout takes seconds as the argument but we gave it milliseconds
* Fix mango index validationPaul J. Davis2019-09-271-6/+1
| | | | | This check fails if Clouseau isn't present. Though we don't need Clouseau to perform the check so just avoid it.
* Update couch_views to use couch_evalPaul J. Davis2019-09-255-20/+58
|
* Add tests for couch_js applicationPaul J. Davis2019-09-253-1/+470
| | | | | These are ported over from the existing couch Eunit suite and updated to be less racey hopefully.
* Implement couch_js callbacks for couch_evalPaul J. Davis2019-09-252-0/+57
|
* Initial creation of couch_js applicationPaul J. Davis2019-09-2511-0/+2221
| | | | | | | | | This commit is mostly a copy paste of the existing modules in the `couch` application. For now I've left the build of the `couchjs` executable in `couch/priv` to avoid having to do the work of moving the build config over. I had contemplated just referencing the modules as they current exist but decided this would prepare us a bit better for when we eventually remove the old modules.
* Add couch_eval abstraction layerGarren Smith2019-09-256-0/+142
|
* Add more deleted docs replication cases to integration testNick Vatamaniuc2019-09-241-5/+57
| | | | | | | | | We test that a newly deleted document is replicated to the target and it bumps the deled doc count but doesn't change doc count. Another thing to test is that an already deleted document is replicated in where its revision path was extended on the source then gets replicated to the target. In that case neither del doc count not doc count are bumped.
* Fix doc counts for replicated deletionsNick Vatamaniuc2019-09-242-2/+36
| | | | | | | | | | | Do not decrement `doc_count` stat if document was previously missing, or if it was already deleted. Deleted documents could be brought in by replication. In that case, if there were more replicated documents than the current `doc_count`, the `doc_count` could even underflow the 64 bit unsigned integer range and end up somewhere in the vicinity of 2^64. The counter, of course, would still be incorrect even if it didn't underflow, the huge value would just make the issue more visible.
* Add revision stemming for interactive docsNick Vatamaniuc2019-09-202-9/+224
|
* Check members after db is openedNick Vatamaniuc2019-09-192-13/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | This brings this to parity with master `couch_db:open/2` logic: https://github.com/apache/couchdb/blob/master/src/couch/src/couch_db.erl#L166 There are two separate cases that have to be handled: 1) Db was already opened and cached. In that case, call `check_is_member(Db)` which reads the security from the Db to ensure we don't authorize against a stale security doc. Otherwise, the delay could be as long as the last write that went through that node. A potential future optimization could be to have a timestamp and only get the new security context if last refresh hasn't happened recently. 2) Db was not cached, and was just opened. To avoid running another two read transactions to get the security doc after the main transaction finished, call a version of check_is_member which gets the security doc passed in as an argument. As a bonus, `check_is_members(Db, SecDoc)` version ends up saving one extra security read since we don't read twice in is_member and is_admin calls. `delete/2` was updated to pass ?ADMIN_CTX to `open/2` since it only cares about getting a `database_does_not_exist` error thrown. There is a check for server admin at the HTTP API level that would care of authentication / authorization.
* Make get_security and get_revs_limit calls consistentNick Vatamaniuc2019-09-173-23/+45
| | | | | | | | | | | | | | There are two fixes: 1) In `fabric2_fdb:get_config/1`, Db was matched before and after `ensure_current/1`. Only the db prefix path was used, which doesn't normally change, but it's worth fixing it anyway. 2) We used a cached version of the security document outside the transaction. Now we force it go through a transaction to call `fabric2_fdb:get_config/1` which call `ensure_current/1`. When done, we also update the cached Db handle. Do the same thing for revs_limit even thought it is not as critical as security.
* Implement setting and getting _revs_limitNick Vatamaniuc2019-09-162-2/+19
|
* Merge pull request #2184 from cloudant/add-extra-arguments-to-beamiilyak2019-09-111-1/+15
|\ | | | | Support `--extra_args` parameter in `dev/run`
| * Support `--extra_args` parameter in `dev/run`ILYA Khlopotov2019-09-101-1/+15
|/ | | | | | | | | | | | | | | | | | | Sometimes there is a need to specify additional arguments for the beam process we start from dev/run. In particular the feature is handy for: - changing emulator flags - simulate OOM via available RAM restrictions - enable module loading tracing - configure number of schedulers - modify applications configuration - run customization script to add extra development deps (such as automatic code reload) Historically developers had to edit dev/run to do it. This PR adds an ability to specify additional arguments via `--extra_args` argument. In order to run customization script create `customization.erl` which exports `start/0` and run it using: ``` dev/run --extra_args='-run customization' ```
* Fix _changes heartbeat optionNick Vatamaniuc2019-09-092-5/+5
| | | | | | | | `TimeoutFun` was already returning `{ok|stop, UserAcc}` so there was no need to wrap it another `{ok, ...} tuple. Also TimeoutFun was calling user with`{timeout, _ResponseType}` not just timeout, so added a clause to handle that as well.
* Fetch docs in parallel for view indexingGarren Smith2019-09-091-13/+45
|
* Handle _changes since=now paramNick Vatamaniuc2019-09-092-5/+6
| | | | | On master this is happening in `fabric_view_changes` but on FDB we don't go through that module anymore, so we do in `chttpd_changes` module instead.
* Handle update_seq for _local_docsNick Vatamaniuc2019-09-062-11/+24
| | | | | | | | | | | On master `_local_docs&update_seq=true` returns the update sequence of the database. Even though it might not make much sense since updating local docs doesn't bump the sequence, it's probably a good idea to stay consistent. It's also worth mentioning another inconsistency is when FDB returns a `total_rows` count for `_local_docs` while master returns `null`. I think that's probably acceptable. Master would return the count if had it available easily and having it seems like a useful thing and an improvement.
* Add a max db name length config optionNick Vatamaniuc2019-09-053-0/+34
| | | | | | | This is done for compatibility with CouchDB < 4.x where this limit was implicitly enforced by the file system's max filename size. The default value enforces the same limit for FDB in case users decide to replicate back and forth between old and new instances with 'create_target = true' option.
* Implement _design_docs and _local_docsNick Vatamaniuc2019-09-044-21/+231
| | | | | | | | | `_design_docs` reuses `_all_docs` logic and adjusts `start_key` and `end_key` to be within the `_design/` prefix range. Namespace setting was simplified to never have an `undefined` value. This way it doesn't need extra case statements to handle it further down in the FDB code.
* Implement _all_docs/queriesNick Vatamaniuc2019-08-303-88/+290
| | | | | Also add mrargs validation to match what master does and provide some helpful feedback to the users.
* Fix starkey_docid, endkey_docid and inclusive_end=false for _all_docsNick Vatamaniuc2019-08-291-9/+24
|
* Remove debug logging in fabric2_db_pluginNick Vatamaniuc2019-08-291-3/+1
|
* Skip before_doc_update processing for local docsNick Vatamaniuc2019-08-292-1/+30
| | | | | | This matches behavior on master: https://github.com/apache/couchdb/blob/master/src/couch/src/couch_db.erl#L1373-L1387
* Fix couch_views encoding testNick Vatamaniuc2019-08-231-51/+74
| | | | | Start stop config and couch_drv in `setup/0` and `teardown/1` instead of the test function itself.
* Move fabric2_dir_prefix_tests to test directoryNick Vatamaniuc2019-08-221-0/+0
|
* Implement update_seq for couch_viewsNick Vatamaniuc2019-08-221-3/+11
|
* Open dbs with admin privileges in couch_views_indexerNick Vatamaniuc2019-08-221-1/+1
| | | | That is needed when indexing authentication dbs like `_users`.
* Fix DB reopen behaviorNick Vatamaniuc2019-08-221-10/+26
| | | | | | | | | | | | | | Previously if a VDU was updated in the `before_doc_update/3` handlers, the Db handle wasn't refreshed soon enough such that the `prepare_and_validate/3` would not see the newly update VDU and would not run it. To fix make a stale Db throws an exception which bubbles up all the way to `fabric2_fdb:transactional/2` where the transaction is retried again with a reopened Db. During `reopen/1` make sure to properly transfer the `user_ctx` property to the new handle, and also make sure `user_ctx` is removed from `db_options` to avoid caching it in `fabric2_server`.
* Implement fabric2_db EPI pluginNick Vatamaniuc2019-08-227-46/+269
| | | | | | This mostly equivalent to the `couch_db` EPI plugin, but using fabric2 calls and without some of the functions that are not relevant to FDB such as on_compact/1 and others.
* add trace test for couch_viewsGarren Smith2019-08-221-0/+140
|
* Merge pull request #2123 from ↵iilyak2019-08-191-11/+21
|\ | | | | | | | | cloudant/add-timeout-to-fold_docs_with_different_keys Add timeout for 'fold_docs-with_different_keys' test
| * Add timeout for 'fold_docs-with_different_keys' testILYA Khlopotov2019-08-161-11/+21
|/
* only fetch 1 query server for indexingGarren Smith2019-08-151-2/+5
|
* Configurable FDB directory prefixes for CouchDB instancesNick Vatamaniuc2019-08-126-6/+111
| | | | | | | Specifying a custom prefix allows having multiple CouchDB instances on a single FDB cluster. This can be used for one form of multi-tenancy. It can also be used for integration testing by creating a temporary prefix then deleting all data in that directory when the test has finished.
* Remember to remove old doc bodiesPaul J. Davis2019-08-061-5/+25
|
* Write large documents in chunksPaul J. Davis2019-08-062-17/+30
| | | | | This opens the max document size to the transaction limits rather than a single 100k value.
* Update to use new fold_range_future/wait functionsPaul J. Davis2019-08-061-6/+7
|
* Fix job handling to halt on errorsPaul J. Davis2019-08-011-8/+20
| | | | | | If the indexing job has timed out and has been requed we need to exit the current indexer. This ensures the errors are logged so that we can keep an eye on failing jobs.
* CouchDB map indexes on FDBGarren Smith2019-07-3128-28/+3189
| | | | | | This adds couch_views which builds map indexes and stores them in FDB. Co-authored-by: Paul J. Davis <paul.joseph.davis@gmail.com>
* Fix default key ranges for fold_rangePaul J. Davis2019-07-311-15/+16
| | | | | If a start or end key is not specified we still need to scope the range read to the given `RangePrefix`.
* Expose the is_replicator_db and is_user_db logicPaul J. Davis2019-07-312-12/+31
| | | | | This exposes a single place where we can check for whether a given database or database name is a replicator or users database.
* Make fabric2.hrl publicPaul J. Davis2019-07-311-0/+0
|
* Fix more elixir testsPaul J. Davis2019-07-313-10/+43
|
* Expose ICU ucol_getSortKeyRussell Branca2019-07-313-8/+219
|
* Disable broken couch_att testsPaul J. Davis2019-07-311-187/+187
|