summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Fix doc order in fabric_doc_update testsfix-doc-order-in-fabric-doc-update-testNick Vatamaniuc2022-01-201-5/+5
| | | | | | | | The previous response order of {accepted, [{Doc1, {accepted, Doc2}}, {Doc2, {accepted, Doc1}} looked a bit odd so update the order to look as expected.
* Do not start test name with "test "Adam Kocoloski2022-01-202-2/+2
| | | | | | | | | | | | The code that generates suite.elixir will repreatedly strip the "test " from the name of the test when writing the file, resulting in a mismatch between the actual test name and what's in suite.elixir. You can see this by searching for e.g. COUCHDB-497 in the suite file. I tried using String.replace_prefix instead of String.replace_leading in test_name() but that function seems to get called multiple times during the test grouping. Simpler to just avoid naming the tests that way for now.
* Enable exunit in check targetJay Doane2022-01-201-0/+2
| | | | Also `export COUCHDB_TEST_ADMIN_PARTY_OVERRIDE=1` so couch will start.
* Merge pull request #3901 from apache/nonzero_instance_start_timeRobert Newson2022-01-194-9/+23
|\ | | | | Non-zero instance start time
| * Non-zero instance_start_timeRobert Newson2022-01-194-9/+23
|/ | | | | | | | | Set instance_start_time to the creation time of the database to restore the ability for the replicator to detect a db recreation event during a replication. Without this, a replication can fail to notice the db was deleted and recreated, write the checkpoint document for the in-progress replication and keep going, whereas what should happen is the replication should reset from sequence 0.
* Drop support for Ubuntu 16.04Adam Kocoloski2022-01-151-47/+0
|
* Use Debian Stable, add Erlang 24 to CI (#3895)Adam Kocoloski2022-01-142-12/+35
| | | | | | | | | | | | | | | | * Use Debian Stable, add Erlang 24 to CI * Use specific images for each Erlang version Instead of building one image with all supported Erlang versions through kerl, this configuration looks for a specific container image for each Erlang version. Decoupling it like this enables us to more easily adopt newer distros for newer Erlang versions, and to build new images with patch releases of Erlang without needing a simultaneous PR to the CouchDB repo to pick them up in CI (although some change to Jenkins might be needed to avoid images being cached for too long when a stable tag changes). * Bump Credo to 1.5.6 for Elixir 1.12 support
* Expose `decode/4` to skip decoding stepsJay Doane2022-01-121-3/+13
| | | | | | | | | | | | Currently, `decode/3` performs various checks on a JWT, and then base64 decodes and finally JSON decodes the token. However, in some cases, it's desirable to skip the decoding steps, and just return the token payload in binary form. This exposes `decode/4` where the 4th argument is a decoder fun that defaults to `decode_b64url_json/1` for `decode/3` to retain existing behavior, but also exposes `decode_passthrough/1` in case a client wants to avoid any decoding steps.
* Fix CI by updating the rebar3 branch from master -> mainNick Vatamaniuc2022-01-071-1/+1
|
* Fix changes feed rewinds after a shard move with no subsequent db updatesNick Vatamaniuc2022-01-071-4/+10
| | | | | | | | | | | | Previously, when a database shard is moved to a new node, and there are no subsequent updates, the changes feed sequence rewound to the previous epoch. In case of a first shard move, it would rewind to 0. To fix the issue, update `owner_of/2` and `start_seq/3` functions to account for the case when epoch sequence can exactly match the current db update sequence. Fixes #3885
* Remove ERL_OPTSncshaw2021-12-173-8/+3
|
* Merge pull request #3860 from apache/sharded_couch_index_serverRobert Newson2021-12-136-92/+173
|\ | | | | Sharded couch index server
| * Add sharding to couch_index_serverRobert Newson2021-12-136-92/+173
|/
* Reformat a few more files which slipped through the cracksNick Vatamaniuc2021-12-135-70/+111
| | | | | | One set of files is from a recent PR which we failed to detect because of the Eralng 20 issue. The others are some test modules which had been skipped during the initial reformatting.
* Run erlfmt-check in a separate build stageNick Vatamaniuc2021-12-132-2/+36
| | | | | | Previously, we ran it in the dist building stage. However, unlike in main, in 3.x that stage is run with Erlang 20 where erlfmt check is skipped. To fix that we run erlfmt in a separate stage with Erlang 23.
* Merge pull request #3869 from apache/always-send-all-cookie-attributes-3.xRobert Newson2021-12-101-15/+22
|\ | | | | Always send all cookie attributes
| * Always send all cookie attributesalways-send-all-cookie-attributes-3.xRobert Newson2021-12-101-15/+22
|/
* Fix TLS custom (couch) dist for Erlang 20Nick Vatamaniuc2021-12-082-1/+44
| | | | | | | | | | | * Fix quoting so that it works with all OTP versions 20 through 24 [1]. * Dist API in 20 [2] did not have a `listen/2` [3] callback. Implement `listen/1` so we're compatible with all the supported OTP versions. [1] https://github.com/apache/couchdb/issues/3821#issuecomment-985089867 [2] https://github.com/erlang/otp/blob/maint-20/lib/kernel/src/inet_tcp_dist.erl#L71-L72 [3] https://github.com/erlang/otp/blob/master/lib/kernel/src/inet_tcp_dist.erl#L79-L80
* Merge pull request #3864 from apache/erlang_ls_improveRobert Newson2021-12-081-1/+4
|\ | | | | improve erlang_ls.config
| * improve erlang_ls.configRobert Newson2021-12-081-1/+4
|/
* add GET _dbs_infojiahuili2021-12-084-28/+194
|
* Merge pull request #3861 from apache/delete-session-cookie-samesite-3.xRobert Newson2021-12-061-1/+1
|\ | | | | Delete session cookie samesite 3.x
| * Add SameSite setting when clearing session cookieRobert Newson2021-12-061-1/+1
|/
* Remove obsolete ddoc_cache_openerAdam Kocoloski2021-12-016-122/+3
|
* Avoid routing events through ddoc_cache_openerAdam Kocoloski2021-12-011-4/+2
| | | | | | | | This flow was presumably in place for some cluster upgrade scenarios several years ago, and no one ever completed the work to finalize the event flow. All the ddoc_cache_opener process does at this point is forward the event back to ddoc_cache_lru, so let's just send that message directly and remove the superfluous gen_server.
* Cleanup couch_ejson_compare C moduleNick Vatamaniuc2021-11-241-2/+6
| | | | | | | | | | | Add `static` modifiers to some of the newer functions. Fix function protype for get_collator to be `void`. This squashes this warning on MS compiler: ``` 'get_collator': no function prototype given: converting '()' to '(void)' ```
* Fix Jenkins email quoting failuresNick Vatamaniuc2021-11-231-7/+6
| | | | | | | Should fix: ``` Warning: A secret was passed to "mail" using Groovy String interpolation, which is insecure. ```
* Apply erlfmt formatting to source treeNick Vatamaniuc2021-11-22553-38647/+45794
| | | | | | | | | | | | | | | | | | | | | | | | | | | These exceptions from main were ported over to 3.x ``` --- a/src/chttpd/src/chttpd.erl +++ b/src/chttpd/src/chttpd.erl @@ -491,6 +491,7 @@ extract_cookie(#httpd{mochi_req = MochiReq}) -> end. %%% end hack +%% erlfmt-ignore set_auth_handlers() -> AuthenticationDefault = "{chttpd_auth, cookie_authentication_handler}, ``` ``` --- a/src/couch/src/couch_debug.erl +++ b/src/couch/src/couch_debug.erl @@ -49,6 +49,7 @@ help() -> ]. -spec help(Function :: atom()) -> ok. +%% erlfmt-ignore help(opened_files) -> ```
* Port erlfmt formatting to 3.xNick Vatamaniuc2021-11-2210-6/+236
| | | | | | | | | | | | | | | | | | From PR: #3568 A few changes to the formatting logic compared to `main`: * rebar3 and `erlfmt` don't work with Erlang 20, so added an OTP version check to skip formatting for Erlang 20 * `make erlfmt-*` commands are faster. Instead of spawning an `erlfmt` process per-file, spawn one per `directory/*.erl` pattern. * Remove the non-0 return code work-around. `erlfmt` was returning an error on `main` because the line width option used for formatting was different than the one used during format checking. Making both the default removed all the spurious non-0 exits with the current version of `erlfmt`. Thanks to Adam Kocoloski for pointing out the issue.
* Reduce skipped integration testsJay Doane2021-11-141-285/+1
| | | | | | | | | The `skip.elixir` file was ported [1] from FDB-based main branch but contains more exclusions than necessary for 3.x. This removes tests from `skip.elixir` that pass on 3.x. [1] https://github.com/apache/couchdb/commit/65fbc6cfdb29bab1072a42f4d6005da452a7fa65#diff-d9485d6f3162015017bc09c9db3f87df0ca489d961c44347241a035854eb8044
* Eliminate elixir test compiler warningsJay Doane2021-11-103-3/+3
| | | | | | | | | | | | | | | | Eliminates the following: warning: variable "epoch" is unused (if the variable is not meant to be used, prefix it with an underscore) src/couchdb/test/elixir/test/replication_test.exs:1756: ReplicationTest.seq_to_shards/1 warning: variable "uuid" is unused (if the variable is not meant to be used, prefix it with an underscore) src/couchdb/test/elixir/test/replication_test.exs:1756: ReplicationTest.seq_to_shards/1 warning: variable "string" is unused (if the variable is not meant to be used, prefix it with an underscore) src/couchdb/test/elixir/test/utf8_test.exs:32: UTF8Test."test UTF8 support"/1 warning: variable "repeated_view_offset_test_fun" does not exist and is being expanded to "repeated_view_offset_test_fun()", please use parentheses to remove the ambiguity or change the variable name src/couchdb/test/elixir/test/view_offsets_test.exs:63: ViewOffsetTest."test repeated view offsets"/1
* Using _bulk_docs with new_edits false and without _rev should responds 400 ↵jiahuili2021-11-082-1/+43
| | | | Bad Request
* Use `weatherreport_check_nodes_connected:check/1`Jay Doane2021-11-061-1/+1
| | | | | | | | | | The arity 0 version does not exist. Fixes dialyzer warning: {warn_callgraph,{"src/weatherreport_node.erl",126}, {call_to_missing,[weatherreport_check_nodes_connected,check, 0]}}
* Bump version to 3.2.1 and update dependencies3.2.1-RC13.2.1Nick Vatamaniuc2021-11-023-4/+4
|
* Fix reduce view row collation with unicode equivalent keysNick Vatamaniuc2021-11-011-3/+18
| | | | | | | | | | | | Previously, view reduce collation with keys relied on the keys in the rows returned from the view shards to exactly match (=:=) the keys specified in the args. However, in the case when there are multiple rows which compare equal with the unicode collator, that may not always be the case. In that case when the rows are fetched from the row dict by key, they should be matched using the same collation algorithm as the one used on the view shards.
* fix(local.ini): straggling s/couch_httpd_auth/chttpd_auth/Jay Doane2021-10-301-1/+1
| | | | | | | | | | | | | | | | | | | | | The appropriately named `hack_local_ini` function in dev/run lives up to its name by adding a random `secret` to the end of whichever section contains the line `"; require_valid_user = false\n"`. Recent commits [1] updated the `setup` application from `couch_httpd_auth` to `chttpd_auth` without also changing this line in local.ini, which had the fun effect of causing `setup` to fail with this error: ``` [error] 2021-10-30T18:50:15.019274Z node1@127.0.0.1 <0.817.0> 85d0421b9c setup sync_admin results [{badrpc,{'EXIT',{function_clause,[{config,set,["chttpd_auth","secret",undefined,true,nil],[{file,"src/config.erl"},{line,189}]},{rpc,'-handle_call_call/6-fun-0-',5,[{file,"rpc.erl"},{line,197}]}]}}},{badrpc,{'EXIT',{function_clause,[{config,set,["chttpd_auth","secret",undefined,true,nil],[{file,"src/config.erl"},{line,189}]},{rpc,'-handle_call_call/6-fun-0-',5,[{file,"rpc.erl"},{line,197}]}]}}}] errors [] [notice] 2021-10-30T18:50:15.019530Z node1@127.0.0.1 <0.817.0> 85d0421b9c 127.0.0.1:15984 127.0.0.1 adm POST /_cluster_setup 500 ok 12 ``` This change "fixes" dev/run by also updating the straggling section name in local.ini. [1] https://github.com/apache/couchdb/pull/3808
* feat(prometheus): allow users with _metrics role to readJan Lehnardt2021-10-301-0/+2
| | | | Closes #3801
* fix(auth): straggling s/couch_httpd_auth/chttpd/Jan Lehnardt2021-10-301-1/+1
|
* fix(setup): straggling s/couch_httpd_auth/chttpd_auth/ closes #3805Jan Lehnardt2021-10-301-2/+2
|
* Eliminate custodian false positive errors for dbs with N < default NNick Vatamaniuc2021-10-281-4/+3
| | | | | | | | | | | | | Previously, dbs with N < cluster default N would pollute logs with critical errors regarding not having enough shards. Instead, use each database's expected N value to emit custodian reports. Note: the expected N value is a bit tricky to understand since with shard splitting feature, shard ranges are not guaranteed to exactly match for all copies. The N value is then defined as the max number of rings which can be completed with the given set of shards -- complete the ring once, remove participating shards, try again, etc. Lucky for us, that function is already written (`mem3_util:calculate_max_n(Shards)` so we are just re-using it.
* Use configured shards db in custodian instead of `"dbs"`Nick Vatamaniuc2021-10-282-2/+3
|
* Mock `couch_log:warning/2`Jay Doane2021-10-271-5/+22
| | | | | | | | | | | | | | | | | Prevent failures like: mem3_rep: find_source_seq_unknown_node_test...*failed* in function gen_server:call/2 (gen_server.erl, line 206) in call from couch_log:log/3 (src/couch_log.erl, line 73) in call from mem3_rep:find_source_seq_int/5 (src/mem3_rep.erl, line 248) in call from mem3_rep:'-find_source_seq_unknown_node_test/0-fun-0-'/0 (src/mem3_rep.erl, line 794) **exit:{noproc,{gen_server,call, [couch_log_server, {log,{log_entry,warning,<0.17426.5>, ["mem3_rep",32,102,105,110,100|...], "--------", ["2021",45,"10",45|...]}}]}} output:<<"">>
* Use unique ddoc id to prevent collisionsJay Doane2021-10-271-2/+4
| | | | | | | | | | | | | | | | | | | | Prevent failures like this from repeated test runs: mem3_bdu_test:73: mem3_bdu_shard_doc_test_ (t_design_docs_are_not_validated)...*failed* in function mem3_bdu_test:'-t_design_docs_are_not_validated/1-fun-0-'/1 (test/eunit/mem3_bdu_test.erl, line 206) in call from mem3_bdu_test:t_design_docs_are_not_validated/1 (test/eunit/mem3_bdu_test.erl, line 206) in call from eunit_test:run_testfun/1 (eunit_test.erl, line 71) in call from eunit_proc:run_test/1 (eunit_proc.erl, line 510) in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 335) in call from eunit_proc:handle_test/2 (eunit_proc.erl, line 493) in call from eunit_proc:tests_inorder/3 (eunit_proc.erl, line 435) in call from eunit_proc:with_timeout/3 (eunit_proc.erl, line 325) **error:{assertEqual,[{module,mem3_bdu_test}, {line,206}, {expression,"Code"}, {expected,201}, {value,409}]} output:<<"">>
* Parameterize shards dbJay Doane2021-10-271-31/+31
| | | | | Depending on configuration, it is possible for the shards db to be different than `_dbs`.
* Mock `couch_log` for config applicationJay Doane2021-10-271-0/+4
| | | | | | | | | | | | | | | Prevent failures like this: mem3_sync_event_listener:267: should_set_sync_delay...*failed* in function gen_server:call/3 (gen_server.erl, line 214) in call from mem3_sync_event_listener:'-should_set_sync_delay/1-fun-1-'/1 (src/mem3_sync_event_listener.erl, line 268) **exit:{{noproc,{gen_server,call, [couch_log_server, {log,{log_entry,notice,<0.31789.5>, ["config",58,32,91,[...]|...], "--------", ["2021",45,[...]|...]}}]}},
* Minimize rewinds when a node is down (#3792)Adam Kocoloski2021-10-271-17/+59
| | | | | | | | | | | | | | | | | | | | | Our existing logic for handling rewinds in the changes feed addresses the following cases: - A node that contributed to a sequence is in maintenance mode - A shard that contributed to a sequence has been split This patch adds support for cases where the node that contributed to a client-supplied sequence is down at the beginning of the request handling. It reuses the same logic as the maintenance mode case as these two situations really ought to be handled the same way. A future improvement would be to unify the "node down" and "shard split" logic so that we could handle the compound case, e.g. replacing a shard from a down node with a pair of shards from nodes that cover the same range. Fixes #3788 Co-authored-by: Nick Vatamaniuc <vatamane@gmail.com>
* Eliminate eunit compiler warningsJay Doane2021-10-256-33/+18
| | | | | | | | | - Unused functions in `couch_util_tests` - Unused variables in `couch_prometheus_e2e_tests` - Unused variable in `dreyfus_blacklist_await_test` - Deprecated BIF `erlang:now/0` in `dreyfus_purge_test` - `export_all` flag in dreyfus tests - Unused variable in `mem3_reshard_test`
* Fix flaky mem3_bdu testNick Vatamaniuc2021-10-251-9/+1
| | | | | | The test only checks that we can update the shard doc so we just verifythat. Apparently, it doesn't mean we can synchronously access the newly created db info right away so we just skip that part to avoid a flaky failure.
* Fix flaky retain_stats replicator testNick Vatamaniuc2021-10-241-6/+14
| | | | | | | | | | | | | | | | Noticed this flaky test show up in a few recent test runs, for [example](https://ci-couchdb.apache.org/blue/organizations/jenkins/jenkins-cm1%2FPullRequests/detail/PR-3799/1/pipeline) The test was flaky as We were only waiting for the replication task or scheduler job to appear in the list but didn't not wait until the value of the task had been updated to an expected value. So the task might have appeared but then only half the docs written (say, 5 instead of 10). Testing the value at that stage is too early and the test would fail. To fix the issue, besides waiting on the task/job to appear in the list, also wait until its `docs_written` value matches the expected value. By that point `docs_read` should have caught up as well.
* Add libicu version fetching and emit it in the _node/_local/_versionsNick Vatamaniuc2021-10-244-10/+98
| | | | | | | Fetch the libicu base version as well as the collator version. The base version may be used to determine which libicu library CouchDB is using. The collator version may be used to debug view behavior in case when collation order had changed from one version ot the next.