| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
We don't want to register replication jobs with the module name as that
prevents us from running more than one job at a time.
This was introduced when we removed the dependence on global registrations and
switched to using pg instead.
Even more embarrassing is we didn't have tests to check multiple replication
jobs running at the same time.
|
| |
|
|
| |
Just make it use the Erlang 24 version like all the other images
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
This adds a test to cover the case when a replication doc contains a
user_ctx, and subsequently executes `get_json_value/3`.
See: https://github.com/apache/couchdb/pull/4343
|
| |
|
|
| |
Closes https://github.com/apache/couchdb-fauxton/issues/1382
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Don't run erlfmt and elixir format checks
We already ran those during the pull request CI phase
|
| | |
|
| | |
|
| |
|
|
|
| |
This is to help unblock 3.3 windows build so make check doesn't choke on
re-formatting elixir files for Windows.
|
| | |
|
| | |
|
| |
|
|
| |
Instead of having it download its own every time
|
| |
|
|
|
|
|
| |
Latest jiffy apparently is blocking windows builds so for now let's
revert to the previous version. The previous version didn't quite work
right off the bat as enc doesn't load on Erlang 25 any longer. So
there is a new 1.0.9-2 release wiht enc rebuilt on Erlang 23.
|
| |
|
|
|
|
|
|
|
| |
At some point it broke - it looks like rebar3, erlfmt and python-black
usage ended up needing the network to install/update themselves.
Make sure rebar3 and erlfmt is in bin/ alongside rebar, and make
python-black is in a separate source formatting stage step in CI and
remove it from `make check`.
|
| |
|
|
| |
It's a deprecated feature but we want to make sure it still works.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Since the replicator optimization issues were a bit scattered, create a
single parent one as a highlight, and also run somewhat more realistic
benchmark tests.
* Add an image macro to keep the tradition going.
* Update docs section with a 5.3.0 PR which just merged.
* Add reference to the newly merged `couchjs -v` enhancement.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
* fix(3517): super-simplistic fix to avoid costly AST transforms when they are not needed
Co-authored-by: Ronny Berndt <ronny@apache.org>
|
| |
|
|
|
| |
Fix wrong rendering of an inline literal with whitespaces.
Use `:literal` as workaround, with esacpes whitespace `\ ` and added
non-break-whitespace chars (U+00A0).
|
| |
|
|
|
|
|
| |
Documentation was mentioning the exponential backoff on `Crashing` and
`Error` states but did not mention how to configure it. This commit
adds note on relation of `max_history` and maximum backoff with link to
appropriate configuration section.
|
| |
|
|
|
|
|
|
|
| |
Give up trying to wait on compaction pids, since even if they finished the
compaction itself isn't done and swapping hasn't happened yet. Instead just
poll db and view sizes until those show a lowered file size.
To avoid other channels possibly getting in the way, by default unpausing only
the "ratio_dbs" one, and as needed unpause others depending on the test.
|
| |
|
|
|
|
| |
This may not be a 100% fix just doing extra asserts that file operations have
succeeded and making to wait for the compactor to exit and match the exit
reason exactly, to make sure we killed and it does as expected.
|
| |
|
|
|
|
|
|
| |
It's already false in 23 and 24 but will start to be enabled in 25+. We don't
rely on global for process registration any more, and have our own
auto-connection module. So we don't want to be caught by surprise in Erlang 25+
since there is some additional coordination and resource usage needed when this
option is true. See https://github.com/erlang/otp/issues/6470 for an example.
|
| |
|
|
| |
While the variable `RelativeFileName` is not really unbound in the case-statement, use a
more up-to-date pattern for binding the value.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
They are allowed only for the "k = v" format and not the "k=v" format. The idea
is to split on " = " first, and if that fails to produce a valid kv pair we
split on "=" as before.
To implement it, simplify the parsing logic and remove the undocumented
multi-line config value feature. The continuation syntax is not documented
anywhere and not used by our default.ini or in documentation.
Fix: https://github.com/apache/couchdb/issues/3319
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To hopefully fix a flaky test I noticed:
```
module 'ddoc_cache_lru_test'
...
ddoc_cache_lru_test:97: with (check_cache_refill)...*failed*
...
in function meck_proc:wait/6 (src/meck_proc.erl, line 171)
in call from ddoc_cache_lru_test:check_cache_refill/1 (test/eunit/ddoc_cache_lru_test.erl, line 204)
in call from eunit_test:run_testfun/1 (eunit_test.erl, line 71)
**error:timeout
```
|
| |
|
|
|
|
| |
Saw this fail 2 in a the last week or so but couldn't reproduce it locally.
Instead of deleting, this is an attempt to emit debug lines to see how far it
goes before it times out in the CI.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Global has a tendency to create deadlocks [1], and when that happens
replication jobs can't start on any of the nodes in the cluster.
We don't really need strict consistent locking for replication jobs. It's
mostly to avoid replication jobs thrashing the same checkpoint docs back and
forth between different session IDs. So, remove global to avoid any issues
around it, and replace it with `pg` -- the new (Erlang 23+) process group
implementation. (Technically `global` is still running in the runtime system as
it's started by the `kernel` app. We just avoid interacting with it and
registering any names to avoid deadlocks).
In `pg` we start a replication `scope`, and then in that scope make every RepId
a process `group`. When replication processes spawn, their Pids becomes
`members` of that group:
```
couch_replicator_pg (scope):
"a12c+create_target" (RepId group):
[Pid1, Pid2] (members)
...
```
As per `pg` implementation, groups are created and remove automatically as
members are added/removed from it, so we don't have to do anything there.
If there are already any running Pids in the same group, we avoid starting the
jobs, and fail like we did before when we used global. In the even more rare
case of a race condition, when 2+ jobs do manage to start, we do a membership
check before each checkpoint. One of the jobs then stops to yield to another.
For simplicity pick the one running on the lowest lexicographically sorted node
name to survive.
[1] https://github.com/erlang/otp/issues/6524
[2] https://www.erlang.org/doc/man/pg.html
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are two main improvements:
* Replace the auto-inserted replicator VDU with a BDU. Replicator already had
a BDU to update the `"owner"` field, so plug right into it and validate
everything we need there. This way, the validation and parsing logic is all
in one module. The previously inserted VDU design doc will be deleted.
* Allow constraining endpoint protocol types and socket options. Previously,
users could create replications with any low level socket options. Some of
those are dangerous and are possible "foot-guns". Restrict those options to
a more usable set.
In addition to those improvements, increase test coverage a bit by explicitly
checking a few more parsing corner cases.
Fixes #4273
|
| |
|
|
|
|
|
|
|
|
| |
Adjust configuration files to improve readability.
Adjust all config keys to use the same pattern (semi-colon w/o ws):
";key = value"
Adjust comments (semi-colon with ws):
"; My comment description"
Added new-line between description and config key, if config-key
has a description
|
| |
|
|
|
|
| |
add `LDFLAGS` and `CFLAGS` environment variables to `IcuEnv` in rebar
config to be able to configure ICU includes and object paths via
environment variable without having to patch `rebar.config.script`.
|
| |\
| |
| | |
add warning about misapprehending the node-local interface
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove the state chart. With activated/not-activated state gone, we don't
need it any longer.
* Describe the cleanup channels.
* Add upgrade db and view channel references in a few places.
* Remove references to `external` or `data_size` and other previous compaction
size metrics used for triggering compactions. Replace references with
`active` size.
* Use double back-ticks in a few places instead of single back-ticks due to
differences in RST vs MD. In RST code litterals need double back-ticks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Clean up, optimize and increase test coverage for smoosh.
* Use the new, simpler persistence module. Remove the extra `activated vs
non-activated` states. Use `handle_continue(...)` gen_server callback to
avoid blocking smoosh application initialization. Then, let channel
unpersist in their init function which happens outside general application
initialization.
* Add an index cleanup channel and enqueue index cleanup jobs by default.
* Remove gen_server bottlenecks for status and last update calls. Instead
rely on ets table lookups and gen_casts only.
* Add a few more `try ... catch`'s to avoid crashing the channels.
* Use maps to keep track of starting and active jobs in the channel.
* Re-check priority again before starting jobs. This is needed when jobs are
un-persisted after restart and database may have been compacted or deleted
already.
* Update periodic channel scheduled checks to have a common scheduling
mechanism.
* Quantize ratio priorities to keep about a single decimal worth of
precision. This should help with churn in the priority queue where the same
db is constantly added and removed for small insignificant ratio changes
like 2.0001 -> 2.0002. This works along a recent optimization in smoosh
priority queue module, where if priority matches that item is not removed
and re-added, it just stays where it is, reducing CPU and heap memory
churn.
* Remove testing-only API functions to avoid cluttering the API.
* Store messages off-heap for channels and smoosh_server.
* Instead of a per-channel `last_updated` gen_server:call(...), use a single
access ets table which is periodically cleaned from stale entries. As an
optimization, before enqueueing the shard, check last access first. This
avoids sending an extra message to smoosh_server.
* As a protection mechanism against overload, cap the access table size at
250k entries. Don't allow enqueueing more than that many entries during the
configured `[smoosh] staleness = Minutes` period.
* Increase test coverage from 60% to 90%
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove `Value` parameter as it was never used.
* Use `make_ref()` instead of `unique_integer([monotonic])` for performance [1].
* Rewrite tests to get 100% coverage. Previous tests didn't actually run due
to a setup error.
* Switch from `size/1` to `qsize/1` as `size/1` is a built-in and we have to
write `?MODULE:size/1` everywhere.
* Do not remove elements from the gb_tree just for inspecting min and max
elements.
* Remove `last_updated/2` logic, as that will be published in an ETS table.
* Replace low level file serialization operations with `to_map/1`, `from_map/3`.
[1]
```
4> timer:tc(fun() -> [make_ref() || _ <- lists:seq(1, 1000000)], ok end ).
{488923,ok}
6> timer:tc(fun() -> [{erlang:monotonic_time(), erlang:unique_integer([monotonic])} || _ <- lists:seq(1, 1000000)], ok end ).
{1178409,ok}
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add functions related to dealing with new index cleanup channel.
Add functions which fetch the channels list from config. Fix an unfortunate
corner case where operators when creating custom smoosh channel may omit by
mistake some of the built-in channel like view upgrades. To improve the
behavior, make built-in channel always available. Channels configured by the
user are always appended to the list of built-in ones.
The new validate_arg/1 function is in charge of validation smoosh's input args.
It centralizes ?b2l, ?l2b and other transforms and checks in one place so they
are not sprinkled throughout the internal functions. After the validation
steps all the db and view names are guaranteed to be binaries.
The most significant changes is probably addition of extra tests to get to 100%
test coverage. Since we cover all of the cases in time window checking, remove
the more heavyweight elixir test, since we now get a nice coverage report in
Erlang and the test is overall smaller as well.
|
