| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Pass error through (usually timeout)
|
| |/ |
|
| |
|
|
| |
Issue #551
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Logging is based on an environment variable:
`COUCHDB_IO_LOG_DIR`
If set, logs will go to that directory.
Logs are per `couch_os_process` Erlang process. There are 3 files saved for
each process:
```
<unixtimestamp>_<erlangpid>.in.log : Input, data coming from the proess
<unixtimestamp>_<erlangpid>.out.log : Output, data going to the process
<unixtimestamp>_<erlangpid>.meta : Error reason
```
Log files are saved as named (visible) files only if an error occurs. If there
is no error, disk space will still be used as long the process is alive. But as
soon as it exists, file will be unlinked and space will be reclaimed.
Issue: #551
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The timeout=1 (1ms) parameter would some times trigger extra newlines to be
included in the response body. The use of `binary:split/2` would then
return different portions of the body depending on timing in the
cluster. This change adds a helper function to split out all newlines in
the response and then returns the last non-empty line.
This also removes introspection of the clustered update sequence since
this is an HTTP API behavior tests and those are defined as opaque
values.
COUCHDB-3415
|
| |
|
|
| |
COUCHDB-3360/FB 85485
|
| |
|
|
|
|
| |
Even if it clean up fails use `kill` to avoid failing the next set of tests.
Issue: #571
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As it turns out, the original change in COUCHDB-3298 ends up hurting
disk usage when a view emits large amounts of data (i.e., more than
half of the btree chunk size). The cause for this is that instead of
writing single element nodes it would instead prefer to write kv nodes
with three elements. While normally we might prefer this in memory, it
turns out that our append only storage this causes a significantly more
amount of trash on disk.
We can show this with a few trivial examples. Imagine we write KV's a
through f. The two following patterns show the nodes as we write each
new kv.
Before 3298:
[]
[a]
[a, b]
[a, b]', [c]
[a, b]', [c, d]
[a, b]', [c, d]', [e]
[a, b]', [c, d]', [e, f]
After 3298:
[]
[a]
[a, b]
[a, b, c]
[a, b]', [c, d]
[a, b]', [c, d, e]
[a, b]', [c, d]', [e, f]
The thing to realize here is which of these nodes end up as garbage. In
the first example we end up with [a], [a, b], [c], [c, d], and [e] nodes
that have been orphaned. Where as in the second case we end up with
[a], [a, b], [a, b, c], [c, d], [c, d, e] as nodes that have been
orphaned. A quick aside, the reason that [a, b] and [c, d] are orphaned
is due to how a btree update works. For instance, when adding c, we read
[a, b] into memory, append c, and then during our node write we call
chunkify which gives us back [a, b], [c] which leads us to writing [a,
b] a second time.
The main benefit of this patch is to realize when its possible to reuse
a node that already exists on disk. It achieves this by looking at the
list of key/values when writing new nodes and comparing it to the old
list of key/values for the node read from disk. By checking to see if
the old list exists unchanged in the new list we can just reuse the old
node. Node reuse is limited to when the old node is larger than 50% of
the chunk threshold to maintain the B+Tree properties.
The disk usage improvements this gives can also be quite dramatic. In
the case above when we have ordered keys with large values (> 50% of the
btree chunk size) we find upwards of 50% less disk usage. Random keys
also benefit as well though to a lesser extent depending on disk size
(as they will often be in the middle of an existing node which prevents
our optimization).
COUCHDB-3298
|
| |
|
|
| |
This reverts commit 8556adbb98e79a09ec254967ee6acf3bef8d1fb6.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously idle dbs, especially sys dbs like _replicator once opened
once for scanning would stay open forever. In a large cluster with many
_replicator shards that can add up to a significant overhead, mostly in terms
of number of active processes.
Add a mechanism to close dbs which have an idle db updater. Before hibernation
was used to limit the memory pressure, however that is often not enough.
Some databases are only read periodically so their updater would time
out. To prevent that from happening keep the last read timestamp in
the couch file process dictionary. Idle check then avoid closing dbs
which have been recently read from.
(Original idea for using timeouts in gen_server replies belongs to
Paul Davis)
COUCHDB-3323
|
| |\
| |
| | |
Avoid using length to detect non empty list
|
| |/
|
|
|
|
| |
length(Values) is O(n) operation. Which could get expensive for long
lists. Change the code to rely on pattern matching to detect non empty
lists.
|
| |
|
|
|
| |
Update to latest Jiffy. Not a whole lot new but this removes a huge test
file that caused my source code analyzer to run slowly.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
There are two implementations for filtering the _changes feed by doc_ids. One
implementation is faster, but requires more server resources. The other is
cheaper but very slow. This commit allows configuration of the threshold at
which couch changes to using the slower implementation using:
[couchdb]
changes_doc_ids_optimization_threshold = 100
COUCHDB-3425
|
| |\
| |
| | |
Fix encoding issues
|
| |/
|
|
|
|
|
| |
This fixes encoding issues in responses on following requests:
- PUT: {db}/{design}/_update/{update}/{doc_id}
- PUT: {db}/{doc_id}/{att_id}
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
There are a race condition in `get_view/4`,
between aquiring index's Pid and getting
its state, that surface when the same
signature DDoc got rapidly deleted
and re-created.
This patch addresses this by adding
retry logic to get_view_index_state.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This test case is failing for the same reason as the failures in #559,
namely a GET on a _show, a PUT to the _show's ddoc to change the _show
function, and a subsequent GET on the same _show that returns a result
that is seemingly outdated. Late ddoc_cache eviction is still the
problem; setting ddoc_cache max_objects to 0 ensures this test always
passes.
Based on the discussion in #559 I am deleting this test as well, direct
on master with approval from @janl and @davisp.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The full discussion is in #559, but here is a summary.
Through instrumentation of ddoc_cache and ets_lru I suspect that this is
caused by cache eviction happening after the second GET. While I didn't
instrument exactly where the GET occurs it's clear that it's fairly
late, certainly after the PUT 201 is returned, and likely after the
subsequent GET actually reads from ddoc_cache.
After applying a change to allow me to completely disable the ddoc_cache
(-ddoc_cache max_objects 0 in vm.args) I ran the test on a loop
overnight, and the test never failed (>1000 executions). Previously the
test would fail every 20-30 executions.
TL;DR: we can't guarantee immediate ddoc_cache eviction on a ddoc
update, even for a single .couch file on a single node. (For obvious
reasons we definitely can't guarantee this in a cluster configuration.)
I will document this as a backwards compatibility change in 2.0 and
forward with a separate checkin to couchdb-documentation.
Thanks to @rnewson @janl and @davisp for helping track this one down!
This checkin also includes an improvement to the output when a JS test
fails a notEquals assertion.
Closes #559
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
restartServer() is still erroring out sometimes in Travis/Jenkins. This
PR both bumps the timeout to 15s as well as changes the detection
mechanism for restart to look for the uptime in _system to reset to a
low number.
This PR also removes the eclipsed redundant restartServer() definition
in couch_test_runner.js.
Closes #553
|
| |
|
|
|
|
|
|
|
|
|
| |
Before when a design doc is updated/deleted, only one couch_index
process was notified - the one which shard contained a design doc.
couch_index processes from other shards still continued to exist,
and indexing activities for these processes were still be going on.
The patch notifies couch_index_processes on all shards
COUCHDB-3400
|
| | |
|
| |\
| |
| |
| | |
COUCHDB-3417
|
| | |
| |
| |
| |
| |
| |
| |
| | |
To predict future compaction results, we log pre-compaction and
post-compaction file sizes. These log results will be used as data
points for regression analysis.
COUCHDB-3417
|
| | | |
|
| |\ \
| | |
| | | |
Export test_request:request/5 function
|
| |/ /
| |
| |
| |
| |
| | |
Currently it is impossible to pass authentication options to
test_request:request. This commit exports request/5 which accept options
argument.
|
| | | |
|
| | | |
|
| |\ \
| | |
| | | |
Send a better error when opening a db without authorisation
|
| | | |
| | |
| | |
| | | |
COUCHDB-3426
|
| |/ /
| |
| |
| | |
This option is no longer available.
|
