| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we were testing if Pos + TotalBytes exceeded the pread
limit. This is the wrong logic entirely. We are trying to prevent an
attempted call to file:pread/3 where the third parameter, the number
of bytes to read, is a very large number (due to a corruption
elsewhere, say). Instead we throw exceed_limit as soon as a file gets
above a certain size.
I switched this to an if statement to make it clear that the "read
past EOF" and "try to read too many bytes" checks are quite distinct
from each other.
|
|\
| |
| | |
Add document_purges counter for stats
|
|/
|
|
| |
COUCHDB-3326
|
|\
| |
| | |
Restrict access to `_active_tasks` to server admin
|
|/ |
|
|\
| |
| | |
Pass user_ctx in _bulk_get
|
|/
|
|
| |
This fixes _bulk_get for _users db and probably others I don't know
|
|\
| |
| | |
Validate database prefix against DBNAME_REGEX for system dbs
|
|/
|
|
|
|
|
|
|
| |
Previously we only checked that the suffix of the database is
matching one of the predefined system databases. We really should
check the prefix against DBNAME_REGEXP to prevent creation of
illegally named databases.
This fixes #1644
|
|
|
|
| |
Fixes #1396
|
|
|
|
|
|
|
|
|
| |
This can now return references that are from NIFs monitoring the
process. This is important for the new file IO NIFs that monitor the
controlling process. For now we'll just take the easy way out by
filtering the references from our returned monitor lists.
Fixes #1396
|
|
|
|
| |
Fixes #1396
|
|
|
|
|
|
|
|
|
|
| |
This is a temporary bandaid to allow us to continue using parameterized
modules with Erlang 21. We'll have to go back and modify every one of
these files to avoid that as well as figuring out how to upgrade
mochiweb to something that doesn't use parameterized modules by the time
they are fully removed from Erlang.
Fixes #1396
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously a user could insert a VDU function into one of the _replicator
databases such that it prevents the replicator application from updating
documents in that db. Replicator application would then crash and prevent
replications from running on the whole cluster.
To avoid crashing the replicator when saving documents, log the error
and return `{ok, forbidden}`. The return might seem odd but we are
asserting that forbidden is an OK value in this context and explicitly
handling it. This shape of the return also conforms to the expected
`{ok, _Rev}` result, noticing that `_Rev` is never actually used.
|
| |
|
|
|
|
| |
Fixes #1612
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This ensures they will be ready to process requests as soon as the application
starts up. This should make the service available sooner and should help tests
which setup and tear down the services repeatedly, where it would avoid an
annoying retry-until-ready loop.
Per-node servers/buffers are started in the init method of the monitors. There
is not chance of deadlock there because per-node supervisors are started before
the monitors.
Issue #1625
|
|
|
|
|
|
|
| |
This has been solid for years and when not enabled can be a performance
bottleneck.
Fixes #1625
|
|
|
|
|
|
|
|
|
|
| |
This used to be the case before the scheduling replicator:
https://github.com/apache/couchdb-couch-replicator/blob/master/src/couch_replicator.erl#L166
This is also how replications backed by a document in a _replicator db behave:
https://github.com/apache/couchdb/blob/master/src/couch_replicator/src/couch_replicator_doc_processor.erl#L283
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously when an owner process crashed before it had a chance to release the
worker to the pool, the worker entry was simply deleted. In some cases that
was ok because ibrowse's inactivity timeout would kick in and connection would
stop itself. In other cases, as observed in practice with _changes feed
connection over TLS protocol, inactivity timeout would never fire, so these
deleted connections would slowly accumulate leaking memory and filling the
process table. TLS connection would keep an associated session
open as well making things even worse.
To prevent the connection leak, explicitly unlink and kill the worker.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Recent Linux distributions start defaulting to Python 3, and require
ambiguous scripts to be more explicit.
For example building for Fedora 30 (not released yet) fails with:
ERROR: ambiguous python shebang in /opt/couchdb/bin/couchup:
#!/usr/bin/env python. Change it to python3 (or python2) explicitly.
So this commit changes the four Python scripts to use `python2`.
Note: They seem to be Python-3-compatible, but I couldn't be sure. If
you know they are, please tell me, I'll change it to `python3`.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Mango match doc on co-ordinating node
This fixes an issue when doing a rolling upgrade of a CouchDB cluster
and adding commit a6bc72e the nodes that were not upgraded yet would
send through all the docs in the index and those would be passed through
to the user because the co-oridnator would assume it was matched at the
node level. This adds in a check to see if it has been matched at the
node level or not. And then performs a match if required.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previosly local node revisions were causing `badmatch` failures in read repair
filter. Node sequences already filtered out local nodes while NodeRevs didn't, so
during matching `{Node, NodeSeq} = lists:keyfind(Node, 1, NodeSeqs)` Node would
not be found in the list and crash.
Example of crash:
```
fabric_rpc:update_docs/3 error:{badmatch,false}
[{fabric_rpc,'-read_repair_filter/3-fun-1-',4,[{file,"src/fabric_rpc.erl"},{line,360}]},
```
|
|\
| |
| | |
Implement couch_file:format_status to log filepath
|
|/ |
|
|\
| |
| | |
Couch server improvements
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The couchdb.update_lru_on_read setting controls whether couch_server
uses read requests as LRU update triggers. Unfortunately, the messages
for update_lru on reads are sent regardless of whether this is enabled
or disabled. While in principle this is harmless, and overloaded
couch_server pid can accumulate a considerable volume of these messages,
even when disabled. This patch prevents the caller from sending an
update_lru message when the setting is disabled.
|
|/
|
|
|
|
|
|
| |
This adds the read_concurrency option to couch_server's ETS table for
couch_dbs which contains the references to open database handles. This
is an obvious improvement as all callers opening database pids interact
with this ETS table concurrently. Conversely, the couch_server pid is
the only writer, so no need for write_concurrency.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Off-heap messages is an Erlang 19 feature:
http://erlang.org/doc/man/erlang.html#process_flag_message_queue_data
It is adviseable to use that setting for processes which expect to receive a
lot of messages. CouchDB sets it for couch_server, couch_log_server and bunch
of others as well.
In some cases the off-heap behavior could alter the timing of message receives
and expose subtle bugs that have been lurking in the code for years. Or could
slightly reduce performance, so a safety measure allow disabling it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Its possible that a busy couch_server and a specific ordering and timing
of events can end up with an open_async message in the mailbox while a
new and unrelated open_async process is spawned. This change just ensure
that if we encounter any old messages in the mailbox that we ignore
them.
The underlying issue here is that a delete request clears out the state
in our couch_dbs ets table while not clearing out state in the message
queue. In some fairly specific circumstances this leads to the message
on in the mailbox satisfying an ets entry for a newer open_async
process. This change just includes a match on the opener process.
Anything unmatched came before the current open_async request which
means it should be ignored.
|
|
|
|
|
|
|
|
| |
A rather uncommon bug found in production. Will write more as this is
just for show and tell.
For now this test case just demonstrates the issue that was discovered.
A fix is still being pondered.
|
|
|
|
|
|
|
| |
If couch_server terminates while there is an active open_async process
it will throw a function_clause exception because `couch_db:get_pid/1`
will fail due to the `#entry.db` member being undefined. Simple fix is
to just filter those out.
|
|\
| |
| | |
Log error when changes forced to rewind to beginning
|
|/ |
|
|\
| |
| | |
Create shard files if missing
|
|/
|
|
|
|
|
|
|
|
| |
If, when a database is created, it was not possible to create any of
the shard files, the database cannot be used. All requests return a
"No DB shards could be opened." error.
This commit changes fabric_util:get_db/2 to create the shard file if
missing. This is correct as that function has already called
mem3:shards(DbName) which only returns shards if the database exists.
|
|
|
|
|
|
|
|
|
|
|
| |
We removed a security call in `do_db_req` to avoid
a duplicate authorization check and as a result
there are now no db validation in noop call
`/db/_ensure_full_commit`. This makes it always
return a success code, even for missing databases.
This fix places the security check back, directly
in _ensure_full_commit call and adds eunit tests
for a good measure.
|
|\
| |
| | |
Implement convinience `mem3:ping/2` function
|
|/
|
|
|
|
|
|
| |
Sometimes in operations it is helpful to re-establish connection between
erlang nodes. Usually it is achieved by calling `net_adm:ping/1`. However
the `ping` function provided by OTP uses `infinity` timeout. Which causes
indefinite hang in some cases. This PR adds convinience function to be
used instead of `net_adm:ping/1`.
|
|\
| |
| | |
Improve cleanup_index_files
|
|/
|
|
|
|
|
|
|
| |
The previous implementation was based on a search using
{view_index_dir}/.shards/*/{db_name}.[0-9]*_design/mrview/*
This wildcard includes all shards for all indexes of all databases.
This PR changes the search to look at index_directory of a database.
|
|\
| |
| | |
Fix dialyzer warning of shard record construction
|
|/
|
|
|
|
|
|
| |
- Fix dialyzer warning that record construction #shard violates
the declared type in fabric_doc_open_revs.erl,
cpse_test_purge_replication.erl and other files
Fixes #1580
|
|\
| |
| | |
Improve validation of database creation parameters
|
|/ |
|
|\
| |
| | |
Fix make warning from cpse_test_purge_seqs.erl
|
|/
|
|
| |
Fixes #1572
|
|
|
|
|
|
|
|
|
|
| |
The builting _sum reduce function has no protection against overflowing
reduce values. Users can emit objects with enough unique keys to cause
the builtin _sum to create objects that are exceedingly large in the
inner nodes of the view B+Tree.
This change adds the same logic that applies to JavaScript reduce
functions to check if a reduce function is properly reducing its input.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, as described in issue #1571, `rexi_server_sup` supervisor could die
and restart. After it restarts `rexi_server_mon` would not respan rexi servers
as it wouldn't notice `rexi_server_sup` went away and come back. That would
leave the cluster in a disabled state.
To fix the issue, switch restart strategy to `rest_for_one`. In this case, if a
child at the top dies it will restart all the children below it in the list.
For example, if `rexi_server` dies, it will restart all the children. If
`rexi_server_sup` dies, it will restart `rexi_server_mon`. And then on restart
`rexi_server_mon` will properly spawn all the rexi servers.
Same for the buffers, if `rexi_buffer_sup` dies, it will restart `rexi_buffer_mon`
and on restart it will spawn buffers as expected.
Fixes: #1571
|
|\
| |
| |
| |
| | |
apache/COUCHDB-3326-clustered-purge-pr5-implementation
[5/5] Clustered Purge Implementation
|