| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Simplify and speedup dev node startup
This patch introduces an escript that generates an Erlang .boot script
to start CouchDB using the in-place .beam files produced by the compile
phase of the build. This allows us to radically simplify the boot
process as Erlang computes the optimal order for loading the necessary
modules.
In addition to the simplification this approach offers a significant
speedup when working inside a container environment. In my test with
the stock .devcontainer it reduces startup time from about 75 seconds
down to under 5 seconds.
* Rename boot_node to monitor_parent
* Add formatting suggestions from python-black
Co-authored-by: Paul J. Davis <paul.joseph.davis@gmail.com>
|
| |\ |
|
| | | |
|
| | |
| |
| |
| | |
* Add support for specify a custom config file for CouchDB startup during testing
* Port proxyauth test from js to elixir
|
| |/ |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Added db initialization for standalone instance
* Test setup
* Updated readme
* Added standalone option to rel/overlay/etc/default.ini
* Simplified test setup
* s/standalone/single_node/
Co-authored-by: Jan Lehnardt <jan@apache.org>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Sometimes there is a need to specify additional arguments for the beam process we start from dev/run.
In particular the feature is handy for:
- changing emulator flags
- simulate OOM via available RAM restrictions
- enable module loading tracing
- configure number of schedulers
- modify applications configuration
- run customization script to add extra development deps (such as automatic code reload)
Historically developers had to edit dev/run to do it.
This PR adds an ability to specify additional arguments via `--extra_args` argument.
In order to run customization script create `customization.erl` which exports `start/0` and run it using:
```
dev/run --extra_args='-run customization'
```
|
| |
|
|
|
|
|
|
|
| |
Sometimes admin party mode causes the 'dev/run' to fail with
```
http.client.RemoteDisconnected: Remote end closed connection without response
```
This PR makes this use case more robust.
|
| | |
|
| |
|
|
| |
Remove couch_compaction_daemon and related tests too.
|
| |
|
|
|
| |
The problem was introduced in 94eff0d8 during rebase of
https://github.com/apache/couchdb/pull/1774
|
| | |
|
| |
|
|
|
| |
The Makefile target builds a python3 venv at .venv and installs
black if possible. Since black is Python 3.6 and up only, we
skip the check on systems with an older Python 3.x.
|
| | |
|
| |
|
|
|
|
| |
Ran 2to3 and fixed a few deprecated warnings
Issue #1632
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CouchDB ships with two default query_servers (javascript and coffeescript)
as well as one default native_query_server (query aka mango). These used
to be configured in default.ini in these sections:
```
[query_servers]
javascript = {{prefix}}/bin/couchjs {{prefix}}/share/server/main.js
coffeescript = {{prefix}}/bin/couchjs {{prefix}}/share/server/main-coffee.js
; enable mango query engine
[native_query_servers]
query = {mango_native_proc, start_link, []}
; erlang query server
; erlang = {couch_native_process, start_link, []}
```
This allowed end-users post-install and even runtime-changes to which
query servers are enabled and where their binaries live.
This patch changes things, so only a post-install, but not at-runtime
changes are possible from now on.
This still allows people to configure their CouchDB to run a third-
party query server like the somewhat popular Python query server,
but it changes the way the setup is done.
Query Servers
The javascript and coffeescript query servers continue to be enabled
by default. Setup differences have been moved from default.ini to
the couchdb and couchdb.cmd start scripts respectively.
Additional query servers can now be configured using environment
variables:
```
export COUCHDB_QUERY_SERVER_PYTHON="/path/to/python/query/server.py with args"
couchdb
```
Where the last segment in the environment variable matches the usual
lowercase(!) query language in the design doc `language` field.
Multiple query servers can be configured by using more environment
variables.
Native Query Servers
The mango query server continues to be enabled by default. The erlang
query server continues to be disabled by default. This patch adds
a `[native_query_servers] enable_erlang_query_server = BOOL` setting
(defaults to `"false"`) to enable the erlang query server.
If the legacy configuration for enabling the query server is detected,
that is counted as a `true` setting as well, so existing configurations
continue to work just fine.
Windows
Since the setting of the `./configure` time `PREFIX` happens during
`make release`, I had to adapt the `couchdb` and `couchdb.cmd` scripts
to have the correct env vars set and the `PREFIX` replaced there.
I did this to the best of my abilities and research, but this needs
review from the Windows team (Hi Joan! :).
OS Daemons
Although deprecated in 2.2.0, we’re keeping support for this until 3.x,
but the configuration changes analogous to query servers.
Previously, configuration looked like this:
```
[os_daemons]
name = /path/to/daemon with args
```
With this patch, setup looks like this:
```
COUCHDB_OS_DAEMON_NAME="/path/to/daemon with args"
couchdb
```
Multiple OS Daemons can be started with multiple env vars. The final
segment in the env var becomes the daemon identifier inside CouchDB
as lowercase(!).
|
| | |
|
| |
|
|
|
|
|
|
| |
Add degrade-cluster option for cluster testing
Add tests for different cluster conditions with/without quorum
Add test-cluster-with-quorum and test-cluster-without-quorum tasks
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
When a JS test requested a restart server we would wip the current log
file. This makes it hard to debug failing tests occasionally when they
happen just after a restart. This change prevents just opens log files
in read/write mode specifically when a test requests a server restart.
The current behavior for interactive use of `dev/run` will continue to
truncate log files on startup.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Glue together all the scheduling replicator pieces.
Scheduler is the main component. It can run a large number of replication jobs
by switching between them, stopping and starting some periodically. Jobs
which fail are backed off exponentially. Normal (non-continuous) jobs will be
allowed to run to completion to preserve their current semantics.
Scheduler behavior can configured by these configuration options in
`[replicator]` sections:
* `max_jobs` : Number of actively running replications. Making this too high
could cause performance issues. Making it too low could mean replications jobs
might not have enough time to make progress before getting unscheduled again.
This parameter can be adjusted at runtime and will take effect during next
reschudling cycle.
* `interval` : Scheduling interval in milliseconds. During each reschedule
cycle scheduler might start or stop up to "max_churn" number of jobs.
* `max_churn` : Maximum number of replications to start and stop during
rescheduling. This parameter along with "interval" defines the rate of job
replacement. During startup, however a much larger number of jobs could be
started (up to max_jobs) in short period of time.
Replication jobs are added to the scheduler by the document processor or from
the `couch_replicator:replicate/2` function when called from `_replicate` HTTP
endpoint handler.
Document processor listens for updates via couch_mutlidb_changes module then
tries to add replication jobs to the scheduler. Sometimes translating a
document update to a replication job could fail, either permantly (if document
is malformed and missing some expected fields for example) or temporarily if
it is a filtered replication and filter cannot be fetched. A failed filter
fetch will be retried with an exponential backoff.
couch_replicator_clustering is in charge of monitoring cluster membership
changes. When membership changes, after a configurable quiet period, a rescan
will be initiated. Rescan will shufle replication jobs to make sure a
replication job is running on only one node.
A new set of stats were added to introspect scheduler and doc processor
internals.
The top replication supervisor structure is `rest_for_one`. This means if
a child crashes, all children to the "right" of it will be restarted (if
visualized supervisor hierarchy as an upside-down tree). Clustering,
connection pool and rate limiter are towards the "left" as they are more
fundamental, if clustering child crashes, most other components will be
restart. Doc process or and multi-db changes children are towards the "right".
If they crash, they can be safely restarted without affecting already running
replication or components like clustering or connection pool.
Jira: COUCHDB-3324
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This code was actually replacing the file_compression = snappy
line, which was preventing snappy from being tested in the test
suite.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If connection succeeds eventually, don't report spurious error when retrying.
Turns `./dev/run` output from this:
```
[ * ] Check node at http://127.0.0.1:15984/ ... failed: [Errno socket error] [Errno 111] Connection refused
[ * ] Check node at http://127.0.0.1:25984/ ... ok
[ * ] Check node at http://127.0.0.1:35984/ ... ok
[ * ] Check node at http://127.0.0.1:15984/ ... ok
```
to this:
```
[ * ] Check node at http://127.0.0.1:15984/ ... ok
[ * ] Check node at http://127.0.0.1:25984/ ... ok
[ * ] Check node at http://127.0.0.1:35984/ ... ok
```
Cleaner and a bit less scary for new developers.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
COUCHDB-2990
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This gives the option of specificing the seed number when generating the
node name and the port for that node.
e.g.
/dev/run --node-number 5
- node5@127.0.0.1 on port 55894
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
