| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Update cli.py to support custom loggers
Allows a config.py file (via rq worker --config) to support things like this for logfmt logging:
DICT_CONFIG = {
"version": 1,
"formatters": {"logfmt": {
"()": "logfmter.Logfmter",
"keys": ["level","when","pid"],
"mapping": {"level":"levelname","when":"asctime","pid":"process"},
"datefmt": "%Y-%m-%dT%H:%M:%S%z"
}},
"handlers": {"console": {"class": "logging.StreamHandler","formatter": "logfmt" }},
"loggers": {
"root": {"handlers":["console"], "level": "INFO"},
"rq": {"handlers":["console"], "level": "INFO", "propagate": False},
}
}
* added simple test and documentation for DICT_CONFIG
* further attempt to get testing right for dictConfig
* move import to correct location
* fix
* remove meaningless options.get() usage
* linting checks and added missing test config file
|
| |
|
|
|
|
|
|
|
| |
* Update linting configuration
This removes flake8 in favor of ruff, which also provides isort support, and
updates all files to be black, isort, and ruff compliant. This also adds black
and ruff checks to the tox and Github linting workflow.
* Tweak the code coverage config and calls
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* First stab at implementating worker pool
* Use process.is_alive() to check whether a process is still live
* Handle shutdown signal
* Check worker loop done
* First working version of `WorkerPool`.
* Added test for check_workers()
* Added test for pool.start()
* Better shutdown process
* Comment out test_start() to see if it fixes CI
* Make tests pass
* Make CI pass
* Comment out some tests
* Comment out more tests
* Re-enable a test
* Re-enable another test
* Uncomment check_workers test
* Added run_worker test
* Minor modification to dead worker detection
* More test cases
* Better process name for workers
* Added back pool.stop_workers() when signal is received
* Cleaned up cli.py
* WIP on worker-pool command
* Fix test
* Test that worker pool ignores consecutive shutdown signals
* Added test for worker-pool CLI command.
* Added timeout to CI jobs
* Fix worker pool test
* Comment out test_scheduler.py
* Fixed worker-pool in burst mode
* Increase test coverage
* Exclude tests directory from coverage.py
* Improve test coverage
* Renamed `Pool(num_workers=2) to `Pool(size=2)`
* Revert "Renamed `Pool(num_workers=2) to `Pool(size=2)`"
This reverts commit a1306f89ad0d8686c6bde447bff75e2f71f0733b.
* Renamed Pool to WorkerPool
* Added a new TestCase that doesn't use LocalStack
* Added job_class, worker_class and serializer arguments to WorkerPool
* Use parse_connection() in WorkerPool.__init__
* Added CLI arguments for worker-pool
* Minor WorkerPool and test fixes
* Fixed failing CLI test
* Document WorkerPool
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* consolidate job failure callback execution
* fix
* success callback as well
* merge fix
* create Callback class and change how callbacks serde
* deprecation
* add tests
* string format
* pr fix
* revert serialization changes
* fix timeout typing
* add documentation
* add test
* fix a bug
* fix tests
* move job heartbeat call to worker and make sure there is always a callback timeout
* fix test
* fix test
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* New dequeue strategy
This implements a new parameter `dequeue_strategy` that
should replace the `RoundRobinWorker` and `RandomWorker`.
Changes includes: feature, docs, tests, deprecation warning.
* Fix dequeue strategy name
* Black & Fix warning
* feat: tests, warnings, refactor naming
* feat: improve worker check
* fix: revert to str subclass
* fix: dequeue strategy into bootstrap
* org: move DequeueStrategy to worker
* refactor: round robin naming
* fix: naming
* fix: type annotation
* fix: typo
* refactor: remove kwarg from worker's init
* fix: typo
* move `dequeue_strategy` from `bootstrap()` into `work()`
|
| |
|
|
|
|
|
|
|
|
| |
* Move common flake8 options into config file
Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.
Move --count and --statistics to config file as well.
* Fix some lints
|
| |
|
|
|
|
|
| |
* Drop syntax required only for Python 2
* Drop python2-style super() calls
Co-authored-by: Selwin Ong <selwin.ong@gmail.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add serializer where missing in code
* Fix cli
* Pass option to command
* Add tests for serializer option
* Merge branch 'master' into respect-serializer
- Update enqueue cli to resp. serializer
* Address @selwin's review
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Allows enqueueing by the cli
#372
* schedule support
* `_` to `-`
* fix flake8
* echo job-id
* Some improvements
- Description as in python jobs
- return result
- quiet mode
- allows `--boolean` and `--integer`
- raises errors if not used correctly
* added tests
* add schedule tests
* add retry test
* use click exceptions
* add error test
* add job_func test
* change messages
https://github.com/rq/rq/pull/1466#discussion_r640211128
https://github.com/rq/rq/pull/1466#discussion_r640210850
* Use different format for arguments
View https://github.com/rq/rq/pull/1466#discussion_r650510889
* Add file support
Usage: @filename
* ast.literal_eval support with `#` instead of `:`
* func -> function
Makes error messages more readable
* click Error
* print function string
* add docs
* increase seconds in test
* Update `parse_function_arg`
Add `ParsingMode` enum (https://github.com/rq/rq/pull/1466#discussion_r656676114)
Change error messages (https://github.com/rq/rq/pull/1466#discussion_r656676800, https://github.com/rq/rq/pull/1466#discussion_r656677082)
* `#` to `%`
`#` is the letter for a comment in bash
* Add some tests
(https://github.com/rq/rq/pull/1466#discussion_r656674539, https://github.com/rq/rq/pull/1466#discussion_r656676543)
* Add some tests
* docs: Add some examples
* catch all literal_eval exceptions
There are some edge cases with other exceptions
* remove job_func
(https://github.com/rq/rq/pull/1466#pullrequestreview-690110118)
* edit docs
https://github.com/rq/rq/pull/1466#pullrequestreview-695758691
* format examples
* format examples
`queue.enqueue(path.to.func, args=['abc'])` to `queue.enqueue(path.to.func, 'abc')`
https://github.com/rq/rq/pull/1466#discussion_r673615464
* add examples
https://github.com/rq/rq/pull/1466#discussion_r673658933
* add doc test
https://github.com/rq/rq/pull/1466#discussion_r673659124
* Update index.md
* Update test_cli.py
* Update test_cli.py
* Add version info
Co-authored-by: rpkak <rpkak@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add environment variables for connection
Use same args as in config file
#1342
* add test
* add suggestions
https://github.com/rq/rq/pull/1472#discussion_r640205865
https://github.com/rq/rq/pull/1472#discussion_r640206565
* remove unused import
|
| |
|
|
|
|
|
|
|
|
|
| |
* Added --serializer option to cli, finishing off PR #1381 and fix #1357
* Update documentation
* Update documentation
* Modified help message
Co-authored-by: f0cker <dturner@trustwave.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
* Added job.worker_name
* Fix compatibility with Redis server 3.x
* Document job.worker_name
* Removed some Python 2 compatibility stuff.
* Remove unused codes
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* First RQScheduler prototype
* WIP job scheduling
* Fixed Python 2.7 tests
* Added ScheduledJobRegistry.get_scheduled_time(job)
* WIP on scheduler's threading mechanism
* Fixed test errors
* Changed scheduler.acquire_locks() to instance method
* Added scheduler.prepare_registries()
* Somewhat working implementation of RQ scheduler
* Only call stop_scheduler if there's a scheduler present
* Use OSError rather than ProcessLookupError for PyPy compatibility
* Added `auto_start` argument to scheduler.acquire_locks()
* Make RQScheduler play better with timezone
* Fixed test error
* Added --with-scheduler flag to rq worker CLI
* Fix tests on Python 2.x
* More Python 2 fixes
* Only call `scheduler.start` if worker is run in non burst mode
* Fixed an issue where running worker with scheduler would fail sometimes
* Make `worker.stop_scheduler()` more resilient to errors
* worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks
* Scheduler can now work with worker in both burst and non burst mode
* Fixed scheduler logging message
* Always log scheduler errors when running
* Improve scheduler error logging message
* Removed testing code
* Scheduler should periodically try to acquire locks for other queues it doesn't have
* Added tests for scheduler.should_reacquire_locks
* Added queue.enqueue_in()
* Fixes queue.enqueue_in() in Python 2.7
* First stab at documenting job scheduling
* Remove unused methods
* Remove Python 2.6 logging compatibility code
* Remove more unused imports
* Added convenience methods to access job registries from queue
* Added test for worker.run_maintenance_tasks()
* Simplify worker.queue_names() and worker.queue_keys()
* Updated changelog to mention RQ's new job scheduling mechanism.
|
| |
|
|
|
|
| |
* Fix rq info not found workers information error, Fixes #1139
* Add test(#1149)
|
| |
|
|
|
|
|
|
|
|
| |
* rq info management command now cleans up registries when first run
* Deleted print statement
* Improve CLI test coverage
* Fixed CLI test on Linux
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Added FailedJobRegistry.
* Added job.failure_ttl.
* queue.enqueue() now supports failure_ttl
* Added registry.get_queue().
* FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL.
* StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry.
* Failed jobs are now added to FailedJobRegistry.
* Added FailedJobRegistry.requeue()
* Document the new `FailedJobRegistry` and changes in custom exception handler behavior.
* Added worker.disable_default_exception_handler.
* Document --disable-default-exception-handler option.
* Deleted worker.failed_queue.
* Deleted "move_to_failed_queue" exception handler.
* StartedJobRegistry should no longer move jobs to FailedQueue.
* Deleted requeue_job
* Fixed test error.
* Make requeue cli command work with FailedJobRegistry
* Added .pytest_cache to gitignore.
* Custom exception handlers are no longer run in reverse
* Restored requeue_job function
* Removed get_failed_queue
* Deleted FailedQueue
* Updated changelog.
* Document `failure_ttl`
* Updated docs.
* Remove job.status
* Fixed typo in test_registry.py
* Replaced _pipeline() with pipeline()
* FailedJobRegistry no longer fails on redis-py>=3
* Fixes test_clean_registries
* Worker names are now randomized
* Added a note about random worker names in CHANGES.md
* Worker will now stop working when encountering an unhandled exception.
* Worker should reraise SystemExit on cold shutdowns
* Added anchor.js to docs
* Support for Sentry-SDK (#1045)
* Updated RQ to support sentry-sdk
* Document Sentry integration
* Install sentry-sdk before running tests
* Improved rq info CLI command to be more efficient when displaying lar… (#1046)
* Improved rq info CLI command to be more efficient when displaying large number of workers
* Fixed an rq info --by-queue bug
* Fixed worker.total_working_time bug (#1047)
* queue.enqueue() no longer accepts `timeout` argument (#1055)
* Clean worker registry (#1056)
* queue.enqueue() no longer accepts `timeout` argument
* Added clean_worker_registry()
* Show worker hostname and PID on cli (#1058)
* Show worker hostname and PID on cli
* Improve test coverage
* Remove Redis version check when SSL is used
* Bump version to 1.0
* Removed pytest_cache/README.md
* Changed worker logging to use exc_info=True
* Removed unused queue.dequeue()
* Fixed typo in CHANGES.md
* setup_loghandlers() should always call logger.setLevel() if specified
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
other APIs
This includes:
- a partial refactor of the CLI to organize the shared options
- extends the tests in areas where passing custom backend classes makes sense
- allow setting the core CLI options as env vars
- minor cosmetic changes here and there
|
| |
|
|
|
| |
syntax: assertEquals -> assertEqual, assertNotEquals -> assertNotEqual
usage: status of worker and job now will use get/set method instead of property method
|
| | |
|
| |
|
|
| |
default behavior.
|
| |\ |
|
| | | |
|
| |/
|
|
|
| |
Adding command line --exception_handler option (with multiple entries allowed) to `rq worker`
Added tests for command line options.
|
| | |
|
| | |
|
| |
|
