diff options
author | Jordan Cook <jordan.cook@pioneer.com> | 2021-08-26 14:48:33 -0500 |
---|---|---|
committer | Jordan Cook <jordan.cook@pioneer.com> | 2021-08-26 17:04:15 -0500 |
commit | e203a48cdf58eda51f85b27d3eaf852bd9da941c (patch) | |
tree | a384f2ab226fecc66b81bf288d51f2acb0b3b960 | |
parent | 31a760cfb998d31cdeb8f79e65e1f1765cb2e3c9 (diff) | |
download | requests-cache-e203a48cdf58eda51f85b27d3eaf852bd9da941c.tar.gz |
Reorganize user docs: break down User Guide and Advanced Usage sections into smaller pages
37 files changed, 1268 insertions, 1257 deletions
diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md index 92206cb..7ba48ba 100644 --- a/.github/CODE_OF_CONDUCT.md +++ b/.github/CODE_OF_CONDUCT.md @@ -1,4 +1,7 @@ -# Contributor Covenant Code of Conduct +# Code of Conduct +This Code of Conduct is adapted from +[Contributor Covenant, version 1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct.html). +See [FAQ](https://www.contributor-covenant.org/faq) for answers to common questions. ## Our Pledge @@ -66,11 +69,3 @@ faith may face temporary or permanent repercussions as determined by other members of the project's leadership. ## Attribution - -This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, -available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html - -[homepage]: https://www.contributor-covenant.org - -For answers to common questions about this code of conduct, see -https://www.contributor-covenant.org/faq diff --git a/docs/advanced_usage.md b/docs/advanced_usage.md index ea56184..95bb011 100644 --- a/docs/advanced_usage.md +++ b/docs/advanced_usage.md @@ -1,480 +1,3 @@ -(advanced-usage)= +<!-- Placeholder to avoid broken links --> # Advanced Usage -This section covers some more advanced and use-case-specific features. - -## Cache Inspection -Here are some ways to get additional information out of the cache session, backend, and responses: - -### Response Details -The following attributes are available on responses: -- `from_cache`: indicates if the response came from the cache -- `created_at`: {py:class}`~datetime.datetime` of when the cached response was created or last updated -- `expires`: {py:class}`~datetime.datetime` after which the cached response will expire -- `is_expired`: indicates if the cached response is expired (if an old response was returned due to a request error) - -Examples: -:::{admonition} Example code -:class: toggle -```python ->>> from requests_cache import CachedSession ->>> session = CachedSession(expire_after=timedelta(days=1)) - ->>> # Placeholders are added for non-cached responses ->>> response = session.get('http://httpbin.org/get') ->>> print(response.from_cache, response.created_at, response.expires, response.is_expired) -False None None None - ->>> # Values will be populated for cached responses ->>> response = session.get('http://httpbin.org/get') ->>> print(response.from_cache, response.created_at, response.expires, response.is_expired) -True 2021-01-01 18:00:00 2021-01-02 18:00:00 False - ->>> # Print a response object to get general information about it ->>> print(response) -'request: GET https://httpbin.org/get, response: 200 (308 bytes), created: 2021-01-01 22:45:00 IST, expires: 2021-01-02 18:45:00 IST (fresh)' -``` -::: - -### Cache Contents -You can use `CachedSession.cache.urls` to see all URLs currently in the cache: -```python ->>> session = CachedSession() ->>> print(session.cache.urls) -['https://httpbin.org/get', 'https://httpbin.org/stream/100'] -``` - -If needed, you can get more details on cached responses via `CachedSession.cache.responses`, which -is a dict-like interface to the cache backend. See {py:class}`.CachedResponse` for a full list of -attributes available. - -For example, if you wanted to to see all URLs requested with a specific method: -```python ->>> post_urls = [ -... response.url for response in session.cache.responses.values() -... if response.request.method == 'POST' -... ] -``` - -You can also inspect `CachedSession.cache.redirects`, which maps redirect URLs to keys of the -responses they redirect to. - -Additional `keys()` and `values()` wrapper methods are available on {py:class}`.BaseCache` to get -combined keys and responses. -```python ->>> print('All responses:') ->>> for response in session.cache.values(): ->>> print(response) - ->>> print('All cache keys for redirects and responses combined:') ->>> print(list(session.cache.keys())) -``` - -Both methods also take a `check_expiry` argument to exclude expired responses: -```python ->>> print('All unexpired responses:') ->>> for response in session.cache.values(check_expiry=True): ->>> print(response) -``` - -Similarly, you can get a count of responses with {py:meth}`.BaseCache.response_count`, and optionally -exclude expired responses: -```python ->>> print(f'Total responses: {session.cache.response_count()}') ->>> print(f'Unexpired responses: {session.cache.response_count(check_expiry=True)}') -``` - -## Custom Cache Filtering -If you need more advanced behavior for choosing what to cache, you can provide a custom filtering -function via the `filter_fn` param. This can by any function that takes a -{py:class}`requests.Response` object and returns a boolean indicating whether or not that response -should be cached. It will be applied to both new responses (on write) and previously cached -responses (on read): - -:::{admonition} Example code -:class: toggle -```python ->>> from sys import getsizeof ->>> from requests_cache import CachedSession - ->>> def filter_by_size(response: Response) -> bool: ->>> """Don't cache responses with a body over 1 MB""" ->>> return getsizeof(response.content) <= 1024 * 1024 - ->>> session = CachedSession(filter_fn=filter_by_size) -``` -::: - -```{note} -`filter_fn()` will be used **in addition to** other {ref:`user_guide:cache filtering`} options. -``` - -## Custom Request Matching -Request matching is accomplished using a **cache key**, which uniquely identifies a response in the -cache based on request info. For example, the option `ignored_parameters=['foo']` works by excluding -the `foo` request parameter from the cache key, meaning these three requests will all use the same -cached response: -```python ->>> session = CachedSession(ignored_parameters=['foo']) ->>> response_1 = session.get('https://example.com') # cache miss ->>> response_2 = session.get('https://example.com?foo=bar') # cache hit ->>> response_3 = session.get('https://example.com?foo=qux') # cache hit ->>> assert response_1.cache_key == response_2.cache_key == response_3.cache_key -``` - -If you want to implement your own request matching, you can provide a cache key function which will -take a {py:class}`~requests.PreparedRequest` plus optional keyword args, and return a string: -```python -def create_key(request: requests.PreparedRequest, **kwargs) -> str: - """Generate a custom cache key for the given request""" -``` - -`**kwargs` includes relevant {py:class}`.BaseCache` settings and any other keyword args passed to -{py:meth}`.CachedSession.send()`. See {py:func}`.create_key` for the reference implementation, and -see the rest of the {py:mod}`.cache_keys` module for some potentially useful helper functions. - -You can then pass this function via the `key_fn` param: -```python -session = CachedSession(key_fn=create_key) -``` - -```{note} -`key_fn()` will be used **instead of** any other {ref}`user_guide:request matching` options and -default matching behavior. -``` -```{tip} -See {ref}`Examples<custom_keys>` page for a complete example for custom request matching. -``` -```{tip} -As a general rule, if you include less info in your cache keys, you will have more cache hits and -use less storage space, but risk getting incorrect response data back. For example, if you exclude -all request parameters, you will get the same cached response back for any combination of request -parameters. -``` -```{warning} -If you provide a custom key function for a non-empty cache, any responses previously cached with a -different key function will likely be unused. -``` - -## Custom Backends -If the built-in {py:mod}`Cache Backends <requests_cache.backends>` don't suit your needs, you can -create your own by making subclasses of {py:class}`.BaseCache` and {py:class}`.BaseStorage`: - -:::{admonition} Example code -:class: toggle -```python ->>> from requests_cache import CachedSession ->>> from requests_cache.backends import BaseCache, BaseStorage - ->>> class CustomCache(BaseCache): -... """Wrapper for higher-level cache operations. In most cases, the only thing you need -... to specify here is which storage class(es) to use. -... """ -... def __init__(self, **kwargs): -... super().__init__(**kwargs) -... self.redirects = CustomStorage(**kwargs) -... self.responses = CustomStorage(**kwargs) - ->>> class CustomStorage(BaseStorage): -... """Dict-like interface for lower-level backend storage operations""" -... def __init__(self, **kwargs): -... super().__init__(**kwargs) -... -... def __getitem__(self, key): -... pass -... -... def __setitem__(self, key, value): -... pass -... -... def __delitem__(self, key): -... pass -... -... def __iter__(self): -... pass -... -... def __len__(self): -... pass -... -... def clear(self): -... pass -``` -::: - -You can then use your custom backend in a {py:class}`.CachedSession` with the `backend` parameter: -```python ->>> session = CachedSession(backend=CustomCache()) -``` - -## Custom Serializers -If the built-in {ref}`serializers` don't suit your needs, you can create your own. For example, if -you had a imaginary `custom_pickle` module that provides `dumps` and `loads` functions: -```python ->>> import custom_pickle ->>> from requests_cache import CachedSession ->>> session = CachedSession(serializer=custom_pickle) -``` - -### Serializer Pipelines -More complex serialization can be done with {py:class}`.SerializerPipeline`. Use cases include -text-based serialization, compression, encryption, and any other intermediate steps you might want -to add. - -Any combination of these can be composed with a {py:class}`.SerializerPipeline`, which starts with a -{py:class}`.CachedResponse` and ends with a {py:class}`.str` or {py:class}`.bytes` object. Each stage -of the pipeline can be any object or module with `dumps` and `loads` functions. If the object has -similar methods with different names (e.g. `compress` / `decompress`), those can be aliased using -{py:class}`.Stage`. - -For example, a compressed pickle serializer can be built as: -:::{admonition} Example code -:class: toggle -```python ->>> import pickle, gzip ->>> from requests_cache.serialzers import SerializerPipeline, Stage ->>> compressed_serializer = SerializerPipeline([ -... pickle, -... Stage(gzip, dumps='compress', loads='decompress'), -...]) ->>> session = CachedSession(serializer=compressed_serializer) -``` -::: - -### Text-based Serializers -If you're using a text-based serialization format like JSON or YAML, some extra steps are needed to -encode binary data and non-builtin types. The [cattrs](https://cattrs.readthedocs.io) library can do -the majority of the work here, and some pre-configured converters are included for serveral common -formats in the {py:mod}`.preconf` module. - -For example, a compressed JSON pipeline could be built as follows: -:::{admonition} Example code -:class: toggle -```python ->>> import json, gzip, codecs ->>> from requests_cache.serializers import SerializerPipeline, Stage, json_converter ->>> comp_json_serializer = SerializerPipeline([ -... json_converter, # Serialize to a JSON string -... Stage(codecs.utf_8, dumps='encode', loads='decode'), # Encode to bytes -... Stage(gzip, dumps='compress', loads='decompress'), # Compress -...]) -``` -::: - -```{note} -If you want to use a different format that isn't included in {py:mod}`.preconf`, you can use -{py:class}`.CattrStage` as a starting point. -``` - -```{note} -If you want to convert a string representation to bytes (e.g. for compression), you must use a codec -from {py:mod}`.codecs` (typically `codecs.utf_8`) -``` - -### Additional Serialization Steps -Some other tools that could be used as a stage in a {py:class}`.SerializerPipeline` include: - -Class | loads | dumps ------ | ----- | ----- -{py:mod}`codecs.* <.codecs>` | encode | decode -{py:mod}`.bz2` | compress | decompress -{py:mod}`.gzip` | compress | decompress -{py:mod}`.lzma` | compress | decompress -{py:mod}`.zlib` | compress | decompress -{py:mod}`.pickle` | dumps | loads -{py:class}`itsdangerous.signer.Signer` | sign | unsign -{py:class}`itsdangerous.serializer.Serializer` | loads | dumps -{py:class}`cryptography.fernet.Fernet` | encrypt | decrypt - -## Usage with other requests features - -### Request Hooks -Requests has an [Event Hook](https://requests.readthedocs.io/en/master/user/advanced/#event-hooks) -system that can be used to add custom behavior into different parts of the request process. -It can be used, for example, for request throttling: - -:::{admonition} Example code -:class: toggle -```python ->>> import time ->>> import requests ->>> from requests_cache import CachedSession ->>> ->>> def make_throttle_hook(timeout=1.0): ->>> """Make a request hook function that adds a custom delay for non-cached requests""" ->>> def hook(response, *args, **kwargs): ->>> if not getattr(response, 'from_cache', False): ->>> print('sleeping') ->>> time.sleep(timeout) ->>> return response ->>> return hook ->>> ->>> session = CachedSession() ->>> session.hooks['response'].append(make_throttle_hook(0.1)) ->>> # The first (real) request will have an added delay ->>> session.get('http://httpbin.org/get') ->>> session.get('http://httpbin.org/get') -``` -::: - -### Streaming Requests -If you use [streaming requests](https://2.python-requests.org/en/master/user/advanced/#id9), you -can use the same code to iterate over both cached and non-cached requests. Cached response content -will have already been read (i.e., consumed), but will be available for re-reading so it behaves like -the original streamed response: - -:::{admonition} Example code -:class: toggle -```python ->>> from requests_cache import CachedSession ->>> ->>> session = CachedSession() ->>> for i in range(2): -... response = session.get('https://httpbin.org/stream/20', stream=True) -... for chunk in response.iter_lines(): -... print(chunk.decode('utf-8')) -``` -::: - -(library-compatibility)= -## Usage with other requests-based libraries -This library works by patching and/or extending {py:class}`requests.Session`. Many other libraries -out there do the same thing, making it potentially difficult to combine them. - -For that scenario, a mixin class is provided, so you can create a custom class with behavior from -multiple Session-modifying libraries: -```python ->>> from requests import Session ->>> from requests_cache import CacheMixin ->>> from some_other_lib import SomeOtherMixin ->>> ->>> class CustomSession(CacheMixin, SomeOtherMixin, Session): -... """Session class with features from both some_other_lib and requests-cache""" -``` - -### Requests-HTML -[requests-html](https://github.com/psf/requests-html) is one library that works with this method: -:::{admonition} Example code -:class: toggle -```python ->>> import requests ->>> from requests_cache import CacheMixin, install_cache ->>> from requests_html import HTMLSession ->>> ->>> class CachedHTMLSession(CacheMixin, HTMLSession): -... """Session with features from both CachedSession and HTMLSession""" ->>> ->>> session = CachedHTMLSession() ->>> response = session.get('https://github.com/') ->>> print(response.from_cache, response.html.links) -``` -::: - - -Or if you are using {py:func}`.install_cache`, you can use the `session_factory` argument: -:::{admonition} Example code -:class: toggle -```python ->>> install_cache(session_factory=CachedHTMLSession) ->>> response = requests.get('https://github.com/') ->>> print(response.from_cache, response.html.links) -``` -::: - -The same approach can be used with other libraries that subclass {py:class}`requests.Session`. - -### Requests-futures -Some libraries, including [requests-futures](https://github.com/ross/requests-futures), -support wrapping an existing session object: -```python ->>> session = FutureSession(session=CachedSession()) -``` - -In this case, `FutureSession` must wrap `CachedSession` rather than the other way around, since -`FutureSession` returns (as you might expect) futures rather than response objects. -See [issue #135](https://github.com/reclosedev/requests-cache/issues/135) for more notes on this. - -### Internet Archive -Usage with [internetarchive](https://github.com/jjjake/internetarchive) is the same as other libraries -that subclass `requests.Session`: -:::{admonition} Example code -:class: toggle -```python ->>> from requests_cache import CacheMixin ->>> from internetarchive.session import ArchiveSession ->>> ->>> class CachedArchiveSession(CacheMixin, ArchiveSession): -... """Session with features from both CachedSession and ArchiveSession""" -``` -::: - -### Requests-mock -[requests-mock](https://github.com/jamielennox/requests-mock) has multiple methods for mocking -requests, including a contextmanager, decorator, fixture, and adapter. There are a few different -options for using it with requests-cache, depending on how you want your tests to work. - -#### Disabling requests-cache -If you have an application that uses requests-cache and you just want to use requests-mock in -your tests, the easiest thing to do is to disable requests-cache. - -For example, if you are using {py:func}`.install_cache` in your application and the -requests-mock [pytest fixture](https://requests-mock.readthedocs.io/en/latest/pytest.html) in your -tests, you could wrap it in another fixture that uses {py:func}`.uninstall_cache` or {py:func}`.disabled`: -:::{admonition} Example code -:class: toggle -```{literalinclude} ../tests/compat/test_requests_mock_disable_cache.py -``` -::: - -Or if you use a `CachedSession` object, you could replace it with a regular `Session`, for example: -:::{admonition} Example code -:class: toggle -```python -import unittest -import pytest -import requests - - -@pytest.fixure(scope='function', autouse=True) -def disable_requests_cache(): - """Replace CachedSession with a regular Session for all test functions""" - with unittest.mock.patch('requests_cache.CachedSession', requests.Session): - yield -``` -::: - -#### Combining requests-cache with requests-mock -If you want both caching and mocking features at the same time, you can attach requests-mock's -[adapter](https://requests-mock.readthedocs.io/en/latest/adapter.html) to a `CachedSession`: - -:::{admonition} Example code -:class: toggle -```{literalinclude} ../tests/compat/test_requests_mock_combine_cache.py -``` -::: - -#### Building a mocker using requests-cache data -Another approach is to use cached data to dynamically define mock requests + responses. -This has the advantage of only using request-mock's behavior for -[request matching](https://requests-mock.readthedocs.io/en/latest/matching.html). - -:::{admonition} Example code -:class: toggle -```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py -:lines: 21-40 -``` -::: - -To turn that into a complete example: -:::{admonition} Example code -:class: toggle -```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py -``` -::: - -### Responses -Usage with the [responses](https://github.com/getsentry/responses) library is similar to the -requests-mock examples above. - -:::{admonition} Example code -:class: toggle -```{literalinclude} ../tests/compat/test_responses_load_cache.py -``` -::: +The contents of this section have been moved to the {ref}`user-guide`. diff --git a/docs/contributing.md b/docs/contributing.md deleted file mode 100644 index 88f29b2..0000000 --- a/docs/contributing.md +++ /dev/null @@ -1,3 +0,0 @@ -(contributing)= -```{include} ../CONTRIBUTING.md -``` diff --git a/docs/contributors.md b/docs/contributors.md deleted file mode 100644 index c8e001d..0000000 --- a/docs/contributors.md +++ /dev/null @@ -1,2 +0,0 @@ -```{include} ../CONTRIBUTORS.md -``` diff --git a/docs/examples.md b/docs/examples.md index c59c662..5f9c737 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -1,3 +1,4 @@ +(examples)= # Examples This section contains some complete examples that demonstrate the main features of requests-cache. @@ -93,15 +94,15 @@ The following scripts can also be found in the ::: (custom_keys)= -### Custom cache key function -```{include} ../examples/custom_cache_keys.py +### Custom request matcher +```{include} ../examples/custom_request_matcher.py :start-line: 2 :end-line: 15 ``` :::{admonition} Example code :class: toggle -```{literalinclude} ../examples/custom_cache_keys.py +```{literalinclude} ../examples/custom_request_matcher.py :lines: 1,17- ``` ::: diff --git a/docs/history.md b/docs/history.md deleted file mode 100644 index 856698b..0000000 --- a/docs/history.md +++ /dev/null @@ -1,3 +0,0 @@ -(changelog)= -```{include} ../HISTORY.md -``` diff --git a/docs/index.md b/docs/index.md index 7dd484f..df9f4bd 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,7 @@ <!-- Pre-release warning to reduce confusion on what '/latest' means; -TODO: remove prior to next minor release +TODO: remove prior to next minor release, or add script to make this conditional --> ```{admonition} Note :class: warning @@ -25,13 +25,8 @@ Documentation for the latest stable release can be found at [requests-cache.read ```{toctree} :maxdepth: 2 -user_guide -advanced_usage +user_guide/index examples -security reference -contributing -contributors -related_projects -history +project_info/index ```` diff --git a/docs/project_info/code_of_conduct.md b/docs/project_info/code_of_conduct.md new file mode 100644 index 0000000..36beb23 --- /dev/null +++ b/docs/project_info/code_of_conduct.md @@ -0,0 +1,2 @@ +```{include} ../../.github/CODE_OF_CONDUCT.md +``` diff --git a/docs/project_info/contributing.md b/docs/project_info/contributing.md new file mode 100644 index 0000000..01f6808 --- /dev/null +++ b/docs/project_info/contributing.md @@ -0,0 +1,3 @@ +(contributing)= +```{include} ../../CONTRIBUTING.md +``` diff --git a/docs/project_info/contributors.md b/docs/project_info/contributors.md new file mode 100644 index 0000000..f0c9bb7 --- /dev/null +++ b/docs/project_info/contributors.md @@ -0,0 +1,2 @@ +```{include} ../../CONTRIBUTORS.md +``` diff --git a/docs/project_info/history.md b/docs/project_info/history.md new file mode 100644 index 0000000..ba5ca18 --- /dev/null +++ b/docs/project_info/history.md @@ -0,0 +1,3 @@ +(changelog)= +```{include} ../../HISTORY.md +``` diff --git a/docs/project_info/index.md b/docs/project_info/index.md new file mode 100644 index 0000000..c808ab6 --- /dev/null +++ b/docs/project_info/index.md @@ -0,0 +1,11 @@ +# Project Info + +```{toctree} +:maxdepth: 2 + +contributing +contributors +code_of_conduct +related_projects +history +```` diff --git a/docs/related_projects.md b/docs/project_info/related_projects.md index a9ba653..e9abcdc 100644 --- a/docs/related_projects.md +++ b/docs/project_info/related_projects.md @@ -1,3 +1,4 @@ +(related-projects)= # Related Projects If requests-cache isn't quite what you need, you can help make it better! See the {ref}`Contributing Guide <contributing>` for details. @@ -30,4 +31,4 @@ You can also check out these other python projects related to caching and/or HTT for tests; inspired by Ruby's [VCR](https://github.com/vcr/vcr)]. Works at the `httplib` level and is compatible with multiple HTTP libraries. * [betamax](https://github.com/betamaxpy/betamax): Records responses to local files and plays them back - for tests; also inspired by Ruby's [VCR](https://github.com/vcr/vcr)]. Made specifically for `requests`. + for tests; also inspired by Ruby's [VCR](https://github.com/vcr/vcr). Made specifically for `requests`. diff --git a/docs/sample_response.json b/docs/sample_data/sample_response.json index 4b60354..4b60354 100644 --- a/docs/sample_response.json +++ b/docs/sample_data/sample_response.json diff --git a/docs/sample_response.yaml b/docs/sample_data/sample_response.yaml index 07bc7ca..07bc7ca 100644 --- a/docs/sample_response.yaml +++ b/docs/sample_data/sample_response.yaml diff --git a/docs/user_guide.md b/docs/user_guide.md deleted file mode 100644 index d490133..0000000 --- a/docs/user_guide.md +++ /dev/null @@ -1,737 +0,0 @@ -(user-guide)= -# User Guide -This section covers the main features of requests-cache. - -## Installation -Installation instructions: - -:::{tab} Pip -Install the latest stable version from [PyPI](https://pypi.org/project/requests-cache/): -``` -pip install requests-cache -``` -::: -:::{tab} Conda -Or install from [conda-forge](https://anaconda.org/conda-forge/requests-cache), if you prefer: -``` -conda install -c conda-forge requests-cache -``` -::: -:::{tab} Pre-release -If you would like to use the latest development (pre-release) version: -``` -pip install --pre requests-cache -``` -::: -:::{tab} Local development -See {ref}`Contributing Guide <contributing:dev installation>` for setup steps for local development -::: - -### Requirements -The latest version of requests-cache requires **python 3.7+**. If you need to use an older version -of python, here are the latest compatible versions and their documentation pages: - -:::{admonition} Python version compatibility -:class: toggle, tip -* **python 2.6:** [requests-cache 0.4.13](https://requests-cache.readthedocs.io/en/v0.4.13) -* **python 2.7:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) -* **python 3.4:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) -* **python 3.5:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) -* **python 3.6:** [requests-cache 0.7.4](https://requests-cache.readthedocs.io/en/v0.7.4) -::: - -You may need additional dependencies depending on which backend you want to use. To install with -extra dependencies for all supported {ref}`user_guide:cache backends`: -``` -pip install requests-cache[all] -``` - -## General Usage -There are two main ways of using requests-cache: -- **Sessions:** (recommended) Use {py:class}`.CachedSession` to send your requests -- **Patching:** Globally patch `requests` using {py:func}`.install_cache()` - -### Sessions -{py:class}`.CachedSession` can be used as a drop-in replacement for {py:class}`requests.Session`. -Basic usage looks like this: -```python ->>> from requests_cache import CachedSession ->>> ->>> session = CachedSession() ->>> session.get('http://httpbin.org/get') -``` - -Any {py:class}`requests.Session` method can be used (but see -{ref}`user_guide:cached http methods` section for options): -```python ->>> session.request('GET', 'http://httpbin.org/get') ->>> session.head('http://httpbin.org/get') -``` - -Caching can be temporarily disabled for the session with -{py:meth}`.CachedSession.cache_disabled`: -```python ->>> with session.cache_disabled(): -... session.get('http://httpbin.org/get') -``` - -The best way to clean up your cache is through {ref}`user_guide:cache expiration`, but you can also -clear out everything at once with {py:meth}`.BaseCache.clear`: -```python ->>> session.cache.clear() -``` - -### Patching -In some situations, it may not be possible or convenient to manage your own session object. In those -cases, you can use {py:func}`.install_cache` to add caching to all `requests` functions: -```python ->>> import requests ->>> import requests_cache ->>> ->>> requests_cache.install_cache() ->>> requests.get('http://httpbin.org/get') -``` - -As well as session methods: -```python ->>> session = requests.Session() ->>> session.get('http://httpbin.org/get') -``` - -{py:func}`.install_cache` accepts all the same parameters as {py:class}`.CachedSession`: -```python ->>> requests_cache.install_cache(expire_after=360, allowable_methods=('GET', 'POST')) -``` - -It can be temporarily {py:func}`.enabled`: -```python ->>> with requests_cache.enabled(): -... requests.get('http://httpbin.org/get') # Will be cached -``` - -Or temporarily {py:func}`.disabled`: -```python ->>> requests_cache.install_cache() ->>> with requests_cache.disabled(): -... requests.get('http://httpbin.org/get') # Will not be cached -``` - -Or completely removed with {py:func}`.uninstall_cache`: -```python ->>> requests_cache.uninstall_cache() ->>> requests.get('http://httpbin.org/get') -``` - -You can also clear out all responses in the cache with {py:func}`.clear`, and check if -requests-cache is currently installed with {py:func}`.is_installed`. - -(monkeypatch-issues)= -#### Patching Limitations & Potential Issues -Like any other utility that uses monkey-patching, there are some scenarios where you won't want to -use {py:func}`.install_cache`: -- When using other libraries that patch {py:class}`requests.Session` -- In a multi-threaded or multiprocess application -- In a library that will be imported by other libraries or applications -- In a larger application that makes requests in several different modules, where it may not be - obvious what is and isn't being cached - -In any of these cases, consider using {py:class}`.CachedSession`, the {py:func}`.enabled` -contextmanager, or {ref}`selective-caching`. - -(backends)= -## Cache Backends -![](_static/sqlite_32px.png) -![](_static/redis_32px.png) -![](_static/mongodb_32px.png) -![](_static/dynamodb_32px.png) -![](_static/files-json_32px.png) - -Several cache backends are included. The default is SQLite, since it's generally the simplest to -use, and requires no extra dependencies or configuration. -```{note} -In the rare case that SQLite is not available -(for example, [on Heroku](https://devcenter.heroku.com/articles/sqlite3)), a non-persistent -in-memory cache is used by default. -``` - -See {py:mod}`.requests_cache.backends` for usage details for specific backends. -### Backend Dependencies -Most of the other backends require some extra dependencies, listed below. - -Backend | Class | Alias | Dependencies --------------------------------------------------------|----------------------------|----------------|------------- -[SQLite](https://www.sqlite.org) | {py:class}`.SQLiteCache` | `'sqlite'` | -[Redis](https://redis.io) | {py:class}`.RedisCache` | `'redis'` | [redis-py](https://github.com/andymccurdy/redis-py) -[MongoDB](https://www.mongodb.com) | {py:class}`.MongoCache` | `'mongodb'` | [pymongo](https://github.com/mongodb/mongo-python-driver) -[GridFS](https://docs.mongodb.com/manual/core/gridfs/) | {py:class}`.GridFSCache` | `'gridfs'` | [pymongo](https://github.com/mongodb/mongo-python-driver) -[DynamoDB](https://aws.amazon.com/dynamodb) | {py:class}`.DynamoCache` | `'dynamodb'` | [boto3](https://github.com/boto/boto3) -Filesystem | {py:class}`.FileCache` | `'filesystem'` | -Memory | {py:class}`.BaseCache` | `'memory'` | - -### Specifying a Backend -You can specify which backend to use with the `backend` parameter for either {py:class}`.CachedSession` -or {py:func}`.install_cache`. You can specify one by name, using the aliases listed above: -```python ->>> session = CachedSession('my_cache', backend='redis') -``` - -Or by instance: -```python ->>> backend = RedisCache(host='192.168.1.63', port=6379) ->>> session = CachedSession('my_cache', backend=backend) -``` - -### Backend Options -The `cache_name` parameter has a different use depending on the backend: - -Backend | Cache name used as -----------------|------------------- -SQLite | Database path -Redis | Hash namespace -MongoDB, GridFS | Database name -DynamoDB | Table name -Filesystem | Cache directory - -Each backend class also accepts optional parameters for the underlying connection. For example, -{py:class}`.SQLiteCache` accepts parameters for {py:func}`sqlite3.connect`: -```python ->>> session = CachedSession('my_cache', backend='sqlite', timeout=30) -``` - -### Testing Backends -If you just want to quickly try out all of the available backends for comparison, -[docker-compose](https://docs.docker.com/compose/) config is included for all supported services. -First, [install docker](https://docs.docker.com/get-docker/) if you haven't already. Then, run: - -:::{tab} Bash (Linux/macOS) -```bash -pip install -U requests-cache[all] docker-compose -curl https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -O docker-compose.yml -docker-compose up -d -``` -::: -:::{tab} Powershell (Windows) -```ps1 -pip install -U requests-cache[all] docker-compose -Invoke-WebRequest -Uri https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -Outfile docker-compose.yml -docker-compose up -d -``` -::: - -(exporting)= -### Exporting To A Different Backend -If you have cached data that you want to copy or migrate to a different backend, you can do this -with `CachedSession.cache.update()`. For example, if you want to dump the contents of a Redis cache -to JSON files: -```python ->>> src_session = CachedSession('my_cache', backend='redis') ->>> dest_session = CachedSession('~/workspace/cache_dump', backend='filesystem', serializer='json') ->>> dest_session.cache.update(src_session.cache) - ->>> # List the exported files ->>> print(dest_session.cache.paths()) -'/home/user/workspace/cache_dump/9e7a71a3ff2e.json' -'/home/user/workspace/cache_dump/8a922ff3c53f.json' -``` - -Or, using backend classes directly: -```python ->>> src_cache = RedisCache() ->>> dest_cache = FileCache('~/workspace/cache_dump', serializer='json') ->>> dest_cache.update(src_cache) -``` - -### Custom Backends -See {ref}`advanced_usage:custom backends` for details on creating your own backend implementation. - -## Cache Files -```{note} -This section only applies to the {py:mod}`~requests_cache.backends.sqlite` and -{py:mod}`~requests_cache.backends.filesystem` backends. -``` -For file-based backends, the cache name will be used as a path to the cache file(s). You can use -a relative path, absolute path, or use some additional options for system-specific default paths. - -### Relative Paths -```python ->>> # Database path for SQLite cache ->>> session = CachedSession('http_cache', backend='sqlite') ->>> print(session.cache.db_path) -'<current working dir>/http_cache.sqlite' -``` -```python ->>> # Base directory for Filesystem cache ->>> session = CachedSession('http_cache', backend='filesystem') ->>> print(session.cache.cache_dir) -'<current working dir>/http_cache/' -``` - -```{note} -Parent directories will always be created, if they don't already exist. -``` - -### Absolute Paths -You can also give an absolute path, including user paths (with `~`). -```python ->>> session = CachedSession('~/.myapp/http_cache', backend='sqlite') ->>> print(session.cache.db_path) -'/home/user/.myapp/http_cache.sqlite' -``` - -### System Paths -If you don't know exactly where you want to put your cache files, your system's **temp directory** -or **cache directory** is a good choice. Some options are available as shortcuts to use whatever the -default locations are for your operating system. - -Use the default temp directory with the `use_temp` option: -:::{tab} Linux -```python ->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) ->>> print(session.cache.db_path) -'/tmp/http_cache.sqlite' -``` -::: -:::{tab} macOS -```python ->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) ->>> print(session.cache.db_path) -'/var/folders/xx/http_cache.sqlite' -``` -::: -:::{tab} Windows -```python ->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) ->>> print(session.cache.db_path) -'C:\\Users\\user\\AppData\\Local\\temp\\http_cache.sqlite' -``` -::: - -Or use the default cache directory with the `use_cache_dir` option: -:::{tab} Linux -```python ->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) ->>> print(session.cache.cache_dir) -'/home/user/.cache/http_cache/' -``` -::: -:::{tab} macOS -```python ->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) ->>> print(session.cache.cache_dir) -'/Users/user/Library/Caches/http_cache/' -``` -::: -:::{tab} Windows -```python ->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) ->>> print(session.cache.cache_dir) -'C:\\Users\\user\\AppData\\Local\\http_cache\\' -``` -::: - -```{note} -If the cache name is an absolute path, the `use_temp` and `use_cache_dir` options will be ignored. -If it's a relative path, it will be relative to the temp or cache directory, respectively. -``` - -There are a number of other system default locations that might be appropriate for a cache file. See -the [appdirs](https://github.com/ActiveState/appdirs) library for an easy cross-platform way to get -the most commonly used ones. - -## Cache Filtering -In many cases you will want to choose what you want to cache instead of just caching everything. By -default, all **read-only** (`GET` and `HEAD`) **requests with a 200 response code** are cached. A -few options are available to modify this behavior. - -```{note} -When using {py:class}`.CachedSession`, any requests that you don't want to cache can also be made -with a regular {py:class}`requests.Session` object, or wrapper functions like -{py:func}`requests.get`, etc. -``` - -### Cached HTTP Methods -To cache additional HTTP methods, specify them with `allowable_methods`: -```python ->>> session = CachedSession(allowable_methods=('GET', 'POST')) ->>> session.post('http://httpbin.org/post', json={'param': 'value'}) -``` - -For example, some APIs use the `POST` method to request data via a JSON-formatted request body, for -requests that may exceed the max size of a `GET` request. You may also want to cache `POST` requests -to ensure you don't send the exact same data multiple times. - -### Cached Status Codes -To cache additional status codes, specify them with `allowable_codes` -```python ->>> session = CachedSession(allowable_codes=(200, 418)) ->>> session.get('http://httpbin.org/teapot') -``` - -(selective-caching)= -### Cached URLs -You can use {ref}`URL patterns <url-patterns>` to define an allowlist for selective caching, by -using a expiration value of `0` (or `requests_cache.DO_NOT_CACHE`, to be more explicit) for -non-matching request URLs: -```python ->>> from requests_cache import DO_NOT_CACHE, CachedSession ->>> urls_expire_after = { -... '*.site_1.com': 30, -... 'site_2.com/static': -1, -... '*': DO_NOT_CACHE, -... } ->>> session = CachedSession(urls_expire_after=urls_expire_after) -``` - -Note that the catch-all rule above (`'*'`) will behave the same as setting the session-level -expiration to `0`: -```python ->>> urls_expire_after = {'*.site_1.com': 30, 'site_2.com/static': -1} ->>> session = CachedSession(urls_expire_after=urls_expire_after, expire_after=0) -``` - -### Custom Cache Filtering -If you would like more control over which requests get cached, see -{ref}`advanced_usage:custom cache filtering`. - -## Request Matching -Requests are matched according to the request URL, parameters and body. All of these values are -normalized to account for any variations that do not modify response content. - -There are additional options to match according to request headers, ignore specific request -parameters, or create your own custom request matcher. - -### Matching Request Headers -In some cases, different headers may result in different response data, so you may want to cache -them separately. To enable this, use `include_get_headers`: -```python ->>> session = CachedSession(include_get_headers=True) ->>> # Both of these requests will be sent and cached separately ->>> session.get('http://httpbin.org/headers', {'Accept': 'text/plain'}) ->>> session.get('http://httpbin.org/headers', {'Accept': 'application/json'}) -``` - -### Selective Parameter Matching -By default, all normalized request parameters are matched. In some cases, there may be request -parameters that don't affect the response data, for example authentication tokens or credentials. -If you want to ignore specific parameters, specify them with the `ignored_parameters` option. - -**Request Parameters:** - -In this example, only the first request will be sent, and the second request will be a cache hit -due to the ignored parameters: -```python ->>> session = CachedSession(ignored_parameters=['auth-token']) ->>> session.get('http://httpbin.org/get', params={'auth-token': '2F63E5DF4F44'}) ->>> r = session.get('http://httpbin.org/get', params={'auth-token': 'D9FAEB3449D3'}) ->>> assert r.from_cache is True -``` - -**Request Body Parameters:** - -This also applies to parameters in a JSON-formatted request body: -```python ->>> session = CachedSession(allowable_methods=('GET', 'POST'), ignored_parameters=['auth-token']) ->>> session.post('http://httpbin.org/post', json={'auth-token': '2F63E5DF4F44'}) ->>> r = session.post('http://httpbin.org/post', json={'auth-token': 'D9FAEB3449D3'}) ->>> assert r.from_cache is True -``` - -**Request Headers:** - -As well as headers, if `include_get_headers` is also used: -```python ->>> session = CachedSession(ignored_parameters=['auth-token'], include_get_headers=True) ->>> session.get('http://httpbin.org/get', headers={'auth-token': '2F63E5DF4F44'}) ->>> r = session.get('http://httpbin.org/get', headers={'auth-token': 'D9FAEB3449D3'}) ->>> assert r.from_cache is True -``` - -### Removing Sensitive Request Info -`ignored_parameters` will also be removed from the cached response, including request parameters, -body, and headers. This makes `ignored_parameters` a good way to prevent credentials or other -sensitive info from being saved in the cache backend. - -### Custom Request Matching -If you need more control over request matching behavior, see -{ref}`advanced_usage:custom request matching`. - -## Cache Expiration -By default, cached responses will be stored indefinitely. There are a number of options for -specifying how long to store responses, either with a single expiration value, glob patterns, -or {ref}`cache headers <headers>`. - -The simplest option is to initialize the cache with an `expire_after` value, which will apply to all -reponses: -```python ->>> # Set expiration for the session using a value in seconds ->>> session = CachedSession(expire_after=360) -``` - -### Expiration Precedence -Expiration can be set on a per-session, per-URL, or per-request basis, in addition to cache -headers (see sections below for usage details). When there are multiple values provided for a given -request, the following order of precedence is used: -1. Cache-Control request headers (if enabled) -2. Cache-Control response headers (if enabled) -3. Per-request expiration (`expire_after` argument for {py:meth}`.CachedSession.request`) -4. Per-URL expiration (`urls_expire_after` argument for {py:class}`.CachedSession`) -5. Per-session expiration (`expire_after` argument for {py:class}`.CacheBackend`) - -### Expiration Values -`expire_after` can be any of the following: -- `-1` (to never expire) -- `0` (to "expire immediately," e.g. bypass the cache) -- A positive number (in seconds) -- A {py:class}`~datetime.timedelta` -- A {py:class}`~datetime.datetime` - -Examples: -```python ->>> # To specify a unit of time other than seconds, use a timedelta ->>> from datetime import timedelta ->>> session = CachedSession(expire_after=timedelta(days=30)) - ->>> # Update an existing session to disable expiration (i.e., store indefinitely) ->>> session.expire_after = -1 - ->>> # Disable caching by default, unless enabled by other settings ->>> session = CachedSession(expire_after=0) -``` - -(url-patterns)= -### Expiration With URL Patterns -You can use `urls_expire_after` to set different expiration values based on URL glob patterns. -This allows you to customize caching based on what you know about the resources you're requesting -or how you intend to use them. For example, you might request one resource that gets updated -frequently, another that changes infrequently, and another that never changes. Example: -```python ->>> urls_expire_after = { -... '*.site_1.com': 30, -... 'site_2.com/resource_1': 60 * 2, -... 'site_2.com/resource_2': 60 * 60 * 24, -... 'site_2.com/static': -1, -... } ->>> session = CachedSession(urls_expire_after=urls_expire_after) -``` - -**Notes:** -- `urls_expire_after` should be a dict in the format `{'pattern': expire_after}` -- `expire_after` accepts the same types as `CachedSession.expire_after` -- Patterns will match request **base URLs without the protocol**, so the pattern `site.com/resource/` - is equivalent to `http*://site.com/resource/**` -- If there is more than one match, the first match will be used in the order they are defined -- If no patterns match a request, `CachedSession.expire_after` will be used as a default - -### Expiration and Error Handling -In some cases, you might cache a response, have it expire, but then encounter an error when -retrieving a new response. If you would like to use expired response data in these cases, use the -`old_data_on_error` option. - -For example: -```python ->>> # Cache a test response that will expire immediately ->>> session = CachedSession(old_data_on_error=True) ->>> session.get('https://httpbin.org/get', expire_after=0.0001) ->>> time.sleep(0.0001) -``` - -Afterward, let's say the page has moved and you get a 404, or the site is experiencing downtime and -you get a 500. You will then get the expired cache data instead: -```python ->>> response = session.get('https://httpbin.org/get') ->>> print(response.from_cache, response.is_expired) -True, True -``` - -In addition to HTTP error codes, `old_data_on_error` also applies to python exceptions (typically a -{py:exc}`~requests.RequestException`). See `requests` documentation on -[Errors and Exceptions](https://2.python-requests.org/en/master/user/quickstart/#errors-and-exceptions) -for more details on request errors in general. - -### Removing Expired Responses -For better read performance, expired responses won't be removed immediately, but will be removed -(or replaced) the next time they are requested. -:::{tip} -Implementing one or more cache eviction algorithms is being considered. If this is something you are -interested in, please provide feedback via [issues](https://github.com/reclosedev/requests-cache/issues)! -::: - -To manually clear all expired responses, use -{py:meth}`.CachedSession.remove_expired_responses`: -```python ->>> session.remove_expired_responses() -``` - -Or, when using patching: -```python ->>> requests_cache.remove_expired_responses() -``` - -You can also apply a different `expire_after` to previously cached responses, which will -revalidate the cache with the new expiration time: -```python ->>> session.remove_expired_responses(expire_after=timedelta(days=30)) -``` - -(headers)= -## Cache Headers -Most common request and response headers related to caching are supported, including -[Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control) -and [ETags](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag). - -```{note} -requests-cache is not intended to be strict implementation of HTTP caching according to -[RFC 2616](https://datatracker.ietf.org/doc/html/rfc2616), -[RFC 7234](https://datatracker.ietf.org/doc/html/rfc7234), etc. These RFCs describe many behaviors -that make sense in the context of a browser or proxy cache, but not for a python application. -``` - -### Conditional Requests -[Conditional requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests) are -automatically sent for any servers that support them. Once a cached response expires, it will only -be updated if the remote content has changed. - -Here's an example using the [GitHub API](https://docs.github.com/en/rest) to get info about the -requests-cache repo: -```python ->>> # Cache a response that will expire immediately ->>> url = 'https://api.github.com/repos/reclosedev/requests-cache' ->>> session = CachedSession(expire_after=0.0001) ->>> session.get(url) ->>> time.sleep(0.0001) - ->>> # The cached response will still be used until the remote content actually changes ->>> response = session.get(url) ->>> print(response.from_cache, response.is_expired) -True, True -``` - -### Cache-Control -If enabled, `Cache-Control` directives will take priority over any other `expire_after` value. -See {ref}`user_guide:expiration precedence` for the full order of precedence. - -To enable this behavior, use the `cache_control` option: -```python ->>> session = CachedSession(cache_control=True) -``` - -### Supported Headers -The following headers are currently supported: - -**Request headers:** -- `Cache-Control: max-age`: Used as the expiration time in seconds -- `Cache-Control: no-cache`: Skips reading response data from the cache -- `Cache-Control: no-store`: Skips reading and writing response data from/to the cache -- `If-None-Match`: Automatically added if an `ETag` is available -- `If-Modified-Since`: Automatically added if `Last-Modified` is available - -**Response headers:** -- `Cache-Control: max-age`: Used as the expiration time in seconds -- `Cache-Control: no-store` Skips writing response data to the cache -- `Expires`: Used as an absolute expiration time -- `ETag`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response) -- `Last-Modified`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response) - -```{note} -Unlike a browser or proxy cache, `max-age=0` does not clear previously cached responses. -``` - -(serializers)= -## Serializers -![](_static/file-pickle_32px.png) -![](_static/file-json_32px.png) -![](_static/file-yaml_32px.png) -![](_static/file-toml_32px.png) - -By default, responses are serialized using {py:mod}`pickle`, but some alternative serializers are -also included. These are mainly intended for use with {py:class}`.FileCache`, but are compatible -with the other backends as well. - -:::{note} -Some serializers require additional dependencies -::: - -### Specifying a Serializer -Similar to {ref}`backends`, you can specify which serializer to use with the `serializer` parameter -for either {py:class}`.CachedSession` or {py:func}`.install_cache`. - -### JSON Serializer -Storing responses as JSON gives you the benefit of making them human-readable and editable, in -exchange for a minor reduction in read and write speeds. - -Usage: -```python ->>> session = CachedSession('my_cache', serializer='json') -``` - -:::{admonition} Example JSON-serialized Response -:class: toggle -```{literalinclude} sample_response.json -:language: JSON -``` -::: - -This will use [ultrajson](https://github.com/ultrajson/ultrajson) if installed, otherwise the stdlib -`json` module will be used. You can install the optional dependencies for this serializer with: -```bash -pip install requests-cache[json] -``` - -### YAML Serializer -YAML is another option if you need a human-readable/editable format, with the same tradeoffs as JSON. - -Usage: -```python ->>> session = CachedSession('my_cache', serializer='yaml') -``` - -:::{admonition} Example YAML-serialized Response -:class: toggle -```{literalinclude} sample_response.yaml -:language: YAML -``` -::: - -You can install the extra dependencies for this serializer with: -```bash -pip install requests-cache[yaml] -``` - -### BSON Serializer -[BSON](https://www.mongodb.com/json-and-bson) is a serialization format originally created for -MongoDB, but it can also be used independently. Compared to JSON, it has better performance -(although still not as fast as `pickle`), and adds support for additional data types. It is not -human-readable, but some tools support reading and editing it directly -(for example, [bson-converter](https://atom.io/packages/bson-converter) for Atom). - -Usage: -```python ->>> session = CachedSession('my_cache', serializer='bson') -``` - -You can install the extra dependencies for this serializer with: -```bash -pip install requests-cache[mongo] -``` - -Or if you would like to use the standalone BSON codec for a different backend, without installing -MongoDB dependencies: -```bash -pip install requests-cache[bson] -``` - -### Serializer Security -See {ref}`security` for recommended setup steps for more secure cache serialization, particularly -when using {py:mod}`pickle`. - -### Custom Serializers -See {ref}`advanced_usage:custom serializers` for other possible formats, and options for creating -your own implementation. - -## Potential Issues -- See {ref}`monkeypatch-issues` for issues specific to {py:func}`.install_cache` -- New releases of `requests`, `urllib3` or `requests-cache` itself may change response data and be - be incompatible with previously cached data (see issues - [#56](https://github.com/reclosedev/requests-cache/issues/56) and - [#102](https://github.com/reclosedev/requests-cache/issues/102)). - In these cases, the cached data will simply be invalidated and a new response will be fetched. diff --git a/docs/user_guide/advanced_requests.md b/docs/user_guide/advanced_requests.md new file mode 100644 index 0000000..7115a7f --- /dev/null +++ b/docs/user_guide/advanced_requests.md @@ -0,0 +1,50 @@ +<!-- TODO: Better title for this section --> +# Usage with other requests features + +## Request Hooks +Requests has an [Event Hook](https://requests.readthedocs.io/en/master/user/advanced/#event-hooks) +system that can be used to add custom behavior into different parts of the request process. +It can be used, for example, for request throttling: + +:::{admonition} Example code +:class: toggle +```python +>>> import time +>>> import requests +>>> from requests_cache import CachedSession +>>> +>>> def make_throttle_hook(timeout=1.0): +>>> """Make a request hook function that adds a custom delay for non-cached requests""" +>>> def hook(response, *args, **kwargs): +>>> if not getattr(response, 'from_cache', False): +>>> print('sleeping') +>>> time.sleep(timeout) +>>> return response +>>> return hook +>>> +>>> session = CachedSession() +>>> session.hooks['response'].append(make_throttle_hook(0.1)) +>>> # The first (real) request will have an added delay +>>> session.get('http://httpbin.org/get') +>>> session.get('http://httpbin.org/get') +``` +::: + +## Streaming Requests +If you use [streaming requests](https://2.python-requests.org/en/master/user/advanced/#id9), you +can use the same code to iterate over both cached and non-cached requests. Cached response content +will have already been read (i.e., consumed), but will be available for re-reading so it behaves like +the original streamed response: + +:::{admonition} Example code +:class: toggle +```python +>>> from requests_cache import CachedSession +>>> +>>> session = CachedSession() +>>> for i in range(2): +... response = session.get('https://httpbin.org/stream/20', stream=True) +... for chunk in response.iter_lines(): +... print(chunk.decode('utf-8')) +``` +::: diff --git a/docs/user_guide/backends.md b/docs/user_guide/backends.md new file mode 100644 index 0000000..d95c840 --- /dev/null +++ b/docs/user_guide/backends.md @@ -0,0 +1,152 @@ +(backends)= +# Backends +![](../_static/sqlite_32px.png) +![](../_static/redis_32px.png) +![](../_static/mongodb_32px.png) +![](../_static/dynamodb_32px.png) +![](../_static/files-json_32px.png) + +Several cache backends are included. The default is SQLite, since it's generally the simplest to +use, and requires no extra dependencies or configuration. +```{note} +In the rare case that SQLite is not available +(for example, [on Heroku](https://devcenter.heroku.com/articles/sqlite3)), a non-persistent +in-memory cache is used by default. +``` + +See {py:mod}`.requests_cache.backends` for usage details for specific backends. + +## Backend Dependencies +Most of the other backends require some extra dependencies, listed below. + +Backend | Class | Alias | Dependencies +-------------------------------------------------------|----------------------------|----------------|------------- +[SQLite](https://www.sqlite.org) | {py:class}`.SQLiteCache` | `'sqlite'` | +[Redis](https://redis.io) | {py:class}`.RedisCache` | `'redis'` | [redis-py](https://github.com/andymccurdy/redis-py) +[MongoDB](https://www.mongodb.com) | {py:class}`.MongoCache` | `'mongodb'` | [pymongo](https://github.com/mongodb/mongo-python-driver) +[GridFS](https://docs.mongodb.com/manual/core/gridfs/) | {py:class}`.GridFSCache` | `'gridfs'` | [pymongo](https://github.com/mongodb/mongo-python-driver) +[DynamoDB](https://aws.amazon.com/dynamodb) | {py:class}`.DynamoCache` | `'dynamodb'` | [boto3](https://github.com/boto/boto3) +Filesystem | {py:class}`.FileCache` | `'filesystem'` | +Memory | {py:class}`.BaseCache` | `'memory'` | + +## Specifying a Backend +You can specify which backend to use with the `backend` parameter for either {py:class}`.CachedSession` +or {py:func}`.install_cache`. You can specify one by name, using the aliases listed above: +```python +>>> session = CachedSession('my_cache', backend='redis') +``` + +Or by instance: +```python +>>> backend = RedisCache(host='192.168.1.63', port=6379) +>>> session = CachedSession('my_cache', backend=backend) +``` + +## Backend Options +The `cache_name` parameter has a different use depending on the backend: + +Backend | Cache name used as +----------------|------------------- +SQLite | Database path +Redis | Hash namespace +MongoDB, GridFS | Database name +DynamoDB | Table name +Filesystem | Cache directory + +Each backend class also accepts optional parameters for the underlying connection. For example, +{py:class}`.SQLiteCache` accepts parameters for {py:func}`sqlite3.connect`: +```python +>>> session = CachedSession('my_cache', backend='sqlite', timeout=30) +``` + +## Testing Backends +If you just want to quickly try out all of the available backends for comparison, +[docker-compose](https://docs.docker.com/compose/) config is included for all supported services. +First, [install docker](https://docs.docker.com/get-docker/) if you haven't already. Then, run: + +:::{tab} Bash (Linux/macOS) +```bash +pip install -U requests-cache[all] docker-compose +curl https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -O docker-compose.yml +docker-compose up -d +``` +::: +:::{tab} Powershell (Windows) +```ps1 +pip install -U requests-cache[all] docker-compose +Invoke-WebRequest -Uri https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -Outfile docker-compose.yml +docker-compose up -d +``` +::: + +(exporting)= +## Exporting To A Different Backend +If you have cached data that you want to copy or migrate to a different backend, you can do this +with `CachedSession.cache.update()`. For example, if you want to dump the contents of a Redis cache +to JSON files: +```python +>>> src_session = CachedSession('my_cache', backend='redis') +>>> dest_session = CachedSession('~/workspace/cache_dump', backend='filesystem', serializer='json') +>>> dest_session.cache.update(src_session.cache) + +>>> # List the exported files +>>> print(dest_session.cache.paths()) +'/home/user/workspace/cache_dump/9e7a71a3ff2e.json' +'/home/user/workspace/cache_dump/8a922ff3c53f.json' +``` + +Or, using backend classes directly: +```python +>>> src_cache = RedisCache() +>>> dest_cache = FileCache('~/workspace/cache_dump', serializer='json') +>>> dest_cache.update(src_cache) +``` + +(custom-backends)= +## Custom Backends +If the built-in backends don't suit your needs, you can create your own by making subclasses of {py:class}`.BaseCache` and {py:class}`.BaseStorage`: + +:::{admonition} Example code +:class: toggle +```python +>>> from requests_cache import CachedSession +>>> from requests_cache.backends import BaseCache, BaseStorage + +>>> class CustomCache(BaseCache): +... """Wrapper for higher-level cache operations. In most cases, the only thing you need +... to specify here is which storage class(es) to use. +... """ +... def __init__(self, **kwargs): +... super().__init__(**kwargs) +... self.redirects = CustomStorage(**kwargs) +... self.responses = CustomStorage(**kwargs) + +>>> class CustomStorage(BaseStorage): +... """Dict-like interface for lower-level backend storage operations""" +... def __init__(self, **kwargs): +... super().__init__(**kwargs) +... +... def __getitem__(self, key): +... pass +... +... def __setitem__(self, key, value): +... pass +... +... def __delitem__(self, key): +... pass +... +... def __iter__(self): +... pass +... +... def __len__(self): +... pass +... +... def clear(self): +... pass +``` +::: + +You can then use your custom backend in a {py:class}`.CachedSession` with the `backend` parameter: +```python +>>> session = CachedSession(backend=CustomCache()) +``` diff --git a/docs/user_guide/compatibility.md b/docs/user_guide/compatibility.md new file mode 100644 index 0000000..d52d610 --- /dev/null +++ b/docs/user_guide/compatibility.md @@ -0,0 +1,146 @@ +<!-- TODO: Fix relative links --> +(compatibility)= +# Usage with other requests-based libraries +This library works by patching and/or extending {py:class}`requests.Session`. Many other libraries +out there do the same thing, making it potentially difficult to combine them. + +For that scenario, a mixin class is provided, so you can create a custom class with behavior from +multiple Session-modifying libraries: +```python +>>> from requests import Session +>>> from requests_cache import CacheMixin +>>> from some_other_lib import SomeOtherMixin +>>> +>>> class CustomSession(CacheMixin, SomeOtherMixin, Session): +... """Session class with features from both some_other_lib and requests-cache""" +``` + +## Requests-HTML +[requests-html](https://github.com/psf/requests-html) is one library that works with this method: +:::{admonition} Example code +:class: toggle +```python +>>> import requests +>>> from requests_cache import CacheMixin, install_cache +>>> from requests_html import HTMLSession +>>> +>>> class CachedHTMLSession(CacheMixin, HTMLSession): +... """Session with features from both CachedSession and HTMLSession""" +>>> +>>> session = CachedHTMLSession() +>>> response = session.get('https://github.com/') +>>> print(response.from_cache, response.html.links) +``` +::: + + +Or if you are using {py:func}`.install_cache`, you can use the `session_factory` argument: +:::{admonition} Example code +:class: toggle +```python +>>> install_cache(session_factory=CachedHTMLSession) +>>> response = requests.get('https://github.com/') +>>> print(response.from_cache, response.html.links) +``` +::: + +The same approach can be used with other libraries that subclass {py:class}`requests.Session`. + +## Requests-futures +Some libraries, including [requests-futures](https://github.com/ross/requests-futures), +support wrapping an existing session object: +```python +>>> session = FutureSession(session=CachedSession()) +``` + +In this case, `FutureSession` must wrap `CachedSession` rather than the other way around, since +`FutureSession` returns (as you might expect) futures rather than response objects. +See [issue #135](https://github.com/reclosedev/requests-cache/issues/135) for more notes on this. + +## Internet Archive +Usage with [internetarchive](https://github.com/jjjake/internetarchive) is the same as other libraries +that subclass `requests.Session`: +:::{admonition} Example code +:class: toggle +```python +>>> from requests_cache import CacheMixin +>>> from internetarchive.session import ArchiveSession +>>> +>>> class CachedArchiveSession(CacheMixin, ArchiveSession): +... """Session with features from both CachedSession and ArchiveSession""" +``` +::: + +## Requests-mock +[requests-mock](https://github.com/jamielennox/requests-mock) has multiple methods for mocking +requests, including a contextmanager, decorator, fixture, and adapter. There are a few different +options for using it with requests-cache, depending on how you want your tests to work. + +### Disabling requests-cache +If you have an application that uses requests-cache and you just want to use requests-mock in +your tests, the easiest thing to do is to disable requests-cache. + +For example, if you are using {py:func}`.install_cache` in your application and the +requests-mock [pytest fixture](https://requests-mock.readthedocs.io/en/latest/pytest.html) in your +tests, you could wrap it in another fixture that uses {py:func}`.uninstall_cache` or {py:func}`.disabled`: +:::{admonition} Example code +:class: toggle +```{literalinclude} ../../tests/compat/test_requests_mock_disable_cache.py +``` +::: + +Or if you use a `CachedSession` object, you could replace it with a regular `Session`, for example: +:::{admonition} Example code +:class: toggle +```python +import unittest +import pytest +import requests + + +@pytest.fixure(scope='function', autouse=True) +def disable_requests_cache(): + """Replace CachedSession with a regular Session for all test functions""" + with unittest.mock.patch('requests_cache.CachedSession', requests.Session): + yield +``` +::: + +### Combining requests-cache with requests-mock +If you want both caching and mocking features at the same time, you can attach requests-mock's +[adapter](https://requests-mock.readthedocs.io/en/latest/adapter.html) to a `CachedSession`: + +:::{admonition} Example code +:class: toggle +```{literalinclude} ../../tests/compat/test_requests_mock_combine_cache.py +``` +::: + +### Building a mocker using requests-cache data +Another approach is to use cached data to dynamically define mock requests + responses. +This has the advantage of only using request-mock's behavior for +[request matching](https://requests-mock.readthedocs.io/en/latest/matching.html). + +:::{admonition} Example code +:class: toggle +```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py +:lines: 21-40 +``` +::: + +To turn that into a complete example: +:::{admonition} Example code +:class: toggle +```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py +``` +::: + +## Responses +Usage with the [responses](https://github.com/getsentry/responses) library is similar to the +requests-mock examples above. + +:::{admonition} Example code +:class: toggle +```{literalinclude} ../tests/compat/test_responses_load_cache.py +``` +::: diff --git a/docs/user_guide/expiration.md b/docs/user_guide/expiration.md new file mode 100644 index 0000000..adb28ea --- /dev/null +++ b/docs/user_guide/expiration.md @@ -0,0 +1,119 @@ +(expiration)= +# Expiration +By default, cached responses will be stored indefinitely. There are a number of options for +specifying how long to store responses, either with a single expiration value, glob patterns, +or {ref}`cache headers <headers>`. + +The simplest option is to initialize the cache with an `expire_after` value, which will apply to all +reponses: +```python +>>> # Set expiration for the session using a value in seconds +>>> session = CachedSession(expire_after=360) +``` + +(precedence)= +## Expiration Precedence +Expiration can be set on a per-session, per-URL, or per-request basis, in addition to cache +headers (see sections below for usage details). When there are multiple values provided for a given +request, the following order of precedence is used: +1. Cache-Control request headers (if enabled) +2. Cache-Control response headers (if enabled) +3. Per-request expiration (`expire_after` argument for {py:meth}`.CachedSession.request`) +4. Per-URL expiration (`urls_expire_after` argument for {py:class}`.CachedSession`) +5. Per-session expiration (`expire_after` argument for {py:class}`.CacheBackend`) + +## Expiration Values +`expire_after` can be any of the following: +- `-1` (to never expire) +- `0` (to "expire immediately," e.g. bypass the cache) +- A positive number (in seconds) +- A {py:class}`~datetime.timedelta` +- A {py:class}`~datetime.datetime` + +Examples: +```python +>>> # To specify a unit of time other than seconds, use a timedelta +>>> from datetime import timedelta +>>> session = CachedSession(expire_after=timedelta(days=30)) + +>>> # Update an existing session to disable expiration (i.e., store indefinitely) +>>> session.expire_after = -1 + +>>> # Disable caching by default, unless enabled by other settings +>>> session = CachedSession(expire_after=0) +``` + +(url-patterns)= +## Expiration With URL Patterns +You can use `urls_expire_after` to set different expiration values based on URL glob patterns. +This allows you to customize caching based on what you know about the resources you're requesting +or how you intend to use them. For example, you might request one resource that gets updated +frequently, another that changes infrequently, and another that never changes. Example: +```python +>>> urls_expire_after = { +... '*.site_1.com': 30, +... 'site_2.com/resource_1': 60 * 2, +... 'site_2.com/resource_2': 60 * 60 * 24, +... 'site_2.com/static': -1, +... } +>>> session = CachedSession(urls_expire_after=urls_expire_after) +``` + +**Notes:** +- `urls_expire_after` should be a dict in the format `{'pattern': expire_after}` +- `expire_after` accepts the same types as `CachedSession.expire_after` +- Patterns will match request **base URLs without the protocol**, so the pattern `site.com/resource/` + is equivalent to `http*://site.com/resource/**` +- If there is more than one match, the first match will be used in the order they are defined +- If no patterns match a request, `CachedSession.expire_after` will be used as a default + +## Expiration and Error Handling +In some cases, you might cache a response, have it expire, but then encounter an error when +retrieving a new response. If you would like to use expired response data in these cases, use the +`old_data_on_error` option. + +For example: +```python +>>> # Cache a test response that will expire immediately +>>> session = CachedSession(old_data_on_error=True) +>>> session.get('https://httpbin.org/get', expire_after=0.0001) +>>> time.sleep(0.0001) +``` + +Afterward, let's say the page has moved and you get a 404, or the site is experiencing downtime and +you get a 500. You will then get the expired cache data instead: +```python +>>> response = session.get('https://httpbin.org/get') +>>> print(response.from_cache, response.is_expired) +True, True +``` + +In addition to HTTP error codes, `old_data_on_error` also applies to python exceptions (typically a +{py:exc}`~requests.RequestException`). See `requests` documentation on +[Errors and Exceptions](https://2.python-requests.org/en/master/user/quickstart/#errors-and-exceptions) +for more details on request errors in general. + +## Removing Expired Responses +For better read performance, expired responses won't be removed immediately, but will be removed +(or replaced) the next time they are requested. +:::{tip} +Implementing one or more cache eviction algorithms is being considered. If this is something you are +interested in, please provide feedback via [issues](https://github.com/reclosedev/requests-cache/issues)! +::: + +To manually clear all expired responses, use +{py:meth}`.CachedSession.remove_expired_responses`: +```python +>>> session.remove_expired_responses() +``` + +Or, when using patching: +```python +>>> requests_cache.remove_expired_responses() +``` + +You can also apply a different `expire_after` to previously cached responses, which will +revalidate the cache with the new expiration time: +```python +>>> session.remove_expired_responses(expire_after=timedelta(days=30)) +``` diff --git a/docs/user_guide/files.md b/docs/user_guide/files.md new file mode 100644 index 0000000..de1325d --- /dev/null +++ b/docs/user_guide/files.md @@ -0,0 +1,94 @@ +(files)= +# Cache Files +```{note} +This section only applies to the {py:mod}`~requests_cache.backends.sqlite` and +{py:mod}`~requests_cache.backends.filesystem` backends. +``` +For file-based backends, the cache name will be used as a path to the cache file(s). You can use +a relative path, absolute path, or use some additional options for system-specific default paths. + +## Relative Paths +```python +>>> # Database path for SQLite cache +>>> session = CachedSession('http_cache', backend='sqlite') +>>> print(session.cache.db_path) +'<current working dir>/http_cache.sqlite' +``` +```python +>>> # Base directory for Filesystem cache +>>> session = CachedSession('http_cache', backend='filesystem') +>>> print(session.cache.cache_dir) +'<current working dir>/http_cache/' +``` + +```{note} +Parent directories will always be created, if they don't already exist. +``` + +## Absolute Paths +You can also give an absolute path, including user paths (with `~`). +```python +>>> session = CachedSession('~/.myapp/http_cache', backend='sqlite') +>>> print(session.cache.db_path) +'/home/user/.myapp/http_cache.sqlite' +``` + +## System Paths +If you don't know exactly where you want to put your cache files, your system's **temp directory** +or **cache directory** is a good choice. Some options are available as shortcuts to use whatever the +default locations are for your operating system. + +Use the default temp directory with the `use_temp` option: +:::{tab} Linux +```python +>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) +>>> print(session.cache.db_path) +'/tmp/http_cache.sqlite' +``` +::: +:::{tab} macOS +```python +>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) +>>> print(session.cache.db_path) +'/var/folders/xx/http_cache.sqlite' +``` +::: +:::{tab} Windows +```python +>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True) +>>> print(session.cache.db_path) +'C:\\Users\\user\\AppData\\Local\\temp\\http_cache.sqlite' +``` +::: + +Or use the default cache directory with the `use_cache_dir` option: +:::{tab} Linux +```python +>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) +>>> print(session.cache.cache_dir) +'/home/user/.cache/http_cache/' +``` +::: +:::{tab} macOS +```python +>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) +>>> print(session.cache.cache_dir) +'/Users/user/Library/Caches/http_cache/' +``` +::: +:::{tab} Windows +```python +>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True) +>>> print(session.cache.cache_dir) +'C:\\Users\\user\\AppData\\Local\\http_cache\\' +``` +::: + +```{note} +If the cache name is an absolute path, the `use_temp` and `use_cache_dir` options will be ignored. +If it's a relative path, it will be relative to the temp or cache directory, respectively. +``` + +There are a number of other system default locations that might be appropriate for a cache file. See +the [appdirs](https://github.com/ActiveState/appdirs) library for an easy cross-platform way to get +the most commonly used ones. diff --git a/docs/user_guide/filtering.md b/docs/user_guide/filtering.md new file mode 100644 index 0000000..909a535 --- /dev/null +++ b/docs/user_guide/filtering.md @@ -0,0 +1,77 @@ +(filtering)= +# Cache Filtering +In many cases you will want to choose what you want to cache instead of just caching everything. By +default, all **read-only** (`GET` and `HEAD`) **requests with a 200 response code** are cached. A +few options are available to modify this behavior. + +```{note} +When using {py:class}`.CachedSession`, any requests that you don't want to cache can also be made +with a regular {py:class}`requests.Session` object, or wrapper functions like +{py:func}`requests.get`, etc. +``` + +(http-methods)= +## Cached HTTP Methods +To cache additional HTTP methods, specify them with `allowable_methods`: +```python +>>> session = CachedSession(allowable_methods=('GET', 'POST')) +>>> session.post('http://httpbin.org/post', json={'param': 'value'}) +``` + +For example, some APIs use the `POST` method to request data via a JSON-formatted request body, for +requests that may exceed the max size of a `GET` request. You may also want to cache `POST` requests +to ensure you don't send the exact same data multiple times. + +## Cached Status Codes +To cache additional status codes, specify them with `allowable_codes` +```python +>>> session = CachedSession(allowable_codes=(200, 418)) +>>> session.get('http://httpbin.org/teapot') +``` + +(selective-caching)= +## Cached URLs +You can use {ref}`URL patterns <url-patterns>` to define an allowlist for selective caching, by +using a expiration value of `0` (or `requests_cache.DO_NOT_CACHE`, to be more explicit) for +non-matching request URLs: +```python +>>> from requests_cache import DO_NOT_CACHE, CachedSession +>>> urls_expire_after = { +... '*.site_1.com': 30, +... 'site_2.com/static': -1, +... '*': DO_NOT_CACHE, +... } +>>> session = CachedSession(urls_expire_after=urls_expire_after) +``` + +Note that the catch-all rule above (`'*'`) will behave the same as setting the session-level +expiration to `0`: +```python +>>> urls_expire_after = {'*.site_1.com': 30, 'site_2.com/static': -1} +>>> session = CachedSession(urls_expire_after=urls_expire_after, expire_after=0) +``` + +## Custom Cache Filtering +If you need more advanced behavior for choosing what to cache, you can provide a custom filtering +function via the `filter_fn` param. This can by any function that takes a +{py:class}`requests.Response` object and returns a boolean indicating whether or not that response +should be cached. It will be applied to both new responses (on write) and previously cached +responses (on read): + +:::{admonition} Example code +:class: toggle +```python +>>> from sys import getsizeof +>>> from requests_cache import CachedSession + +>>> def filter_by_size(response: Response) -> bool: +>>> """Don't cache responses with a body over 1 MB""" +>>> return getsizeof(response.content) <= 1024 * 1024 + +>>> session = CachedSession(filter_fn=filter_by_size) +``` +::: + +```{note} +`filter_fn()` will be used **in addition to** other filtering options. +``` diff --git a/docs/user_guide/general.md b/docs/user_guide/general.md new file mode 100644 index 0000000..e5d6562 --- /dev/null +++ b/docs/user_guide/general.md @@ -0,0 +1,92 @@ +(general)= +# General Usage +There are two main ways of using requests-cache: +- **Sessions:** (recommended) Use {py:class}`.CachedSession` to send your requests +- **Patching:** Globally patch `requests` using {py:func}`.install_cache()` + +## Sessions +{py:class}`.CachedSession` can be used as a drop-in replacement for {py:class}`requests.Session`. +Basic usage looks like this: +```python +>>> from requests_cache import CachedSession +>>> +>>> session = CachedSession() +>>> session.get('http://httpbin.org/get') +``` + +Any {py:class}`requests.Session` method can be used (but see {ref}`http-methods` section for +options): +```python +>>> session.request('GET', 'http://httpbin.org/get') +>>> session.head('http://httpbin.org/get') +``` + +Caching can be temporarily disabled for the session with +{py:meth}`.CachedSession.cache_disabled`: +```python +>>> with session.cache_disabled(): +... session.get('http://httpbin.org/get') +``` + +The best way to clean up your cache is through {ref}`expiration` settings, but you can also +clear out everything at once with {py:meth}`.BaseCache.clear`: +```python +>>> session.cache.clear() +``` + +## Patching +In some situations, it may not be possible or convenient to manage your own session object. In those +cases, you can use {py:func}`.install_cache` to add caching to all `requests` functions: +```python +>>> import requests +>>> import requests_cache +>>> +>>> requests_cache.install_cache() +>>> requests.get('http://httpbin.org/get') +``` + +As well as session methods: +```python +>>> session = requests.Session() +>>> session.get('http://httpbin.org/get') +``` + +{py:func}`.install_cache` accepts all the same parameters as {py:class}`.CachedSession`: +```python +>>> requests_cache.install_cache(expire_after=360, allowable_methods=('GET', 'POST')) +``` + +It can be temporarily {py:func}`.enabled`: +```python +>>> with requests_cache.enabled(): +... requests.get('http://httpbin.org/get') # Will be cached +``` + +Or temporarily {py:func}`.disabled`: +```python +>>> requests_cache.install_cache() +>>> with requests_cache.disabled(): +... requests.get('http://httpbin.org/get') # Will not be cached +``` + +Or completely removed with {py:func}`.uninstall_cache`: +```python +>>> requests_cache.uninstall_cache() +>>> requests.get('http://httpbin.org/get') +``` + +You can also clear out all responses in the cache with {py:func}`.clear`, and check if +requests-cache is currently installed with {py:func}`.is_installed`. + +(monkeypatch-issues)= +### Patching Limitations & Potential Issues +Like any other utility that uses monkey-patching, there are some scenarios where you won't want to +use {py:func}`.install_cache`: +- When using other libraries that patch {py:class}`requests.Session` +- In a multi-threaded or multiprocess application +- In a library that will be imported by other libraries or applications +- In a larger application that makes requests in several different modules, where it may not be + obvious what is and isn't being cached + +In any of these cases, consider using {py:class}`.CachedSession`, the {py:func}`.enabled` +contextmanager, or {ref}`selective-caching`. diff --git a/docs/user_guide/headers.md b/docs/user_guide/headers.md new file mode 100644 index 0000000..8d1a795 --- /dev/null +++ b/docs/user_guide/headers.md @@ -0,0 +1,62 @@ +(headers)= +# Cache Headers +Most common request and response headers related to caching are supported, including +[Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control) +and [ETags](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag). + +```{note} +requests-cache is not intended to be strict implementation of HTTP caching according to +[RFC 2616](https://datatracker.ietf.org/doc/html/rfc2616), +[RFC 7234](https://datatracker.ietf.org/doc/html/rfc7234), etc. These RFCs describe many behaviors +that make sense in the context of a browser or proxy cache, but not for a python application. +``` + +## Conditional Requests +[Conditional requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests) are +automatically sent for any servers that support them. Once a cached response expires, it will only +be updated if the remote content has changed. + +Here's an example using the [GitHub API](https://docs.github.com/en/rest) to get info about the +requests-cache repo: +```python +>>> # Cache a response that will expire immediately +>>> url = 'https://api.github.com/repos/reclosedev/requests-cache' +>>> session = CachedSession(expire_after=0.0001) +>>> session.get(url) +>>> time.sleep(0.0001) + +>>> # The cached response will still be used until the remote content actually changes +>>> response = session.get(url) +>>> print(response.from_cache, response.is_expired) +True, True +``` + +## Cache-Control +If enabled, `Cache-Control` directives will take priority over any other `expire_after` value. +See {ref}`precedence` for the full order of precedence. + +To enable this behavior, use the `cache_control` option: +```python +>>> session = CachedSession(cache_control=True) +``` + +## Supported Headers +The following headers are currently supported: + +**Request headers:** +- `Cache-Control: max-age`: Used as the expiration time in seconds +- `Cache-Control: no-cache`: Skips reading response data from the cache +- `Cache-Control: no-store`: Skips reading and writing response data from/to the cache +- `If-None-Match`: Automatically added if an `ETag` is available +- `If-Modified-Since`: Automatically added if `Last-Modified` is available + +**Response headers:** +- `Cache-Control: max-age`: Used as the expiration time in seconds +- `Cache-Control: no-store` Skips writing response data to the cache +- `Expires`: Used as an absolute expiration time +- `ETag`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response) +- `Last-Modified`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response) + +```{note} +Unlike a browser or proxy cache, `max-age=0` does not clear previously cached responses. +``` diff --git a/docs/user_guide/index.md b/docs/user_guide/index.md new file mode 100644 index 0000000..d28a108 --- /dev/null +++ b/docs/user_guide/index.md @@ -0,0 +1,21 @@ +(user-guide)= +# User Guide +This section covers the main features of requests-cache. + +```{toctree} +:maxdepth: 2 + +installation +general +backends +files +filtering +headers +inspection +expiration +matching +security +serializers +troubleshooting +compatibility +```` diff --git a/docs/user_guide/inspection.md b/docs/user_guide/inspection.md new file mode 100644 index 0000000..1ba560d --- /dev/null +++ b/docs/user_guide/inspection.md @@ -0,0 +1,82 @@ +<!-- TODO: This could use some more details and examples --> +(inspection)= +# Cache Inspection +Here are some ways to get additional information out of the cache session, backend, and responses: + +## Response Details +The following attributes are available on responses: +- `from_cache`: indicates if the response came from the cache +- `created_at`: {py:class}`~datetime.datetime` of when the cached response was created or last updated +- `expires`: {py:class}`~datetime.datetime` after which the cached response will expire +- `is_expired`: indicates if the cached response is expired (if an old response was returned due to a request error) + +Examples: +:::{admonition} Example code +:class: toggle +```python +>>> from requests_cache import CachedSession +>>> session = CachedSession(expire_after=timedelta(days=1)) + +>>> # Placeholders are added for non-cached responses +>>> response = session.get('http://httpbin.org/get') +>>> print(response.from_cache, response.created_at, response.expires, response.is_expired) +False None None None + +>>> # Values will be populated for cached responses +>>> response = session.get('http://httpbin.org/get') +>>> print(response.from_cache, response.created_at, response.expires, response.is_expired) +True 2021-01-01 18:00:00 2021-01-02 18:00:00 False + +>>> # Print a response object to get general information about it +>>> print(response) +'request: GET https://httpbin.org/get, response: 200 (308 bytes), created: 2021-01-01 22:45:00 IST, expires: 2021-01-02 18:45:00 IST (fresh)' +``` +::: + +## Cache Contents +You can use `CachedSession.cache.urls` to see all URLs currently in the cache: +```python +>>> session = CachedSession() +>>> print(session.cache.urls) +['https://httpbin.org/get', 'https://httpbin.org/stream/100'] +``` + +If needed, you can get more details on cached responses via `CachedSession.cache.responses`, which +is a dict-like interface to the cache backend. See {py:class}`.CachedResponse` for a full list of +attributes available. + +For example, if you wanted to to see all URLs requested with a specific method: +```python +>>> post_urls = [ +... response.url for response in session.cache.responses.values() +... if response.request.method == 'POST' +... ] +``` + +You can also inspect `CachedSession.cache.redirects`, which maps redirect URLs to keys of the +responses they redirect to. + +Additional `keys()` and `values()` wrapper methods are available on {py:class}`.BaseCache` to get +combined keys and responses. +```python +>>> print('All responses:') +>>> for response in session.cache.values(): +>>> print(response) + +>>> print('All cache keys for redirects and responses combined:') +>>> print(list(session.cache.keys())) +``` + +Both methods also take a `check_expiry` argument to exclude expired responses: +```python +>>> print('All unexpired responses:') +>>> for response in session.cache.values(check_expiry=True): +>>> print(response) +``` + +Similarly, you can get a count of responses with {py:meth}`.BaseCache.response_count`, and optionally +exclude expired responses: +```python +>>> print(f'Total responses: {session.cache.response_count()}') +>>> print(f'Unexpired responses: {session.cache.response_count(check_expiry=True)}') +``` diff --git a/docs/user_guide/installation.md b/docs/user_guide/installation.md new file mode 100644 index 0000000..7435c57 --- /dev/null +++ b/docs/user_guide/installation.md @@ -0,0 +1,41 @@ +# Installation +Installation instructions: + +:::{tab} Pip +Install the latest stable version from [PyPI](https://pypi.org/project/requests-cache/): +``` +pip install requests-cache +``` +::: +:::{tab} Conda +Or install from [conda-forge](https://anaconda.org/conda-forge/requests-cache), if you prefer: +``` +conda install -c conda-forge requests-cache +``` +::: +:::{tab} Pre-release +If you would like to use the latest development (pre-release) version: +``` +pip install --pre requests-cache +``` +::: +:::{tab} Local development +See {ref}`contributing` for setup steps for local development +::: + +## Requirements +You may need additional dependencies depending on which backend you want to use. To install with +extra dependencies for all supported {ref}`backends`: +``` +pip install requests-cache[all] +``` + +## Python Version Compatibility +The latest version of requests-cache requires **python 3.7+**. If you need to use an older version +of python, here are the latest compatible versions and their documentation pages: + +* **python 2.6:** [requests-cache 0.4.13](https://requests-cache.readthedocs.io/en/v0.4.13) +* **python 2.7:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) +* **python 3.4:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) +* **python 3.5:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0) +* **python 3.6:** [requests-cache 0.7.4](https://requests-cache.readthedocs.io/en/v0.7.4) diff --git a/docs/user_guide/matching.md b/docs/user_guide/matching.md new file mode 100644 index 0000000..333d868 --- /dev/null +++ b/docs/user_guide/matching.md @@ -0,0 +1,105 @@ +(matching)= +# Request Matching +Requests are matched according to the request URL, parameters and body. All of these values are +normalized to account for any variations that do not modify response content. + +There are additional options to match according to request headers, ignore specific request +parameters, or create your own custom request matcher. + +## Matching Request Headers +In some cases, different headers may result in different response data, so you may want to cache +them separately. To enable this, use `include_get_headers`: +```python +>>> session = CachedSession(include_get_headers=True) +>>> # Both of these requests will be sent and cached separately +>>> session.get('http://httpbin.org/headers', {'Accept': 'text/plain'}) +>>> session.get('http://httpbin.org/headers', {'Accept': 'application/json'}) +``` + +(filter-params)= +## Selective Parameter Matching +By default, all normalized request parameters are matched. In some cases, there may be request +parameters that don't affect the response data, for example authentication tokens or credentials. +If you want to ignore specific parameters, specify them with the `ignored_parameters` option. + +**Request Parameters:** + +In this example, only the first request will be sent, and the second request will be a cache hit +due to the ignored parameters: +```python +>>> session = CachedSession(ignored_parameters=['auth-token']) +>>> session.get('http://httpbin.org/get', params={'auth-token': '2F63E5DF4F44'}) +>>> r = session.get('http://httpbin.org/get', params={'auth-token': 'D9FAEB3449D3'}) +>>> assert r.from_cache is True +``` + +**Request Body Parameters:** + +This also applies to parameters in a JSON-formatted request body: +```python +>>> session = CachedSession(allowable_methods=('GET', 'POST'), ignored_parameters=['auth-token']) +>>> session.post('http://httpbin.org/post', json={'auth-token': '2F63E5DF4F44'}) +>>> r = session.post('http://httpbin.org/post', json={'auth-token': 'D9FAEB3449D3'}) +>>> assert r.from_cache is True +``` + +**Request Headers:** + +As well as headers, if `include_get_headers` is also used: +```python +>>> session = CachedSession(ignored_parameters=['auth-token'], include_get_headers=True) +>>> session.get('http://httpbin.org/get', headers={'auth-token': '2F63E5DF4F44'}) +>>> r = session.get('http://httpbin.org/get', headers={'auth-token': 'D9FAEB3449D3'}) +>>> assert r.from_cache is True +``` +```{note} +Since `ignored_parameters` is most often used for sensitive info like credentials, these values will also be removed from the cached request parameters, body, and headers. +``` + +## Custom Request Matching +If you need more advanced behavior, you can implement your own custom request matching. + +Request matching is accomplished using a **cache key**, which uniquely identifies a response in the +cache based on request info. For example, the option `ignored_parameters=['foo']` works by excluding +the `foo` request parameter from the cache key, meaning these three requests will all use the same +cached response: +```python +>>> session = CachedSession(ignored_parameters=['foo']) +>>> response_1 = session.get('https://example.com') # cache miss +>>> response_2 = session.get('https://example.com?foo=bar') # cache hit +>>> response_3 = session.get('https://example.com?foo=qux') # cache hit +>>> assert response_1.cache_key == response_2.cache_key == response_3.cache_key +``` + +If you want to implement your own request matching, you can provide a cache key function which will +take a {py:class}`~requests.PreparedRequest` plus optional keyword args, and return a string: +```python +def create_key(request: requests.PreparedRequest, **kwargs) -> str: + """Generate a custom cache key for the given request""" +``` + +`**kwargs` includes relevant {py:class}`.BaseCache` settings and any other keyword args passed to +{py:meth}`.CachedSession.send()`. See {py:func}`.create_key` for the reference implementation, and +see the rest of the {py:mod}`.cache_keys` module for some potentially useful helper functions. + +You can then pass this function via the `key_fn` param: +```python +session = CachedSession(key_fn=create_key) +``` + +```{note} +`key_fn()` will be used **instead of** any other {ref}`matching` options and default matching behavior. +``` +```{tip} +See {ref}`Examples<custom_keys>` page for a complete example for custom request matching. +``` +```{tip} +As a general rule, if you include less info in your cache keys, you will have more cache hits and +use less storage space, but risk getting incorrect response data back. For example, if you exclude +all request parameters, you will get the same cached response back for any combination of request +parameters. +``` +```{warning} +If you provide a custom key function for a non-empty cache, any responses previously cached with a +different key function will likely be unused. +``` diff --git a/docs/security.md b/docs/user_guide/security.md index 4c5218f..45fbd0a 100644 --- a/docs/security.md +++ b/docs/user_guide/security.md @@ -48,8 +48,8 @@ Once you have your key, create a {py:func}`.safe_pickle_serializer` with it: ``` :::{note} -You can also make your own {ref}`custom serializer <advanced_usage:custom serializers>` -using `itsdangerous`, if you would like more control over how responses are serialized. +You can also make your own {ref}`custom-serializers`, if you would like more control over how +responses are serialized. ::: You can verify that it's working by modifying the cached item (*without* your key): @@ -65,3 +65,7 @@ Then, if you try to get that cached response again (*with* your key), you will g >>> session.get('https://httpbin.org/get') BadSignature: Signature b'iFNmzdUOSw5vqrR9Cb_wfI1EoZ8' does not match ``` + +## Removing Sensitive Info +The {ref}`ignored_parameters <filter-params>` option can be used to prevent credentials and other +sensitive info from being saved to the cache. It applies to request parameters, body, and headers. diff --git a/docs/user_guide/serializers.md b/docs/user_guide/serializers.md new file mode 100644 index 0000000..46a94b7 --- /dev/null +++ b/docs/user_guide/serializers.md @@ -0,0 +1,167 @@ +(serializers)= +# Serializers +![](../_static/file-pickle_32px.png) +![](../_static/file-json_32px.png) +![](../_static/file-yaml_32px.png) +![](../_static/file-toml_32px.png) + +By default, responses are serialized using {py:mod}`pickle`, but some alternative serializers are +also included. These are mainly intended for use with {py:class}`.FileCache`, but are compatible +with the other backends as well. + +:::{note} +Some serializers require additional dependencies +::: + +## Specifying a Serializer +Similar to {ref}`backends`, you can specify which serializer to use with the `serializer` parameter +for either {py:class}`.CachedSession` or {py:func}`.install_cache`. + +## JSON Serializer +Storing responses as JSON gives you the benefit of making them human-readable and editable, in +exchange for a minor reduction in read and write speeds. + +Usage: +```python +>>> session = CachedSession('my_cache', serializer='json') +``` + +:::{admonition} Example JSON-serialized Response +:class: toggle +```{literalinclude} ../sample_data/sample_response.json +:language: JSON +``` +::: + +This will use [ultrajson](https://github.com/ultrajson/ultrajson) if installed, otherwise the stdlib +`json` module will be used. You can install the optional dependencies for this serializer with: +```bash +pip install requests-cache[json] +``` + +## YAML Serializer +YAML is another option if you need a human-readable/editable format, with the same tradeoffs as JSON. + +Usage: +```python +>>> session = CachedSession('my_cache', serializer='yaml') +``` + +:::{admonition} Example YAML-serialized Response +:class: toggle +```{literalinclude} ../sample_data/sample_response.yaml +:language: YAML +``` +::: + +You can install the extra dependencies for this serializer with: +```bash +pip install requests-cache[yaml] +``` + +## BSON Serializer +[BSON](https://www.mongodb.com/json-and-bson) is a serialization format originally created for +MongoDB, but it can also be used independently. Compared to JSON, it has better performance +(although still not as fast as `pickle`), and adds support for additional data types. It is not +human-readable, but some tools support reading and editing it directly +(for example, [bson-converter](https://atom.io/packages/bson-converter) for Atom). + +Usage: +```python +>>> session = CachedSession('my_cache', serializer='bson') +``` + +You can install the extra dependencies for this serializer with: +```bash +pip install requests-cache[mongo] +``` + +Or if you would like to use the standalone BSON codec for a different backend, without installing +MongoDB dependencies: +```bash +pip install requests-cache[bson] +``` + +## Serializer Security +See {ref}`security` for recommended setup steps for more secure cache serialization, particularly +when using {py:mod}`pickle`. + +(custom-serializers)= +## Custom Serializers +If the built-in serializers don't suit your needs, you can create your own. For example, if +you had a imaginary `custom_pickle` module that provides `dumps` and `loads` functions: +```python +>>> import custom_pickle +>>> from requests_cache import CachedSession +>>> session = CachedSession(serializer=custom_pickle) +``` + +### Serializer Pipelines +More complex serialization can be done with {py:class}`.SerializerPipeline`. Use cases include +text-based serialization, compression, encryption, and any other intermediate steps you might want +to add. + +Any combination of these can be composed with a {py:class}`.SerializerPipeline`, which starts with a +{py:class}`.CachedResponse` and ends with a {py:class}`.str` or {py:class}`.bytes` object. Each stage +of the pipeline can be any object or module with `dumps` and `loads` functions. If the object has +similar methods with different names (e.g. `compress` / `decompress`), those can be aliased using +{py:class}`.Stage`. + +For example, a compressed pickle serializer can be built as: +:::{admonition} Example code +:class: toggle +```python +>>> import pickle, gzip +>>> from requests_cache.serialzers import SerializerPipeline, Stage +>>> compressed_serializer = SerializerPipeline([ +... pickle, +... Stage(gzip, dumps='compress', loads='decompress'), +...]) +>>> session = CachedSession(serializer=compressed_serializer) +``` +::: + +### Text-based Serializers +If you're using a text-based serialization format like JSON or YAML, some extra steps are needed to +encode binary data and non-builtin types. The [cattrs](https://cattrs.readthedocs.io) library can do +the majority of the work here, and some pre-configured converters are included for serveral common +formats in the {py:mod}`.preconf` module. + +For example, a compressed JSON pipeline could be built as follows: +:::{admonition} Example code +:class: toggle +```python +>>> import json, gzip, codecs +>>> from requests_cache.serializers import SerializerPipeline, Stage, json_converter +>>> comp_json_serializer = SerializerPipeline([ +... json_converter, # Serialize to a JSON string +... Stage(codecs.utf_8, dumps='encode', loads='decode'), # Encode to bytes +... Stage(gzip, dumps='compress', loads='decompress'), # Compress +...]) +``` +::: + +```{note} +If you want to use a different format that isn't included in {py:mod}`.preconf`, you can use +{py:class}`.CattrStage` as a starting point. +``` + +```{note} +If you want to convert a string representation to bytes (e.g. for compression), you must use a codec +from {py:mod}`.codecs` (typically `codecs.utf_8`) +``` + +### Additional Serialization Steps +Some other tools that could be used as a stage in a {py:class}`.SerializerPipeline` include: + +Class | loads | dumps +----- | ----- | ----- +{py:mod}`codecs.* <.codecs>` | encode | decode +{py:mod}`.bz2` | compress | decompress +{py:mod}`.gzip` | compress | decompress +{py:mod}`.lzma` | compress | decompress +{py:mod}`.zlib` | compress | decompress +{py:mod}`.pickle` | dumps | loads +{py:class}`itsdangerous.signer.Signer` | sign | unsign +{py:class}`itsdangerous.serializer.Serializer` | loads | dumps +{py:class}`cryptography.fernet.Fernet` | encrypt | decrypt diff --git a/docs/user_guide/troubleshooting.md b/docs/user_guide/troubleshooting.md new file mode 100644 index 0000000..e61bf84 --- /dev/null +++ b/docs/user_guide/troubleshooting.md @@ -0,0 +1,10 @@ +<!-- TODO: Logging, tracebacks, submitting issues, etc. --> +# Troubleshooting + +## Potential Issues +- See {ref}`monkeypatch-issues` for issues specific to {py:func}`.install_cache` +- New releases of `requests`, `urllib3` or `requests-cache` itself may change response data and be + be incompatible with previously cached data (see issues + [#56](https://github.com/reclosedev/requests-cache/issues/56) and + [#102](https://github.com/reclosedev/requests-cache/issues/102)). + In these cases, the cached data will simply be invalidated and a new response will be fetched. diff --git a/examples/custom_cache_keys.py b/examples/custom_request_matcher.py index 5ba6dc0..a63139b 100644 --- a/examples/custom_cache_keys.py +++ b/examples/custom_request_matcher.py @@ -1,7 +1,7 @@ #!/usr/bin/env python """ -Example of a custom cache key function that caches a new response if the version of requests-cache, -requests, or urllib3 changes. +Example of a custom {ref}`request matcher <matching>` that caches a new response if the version of +requests-cache, requests, or urllib3 changes. This generally isn't needed, since anything that causes a deserialization error will simply result in a new request being sent and cached. But you might want to include a library version in your cache diff --git a/requests_cache/backends/__init__.py b/requests_cache/backends/__init__.py index bc48161..870511c 100644 --- a/requests_cache/backends/__init__.py +++ b/requests_cache/backends/__init__.py @@ -1,6 +1,4 @@ -"""Classes and functions for cache persistence. See :ref:`user_guide:cache backends` for general -usage info. -""" +"""Classes and functions for cache persistence. See :ref:`backends` for general usage info.""" # flake8: noqa: F401 from inspect import signature from logging import getLogger diff --git a/requests_cache/backends/base.py b/requests_cache/backends/base.py index e580d8b..4186f60 100644 --- a/requests_cache/backends/base.py +++ b/requests_cache/backends/base.py @@ -39,7 +39,7 @@ class BaseCache: Lower-level storage operations are handled by :py:class:`.BaseStorage`. - To extend this with your own custom backend, see :ref:`advanced_usage:custom backends`. + To extend this with your own custom backend, see :ref:`custom-backends`. """ def __init__( @@ -247,7 +247,7 @@ class BaseStorage(MutableMapping, ABC): ``BaseStorage`` also contains a serializer module or instance (defaulting to :py:mod:`pickle`), which determines how :py:class:`.CachedResponse` objects are saved internally. See - :ref:`user_guide:serializers` for details. + :ref:`serializers` for details. Args: serializer: Custom serializer that provides ``loads`` and ``dumps`` methods diff --git a/requests_cache/backends/filesystem.py b/requests_cache/backends/filesystem.py index 687dbf6..6ab23d4 100644 --- a/requests_cache/backends/filesystem.py +++ b/requests_cache/backends/filesystem.py @@ -24,7 +24,7 @@ Or as YAML (requires ``pyyaml``): Cache Files ^^^^^^^^^^^ -* See :ref:`user_guide:cache files` for general info on cache files +* See :ref:`files` for general info on specifying cache paths * The path for a given response will be in the format ``<cache_name>/<cache_key>`` * Redirects are stored in a separate SQLite database, located at ``<cache_name>/redirects.sqlite`` * Use :py:meth:`.FileCache.paths` to get a list of all cached response paths diff --git a/requests_cache/backends/sqlite.py b/requests_cache/backends/sqlite.py index 41e63ad..2ae5e40 100644 --- a/requests_cache/backends/sqlite.py +++ b/requests_cache/backends/sqlite.py @@ -12,7 +12,7 @@ for requests-cache. Cache Files ^^^^^^^^^^^ -* See :ref:`user_guide:cache files` for general info on cache files +* See :ref:`files` for general info on specifying cache paths * If you specify a name without an extension, the default extension ``.sqlite`` will be used In-Memory Caching diff --git a/requests_cache/serializers/__init__.py b/requests_cache/serializers/__init__.py index eeceecb..68ead1b 100644 --- a/requests_cache/serializers/__init__.py +++ b/requests_cache/serializers/__init__.py @@ -1,4 +1,4 @@ -"""Response serialization utilities. See :ref:`user_guide:serializers` for general usage info. +"""Response serialization utilities. See :ref:`serializers` for general usage info. """ # flake8: noqa: F401 from .cattrs import CattrStage |