summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJordan Cook <jordan.cook@pioneer.com>2021-08-26 14:48:33 -0500
committerJordan Cook <jordan.cook@pioneer.com>2021-08-26 17:04:15 -0500
commite203a48cdf58eda51f85b27d3eaf852bd9da941c (patch)
treea384f2ab226fecc66b81bf288d51f2acb0b3b960
parent31a760cfb998d31cdeb8f79e65e1f1765cb2e3c9 (diff)
downloadrequests-cache-e203a48cdf58eda51f85b27d3eaf852bd9da941c.tar.gz
Reorganize user docs: break down User Guide and Advanced Usage sections into smaller pages
-rw-r--r--.github/CODE_OF_CONDUCT.md13
-rw-r--r--docs/advanced_usage.md481
-rw-r--r--docs/contributing.md3
-rw-r--r--docs/contributors.md2
-rw-r--r--docs/examples.md7
-rw-r--r--docs/history.md3
-rw-r--r--docs/index.md11
-rw-r--r--docs/project_info/code_of_conduct.md2
-rw-r--r--docs/project_info/contributing.md3
-rw-r--r--docs/project_info/contributors.md2
-rw-r--r--docs/project_info/history.md3
-rw-r--r--docs/project_info/index.md11
-rw-r--r--docs/project_info/related_projects.md (renamed from docs/related_projects.md)3
-rw-r--r--docs/sample_data/sample_response.json (renamed from docs/sample_response.json)0
-rw-r--r--docs/sample_data/sample_response.yaml (renamed from docs/sample_response.yaml)0
-rw-r--r--docs/user_guide.md737
-rw-r--r--docs/user_guide/advanced_requests.md50
-rw-r--r--docs/user_guide/backends.md152
-rw-r--r--docs/user_guide/compatibility.md146
-rw-r--r--docs/user_guide/expiration.md119
-rw-r--r--docs/user_guide/files.md94
-rw-r--r--docs/user_guide/filtering.md77
-rw-r--r--docs/user_guide/general.md92
-rw-r--r--docs/user_guide/headers.md62
-rw-r--r--docs/user_guide/index.md21
-rw-r--r--docs/user_guide/inspection.md82
-rw-r--r--docs/user_guide/installation.md41
-rw-r--r--docs/user_guide/matching.md105
-rw-r--r--docs/user_guide/security.md (renamed from docs/security.md)8
-rw-r--r--docs/user_guide/serializers.md167
-rw-r--r--docs/user_guide/troubleshooting.md10
-rw-r--r--examples/custom_request_matcher.py (renamed from examples/custom_cache_keys.py)4
-rw-r--r--requests_cache/backends/__init__.py4
-rw-r--r--requests_cache/backends/base.py4
-rw-r--r--requests_cache/backends/filesystem.py2
-rw-r--r--requests_cache/backends/sqlite.py2
-rw-r--r--requests_cache/serializers/__init__.py2
37 files changed, 1268 insertions, 1257 deletions
diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md
index 92206cb..7ba48ba 100644
--- a/.github/CODE_OF_CONDUCT.md
+++ b/.github/CODE_OF_CONDUCT.md
@@ -1,4 +1,7 @@
-# Contributor Covenant Code of Conduct
+# Code of Conduct
+This Code of Conduct is adapted from
+[Contributor Covenant, version 1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct.html).
+See [FAQ](https://www.contributor-covenant.org/faq) for answers to common questions.
## Our Pledge
@@ -66,11 +69,3 @@ faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
-
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
-available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
-
-[homepage]: https://www.contributor-covenant.org
-
-For answers to common questions about this code of conduct, see
-https://www.contributor-covenant.org/faq
diff --git a/docs/advanced_usage.md b/docs/advanced_usage.md
index ea56184..95bb011 100644
--- a/docs/advanced_usage.md
+++ b/docs/advanced_usage.md
@@ -1,480 +1,3 @@
-(advanced-usage)=
+<!-- Placeholder to avoid broken links -->
# Advanced Usage
-This section covers some more advanced and use-case-specific features.
-
-## Cache Inspection
-Here are some ways to get additional information out of the cache session, backend, and responses:
-
-### Response Details
-The following attributes are available on responses:
-- `from_cache`: indicates if the response came from the cache
-- `created_at`: {py:class}`~datetime.datetime` of when the cached response was created or last updated
-- `expires`: {py:class}`~datetime.datetime` after which the cached response will expire
-- `is_expired`: indicates if the cached response is expired (if an old response was returned due to a request error)
-
-Examples:
-:::{admonition} Example code
-:class: toggle
-```python
->>> from requests_cache import CachedSession
->>> session = CachedSession(expire_after=timedelta(days=1))
-
->>> # Placeholders are added for non-cached responses
->>> response = session.get('http://httpbin.org/get')
->>> print(response.from_cache, response.created_at, response.expires, response.is_expired)
-False None None None
-
->>> # Values will be populated for cached responses
->>> response = session.get('http://httpbin.org/get')
->>> print(response.from_cache, response.created_at, response.expires, response.is_expired)
-True 2021-01-01 18:00:00 2021-01-02 18:00:00 False
-
->>> # Print a response object to get general information about it
->>> print(response)
-'request: GET https://httpbin.org/get, response: 200 (308 bytes), created: 2021-01-01 22:45:00 IST, expires: 2021-01-02 18:45:00 IST (fresh)'
-```
-:::
-
-### Cache Contents
-You can use `CachedSession.cache.urls` to see all URLs currently in the cache:
-```python
->>> session = CachedSession()
->>> print(session.cache.urls)
-['https://httpbin.org/get', 'https://httpbin.org/stream/100']
-```
-
-If needed, you can get more details on cached responses via `CachedSession.cache.responses`, which
-is a dict-like interface to the cache backend. See {py:class}`.CachedResponse` for a full list of
-attributes available.
-
-For example, if you wanted to to see all URLs requested with a specific method:
-```python
->>> post_urls = [
-... response.url for response in session.cache.responses.values()
-... if response.request.method == 'POST'
-... ]
-```
-
-You can also inspect `CachedSession.cache.redirects`, which maps redirect URLs to keys of the
-responses they redirect to.
-
-Additional `keys()` and `values()` wrapper methods are available on {py:class}`.BaseCache` to get
-combined keys and responses.
-```python
->>> print('All responses:')
->>> for response in session.cache.values():
->>> print(response)
-
->>> print('All cache keys for redirects and responses combined:')
->>> print(list(session.cache.keys()))
-```
-
-Both methods also take a `check_expiry` argument to exclude expired responses:
-```python
->>> print('All unexpired responses:')
->>> for response in session.cache.values(check_expiry=True):
->>> print(response)
-```
-
-Similarly, you can get a count of responses with {py:meth}`.BaseCache.response_count`, and optionally
-exclude expired responses:
-```python
->>> print(f'Total responses: {session.cache.response_count()}')
->>> print(f'Unexpired responses: {session.cache.response_count(check_expiry=True)}')
-```
-
-## Custom Cache Filtering
-If you need more advanced behavior for choosing what to cache, you can provide a custom filtering
-function via the `filter_fn` param. This can by any function that takes a
-{py:class}`requests.Response` object and returns a boolean indicating whether or not that response
-should be cached. It will be applied to both new responses (on write) and previously cached
-responses (on read):
-
-:::{admonition} Example code
-:class: toggle
-```python
->>> from sys import getsizeof
->>> from requests_cache import CachedSession
-
->>> def filter_by_size(response: Response) -> bool:
->>> """Don't cache responses with a body over 1 MB"""
->>> return getsizeof(response.content) <= 1024 * 1024
-
->>> session = CachedSession(filter_fn=filter_by_size)
-```
-:::
-
-```{note}
-`filter_fn()` will be used **in addition to** other {ref:`user_guide:cache filtering`} options.
-```
-
-## Custom Request Matching
-Request matching is accomplished using a **cache key**, which uniquely identifies a response in the
-cache based on request info. For example, the option `ignored_parameters=['foo']` works by excluding
-the `foo` request parameter from the cache key, meaning these three requests will all use the same
-cached response:
-```python
->>> session = CachedSession(ignored_parameters=['foo'])
->>> response_1 = session.get('https://example.com') # cache miss
->>> response_2 = session.get('https://example.com?foo=bar') # cache hit
->>> response_3 = session.get('https://example.com?foo=qux') # cache hit
->>> assert response_1.cache_key == response_2.cache_key == response_3.cache_key
-```
-
-If you want to implement your own request matching, you can provide a cache key function which will
-take a {py:class}`~requests.PreparedRequest` plus optional keyword args, and return a string:
-```python
-def create_key(request: requests.PreparedRequest, **kwargs) -> str:
- """Generate a custom cache key for the given request"""
-```
-
-`**kwargs` includes relevant {py:class}`.BaseCache` settings and any other keyword args passed to
-{py:meth}`.CachedSession.send()`. See {py:func}`.create_key` for the reference implementation, and
-see the rest of the {py:mod}`.cache_keys` module for some potentially useful helper functions.
-
-You can then pass this function via the `key_fn` param:
-```python
-session = CachedSession(key_fn=create_key)
-```
-
-```{note}
-`key_fn()` will be used **instead of** any other {ref}`user_guide:request matching` options and
-default matching behavior.
-```
-```{tip}
-See {ref}`Examples<custom_keys>` page for a complete example for custom request matching.
-```
-```{tip}
-As a general rule, if you include less info in your cache keys, you will have more cache hits and
-use less storage space, but risk getting incorrect response data back. For example, if you exclude
-all request parameters, you will get the same cached response back for any combination of request
-parameters.
-```
-```{warning}
-If you provide a custom key function for a non-empty cache, any responses previously cached with a
-different key function will likely be unused.
-```
-
-## Custom Backends
-If the built-in {py:mod}`Cache Backends <requests_cache.backends>` don't suit your needs, you can
-create your own by making subclasses of {py:class}`.BaseCache` and {py:class}`.BaseStorage`:
-
-:::{admonition} Example code
-:class: toggle
-```python
->>> from requests_cache import CachedSession
->>> from requests_cache.backends import BaseCache, BaseStorage
-
->>> class CustomCache(BaseCache):
-... """Wrapper for higher-level cache operations. In most cases, the only thing you need
-... to specify here is which storage class(es) to use.
-... """
-... def __init__(self, **kwargs):
-... super().__init__(**kwargs)
-... self.redirects = CustomStorage(**kwargs)
-... self.responses = CustomStorage(**kwargs)
-
->>> class CustomStorage(BaseStorage):
-... """Dict-like interface for lower-level backend storage operations"""
-... def __init__(self, **kwargs):
-... super().__init__(**kwargs)
-...
-... def __getitem__(self, key):
-... pass
-...
-... def __setitem__(self, key, value):
-... pass
-...
-... def __delitem__(self, key):
-... pass
-...
-... def __iter__(self):
-... pass
-...
-... def __len__(self):
-... pass
-...
-... def clear(self):
-... pass
-```
-:::
-
-You can then use your custom backend in a {py:class}`.CachedSession` with the `backend` parameter:
-```python
->>> session = CachedSession(backend=CustomCache())
-```
-
-## Custom Serializers
-If the built-in {ref}`serializers` don't suit your needs, you can create your own. For example, if
-you had a imaginary `custom_pickle` module that provides `dumps` and `loads` functions:
-```python
->>> import custom_pickle
->>> from requests_cache import CachedSession
->>> session = CachedSession(serializer=custom_pickle)
-```
-
-### Serializer Pipelines
-More complex serialization can be done with {py:class}`.SerializerPipeline`. Use cases include
-text-based serialization, compression, encryption, and any other intermediate steps you might want
-to add.
-
-Any combination of these can be composed with a {py:class}`.SerializerPipeline`, which starts with a
-{py:class}`.CachedResponse` and ends with a {py:class}`.str` or {py:class}`.bytes` object. Each stage
-of the pipeline can be any object or module with `dumps` and `loads` functions. If the object has
-similar methods with different names (e.g. `compress` / `decompress`), those can be aliased using
-{py:class}`.Stage`.
-
-For example, a compressed pickle serializer can be built as:
-:::{admonition} Example code
-:class: toggle
-```python
->>> import pickle, gzip
->>> from requests_cache.serialzers import SerializerPipeline, Stage
->>> compressed_serializer = SerializerPipeline([
-... pickle,
-... Stage(gzip, dumps='compress', loads='decompress'),
-...])
->>> session = CachedSession(serializer=compressed_serializer)
-```
-:::
-
-### Text-based Serializers
-If you're using a text-based serialization format like JSON or YAML, some extra steps are needed to
-encode binary data and non-builtin types. The [cattrs](https://cattrs.readthedocs.io) library can do
-the majority of the work here, and some pre-configured converters are included for serveral common
-formats in the {py:mod}`.preconf` module.
-
-For example, a compressed JSON pipeline could be built as follows:
-:::{admonition} Example code
-:class: toggle
-```python
->>> import json, gzip, codecs
->>> from requests_cache.serializers import SerializerPipeline, Stage, json_converter
->>> comp_json_serializer = SerializerPipeline([
-... json_converter, # Serialize to a JSON string
-... Stage(codecs.utf_8, dumps='encode', loads='decode'), # Encode to bytes
-... Stage(gzip, dumps='compress', loads='decompress'), # Compress
-...])
-```
-:::
-
-```{note}
-If you want to use a different format that isn't included in {py:mod}`.preconf`, you can use
-{py:class}`.CattrStage` as a starting point.
-```
-
-```{note}
-If you want to convert a string representation to bytes (e.g. for compression), you must use a codec
-from {py:mod}`.codecs` (typically `codecs.utf_8`)
-```
-
-### Additional Serialization Steps
-Some other tools that could be used as a stage in a {py:class}`.SerializerPipeline` include:
-
-Class | loads | dumps
------ | ----- | -----
-{py:mod}`codecs.* <.codecs>` | encode | decode
-{py:mod}`.bz2` | compress | decompress
-{py:mod}`.gzip` | compress | decompress
-{py:mod}`.lzma` | compress | decompress
-{py:mod}`.zlib` | compress | decompress
-{py:mod}`.pickle` | dumps | loads
-{py:class}`itsdangerous.signer.Signer` | sign | unsign
-{py:class}`itsdangerous.serializer.Serializer` | loads | dumps
-{py:class}`cryptography.fernet.Fernet` | encrypt | decrypt
-
-## Usage with other requests features
-
-### Request Hooks
-Requests has an [Event Hook](https://requests.readthedocs.io/en/master/user/advanced/#event-hooks)
-system that can be used to add custom behavior into different parts of the request process.
-It can be used, for example, for request throttling:
-
-:::{admonition} Example code
-:class: toggle
-```python
->>> import time
->>> import requests
->>> from requests_cache import CachedSession
->>>
->>> def make_throttle_hook(timeout=1.0):
->>> """Make a request hook function that adds a custom delay for non-cached requests"""
->>> def hook(response, *args, **kwargs):
->>> if not getattr(response, 'from_cache', False):
->>> print('sleeping')
->>> time.sleep(timeout)
->>> return response
->>> return hook
->>>
->>> session = CachedSession()
->>> session.hooks['response'].append(make_throttle_hook(0.1))
->>> # The first (real) request will have an added delay
->>> session.get('http://httpbin.org/get')
->>> session.get('http://httpbin.org/get')
-```
-:::
-
-### Streaming Requests
-If you use [streaming requests](https://2.python-requests.org/en/master/user/advanced/#id9), you
-can use the same code to iterate over both cached and non-cached requests. Cached response content
-will have already been read (i.e., consumed), but will be available for re-reading so it behaves like
-the original streamed response:
-
-:::{admonition} Example code
-:class: toggle
-```python
->>> from requests_cache import CachedSession
->>>
->>> session = CachedSession()
->>> for i in range(2):
-... response = session.get('https://httpbin.org/stream/20', stream=True)
-... for chunk in response.iter_lines():
-... print(chunk.decode('utf-8'))
-```
-:::
-
-(library-compatibility)=
-## Usage with other requests-based libraries
-This library works by patching and/or extending {py:class}`requests.Session`. Many other libraries
-out there do the same thing, making it potentially difficult to combine them.
-
-For that scenario, a mixin class is provided, so you can create a custom class with behavior from
-multiple Session-modifying libraries:
-```python
->>> from requests import Session
->>> from requests_cache import CacheMixin
->>> from some_other_lib import SomeOtherMixin
->>>
->>> class CustomSession(CacheMixin, SomeOtherMixin, Session):
-... """Session class with features from both some_other_lib and requests-cache"""
-```
-
-### Requests-HTML
-[requests-html](https://github.com/psf/requests-html) is one library that works with this method:
-:::{admonition} Example code
-:class: toggle
-```python
->>> import requests
->>> from requests_cache import CacheMixin, install_cache
->>> from requests_html import HTMLSession
->>>
->>> class CachedHTMLSession(CacheMixin, HTMLSession):
-... """Session with features from both CachedSession and HTMLSession"""
->>>
->>> session = CachedHTMLSession()
->>> response = session.get('https://github.com/')
->>> print(response.from_cache, response.html.links)
-```
-:::
-
-
-Or if you are using {py:func}`.install_cache`, you can use the `session_factory` argument:
-:::{admonition} Example code
-:class: toggle
-```python
->>> install_cache(session_factory=CachedHTMLSession)
->>> response = requests.get('https://github.com/')
->>> print(response.from_cache, response.html.links)
-```
-:::
-
-The same approach can be used with other libraries that subclass {py:class}`requests.Session`.
-
-### Requests-futures
-Some libraries, including [requests-futures](https://github.com/ross/requests-futures),
-support wrapping an existing session object:
-```python
->>> session = FutureSession(session=CachedSession())
-```
-
-In this case, `FutureSession` must wrap `CachedSession` rather than the other way around, since
-`FutureSession` returns (as you might expect) futures rather than response objects.
-See [issue #135](https://github.com/reclosedev/requests-cache/issues/135) for more notes on this.
-
-### Internet Archive
-Usage with [internetarchive](https://github.com/jjjake/internetarchive) is the same as other libraries
-that subclass `requests.Session`:
-:::{admonition} Example code
-:class: toggle
-```python
->>> from requests_cache import CacheMixin
->>> from internetarchive.session import ArchiveSession
->>>
->>> class CachedArchiveSession(CacheMixin, ArchiveSession):
-... """Session with features from both CachedSession and ArchiveSession"""
-```
-:::
-
-### Requests-mock
-[requests-mock](https://github.com/jamielennox/requests-mock) has multiple methods for mocking
-requests, including a contextmanager, decorator, fixture, and adapter. There are a few different
-options for using it with requests-cache, depending on how you want your tests to work.
-
-#### Disabling requests-cache
-If you have an application that uses requests-cache and you just want to use requests-mock in
-your tests, the easiest thing to do is to disable requests-cache.
-
-For example, if you are using {py:func}`.install_cache` in your application and the
-requests-mock [pytest fixture](https://requests-mock.readthedocs.io/en/latest/pytest.html) in your
-tests, you could wrap it in another fixture that uses {py:func}`.uninstall_cache` or {py:func}`.disabled`:
-:::{admonition} Example code
-:class: toggle
-```{literalinclude} ../tests/compat/test_requests_mock_disable_cache.py
-```
-:::
-
-Or if you use a `CachedSession` object, you could replace it with a regular `Session`, for example:
-:::{admonition} Example code
-:class: toggle
-```python
-import unittest
-import pytest
-import requests
-
-
-@pytest.fixure(scope='function', autouse=True)
-def disable_requests_cache():
- """Replace CachedSession with a regular Session for all test functions"""
- with unittest.mock.patch('requests_cache.CachedSession', requests.Session):
- yield
-```
-:::
-
-#### Combining requests-cache with requests-mock
-If you want both caching and mocking features at the same time, you can attach requests-mock's
-[adapter](https://requests-mock.readthedocs.io/en/latest/adapter.html) to a `CachedSession`:
-
-:::{admonition} Example code
-:class: toggle
-```{literalinclude} ../tests/compat/test_requests_mock_combine_cache.py
-```
-:::
-
-#### Building a mocker using requests-cache data
-Another approach is to use cached data to dynamically define mock requests + responses.
-This has the advantage of only using request-mock's behavior for
-[request matching](https://requests-mock.readthedocs.io/en/latest/matching.html).
-
-:::{admonition} Example code
-:class: toggle
-```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py
-:lines: 21-40
-```
-:::
-
-To turn that into a complete example:
-:::{admonition} Example code
-:class: toggle
-```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py
-```
-:::
-
-### Responses
-Usage with the [responses](https://github.com/getsentry/responses) library is similar to the
-requests-mock examples above.
-
-:::{admonition} Example code
-:class: toggle
-```{literalinclude} ../tests/compat/test_responses_load_cache.py
-```
-:::
+The contents of this section have been moved to the {ref}`user-guide`.
diff --git a/docs/contributing.md b/docs/contributing.md
deleted file mode 100644
index 88f29b2..0000000
--- a/docs/contributing.md
+++ /dev/null
@@ -1,3 +0,0 @@
-(contributing)=
-```{include} ../CONTRIBUTING.md
-```
diff --git a/docs/contributors.md b/docs/contributors.md
deleted file mode 100644
index c8e001d..0000000
--- a/docs/contributors.md
+++ /dev/null
@@ -1,2 +0,0 @@
-```{include} ../CONTRIBUTORS.md
-```
diff --git a/docs/examples.md b/docs/examples.md
index c59c662..5f9c737 100644
--- a/docs/examples.md
+++ b/docs/examples.md
@@ -1,3 +1,4 @@
+(examples)=
# Examples
This section contains some complete examples that demonstrate the main features of requests-cache.
@@ -93,15 +94,15 @@ The following scripts can also be found in the
:::
(custom_keys)=
-### Custom cache key function
-```{include} ../examples/custom_cache_keys.py
+### Custom request matcher
+```{include} ../examples/custom_request_matcher.py
:start-line: 2
:end-line: 15
```
:::{admonition} Example code
:class: toggle
-```{literalinclude} ../examples/custom_cache_keys.py
+```{literalinclude} ../examples/custom_request_matcher.py
:lines: 1,17-
```
:::
diff --git a/docs/history.md b/docs/history.md
deleted file mode 100644
index 856698b..0000000
--- a/docs/history.md
+++ /dev/null
@@ -1,3 +0,0 @@
-(changelog)=
-```{include} ../HISTORY.md
-```
diff --git a/docs/index.md b/docs/index.md
index 7dd484f..df9f4bd 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,7 +1,7 @@
<!--
Pre-release warning to reduce confusion on what '/latest' means;
-TODO: remove prior to next minor release
+TODO: remove prior to next minor release, or add script to make this conditional
-->
```{admonition} Note
:class: warning
@@ -25,13 +25,8 @@ Documentation for the latest stable release can be found at [requests-cache.read
```{toctree}
:maxdepth: 2
-user_guide
-advanced_usage
+user_guide/index
examples
-security
reference
-contributing
-contributors
-related_projects
-history
+project_info/index
````
diff --git a/docs/project_info/code_of_conduct.md b/docs/project_info/code_of_conduct.md
new file mode 100644
index 0000000..36beb23
--- /dev/null
+++ b/docs/project_info/code_of_conduct.md
@@ -0,0 +1,2 @@
+```{include} ../../.github/CODE_OF_CONDUCT.md
+```
diff --git a/docs/project_info/contributing.md b/docs/project_info/contributing.md
new file mode 100644
index 0000000..01f6808
--- /dev/null
+++ b/docs/project_info/contributing.md
@@ -0,0 +1,3 @@
+(contributing)=
+```{include} ../../CONTRIBUTING.md
+```
diff --git a/docs/project_info/contributors.md b/docs/project_info/contributors.md
new file mode 100644
index 0000000..f0c9bb7
--- /dev/null
+++ b/docs/project_info/contributors.md
@@ -0,0 +1,2 @@
+```{include} ../../CONTRIBUTORS.md
+```
diff --git a/docs/project_info/history.md b/docs/project_info/history.md
new file mode 100644
index 0000000..ba5ca18
--- /dev/null
+++ b/docs/project_info/history.md
@@ -0,0 +1,3 @@
+(changelog)=
+```{include} ../../HISTORY.md
+```
diff --git a/docs/project_info/index.md b/docs/project_info/index.md
new file mode 100644
index 0000000..c808ab6
--- /dev/null
+++ b/docs/project_info/index.md
@@ -0,0 +1,11 @@
+# Project Info
+
+```{toctree}
+:maxdepth: 2
+
+contributing
+contributors
+code_of_conduct
+related_projects
+history
+````
diff --git a/docs/related_projects.md b/docs/project_info/related_projects.md
index a9ba653..e9abcdc 100644
--- a/docs/related_projects.md
+++ b/docs/project_info/related_projects.md
@@ -1,3 +1,4 @@
+(related-projects)=
# Related Projects
If requests-cache isn't quite what you need, you can help make it better! See the
{ref}`Contributing Guide <contributing>` for details.
@@ -30,4 +31,4 @@ You can also check out these other python projects related to caching and/or HTT
for tests; inspired by Ruby's [VCR](https://github.com/vcr/vcr)]. Works at the `httplib` level and
is compatible with multiple HTTP libraries.
* [betamax](https://github.com/betamaxpy/betamax): Records responses to local files and plays them back
- for tests; also inspired by Ruby's [VCR](https://github.com/vcr/vcr)]. Made specifically for `requests`.
+ for tests; also inspired by Ruby's [VCR](https://github.com/vcr/vcr). Made specifically for `requests`.
diff --git a/docs/sample_response.json b/docs/sample_data/sample_response.json
index 4b60354..4b60354 100644
--- a/docs/sample_response.json
+++ b/docs/sample_data/sample_response.json
diff --git a/docs/sample_response.yaml b/docs/sample_data/sample_response.yaml
index 07bc7ca..07bc7ca 100644
--- a/docs/sample_response.yaml
+++ b/docs/sample_data/sample_response.yaml
diff --git a/docs/user_guide.md b/docs/user_guide.md
deleted file mode 100644
index d490133..0000000
--- a/docs/user_guide.md
+++ /dev/null
@@ -1,737 +0,0 @@
-(user-guide)=
-# User Guide
-This section covers the main features of requests-cache.
-
-## Installation
-Installation instructions:
-
-:::{tab} Pip
-Install the latest stable version from [PyPI](https://pypi.org/project/requests-cache/):
-```
-pip install requests-cache
-```
-:::
-:::{tab} Conda
-Or install from [conda-forge](https://anaconda.org/conda-forge/requests-cache), if you prefer:
-```
-conda install -c conda-forge requests-cache
-```
-:::
-:::{tab} Pre-release
-If you would like to use the latest development (pre-release) version:
-```
-pip install --pre requests-cache
-```
-:::
-:::{tab} Local development
-See {ref}`Contributing Guide <contributing:dev installation>` for setup steps for local development
-:::
-
-### Requirements
-The latest version of requests-cache requires **python 3.7+**. If you need to use an older version
-of python, here are the latest compatible versions and their documentation pages:
-
-:::{admonition} Python version compatibility
-:class: toggle, tip
-* **python 2.6:** [requests-cache 0.4.13](https://requests-cache.readthedocs.io/en/v0.4.13)
-* **python 2.7:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
-* **python 3.4:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
-* **python 3.5:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
-* **python 3.6:** [requests-cache 0.7.4](https://requests-cache.readthedocs.io/en/v0.7.4)
-:::
-
-You may need additional dependencies depending on which backend you want to use. To install with
-extra dependencies for all supported {ref}`user_guide:cache backends`:
-```
-pip install requests-cache[all]
-```
-
-## General Usage
-There are two main ways of using requests-cache:
-- **Sessions:** (recommended) Use {py:class}`.CachedSession` to send your requests
-- **Patching:** Globally patch `requests` using {py:func}`.install_cache()`
-
-### Sessions
-{py:class}`.CachedSession` can be used as a drop-in replacement for {py:class}`requests.Session`.
-Basic usage looks like this:
-```python
->>> from requests_cache import CachedSession
->>>
->>> session = CachedSession()
->>> session.get('http://httpbin.org/get')
-```
-
-Any {py:class}`requests.Session` method can be used (but see
-{ref}`user_guide:cached http methods` section for options):
-```python
->>> session.request('GET', 'http://httpbin.org/get')
->>> session.head('http://httpbin.org/get')
-```
-
-Caching can be temporarily disabled for the session with
-{py:meth}`.CachedSession.cache_disabled`:
-```python
->>> with session.cache_disabled():
-... session.get('http://httpbin.org/get')
-```
-
-The best way to clean up your cache is through {ref}`user_guide:cache expiration`, but you can also
-clear out everything at once with {py:meth}`.BaseCache.clear`:
-```python
->>> session.cache.clear()
-```
-
-### Patching
-In some situations, it may not be possible or convenient to manage your own session object. In those
-cases, you can use {py:func}`.install_cache` to add caching to all `requests` functions:
-```python
->>> import requests
->>> import requests_cache
->>>
->>> requests_cache.install_cache()
->>> requests.get('http://httpbin.org/get')
-```
-
-As well as session methods:
-```python
->>> session = requests.Session()
->>> session.get('http://httpbin.org/get')
-```
-
-{py:func}`.install_cache` accepts all the same parameters as {py:class}`.CachedSession`:
-```python
->>> requests_cache.install_cache(expire_after=360, allowable_methods=('GET', 'POST'))
-```
-
-It can be temporarily {py:func}`.enabled`:
-```python
->>> with requests_cache.enabled():
-... requests.get('http://httpbin.org/get') # Will be cached
-```
-
-Or temporarily {py:func}`.disabled`:
-```python
->>> requests_cache.install_cache()
->>> with requests_cache.disabled():
-... requests.get('http://httpbin.org/get') # Will not be cached
-```
-
-Or completely removed with {py:func}`.uninstall_cache`:
-```python
->>> requests_cache.uninstall_cache()
->>> requests.get('http://httpbin.org/get')
-```
-
-You can also clear out all responses in the cache with {py:func}`.clear`, and check if
-requests-cache is currently installed with {py:func}`.is_installed`.
-
-(monkeypatch-issues)=
-#### Patching Limitations & Potential Issues
-Like any other utility that uses monkey-patching, there are some scenarios where you won't want to
-use {py:func}`.install_cache`:
-- When using other libraries that patch {py:class}`requests.Session`
-- In a multi-threaded or multiprocess application
-- In a library that will be imported by other libraries or applications
-- In a larger application that makes requests in several different modules, where it may not be
- obvious what is and isn't being cached
-
-In any of these cases, consider using {py:class}`.CachedSession`, the {py:func}`.enabled`
-contextmanager, or {ref}`selective-caching`.
-
-(backends)=
-## Cache Backends
-![](_static/sqlite_32px.png)
-![](_static/redis_32px.png)
-![](_static/mongodb_32px.png)
-![](_static/dynamodb_32px.png)
-![](_static/files-json_32px.png)
-
-Several cache backends are included. The default is SQLite, since it's generally the simplest to
-use, and requires no extra dependencies or configuration.
-```{note}
-In the rare case that SQLite is not available
-(for example, [on Heroku](https://devcenter.heroku.com/articles/sqlite3)), a non-persistent
-in-memory cache is used by default.
-```
-
-See {py:mod}`.requests_cache.backends` for usage details for specific backends.
-### Backend Dependencies
-Most of the other backends require some extra dependencies, listed below.
-
-Backend | Class | Alias | Dependencies
--------------------------------------------------------|----------------------------|----------------|-------------
-[SQLite](https://www.sqlite.org) | {py:class}`.SQLiteCache` | `'sqlite'` |
-[Redis](https://redis.io) | {py:class}`.RedisCache` | `'redis'` | [redis-py](https://github.com/andymccurdy/redis-py)
-[MongoDB](https://www.mongodb.com) | {py:class}`.MongoCache` | `'mongodb'` | [pymongo](https://github.com/mongodb/mongo-python-driver)
-[GridFS](https://docs.mongodb.com/manual/core/gridfs/) | {py:class}`.GridFSCache` | `'gridfs'` | [pymongo](https://github.com/mongodb/mongo-python-driver)
-[DynamoDB](https://aws.amazon.com/dynamodb) | {py:class}`.DynamoCache` | `'dynamodb'` | [boto3](https://github.com/boto/boto3)
-Filesystem | {py:class}`.FileCache` | `'filesystem'` |
-Memory | {py:class}`.BaseCache` | `'memory'` |
-
-### Specifying a Backend
-You can specify which backend to use with the `backend` parameter for either {py:class}`.CachedSession`
-or {py:func}`.install_cache`. You can specify one by name, using the aliases listed above:
-```python
->>> session = CachedSession('my_cache', backend='redis')
-```
-
-Or by instance:
-```python
->>> backend = RedisCache(host='192.168.1.63', port=6379)
->>> session = CachedSession('my_cache', backend=backend)
-```
-
-### Backend Options
-The `cache_name` parameter has a different use depending on the backend:
-
-Backend | Cache name used as
-----------------|-------------------
-SQLite | Database path
-Redis | Hash namespace
-MongoDB, GridFS | Database name
-DynamoDB | Table name
-Filesystem | Cache directory
-
-Each backend class also accepts optional parameters for the underlying connection. For example,
-{py:class}`.SQLiteCache` accepts parameters for {py:func}`sqlite3.connect`:
-```python
->>> session = CachedSession('my_cache', backend='sqlite', timeout=30)
-```
-
-### Testing Backends
-If you just want to quickly try out all of the available backends for comparison,
-[docker-compose](https://docs.docker.com/compose/) config is included for all supported services.
-First, [install docker](https://docs.docker.com/get-docker/) if you haven't already. Then, run:
-
-:::{tab} Bash (Linux/macOS)
-```bash
-pip install -U requests-cache[all] docker-compose
-curl https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -O docker-compose.yml
-docker-compose up -d
-```
-:::
-:::{tab} Powershell (Windows)
-```ps1
-pip install -U requests-cache[all] docker-compose
-Invoke-WebRequest -Uri https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -Outfile docker-compose.yml
-docker-compose up -d
-```
-:::
-
-(exporting)=
-### Exporting To A Different Backend
-If you have cached data that you want to copy or migrate to a different backend, you can do this
-with `CachedSession.cache.update()`. For example, if you want to dump the contents of a Redis cache
-to JSON files:
-```python
->>> src_session = CachedSession('my_cache', backend='redis')
->>> dest_session = CachedSession('~/workspace/cache_dump', backend='filesystem', serializer='json')
->>> dest_session.cache.update(src_session.cache)
-
->>> # List the exported files
->>> print(dest_session.cache.paths())
-'/home/user/workspace/cache_dump/9e7a71a3ff2e.json'
-'/home/user/workspace/cache_dump/8a922ff3c53f.json'
-```
-
-Or, using backend classes directly:
-```python
->>> src_cache = RedisCache()
->>> dest_cache = FileCache('~/workspace/cache_dump', serializer='json')
->>> dest_cache.update(src_cache)
-```
-
-### Custom Backends
-See {ref}`advanced_usage:custom backends` for details on creating your own backend implementation.
-
-## Cache Files
-```{note}
-This section only applies to the {py:mod}`~requests_cache.backends.sqlite` and
-{py:mod}`~requests_cache.backends.filesystem` backends.
-```
-For file-based backends, the cache name will be used as a path to the cache file(s). You can use
-a relative path, absolute path, or use some additional options for system-specific default paths.
-
-### Relative Paths
-```python
->>> # Database path for SQLite cache
->>> session = CachedSession('http_cache', backend='sqlite')
->>> print(session.cache.db_path)
-'<current working dir>/http_cache.sqlite'
-```
-```python
->>> # Base directory for Filesystem cache
->>> session = CachedSession('http_cache', backend='filesystem')
->>> print(session.cache.cache_dir)
-'<current working dir>/http_cache/'
-```
-
-```{note}
-Parent directories will always be created, if they don't already exist.
-```
-
-### Absolute Paths
-You can also give an absolute path, including user paths (with `~`).
-```python
->>> session = CachedSession('~/.myapp/http_cache', backend='sqlite')
->>> print(session.cache.db_path)
-'/home/user/.myapp/http_cache.sqlite'
-```
-
-### System Paths
-If you don't know exactly where you want to put your cache files, your system's **temp directory**
-or **cache directory** is a good choice. Some options are available as shortcuts to use whatever the
-default locations are for your operating system.
-
-Use the default temp directory with the `use_temp` option:
-:::{tab} Linux
-```python
->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
->>> print(session.cache.db_path)
-'/tmp/http_cache.sqlite'
-```
-:::
-:::{tab} macOS
-```python
->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
->>> print(session.cache.db_path)
-'/var/folders/xx/http_cache.sqlite'
-```
-:::
-:::{tab} Windows
-```python
->>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
->>> print(session.cache.db_path)
-'C:\\Users\\user\\AppData\\Local\\temp\\http_cache.sqlite'
-```
-:::
-
-Or use the default cache directory with the `use_cache_dir` option:
-:::{tab} Linux
-```python
->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
->>> print(session.cache.cache_dir)
-'/home/user/.cache/http_cache/'
-```
-:::
-:::{tab} macOS
-```python
->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
->>> print(session.cache.cache_dir)
-'/Users/user/Library/Caches/http_cache/'
-```
-:::
-:::{tab} Windows
-```python
->>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
->>> print(session.cache.cache_dir)
-'C:\\Users\\user\\AppData\\Local\\http_cache\\'
-```
-:::
-
-```{note}
-If the cache name is an absolute path, the `use_temp` and `use_cache_dir` options will be ignored.
-If it's a relative path, it will be relative to the temp or cache directory, respectively.
-```
-
-There are a number of other system default locations that might be appropriate for a cache file. See
-the [appdirs](https://github.com/ActiveState/appdirs) library for an easy cross-platform way to get
-the most commonly used ones.
-
-## Cache Filtering
-In many cases you will want to choose what you want to cache instead of just caching everything. By
-default, all **read-only** (`GET` and `HEAD`) **requests with a 200 response code** are cached. A
-few options are available to modify this behavior.
-
-```{note}
-When using {py:class}`.CachedSession`, any requests that you don't want to cache can also be made
-with a regular {py:class}`requests.Session` object, or wrapper functions like
-{py:func}`requests.get`, etc.
-```
-
-### Cached HTTP Methods
-To cache additional HTTP methods, specify them with `allowable_methods`:
-```python
->>> session = CachedSession(allowable_methods=('GET', 'POST'))
->>> session.post('http://httpbin.org/post', json={'param': 'value'})
-```
-
-For example, some APIs use the `POST` method to request data via a JSON-formatted request body, for
-requests that may exceed the max size of a `GET` request. You may also want to cache `POST` requests
-to ensure you don't send the exact same data multiple times.
-
-### Cached Status Codes
-To cache additional status codes, specify them with `allowable_codes`
-```python
->>> session = CachedSession(allowable_codes=(200, 418))
->>> session.get('http://httpbin.org/teapot')
-```
-
-(selective-caching)=
-### Cached URLs
-You can use {ref}`URL patterns <url-patterns>` to define an allowlist for selective caching, by
-using a expiration value of `0` (or `requests_cache.DO_NOT_CACHE`, to be more explicit) for
-non-matching request URLs:
-```python
->>> from requests_cache import DO_NOT_CACHE, CachedSession
->>> urls_expire_after = {
-... '*.site_1.com': 30,
-... 'site_2.com/static': -1,
-... '*': DO_NOT_CACHE,
-... }
->>> session = CachedSession(urls_expire_after=urls_expire_after)
-```
-
-Note that the catch-all rule above (`'*'`) will behave the same as setting the session-level
-expiration to `0`:
-```python
->>> urls_expire_after = {'*.site_1.com': 30, 'site_2.com/static': -1}
->>> session = CachedSession(urls_expire_after=urls_expire_after, expire_after=0)
-```
-
-### Custom Cache Filtering
-If you would like more control over which requests get cached, see
-{ref}`advanced_usage:custom cache filtering`.
-
-## Request Matching
-Requests are matched according to the request URL, parameters and body. All of these values are
-normalized to account for any variations that do not modify response content.
-
-There are additional options to match according to request headers, ignore specific request
-parameters, or create your own custom request matcher.
-
-### Matching Request Headers
-In some cases, different headers may result in different response data, so you may want to cache
-them separately. To enable this, use `include_get_headers`:
-```python
->>> session = CachedSession(include_get_headers=True)
->>> # Both of these requests will be sent and cached separately
->>> session.get('http://httpbin.org/headers', {'Accept': 'text/plain'})
->>> session.get('http://httpbin.org/headers', {'Accept': 'application/json'})
-```
-
-### Selective Parameter Matching
-By default, all normalized request parameters are matched. In some cases, there may be request
-parameters that don't affect the response data, for example authentication tokens or credentials.
-If you want to ignore specific parameters, specify them with the `ignored_parameters` option.
-
-**Request Parameters:**
-
-In this example, only the first request will be sent, and the second request will be a cache hit
-due to the ignored parameters:
-```python
->>> session = CachedSession(ignored_parameters=['auth-token'])
->>> session.get('http://httpbin.org/get', params={'auth-token': '2F63E5DF4F44'})
->>> r = session.get('http://httpbin.org/get', params={'auth-token': 'D9FAEB3449D3'})
->>> assert r.from_cache is True
-```
-
-**Request Body Parameters:**
-
-This also applies to parameters in a JSON-formatted request body:
-```python
->>> session = CachedSession(allowable_methods=('GET', 'POST'), ignored_parameters=['auth-token'])
->>> session.post('http://httpbin.org/post', json={'auth-token': '2F63E5DF4F44'})
->>> r = session.post('http://httpbin.org/post', json={'auth-token': 'D9FAEB3449D3'})
->>> assert r.from_cache is True
-```
-
-**Request Headers:**
-
-As well as headers, if `include_get_headers` is also used:
-```python
->>> session = CachedSession(ignored_parameters=['auth-token'], include_get_headers=True)
->>> session.get('http://httpbin.org/get', headers={'auth-token': '2F63E5DF4F44'})
->>> r = session.get('http://httpbin.org/get', headers={'auth-token': 'D9FAEB3449D3'})
->>> assert r.from_cache is True
-```
-
-### Removing Sensitive Request Info
-`ignored_parameters` will also be removed from the cached response, including request parameters,
-body, and headers. This makes `ignored_parameters` a good way to prevent credentials or other
-sensitive info from being saved in the cache backend.
-
-### Custom Request Matching
-If you need more control over request matching behavior, see
-{ref}`advanced_usage:custom request matching`.
-
-## Cache Expiration
-By default, cached responses will be stored indefinitely. There are a number of options for
-specifying how long to store responses, either with a single expiration value, glob patterns,
-or {ref}`cache headers <headers>`.
-
-The simplest option is to initialize the cache with an `expire_after` value, which will apply to all
-reponses:
-```python
->>> # Set expiration for the session using a value in seconds
->>> session = CachedSession(expire_after=360)
-```
-
-### Expiration Precedence
-Expiration can be set on a per-session, per-URL, or per-request basis, in addition to cache
-headers (see sections below for usage details). When there are multiple values provided for a given
-request, the following order of precedence is used:
-1. Cache-Control request headers (if enabled)
-2. Cache-Control response headers (if enabled)
-3. Per-request expiration (`expire_after` argument for {py:meth}`.CachedSession.request`)
-4. Per-URL expiration (`urls_expire_after` argument for {py:class}`.CachedSession`)
-5. Per-session expiration (`expire_after` argument for {py:class}`.CacheBackend`)
-
-### Expiration Values
-`expire_after` can be any of the following:
-- `-1` (to never expire)
-- `0` (to "expire immediately," e.g. bypass the cache)
-- A positive number (in seconds)
-- A {py:class}`~datetime.timedelta`
-- A {py:class}`~datetime.datetime`
-
-Examples:
-```python
->>> # To specify a unit of time other than seconds, use a timedelta
->>> from datetime import timedelta
->>> session = CachedSession(expire_after=timedelta(days=30))
-
->>> # Update an existing session to disable expiration (i.e., store indefinitely)
->>> session.expire_after = -1
-
->>> # Disable caching by default, unless enabled by other settings
->>> session = CachedSession(expire_after=0)
-```
-
-(url-patterns)=
-### Expiration With URL Patterns
-You can use `urls_expire_after` to set different expiration values based on URL glob patterns.
-This allows you to customize caching based on what you know about the resources you're requesting
-or how you intend to use them. For example, you might request one resource that gets updated
-frequently, another that changes infrequently, and another that never changes. Example:
-```python
->>> urls_expire_after = {
-... '*.site_1.com': 30,
-... 'site_2.com/resource_1': 60 * 2,
-... 'site_2.com/resource_2': 60 * 60 * 24,
-... 'site_2.com/static': -1,
-... }
->>> session = CachedSession(urls_expire_after=urls_expire_after)
-```
-
-**Notes:**
-- `urls_expire_after` should be a dict in the format `{'pattern': expire_after}`
-- `expire_after` accepts the same types as `CachedSession.expire_after`
-- Patterns will match request **base URLs without the protocol**, so the pattern `site.com/resource/`
- is equivalent to `http*://site.com/resource/**`
-- If there is more than one match, the first match will be used in the order they are defined
-- If no patterns match a request, `CachedSession.expire_after` will be used as a default
-
-### Expiration and Error Handling
-In some cases, you might cache a response, have it expire, but then encounter an error when
-retrieving a new response. If you would like to use expired response data in these cases, use the
-`old_data_on_error` option.
-
-For example:
-```python
->>> # Cache a test response that will expire immediately
->>> session = CachedSession(old_data_on_error=True)
->>> session.get('https://httpbin.org/get', expire_after=0.0001)
->>> time.sleep(0.0001)
-```
-
-Afterward, let's say the page has moved and you get a 404, or the site is experiencing downtime and
-you get a 500. You will then get the expired cache data instead:
-```python
->>> response = session.get('https://httpbin.org/get')
->>> print(response.from_cache, response.is_expired)
-True, True
-```
-
-In addition to HTTP error codes, `old_data_on_error` also applies to python exceptions (typically a
-{py:exc}`~requests.RequestException`). See `requests` documentation on
-[Errors and Exceptions](https://2.python-requests.org/en/master/user/quickstart/#errors-and-exceptions)
-for more details on request errors in general.
-
-### Removing Expired Responses
-For better read performance, expired responses won't be removed immediately, but will be removed
-(or replaced) the next time they are requested.
-:::{tip}
-Implementing one or more cache eviction algorithms is being considered. If this is something you are
-interested in, please provide feedback via [issues](https://github.com/reclosedev/requests-cache/issues)!
-:::
-
-To manually clear all expired responses, use
-{py:meth}`.CachedSession.remove_expired_responses`:
-```python
->>> session.remove_expired_responses()
-```
-
-Or, when using patching:
-```python
->>> requests_cache.remove_expired_responses()
-```
-
-You can also apply a different `expire_after` to previously cached responses, which will
-revalidate the cache with the new expiration time:
-```python
->>> session.remove_expired_responses(expire_after=timedelta(days=30))
-```
-
-(headers)=
-## Cache Headers
-Most common request and response headers related to caching are supported, including
-[Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control)
-and [ETags](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag).
-
-```{note}
-requests-cache is not intended to be strict implementation of HTTP caching according to
-[RFC 2616](https://datatracker.ietf.org/doc/html/rfc2616),
-[RFC 7234](https://datatracker.ietf.org/doc/html/rfc7234), etc. These RFCs describe many behaviors
-that make sense in the context of a browser or proxy cache, but not for a python application.
-```
-
-### Conditional Requests
-[Conditional requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests) are
-automatically sent for any servers that support them. Once a cached response expires, it will only
-be updated if the remote content has changed.
-
-Here's an example using the [GitHub API](https://docs.github.com/en/rest) to get info about the
-requests-cache repo:
-```python
->>> # Cache a response that will expire immediately
->>> url = 'https://api.github.com/repos/reclosedev/requests-cache'
->>> session = CachedSession(expire_after=0.0001)
->>> session.get(url)
->>> time.sleep(0.0001)
-
->>> # The cached response will still be used until the remote content actually changes
->>> response = session.get(url)
->>> print(response.from_cache, response.is_expired)
-True, True
-```
-
-### Cache-Control
-If enabled, `Cache-Control` directives will take priority over any other `expire_after` value.
-See {ref}`user_guide:expiration precedence` for the full order of precedence.
-
-To enable this behavior, use the `cache_control` option:
-```python
->>> session = CachedSession(cache_control=True)
-```
-
-### Supported Headers
-The following headers are currently supported:
-
-**Request headers:**
-- `Cache-Control: max-age`: Used as the expiration time in seconds
-- `Cache-Control: no-cache`: Skips reading response data from the cache
-- `Cache-Control: no-store`: Skips reading and writing response data from/to the cache
-- `If-None-Match`: Automatically added if an `ETag` is available
-- `If-Modified-Since`: Automatically added if `Last-Modified` is available
-
-**Response headers:**
-- `Cache-Control: max-age`: Used as the expiration time in seconds
-- `Cache-Control: no-store` Skips writing response data to the cache
-- `Expires`: Used as an absolute expiration time
-- `ETag`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response)
-- `Last-Modified`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response)
-
-```{note}
-Unlike a browser or proxy cache, `max-age=0` does not clear previously cached responses.
-```
-
-(serializers)=
-## Serializers
-![](_static/file-pickle_32px.png)
-![](_static/file-json_32px.png)
-![](_static/file-yaml_32px.png)
-![](_static/file-toml_32px.png)
-
-By default, responses are serialized using {py:mod}`pickle`, but some alternative serializers are
-also included. These are mainly intended for use with {py:class}`.FileCache`, but are compatible
-with the other backends as well.
-
-:::{note}
-Some serializers require additional dependencies
-:::
-
-### Specifying a Serializer
-Similar to {ref}`backends`, you can specify which serializer to use with the `serializer` parameter
-for either {py:class}`.CachedSession` or {py:func}`.install_cache`.
-
-### JSON Serializer
-Storing responses as JSON gives you the benefit of making them human-readable and editable, in
-exchange for a minor reduction in read and write speeds.
-
-Usage:
-```python
->>> session = CachedSession('my_cache', serializer='json')
-```
-
-:::{admonition} Example JSON-serialized Response
-:class: toggle
-```{literalinclude} sample_response.json
-:language: JSON
-```
-:::
-
-This will use [ultrajson](https://github.com/ultrajson/ultrajson) if installed, otherwise the stdlib
-`json` module will be used. You can install the optional dependencies for this serializer with:
-```bash
-pip install requests-cache[json]
-```
-
-### YAML Serializer
-YAML is another option if you need a human-readable/editable format, with the same tradeoffs as JSON.
-
-Usage:
-```python
->>> session = CachedSession('my_cache', serializer='yaml')
-```
-
-:::{admonition} Example YAML-serialized Response
-:class: toggle
-```{literalinclude} sample_response.yaml
-:language: YAML
-```
-:::
-
-You can install the extra dependencies for this serializer with:
-```bash
-pip install requests-cache[yaml]
-```
-
-### BSON Serializer
-[BSON](https://www.mongodb.com/json-and-bson) is a serialization format originally created for
-MongoDB, but it can also be used independently. Compared to JSON, it has better performance
-(although still not as fast as `pickle`), and adds support for additional data types. It is not
-human-readable, but some tools support reading and editing it directly
-(for example, [bson-converter](https://atom.io/packages/bson-converter) for Atom).
-
-Usage:
-```python
->>> session = CachedSession('my_cache', serializer='bson')
-```
-
-You can install the extra dependencies for this serializer with:
-```bash
-pip install requests-cache[mongo]
-```
-
-Or if you would like to use the standalone BSON codec for a different backend, without installing
-MongoDB dependencies:
-```bash
-pip install requests-cache[bson]
-```
-
-### Serializer Security
-See {ref}`security` for recommended setup steps for more secure cache serialization, particularly
-when using {py:mod}`pickle`.
-
-### Custom Serializers
-See {ref}`advanced_usage:custom serializers` for other possible formats, and options for creating
-your own implementation.
-
-## Potential Issues
-- See {ref}`monkeypatch-issues` for issues specific to {py:func}`.install_cache`
-- New releases of `requests`, `urllib3` or `requests-cache` itself may change response data and be
- be incompatible with previously cached data (see issues
- [#56](https://github.com/reclosedev/requests-cache/issues/56) and
- [#102](https://github.com/reclosedev/requests-cache/issues/102)).
- In these cases, the cached data will simply be invalidated and a new response will be fetched.
diff --git a/docs/user_guide/advanced_requests.md b/docs/user_guide/advanced_requests.md
new file mode 100644
index 0000000..7115a7f
--- /dev/null
+++ b/docs/user_guide/advanced_requests.md
@@ -0,0 +1,50 @@
+<!-- TODO: Better title for this section -->
+# Usage with other requests features
+
+## Request Hooks
+Requests has an [Event Hook](https://requests.readthedocs.io/en/master/user/advanced/#event-hooks)
+system that can be used to add custom behavior into different parts of the request process.
+It can be used, for example, for request throttling:
+
+:::{admonition} Example code
+:class: toggle
+```python
+>>> import time
+>>> import requests
+>>> from requests_cache import CachedSession
+>>>
+>>> def make_throttle_hook(timeout=1.0):
+>>> """Make a request hook function that adds a custom delay for non-cached requests"""
+>>> def hook(response, *args, **kwargs):
+>>> if not getattr(response, 'from_cache', False):
+>>> print('sleeping')
+>>> time.sleep(timeout)
+>>> return response
+>>> return hook
+>>>
+>>> session = CachedSession()
+>>> session.hooks['response'].append(make_throttle_hook(0.1))
+>>> # The first (real) request will have an added delay
+>>> session.get('http://httpbin.org/get')
+>>> session.get('http://httpbin.org/get')
+```
+:::
+
+## Streaming Requests
+If you use [streaming requests](https://2.python-requests.org/en/master/user/advanced/#id9), you
+can use the same code to iterate over both cached and non-cached requests. Cached response content
+will have already been read (i.e., consumed), but will be available for re-reading so it behaves like
+the original streamed response:
+
+:::{admonition} Example code
+:class: toggle
+```python
+>>> from requests_cache import CachedSession
+>>>
+>>> session = CachedSession()
+>>> for i in range(2):
+... response = session.get('https://httpbin.org/stream/20', stream=True)
+... for chunk in response.iter_lines():
+... print(chunk.decode('utf-8'))
+```
+:::
diff --git a/docs/user_guide/backends.md b/docs/user_guide/backends.md
new file mode 100644
index 0000000..d95c840
--- /dev/null
+++ b/docs/user_guide/backends.md
@@ -0,0 +1,152 @@
+(backends)=
+# Backends
+![](../_static/sqlite_32px.png)
+![](../_static/redis_32px.png)
+![](../_static/mongodb_32px.png)
+![](../_static/dynamodb_32px.png)
+![](../_static/files-json_32px.png)
+
+Several cache backends are included. The default is SQLite, since it's generally the simplest to
+use, and requires no extra dependencies or configuration.
+```{note}
+In the rare case that SQLite is not available
+(for example, [on Heroku](https://devcenter.heroku.com/articles/sqlite3)), a non-persistent
+in-memory cache is used by default.
+```
+
+See {py:mod}`.requests_cache.backends` for usage details for specific backends.
+
+## Backend Dependencies
+Most of the other backends require some extra dependencies, listed below.
+
+Backend | Class | Alias | Dependencies
+-------------------------------------------------------|----------------------------|----------------|-------------
+[SQLite](https://www.sqlite.org) | {py:class}`.SQLiteCache` | `'sqlite'` |
+[Redis](https://redis.io) | {py:class}`.RedisCache` | `'redis'` | [redis-py](https://github.com/andymccurdy/redis-py)
+[MongoDB](https://www.mongodb.com) | {py:class}`.MongoCache` | `'mongodb'` | [pymongo](https://github.com/mongodb/mongo-python-driver)
+[GridFS](https://docs.mongodb.com/manual/core/gridfs/) | {py:class}`.GridFSCache` | `'gridfs'` | [pymongo](https://github.com/mongodb/mongo-python-driver)
+[DynamoDB](https://aws.amazon.com/dynamodb) | {py:class}`.DynamoCache` | `'dynamodb'` | [boto3](https://github.com/boto/boto3)
+Filesystem | {py:class}`.FileCache` | `'filesystem'` |
+Memory | {py:class}`.BaseCache` | `'memory'` |
+
+## Specifying a Backend
+You can specify which backend to use with the `backend` parameter for either {py:class}`.CachedSession`
+or {py:func}`.install_cache`. You can specify one by name, using the aliases listed above:
+```python
+>>> session = CachedSession('my_cache', backend='redis')
+```
+
+Or by instance:
+```python
+>>> backend = RedisCache(host='192.168.1.63', port=6379)
+>>> session = CachedSession('my_cache', backend=backend)
+```
+
+## Backend Options
+The `cache_name` parameter has a different use depending on the backend:
+
+Backend | Cache name used as
+----------------|-------------------
+SQLite | Database path
+Redis | Hash namespace
+MongoDB, GridFS | Database name
+DynamoDB | Table name
+Filesystem | Cache directory
+
+Each backend class also accepts optional parameters for the underlying connection. For example,
+{py:class}`.SQLiteCache` accepts parameters for {py:func}`sqlite3.connect`:
+```python
+>>> session = CachedSession('my_cache', backend='sqlite', timeout=30)
+```
+
+## Testing Backends
+If you just want to quickly try out all of the available backends for comparison,
+[docker-compose](https://docs.docker.com/compose/) config is included for all supported services.
+First, [install docker](https://docs.docker.com/get-docker/) if you haven't already. Then, run:
+
+:::{tab} Bash (Linux/macOS)
+```bash
+pip install -U requests-cache[all] docker-compose
+curl https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -O docker-compose.yml
+docker-compose up -d
+```
+:::
+:::{tab} Powershell (Windows)
+```ps1
+pip install -U requests-cache[all] docker-compose
+Invoke-WebRequest -Uri https://raw.githubusercontent.com/reclosedev/requests-cache/master/docker-compose.yml -Outfile docker-compose.yml
+docker-compose up -d
+```
+:::
+
+(exporting)=
+## Exporting To A Different Backend
+If you have cached data that you want to copy or migrate to a different backend, you can do this
+with `CachedSession.cache.update()`. For example, if you want to dump the contents of a Redis cache
+to JSON files:
+```python
+>>> src_session = CachedSession('my_cache', backend='redis')
+>>> dest_session = CachedSession('~/workspace/cache_dump', backend='filesystem', serializer='json')
+>>> dest_session.cache.update(src_session.cache)
+
+>>> # List the exported files
+>>> print(dest_session.cache.paths())
+'/home/user/workspace/cache_dump/9e7a71a3ff2e.json'
+'/home/user/workspace/cache_dump/8a922ff3c53f.json'
+```
+
+Or, using backend classes directly:
+```python
+>>> src_cache = RedisCache()
+>>> dest_cache = FileCache('~/workspace/cache_dump', serializer='json')
+>>> dest_cache.update(src_cache)
+```
+
+(custom-backends)=
+## Custom Backends
+If the built-in backends don't suit your needs, you can create your own by making subclasses of {py:class}`.BaseCache` and {py:class}`.BaseStorage`:
+
+:::{admonition} Example code
+:class: toggle
+```python
+>>> from requests_cache import CachedSession
+>>> from requests_cache.backends import BaseCache, BaseStorage
+
+>>> class CustomCache(BaseCache):
+... """Wrapper for higher-level cache operations. In most cases, the only thing you need
+... to specify here is which storage class(es) to use.
+... """
+... def __init__(self, **kwargs):
+... super().__init__(**kwargs)
+... self.redirects = CustomStorage(**kwargs)
+... self.responses = CustomStorage(**kwargs)
+
+>>> class CustomStorage(BaseStorage):
+... """Dict-like interface for lower-level backend storage operations"""
+... def __init__(self, **kwargs):
+... super().__init__(**kwargs)
+...
+... def __getitem__(self, key):
+... pass
+...
+... def __setitem__(self, key, value):
+... pass
+...
+... def __delitem__(self, key):
+... pass
+...
+... def __iter__(self):
+... pass
+...
+... def __len__(self):
+... pass
+...
+... def clear(self):
+... pass
+```
+:::
+
+You can then use your custom backend in a {py:class}`.CachedSession` with the `backend` parameter:
+```python
+>>> session = CachedSession(backend=CustomCache())
+```
diff --git a/docs/user_guide/compatibility.md b/docs/user_guide/compatibility.md
new file mode 100644
index 0000000..d52d610
--- /dev/null
+++ b/docs/user_guide/compatibility.md
@@ -0,0 +1,146 @@
+<!-- TODO: Fix relative links -->
+(compatibility)=
+# Usage with other requests-based libraries
+This library works by patching and/or extending {py:class}`requests.Session`. Many other libraries
+out there do the same thing, making it potentially difficult to combine them.
+
+For that scenario, a mixin class is provided, so you can create a custom class with behavior from
+multiple Session-modifying libraries:
+```python
+>>> from requests import Session
+>>> from requests_cache import CacheMixin
+>>> from some_other_lib import SomeOtherMixin
+>>>
+>>> class CustomSession(CacheMixin, SomeOtherMixin, Session):
+... """Session class with features from both some_other_lib and requests-cache"""
+```
+
+## Requests-HTML
+[requests-html](https://github.com/psf/requests-html) is one library that works with this method:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> import requests
+>>> from requests_cache import CacheMixin, install_cache
+>>> from requests_html import HTMLSession
+>>>
+>>> class CachedHTMLSession(CacheMixin, HTMLSession):
+... """Session with features from both CachedSession and HTMLSession"""
+>>>
+>>> session = CachedHTMLSession()
+>>> response = session.get('https://github.com/')
+>>> print(response.from_cache, response.html.links)
+```
+:::
+
+
+Or if you are using {py:func}`.install_cache`, you can use the `session_factory` argument:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> install_cache(session_factory=CachedHTMLSession)
+>>> response = requests.get('https://github.com/')
+>>> print(response.from_cache, response.html.links)
+```
+:::
+
+The same approach can be used with other libraries that subclass {py:class}`requests.Session`.
+
+## Requests-futures
+Some libraries, including [requests-futures](https://github.com/ross/requests-futures),
+support wrapping an existing session object:
+```python
+>>> session = FutureSession(session=CachedSession())
+```
+
+In this case, `FutureSession` must wrap `CachedSession` rather than the other way around, since
+`FutureSession` returns (as you might expect) futures rather than response objects.
+See [issue #135](https://github.com/reclosedev/requests-cache/issues/135) for more notes on this.
+
+## Internet Archive
+Usage with [internetarchive](https://github.com/jjjake/internetarchive) is the same as other libraries
+that subclass `requests.Session`:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> from requests_cache import CacheMixin
+>>> from internetarchive.session import ArchiveSession
+>>>
+>>> class CachedArchiveSession(CacheMixin, ArchiveSession):
+... """Session with features from both CachedSession and ArchiveSession"""
+```
+:::
+
+## Requests-mock
+[requests-mock](https://github.com/jamielennox/requests-mock) has multiple methods for mocking
+requests, including a contextmanager, decorator, fixture, and adapter. There are a few different
+options for using it with requests-cache, depending on how you want your tests to work.
+
+### Disabling requests-cache
+If you have an application that uses requests-cache and you just want to use requests-mock in
+your tests, the easiest thing to do is to disable requests-cache.
+
+For example, if you are using {py:func}`.install_cache` in your application and the
+requests-mock [pytest fixture](https://requests-mock.readthedocs.io/en/latest/pytest.html) in your
+tests, you could wrap it in another fixture that uses {py:func}`.uninstall_cache` or {py:func}`.disabled`:
+:::{admonition} Example code
+:class: toggle
+```{literalinclude} ../../tests/compat/test_requests_mock_disable_cache.py
+```
+:::
+
+Or if you use a `CachedSession` object, you could replace it with a regular `Session`, for example:
+:::{admonition} Example code
+:class: toggle
+```python
+import unittest
+import pytest
+import requests
+
+
+@pytest.fixure(scope='function', autouse=True)
+def disable_requests_cache():
+ """Replace CachedSession with a regular Session for all test functions"""
+ with unittest.mock.patch('requests_cache.CachedSession', requests.Session):
+ yield
+```
+:::
+
+### Combining requests-cache with requests-mock
+If you want both caching and mocking features at the same time, you can attach requests-mock's
+[adapter](https://requests-mock.readthedocs.io/en/latest/adapter.html) to a `CachedSession`:
+
+:::{admonition} Example code
+:class: toggle
+```{literalinclude} ../../tests/compat/test_requests_mock_combine_cache.py
+```
+:::
+
+### Building a mocker using requests-cache data
+Another approach is to use cached data to dynamically define mock requests + responses.
+This has the advantage of only using request-mock's behavior for
+[request matching](https://requests-mock.readthedocs.io/en/latest/matching.html).
+
+:::{admonition} Example code
+:class: toggle
+```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py
+:lines: 21-40
+```
+:::
+
+To turn that into a complete example:
+:::{admonition} Example code
+:class: toggle
+```{literalinclude} ../tests/compat/test_requests_mock_load_cache.py
+```
+:::
+
+## Responses
+Usage with the [responses](https://github.com/getsentry/responses) library is similar to the
+requests-mock examples above.
+
+:::{admonition} Example code
+:class: toggle
+```{literalinclude} ../tests/compat/test_responses_load_cache.py
+```
+:::
diff --git a/docs/user_guide/expiration.md b/docs/user_guide/expiration.md
new file mode 100644
index 0000000..adb28ea
--- /dev/null
+++ b/docs/user_guide/expiration.md
@@ -0,0 +1,119 @@
+(expiration)=
+# Expiration
+By default, cached responses will be stored indefinitely. There are a number of options for
+specifying how long to store responses, either with a single expiration value, glob patterns,
+or {ref}`cache headers <headers>`.
+
+The simplest option is to initialize the cache with an `expire_after` value, which will apply to all
+reponses:
+```python
+>>> # Set expiration for the session using a value in seconds
+>>> session = CachedSession(expire_after=360)
+```
+
+(precedence)=
+## Expiration Precedence
+Expiration can be set on a per-session, per-URL, or per-request basis, in addition to cache
+headers (see sections below for usage details). When there are multiple values provided for a given
+request, the following order of precedence is used:
+1. Cache-Control request headers (if enabled)
+2. Cache-Control response headers (if enabled)
+3. Per-request expiration (`expire_after` argument for {py:meth}`.CachedSession.request`)
+4. Per-URL expiration (`urls_expire_after` argument for {py:class}`.CachedSession`)
+5. Per-session expiration (`expire_after` argument for {py:class}`.CacheBackend`)
+
+## Expiration Values
+`expire_after` can be any of the following:
+- `-1` (to never expire)
+- `0` (to "expire immediately," e.g. bypass the cache)
+- A positive number (in seconds)
+- A {py:class}`~datetime.timedelta`
+- A {py:class}`~datetime.datetime`
+
+Examples:
+```python
+>>> # To specify a unit of time other than seconds, use a timedelta
+>>> from datetime import timedelta
+>>> session = CachedSession(expire_after=timedelta(days=30))
+
+>>> # Update an existing session to disable expiration (i.e., store indefinitely)
+>>> session.expire_after = -1
+
+>>> # Disable caching by default, unless enabled by other settings
+>>> session = CachedSession(expire_after=0)
+```
+
+(url-patterns)=
+## Expiration With URL Patterns
+You can use `urls_expire_after` to set different expiration values based on URL glob patterns.
+This allows you to customize caching based on what you know about the resources you're requesting
+or how you intend to use them. For example, you might request one resource that gets updated
+frequently, another that changes infrequently, and another that never changes. Example:
+```python
+>>> urls_expire_after = {
+... '*.site_1.com': 30,
+... 'site_2.com/resource_1': 60 * 2,
+... 'site_2.com/resource_2': 60 * 60 * 24,
+... 'site_2.com/static': -1,
+... }
+>>> session = CachedSession(urls_expire_after=urls_expire_after)
+```
+
+**Notes:**
+- `urls_expire_after` should be a dict in the format `{'pattern': expire_after}`
+- `expire_after` accepts the same types as `CachedSession.expire_after`
+- Patterns will match request **base URLs without the protocol**, so the pattern `site.com/resource/`
+ is equivalent to `http*://site.com/resource/**`
+- If there is more than one match, the first match will be used in the order they are defined
+- If no patterns match a request, `CachedSession.expire_after` will be used as a default
+
+## Expiration and Error Handling
+In some cases, you might cache a response, have it expire, but then encounter an error when
+retrieving a new response. If you would like to use expired response data in these cases, use the
+`old_data_on_error` option.
+
+For example:
+```python
+>>> # Cache a test response that will expire immediately
+>>> session = CachedSession(old_data_on_error=True)
+>>> session.get('https://httpbin.org/get', expire_after=0.0001)
+>>> time.sleep(0.0001)
+```
+
+Afterward, let's say the page has moved and you get a 404, or the site is experiencing downtime and
+you get a 500. You will then get the expired cache data instead:
+```python
+>>> response = session.get('https://httpbin.org/get')
+>>> print(response.from_cache, response.is_expired)
+True, True
+```
+
+In addition to HTTP error codes, `old_data_on_error` also applies to python exceptions (typically a
+{py:exc}`~requests.RequestException`). See `requests` documentation on
+[Errors and Exceptions](https://2.python-requests.org/en/master/user/quickstart/#errors-and-exceptions)
+for more details on request errors in general.
+
+## Removing Expired Responses
+For better read performance, expired responses won't be removed immediately, but will be removed
+(or replaced) the next time they are requested.
+:::{tip}
+Implementing one or more cache eviction algorithms is being considered. If this is something you are
+interested in, please provide feedback via [issues](https://github.com/reclosedev/requests-cache/issues)!
+:::
+
+To manually clear all expired responses, use
+{py:meth}`.CachedSession.remove_expired_responses`:
+```python
+>>> session.remove_expired_responses()
+```
+
+Or, when using patching:
+```python
+>>> requests_cache.remove_expired_responses()
+```
+
+You can also apply a different `expire_after` to previously cached responses, which will
+revalidate the cache with the new expiration time:
+```python
+>>> session.remove_expired_responses(expire_after=timedelta(days=30))
+```
diff --git a/docs/user_guide/files.md b/docs/user_guide/files.md
new file mode 100644
index 0000000..de1325d
--- /dev/null
+++ b/docs/user_guide/files.md
@@ -0,0 +1,94 @@
+(files)=
+# Cache Files
+```{note}
+This section only applies to the {py:mod}`~requests_cache.backends.sqlite` and
+{py:mod}`~requests_cache.backends.filesystem` backends.
+```
+For file-based backends, the cache name will be used as a path to the cache file(s). You can use
+a relative path, absolute path, or use some additional options for system-specific default paths.
+
+## Relative Paths
+```python
+>>> # Database path for SQLite cache
+>>> session = CachedSession('http_cache', backend='sqlite')
+>>> print(session.cache.db_path)
+'<current working dir>/http_cache.sqlite'
+```
+```python
+>>> # Base directory for Filesystem cache
+>>> session = CachedSession('http_cache', backend='filesystem')
+>>> print(session.cache.cache_dir)
+'<current working dir>/http_cache/'
+```
+
+```{note}
+Parent directories will always be created, if they don't already exist.
+```
+
+## Absolute Paths
+You can also give an absolute path, including user paths (with `~`).
+```python
+>>> session = CachedSession('~/.myapp/http_cache', backend='sqlite')
+>>> print(session.cache.db_path)
+'/home/user/.myapp/http_cache.sqlite'
+```
+
+## System Paths
+If you don't know exactly where you want to put your cache files, your system's **temp directory**
+or **cache directory** is a good choice. Some options are available as shortcuts to use whatever the
+default locations are for your operating system.
+
+Use the default temp directory with the `use_temp` option:
+:::{tab} Linux
+```python
+>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
+>>> print(session.cache.db_path)
+'/tmp/http_cache.sqlite'
+```
+:::
+:::{tab} macOS
+```python
+>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
+>>> print(session.cache.db_path)
+'/var/folders/xx/http_cache.sqlite'
+```
+:::
+:::{tab} Windows
+```python
+>>> session = CachedSession('http_cache', backend='sqlite', use_temp=True)
+>>> print(session.cache.db_path)
+'C:\\Users\\user\\AppData\\Local\\temp\\http_cache.sqlite'
+```
+:::
+
+Or use the default cache directory with the `use_cache_dir` option:
+:::{tab} Linux
+```python
+>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
+>>> print(session.cache.cache_dir)
+'/home/user/.cache/http_cache/'
+```
+:::
+:::{tab} macOS
+```python
+>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
+>>> print(session.cache.cache_dir)
+'/Users/user/Library/Caches/http_cache/'
+```
+:::
+:::{tab} Windows
+```python
+>>> session = CachedSession('http_cache', backend='filesystem', use_cache_dir=True)
+>>> print(session.cache.cache_dir)
+'C:\\Users\\user\\AppData\\Local\\http_cache\\'
+```
+:::
+
+```{note}
+If the cache name is an absolute path, the `use_temp` and `use_cache_dir` options will be ignored.
+If it's a relative path, it will be relative to the temp or cache directory, respectively.
+```
+
+There are a number of other system default locations that might be appropriate for a cache file. See
+the [appdirs](https://github.com/ActiveState/appdirs) library for an easy cross-platform way to get
+the most commonly used ones.
diff --git a/docs/user_guide/filtering.md b/docs/user_guide/filtering.md
new file mode 100644
index 0000000..909a535
--- /dev/null
+++ b/docs/user_guide/filtering.md
@@ -0,0 +1,77 @@
+(filtering)=
+# Cache Filtering
+In many cases you will want to choose what you want to cache instead of just caching everything. By
+default, all **read-only** (`GET` and `HEAD`) **requests with a 200 response code** are cached. A
+few options are available to modify this behavior.
+
+```{note}
+When using {py:class}`.CachedSession`, any requests that you don't want to cache can also be made
+with a regular {py:class}`requests.Session` object, or wrapper functions like
+{py:func}`requests.get`, etc.
+```
+
+(http-methods)=
+## Cached HTTP Methods
+To cache additional HTTP methods, specify them with `allowable_methods`:
+```python
+>>> session = CachedSession(allowable_methods=('GET', 'POST'))
+>>> session.post('http://httpbin.org/post', json={'param': 'value'})
+```
+
+For example, some APIs use the `POST` method to request data via a JSON-formatted request body, for
+requests that may exceed the max size of a `GET` request. You may also want to cache `POST` requests
+to ensure you don't send the exact same data multiple times.
+
+## Cached Status Codes
+To cache additional status codes, specify them with `allowable_codes`
+```python
+>>> session = CachedSession(allowable_codes=(200, 418))
+>>> session.get('http://httpbin.org/teapot')
+```
+
+(selective-caching)=
+## Cached URLs
+You can use {ref}`URL patterns <url-patterns>` to define an allowlist for selective caching, by
+using a expiration value of `0` (or `requests_cache.DO_NOT_CACHE`, to be more explicit) for
+non-matching request URLs:
+```python
+>>> from requests_cache import DO_NOT_CACHE, CachedSession
+>>> urls_expire_after = {
+... '*.site_1.com': 30,
+... 'site_2.com/static': -1,
+... '*': DO_NOT_CACHE,
+... }
+>>> session = CachedSession(urls_expire_after=urls_expire_after)
+```
+
+Note that the catch-all rule above (`'*'`) will behave the same as setting the session-level
+expiration to `0`:
+```python
+>>> urls_expire_after = {'*.site_1.com': 30, 'site_2.com/static': -1}
+>>> session = CachedSession(urls_expire_after=urls_expire_after, expire_after=0)
+```
+
+## Custom Cache Filtering
+If you need more advanced behavior for choosing what to cache, you can provide a custom filtering
+function via the `filter_fn` param. This can by any function that takes a
+{py:class}`requests.Response` object and returns a boolean indicating whether or not that response
+should be cached. It will be applied to both new responses (on write) and previously cached
+responses (on read):
+
+:::{admonition} Example code
+:class: toggle
+```python
+>>> from sys import getsizeof
+>>> from requests_cache import CachedSession
+
+>>> def filter_by_size(response: Response) -> bool:
+>>> """Don't cache responses with a body over 1 MB"""
+>>> return getsizeof(response.content) <= 1024 * 1024
+
+>>> session = CachedSession(filter_fn=filter_by_size)
+```
+:::
+
+```{note}
+`filter_fn()` will be used **in addition to** other filtering options.
+```
diff --git a/docs/user_guide/general.md b/docs/user_guide/general.md
new file mode 100644
index 0000000..e5d6562
--- /dev/null
+++ b/docs/user_guide/general.md
@@ -0,0 +1,92 @@
+(general)=
+# General Usage
+There are two main ways of using requests-cache:
+- **Sessions:** (recommended) Use {py:class}`.CachedSession` to send your requests
+- **Patching:** Globally patch `requests` using {py:func}`.install_cache()`
+
+## Sessions
+{py:class}`.CachedSession` can be used as a drop-in replacement for {py:class}`requests.Session`.
+Basic usage looks like this:
+```python
+>>> from requests_cache import CachedSession
+>>>
+>>> session = CachedSession()
+>>> session.get('http://httpbin.org/get')
+```
+
+Any {py:class}`requests.Session` method can be used (but see {ref}`http-methods` section for
+options):
+```python
+>>> session.request('GET', 'http://httpbin.org/get')
+>>> session.head('http://httpbin.org/get')
+```
+
+Caching can be temporarily disabled for the session with
+{py:meth}`.CachedSession.cache_disabled`:
+```python
+>>> with session.cache_disabled():
+... session.get('http://httpbin.org/get')
+```
+
+The best way to clean up your cache is through {ref}`expiration` settings, but you can also
+clear out everything at once with {py:meth}`.BaseCache.clear`:
+```python
+>>> session.cache.clear()
+```
+
+## Patching
+In some situations, it may not be possible or convenient to manage your own session object. In those
+cases, you can use {py:func}`.install_cache` to add caching to all `requests` functions:
+```python
+>>> import requests
+>>> import requests_cache
+>>>
+>>> requests_cache.install_cache()
+>>> requests.get('http://httpbin.org/get')
+```
+
+As well as session methods:
+```python
+>>> session = requests.Session()
+>>> session.get('http://httpbin.org/get')
+```
+
+{py:func}`.install_cache` accepts all the same parameters as {py:class}`.CachedSession`:
+```python
+>>> requests_cache.install_cache(expire_after=360, allowable_methods=('GET', 'POST'))
+```
+
+It can be temporarily {py:func}`.enabled`:
+```python
+>>> with requests_cache.enabled():
+... requests.get('http://httpbin.org/get') # Will be cached
+```
+
+Or temporarily {py:func}`.disabled`:
+```python
+>>> requests_cache.install_cache()
+>>> with requests_cache.disabled():
+... requests.get('http://httpbin.org/get') # Will not be cached
+```
+
+Or completely removed with {py:func}`.uninstall_cache`:
+```python
+>>> requests_cache.uninstall_cache()
+>>> requests.get('http://httpbin.org/get')
+```
+
+You can also clear out all responses in the cache with {py:func}`.clear`, and check if
+requests-cache is currently installed with {py:func}`.is_installed`.
+
+(monkeypatch-issues)=
+### Patching Limitations & Potential Issues
+Like any other utility that uses monkey-patching, there are some scenarios where you won't want to
+use {py:func}`.install_cache`:
+- When using other libraries that patch {py:class}`requests.Session`
+- In a multi-threaded or multiprocess application
+- In a library that will be imported by other libraries or applications
+- In a larger application that makes requests in several different modules, where it may not be
+ obvious what is and isn't being cached
+
+In any of these cases, consider using {py:class}`.CachedSession`, the {py:func}`.enabled`
+contextmanager, or {ref}`selective-caching`.
diff --git a/docs/user_guide/headers.md b/docs/user_guide/headers.md
new file mode 100644
index 0000000..8d1a795
--- /dev/null
+++ b/docs/user_guide/headers.md
@@ -0,0 +1,62 @@
+(headers)=
+# Cache Headers
+Most common request and response headers related to caching are supported, including
+[Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control)
+and [ETags](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag).
+
+```{note}
+requests-cache is not intended to be strict implementation of HTTP caching according to
+[RFC 2616](https://datatracker.ietf.org/doc/html/rfc2616),
+[RFC 7234](https://datatracker.ietf.org/doc/html/rfc7234), etc. These RFCs describe many behaviors
+that make sense in the context of a browser or proxy cache, but not for a python application.
+```
+
+## Conditional Requests
+[Conditional requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests) are
+automatically sent for any servers that support them. Once a cached response expires, it will only
+be updated if the remote content has changed.
+
+Here's an example using the [GitHub API](https://docs.github.com/en/rest) to get info about the
+requests-cache repo:
+```python
+>>> # Cache a response that will expire immediately
+>>> url = 'https://api.github.com/repos/reclosedev/requests-cache'
+>>> session = CachedSession(expire_after=0.0001)
+>>> session.get(url)
+>>> time.sleep(0.0001)
+
+>>> # The cached response will still be used until the remote content actually changes
+>>> response = session.get(url)
+>>> print(response.from_cache, response.is_expired)
+True, True
+```
+
+## Cache-Control
+If enabled, `Cache-Control` directives will take priority over any other `expire_after` value.
+See {ref}`precedence` for the full order of precedence.
+
+To enable this behavior, use the `cache_control` option:
+```python
+>>> session = CachedSession(cache_control=True)
+```
+
+## Supported Headers
+The following headers are currently supported:
+
+**Request headers:**
+- `Cache-Control: max-age`: Used as the expiration time in seconds
+- `Cache-Control: no-cache`: Skips reading response data from the cache
+- `Cache-Control: no-store`: Skips reading and writing response data from/to the cache
+- `If-None-Match`: Automatically added if an `ETag` is available
+- `If-Modified-Since`: Automatically added if `Last-Modified` is available
+
+**Response headers:**
+- `Cache-Control: max-age`: Used as the expiration time in seconds
+- `Cache-Control: no-store` Skips writing response data to the cache
+- `Expires`: Used as an absolute expiration time
+- `ETag`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response)
+- `Last-Modified`: Returns expired cache data if the remote content has not changed (`304 Not Modified` response)
+
+```{note}
+Unlike a browser or proxy cache, `max-age=0` does not clear previously cached responses.
+```
diff --git a/docs/user_guide/index.md b/docs/user_guide/index.md
new file mode 100644
index 0000000..d28a108
--- /dev/null
+++ b/docs/user_guide/index.md
@@ -0,0 +1,21 @@
+(user-guide)=
+# User Guide
+This section covers the main features of requests-cache.
+
+```{toctree}
+:maxdepth: 2
+
+installation
+general
+backends
+files
+filtering
+headers
+inspection
+expiration
+matching
+security
+serializers
+troubleshooting
+compatibility
+````
diff --git a/docs/user_guide/inspection.md b/docs/user_guide/inspection.md
new file mode 100644
index 0000000..1ba560d
--- /dev/null
+++ b/docs/user_guide/inspection.md
@@ -0,0 +1,82 @@
+<!-- TODO: This could use some more details and examples -->
+(inspection)=
+# Cache Inspection
+Here are some ways to get additional information out of the cache session, backend, and responses:
+
+## Response Details
+The following attributes are available on responses:
+- `from_cache`: indicates if the response came from the cache
+- `created_at`: {py:class}`~datetime.datetime` of when the cached response was created or last updated
+- `expires`: {py:class}`~datetime.datetime` after which the cached response will expire
+- `is_expired`: indicates if the cached response is expired (if an old response was returned due to a request error)
+
+Examples:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> from requests_cache import CachedSession
+>>> session = CachedSession(expire_after=timedelta(days=1))
+
+>>> # Placeholders are added for non-cached responses
+>>> response = session.get('http://httpbin.org/get')
+>>> print(response.from_cache, response.created_at, response.expires, response.is_expired)
+False None None None
+
+>>> # Values will be populated for cached responses
+>>> response = session.get('http://httpbin.org/get')
+>>> print(response.from_cache, response.created_at, response.expires, response.is_expired)
+True 2021-01-01 18:00:00 2021-01-02 18:00:00 False
+
+>>> # Print a response object to get general information about it
+>>> print(response)
+'request: GET https://httpbin.org/get, response: 200 (308 bytes), created: 2021-01-01 22:45:00 IST, expires: 2021-01-02 18:45:00 IST (fresh)'
+```
+:::
+
+## Cache Contents
+You can use `CachedSession.cache.urls` to see all URLs currently in the cache:
+```python
+>>> session = CachedSession()
+>>> print(session.cache.urls)
+['https://httpbin.org/get', 'https://httpbin.org/stream/100']
+```
+
+If needed, you can get more details on cached responses via `CachedSession.cache.responses`, which
+is a dict-like interface to the cache backend. See {py:class}`.CachedResponse` for a full list of
+attributes available.
+
+For example, if you wanted to to see all URLs requested with a specific method:
+```python
+>>> post_urls = [
+... response.url for response in session.cache.responses.values()
+... if response.request.method == 'POST'
+... ]
+```
+
+You can also inspect `CachedSession.cache.redirects`, which maps redirect URLs to keys of the
+responses they redirect to.
+
+Additional `keys()` and `values()` wrapper methods are available on {py:class}`.BaseCache` to get
+combined keys and responses.
+```python
+>>> print('All responses:')
+>>> for response in session.cache.values():
+>>> print(response)
+
+>>> print('All cache keys for redirects and responses combined:')
+>>> print(list(session.cache.keys()))
+```
+
+Both methods also take a `check_expiry` argument to exclude expired responses:
+```python
+>>> print('All unexpired responses:')
+>>> for response in session.cache.values(check_expiry=True):
+>>> print(response)
+```
+
+Similarly, you can get a count of responses with {py:meth}`.BaseCache.response_count`, and optionally
+exclude expired responses:
+```python
+>>> print(f'Total responses: {session.cache.response_count()}')
+>>> print(f'Unexpired responses: {session.cache.response_count(check_expiry=True)}')
+```
diff --git a/docs/user_guide/installation.md b/docs/user_guide/installation.md
new file mode 100644
index 0000000..7435c57
--- /dev/null
+++ b/docs/user_guide/installation.md
@@ -0,0 +1,41 @@
+# Installation
+Installation instructions:
+
+:::{tab} Pip
+Install the latest stable version from [PyPI](https://pypi.org/project/requests-cache/):
+```
+pip install requests-cache
+```
+:::
+:::{tab} Conda
+Or install from [conda-forge](https://anaconda.org/conda-forge/requests-cache), if you prefer:
+```
+conda install -c conda-forge requests-cache
+```
+:::
+:::{tab} Pre-release
+If you would like to use the latest development (pre-release) version:
+```
+pip install --pre requests-cache
+```
+:::
+:::{tab} Local development
+See {ref}`contributing` for setup steps for local development
+:::
+
+## Requirements
+You may need additional dependencies depending on which backend you want to use. To install with
+extra dependencies for all supported {ref}`backends`:
+```
+pip install requests-cache[all]
+```
+
+## Python Version Compatibility
+The latest version of requests-cache requires **python 3.7+**. If you need to use an older version
+of python, here are the latest compatible versions and their documentation pages:
+
+* **python 2.6:** [requests-cache 0.4.13](https://requests-cache.readthedocs.io/en/v0.4.13)
+* **python 2.7:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
+* **python 3.4:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
+* **python 3.5:** [requests-cache 0.5.2](https://requests-cache.readthedocs.io/en/v0.5.0)
+* **python 3.6:** [requests-cache 0.7.4](https://requests-cache.readthedocs.io/en/v0.7.4)
diff --git a/docs/user_guide/matching.md b/docs/user_guide/matching.md
new file mode 100644
index 0000000..333d868
--- /dev/null
+++ b/docs/user_guide/matching.md
@@ -0,0 +1,105 @@
+(matching)=
+# Request Matching
+Requests are matched according to the request URL, parameters and body. All of these values are
+normalized to account for any variations that do not modify response content.
+
+There are additional options to match according to request headers, ignore specific request
+parameters, or create your own custom request matcher.
+
+## Matching Request Headers
+In some cases, different headers may result in different response data, so you may want to cache
+them separately. To enable this, use `include_get_headers`:
+```python
+>>> session = CachedSession(include_get_headers=True)
+>>> # Both of these requests will be sent and cached separately
+>>> session.get('http://httpbin.org/headers', {'Accept': 'text/plain'})
+>>> session.get('http://httpbin.org/headers', {'Accept': 'application/json'})
+```
+
+(filter-params)=
+## Selective Parameter Matching
+By default, all normalized request parameters are matched. In some cases, there may be request
+parameters that don't affect the response data, for example authentication tokens or credentials.
+If you want to ignore specific parameters, specify them with the `ignored_parameters` option.
+
+**Request Parameters:**
+
+In this example, only the first request will be sent, and the second request will be a cache hit
+due to the ignored parameters:
+```python
+>>> session = CachedSession(ignored_parameters=['auth-token'])
+>>> session.get('http://httpbin.org/get', params={'auth-token': '2F63E5DF4F44'})
+>>> r = session.get('http://httpbin.org/get', params={'auth-token': 'D9FAEB3449D3'})
+>>> assert r.from_cache is True
+```
+
+**Request Body Parameters:**
+
+This also applies to parameters in a JSON-formatted request body:
+```python
+>>> session = CachedSession(allowable_methods=('GET', 'POST'), ignored_parameters=['auth-token'])
+>>> session.post('http://httpbin.org/post', json={'auth-token': '2F63E5DF4F44'})
+>>> r = session.post('http://httpbin.org/post', json={'auth-token': 'D9FAEB3449D3'})
+>>> assert r.from_cache is True
+```
+
+**Request Headers:**
+
+As well as headers, if `include_get_headers` is also used:
+```python
+>>> session = CachedSession(ignored_parameters=['auth-token'], include_get_headers=True)
+>>> session.get('http://httpbin.org/get', headers={'auth-token': '2F63E5DF4F44'})
+>>> r = session.get('http://httpbin.org/get', headers={'auth-token': 'D9FAEB3449D3'})
+>>> assert r.from_cache is True
+```
+```{note}
+Since `ignored_parameters` is most often used for sensitive info like credentials, these values will also be removed from the cached request parameters, body, and headers.
+```
+
+## Custom Request Matching
+If you need more advanced behavior, you can implement your own custom request matching.
+
+Request matching is accomplished using a **cache key**, which uniquely identifies a response in the
+cache based on request info. For example, the option `ignored_parameters=['foo']` works by excluding
+the `foo` request parameter from the cache key, meaning these three requests will all use the same
+cached response:
+```python
+>>> session = CachedSession(ignored_parameters=['foo'])
+>>> response_1 = session.get('https://example.com') # cache miss
+>>> response_2 = session.get('https://example.com?foo=bar') # cache hit
+>>> response_3 = session.get('https://example.com?foo=qux') # cache hit
+>>> assert response_1.cache_key == response_2.cache_key == response_3.cache_key
+```
+
+If you want to implement your own request matching, you can provide a cache key function which will
+take a {py:class}`~requests.PreparedRequest` plus optional keyword args, and return a string:
+```python
+def create_key(request: requests.PreparedRequest, **kwargs) -> str:
+ """Generate a custom cache key for the given request"""
+```
+
+`**kwargs` includes relevant {py:class}`.BaseCache` settings and any other keyword args passed to
+{py:meth}`.CachedSession.send()`. See {py:func}`.create_key` for the reference implementation, and
+see the rest of the {py:mod}`.cache_keys` module for some potentially useful helper functions.
+
+You can then pass this function via the `key_fn` param:
+```python
+session = CachedSession(key_fn=create_key)
+```
+
+```{note}
+`key_fn()` will be used **instead of** any other {ref}`matching` options and default matching behavior.
+```
+```{tip}
+See {ref}`Examples<custom_keys>` page for a complete example for custom request matching.
+```
+```{tip}
+As a general rule, if you include less info in your cache keys, you will have more cache hits and
+use less storage space, but risk getting incorrect response data back. For example, if you exclude
+all request parameters, you will get the same cached response back for any combination of request
+parameters.
+```
+```{warning}
+If you provide a custom key function for a non-empty cache, any responses previously cached with a
+different key function will likely be unused.
+```
diff --git a/docs/security.md b/docs/user_guide/security.md
index 4c5218f..45fbd0a 100644
--- a/docs/security.md
+++ b/docs/user_guide/security.md
@@ -48,8 +48,8 @@ Once you have your key, create a {py:func}`.safe_pickle_serializer` with it:
```
:::{note}
-You can also make your own {ref}`custom serializer <advanced_usage:custom serializers>`
-using `itsdangerous`, if you would like more control over how responses are serialized.
+You can also make your own {ref}`custom-serializers`, if you would like more control over how
+responses are serialized.
:::
You can verify that it's working by modifying the cached item (*without* your key):
@@ -65,3 +65,7 @@ Then, if you try to get that cached response again (*with* your key), you will g
>>> session.get('https://httpbin.org/get')
BadSignature: Signature b'iFNmzdUOSw5vqrR9Cb_wfI1EoZ8' does not match
```
+
+## Removing Sensitive Info
+The {ref}`ignored_parameters <filter-params>` option can be used to prevent credentials and other
+sensitive info from being saved to the cache. It applies to request parameters, body, and headers.
diff --git a/docs/user_guide/serializers.md b/docs/user_guide/serializers.md
new file mode 100644
index 0000000..46a94b7
--- /dev/null
+++ b/docs/user_guide/serializers.md
@@ -0,0 +1,167 @@
+(serializers)=
+# Serializers
+![](../_static/file-pickle_32px.png)
+![](../_static/file-json_32px.png)
+![](../_static/file-yaml_32px.png)
+![](../_static/file-toml_32px.png)
+
+By default, responses are serialized using {py:mod}`pickle`, but some alternative serializers are
+also included. These are mainly intended for use with {py:class}`.FileCache`, but are compatible
+with the other backends as well.
+
+:::{note}
+Some serializers require additional dependencies
+:::
+
+## Specifying a Serializer
+Similar to {ref}`backends`, you can specify which serializer to use with the `serializer` parameter
+for either {py:class}`.CachedSession` or {py:func}`.install_cache`.
+
+## JSON Serializer
+Storing responses as JSON gives you the benefit of making them human-readable and editable, in
+exchange for a minor reduction in read and write speeds.
+
+Usage:
+```python
+>>> session = CachedSession('my_cache', serializer='json')
+```
+
+:::{admonition} Example JSON-serialized Response
+:class: toggle
+```{literalinclude} ../sample_data/sample_response.json
+:language: JSON
+```
+:::
+
+This will use [ultrajson](https://github.com/ultrajson/ultrajson) if installed, otherwise the stdlib
+`json` module will be used. You can install the optional dependencies for this serializer with:
+```bash
+pip install requests-cache[json]
+```
+
+## YAML Serializer
+YAML is another option if you need a human-readable/editable format, with the same tradeoffs as JSON.
+
+Usage:
+```python
+>>> session = CachedSession('my_cache', serializer='yaml')
+```
+
+:::{admonition} Example YAML-serialized Response
+:class: toggle
+```{literalinclude} ../sample_data/sample_response.yaml
+:language: YAML
+```
+:::
+
+You can install the extra dependencies for this serializer with:
+```bash
+pip install requests-cache[yaml]
+```
+
+## BSON Serializer
+[BSON](https://www.mongodb.com/json-and-bson) is a serialization format originally created for
+MongoDB, but it can also be used independently. Compared to JSON, it has better performance
+(although still not as fast as `pickle`), and adds support for additional data types. It is not
+human-readable, but some tools support reading and editing it directly
+(for example, [bson-converter](https://atom.io/packages/bson-converter) for Atom).
+
+Usage:
+```python
+>>> session = CachedSession('my_cache', serializer='bson')
+```
+
+You can install the extra dependencies for this serializer with:
+```bash
+pip install requests-cache[mongo]
+```
+
+Or if you would like to use the standalone BSON codec for a different backend, without installing
+MongoDB dependencies:
+```bash
+pip install requests-cache[bson]
+```
+
+## Serializer Security
+See {ref}`security` for recommended setup steps for more secure cache serialization, particularly
+when using {py:mod}`pickle`.
+
+(custom-serializers)=
+## Custom Serializers
+If the built-in serializers don't suit your needs, you can create your own. For example, if
+you had a imaginary `custom_pickle` module that provides `dumps` and `loads` functions:
+```python
+>>> import custom_pickle
+>>> from requests_cache import CachedSession
+>>> session = CachedSession(serializer=custom_pickle)
+```
+
+### Serializer Pipelines
+More complex serialization can be done with {py:class}`.SerializerPipeline`. Use cases include
+text-based serialization, compression, encryption, and any other intermediate steps you might want
+to add.
+
+Any combination of these can be composed with a {py:class}`.SerializerPipeline`, which starts with a
+{py:class}`.CachedResponse` and ends with a {py:class}`.str` or {py:class}`.bytes` object. Each stage
+of the pipeline can be any object or module with `dumps` and `loads` functions. If the object has
+similar methods with different names (e.g. `compress` / `decompress`), those can be aliased using
+{py:class}`.Stage`.
+
+For example, a compressed pickle serializer can be built as:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> import pickle, gzip
+>>> from requests_cache.serialzers import SerializerPipeline, Stage
+>>> compressed_serializer = SerializerPipeline([
+... pickle,
+... Stage(gzip, dumps='compress', loads='decompress'),
+...])
+>>> session = CachedSession(serializer=compressed_serializer)
+```
+:::
+
+### Text-based Serializers
+If you're using a text-based serialization format like JSON or YAML, some extra steps are needed to
+encode binary data and non-builtin types. The [cattrs](https://cattrs.readthedocs.io) library can do
+the majority of the work here, and some pre-configured converters are included for serveral common
+formats in the {py:mod}`.preconf` module.
+
+For example, a compressed JSON pipeline could be built as follows:
+:::{admonition} Example code
+:class: toggle
+```python
+>>> import json, gzip, codecs
+>>> from requests_cache.serializers import SerializerPipeline, Stage, json_converter
+>>> comp_json_serializer = SerializerPipeline([
+... json_converter, # Serialize to a JSON string
+... Stage(codecs.utf_8, dumps='encode', loads='decode'), # Encode to bytes
+... Stage(gzip, dumps='compress', loads='decompress'), # Compress
+...])
+```
+:::
+
+```{note}
+If you want to use a different format that isn't included in {py:mod}`.preconf`, you can use
+{py:class}`.CattrStage` as a starting point.
+```
+
+```{note}
+If you want to convert a string representation to bytes (e.g. for compression), you must use a codec
+from {py:mod}`.codecs` (typically `codecs.utf_8`)
+```
+
+### Additional Serialization Steps
+Some other tools that could be used as a stage in a {py:class}`.SerializerPipeline` include:
+
+Class | loads | dumps
+----- | ----- | -----
+{py:mod}`codecs.* <.codecs>` | encode | decode
+{py:mod}`.bz2` | compress | decompress
+{py:mod}`.gzip` | compress | decompress
+{py:mod}`.lzma` | compress | decompress
+{py:mod}`.zlib` | compress | decompress
+{py:mod}`.pickle` | dumps | loads
+{py:class}`itsdangerous.signer.Signer` | sign | unsign
+{py:class}`itsdangerous.serializer.Serializer` | loads | dumps
+{py:class}`cryptography.fernet.Fernet` | encrypt | decrypt
diff --git a/docs/user_guide/troubleshooting.md b/docs/user_guide/troubleshooting.md
new file mode 100644
index 0000000..e61bf84
--- /dev/null
+++ b/docs/user_guide/troubleshooting.md
@@ -0,0 +1,10 @@
+<!-- TODO: Logging, tracebacks, submitting issues, etc. -->
+# Troubleshooting
+
+## Potential Issues
+- See {ref}`monkeypatch-issues` for issues specific to {py:func}`.install_cache`
+- New releases of `requests`, `urllib3` or `requests-cache` itself may change response data and be
+ be incompatible with previously cached data (see issues
+ [#56](https://github.com/reclosedev/requests-cache/issues/56) and
+ [#102](https://github.com/reclosedev/requests-cache/issues/102)).
+ In these cases, the cached data will simply be invalidated and a new response will be fetched.
diff --git a/examples/custom_cache_keys.py b/examples/custom_request_matcher.py
index 5ba6dc0..a63139b 100644
--- a/examples/custom_cache_keys.py
+++ b/examples/custom_request_matcher.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python
"""
-Example of a custom cache key function that caches a new response if the version of requests-cache,
-requests, or urllib3 changes.
+Example of a custom {ref}`request matcher <matching>` that caches a new response if the version of
+requests-cache, requests, or urllib3 changes.
This generally isn't needed, since anything that causes a deserialization error will simply result
in a new request being sent and cached. But you might want to include a library version in your cache
diff --git a/requests_cache/backends/__init__.py b/requests_cache/backends/__init__.py
index bc48161..870511c 100644
--- a/requests_cache/backends/__init__.py
+++ b/requests_cache/backends/__init__.py
@@ -1,6 +1,4 @@
-"""Classes and functions for cache persistence. See :ref:`user_guide:cache backends` for general
-usage info.
-"""
+"""Classes and functions for cache persistence. See :ref:`backends` for general usage info."""
# flake8: noqa: F401
from inspect import signature
from logging import getLogger
diff --git a/requests_cache/backends/base.py b/requests_cache/backends/base.py
index e580d8b..4186f60 100644
--- a/requests_cache/backends/base.py
+++ b/requests_cache/backends/base.py
@@ -39,7 +39,7 @@ class BaseCache:
Lower-level storage operations are handled by :py:class:`.BaseStorage`.
- To extend this with your own custom backend, see :ref:`advanced_usage:custom backends`.
+ To extend this with your own custom backend, see :ref:`custom-backends`.
"""
def __init__(
@@ -247,7 +247,7 @@ class BaseStorage(MutableMapping, ABC):
``BaseStorage`` also contains a serializer module or instance (defaulting to :py:mod:`pickle`),
which determines how :py:class:`.CachedResponse` objects are saved internally. See
- :ref:`user_guide:serializers` for details.
+ :ref:`serializers` for details.
Args:
serializer: Custom serializer that provides ``loads`` and ``dumps`` methods
diff --git a/requests_cache/backends/filesystem.py b/requests_cache/backends/filesystem.py
index 687dbf6..6ab23d4 100644
--- a/requests_cache/backends/filesystem.py
+++ b/requests_cache/backends/filesystem.py
@@ -24,7 +24,7 @@ Or as YAML (requires ``pyyaml``):
Cache Files
^^^^^^^^^^^
-* See :ref:`user_guide:cache files` for general info on cache files
+* See :ref:`files` for general info on specifying cache paths
* The path for a given response will be in the format ``<cache_name>/<cache_key>``
* Redirects are stored in a separate SQLite database, located at ``<cache_name>/redirects.sqlite``
* Use :py:meth:`.FileCache.paths` to get a list of all cached response paths
diff --git a/requests_cache/backends/sqlite.py b/requests_cache/backends/sqlite.py
index 41e63ad..2ae5e40 100644
--- a/requests_cache/backends/sqlite.py
+++ b/requests_cache/backends/sqlite.py
@@ -12,7 +12,7 @@ for requests-cache.
Cache Files
^^^^^^^^^^^
-* See :ref:`user_guide:cache files` for general info on cache files
+* See :ref:`files` for general info on specifying cache paths
* If you specify a name without an extension, the default extension ``.sqlite`` will be used
In-Memory Caching
diff --git a/requests_cache/serializers/__init__.py b/requests_cache/serializers/__init__.py
index eeceecb..68ead1b 100644
--- a/requests_cache/serializers/__init__.py
+++ b/requests_cache/serializers/__init__.py
@@ -1,4 +1,4 @@
-"""Response serialization utilities. See :ref:`user_guide:serializers` for general usage info.
+"""Response serialization utilities. See :ref:`serializers` for general usage info.
"""
# flake8: noqa: F401
from .cattrs import CattrStage