| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change the handling of HTTP status code 308 to behave more like
`urllib.request.HTTPRedirectHandler`, most critically, the new 308 handling will
create a new `urllib.request.Request` object with the new URL, which will
prevent state from being carried over from the original request.
One case where this is important is when the domain name changes, for example,
when the original URL is `http://www.w3.org/ns/adms.ttl` and the redirect URL is
`https://uri.semic.eu/w3c/ns/adms.ttl`. With the previous behaviour, the redirect
would contain a `Host` header with the value `www.w3.org` instead of
`uri.semic.eu` because the `Host` header is placed in
`Request.unredirected_hdrs` and takes precedence over the `Host` header in
`Request.headers`.
Other changes:
- Only handle HTTP status code 308 on Python versions before 3.11 as Python 3.11
will handle 308 by default [[ref](https://docs.python.org/3.11/whatsnew/changelog.html#id128)].
- Move code which uses `http://www.w3.org/ns/adms.ttl` and
`http://www.w3.org/ns/adms.rdf` out of `test_guess_format_for_parse` into a
separate parameterized test, which instead uses the embedded http server.
This allows the test to fully control the `Content-Type` header in the
response instead of relying on the value that the server is sending.
This is needed because the server is sending `Content-Type: text/plain` for
the `adms.ttl` file, which is not a valid RDF format, and the test is
expecting `Content-Type: text/turtle`.
Fixes:
- <https://github.com/RDFLib/rdflib/issues/2382>.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Eliminate some occurrences of the following flake8 errors in tests:
* E265 block comment should start with '# '
* E266 too many leading '#' for block comment
* E402 module level import not at top of file
* E712 comparison to False should be 'if cond is False:' or 'if not cond:'
* E712 comparison to True should be 'if cond is True:' or 'if cond:'
* E722 do not use bare 'except'
* F401 ... imported but unused
* F403 ... used; unable to detect undefined names
* F405 ... may be undefined, or defined from star imports: ...
* F541 f-string is missing placeholders
* F841 local variable 'result' is assigned to but never used
* N806 variable 'TEST_DIR' in function should be lowercase
This is pursuant to eliminating
[flakeheaven](https://github.com/flakeheaven/flakeheaven), as it no longer
supports the latest version of flake8
[[ref](https://github.com/flakeheaven/flakeheaven/issues/132)].
|
|
|
|
|
|
|
|
| |
Disable
[`implicit_reexport`](https://mypy.readthedocs.io/en/stable/config_file.html#confval-implicit_reexport)
and eliminate all errors reported by mypy after this.
This helps ensure that import statements import from the right module and that
the `__all__` variable is correct.
|
|
|
|
|
|
|
|
|
| |
DESCRIBE queries (#2322)
Add optional keyword only `target_graph` argument to `rdflib.graph.Graph.cbd` and use this new argument in `evalDescribeQuery`.
This makes it possible to compute a concise bounded description without creating a new graph to hold the result, and also without potentially having to copy it to another final graph.
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
| |
This change removes the redundant inheritance from `object` (i.e. `class
Foo(object): pass`) that is no longer needed in Python 3 and is a relic from
Python 2.
|
|
|
|
|
|
|
|
|
| |
`ROUND` was not correctly rounding negative numbers towards positive infinity,
`ENCODE_FOR_URI` incorrectly treated `/` as safe, and `SECONDS` did not include
fractional seconds.
This change corrects these issues.
- Closes <https://github.com/RDFLib/rdflib/issues/2151>.
|
|
|
|
|
|
|
|
|
|
|
|
| |
The namespaces bound by default by `rdflib.graph.Graph` and
`rdflib.namespace.NamespaceManager` was reduced in version 6.2.0 of RDFLib,
however, this also would cause code that worked with 6.1.1 to break, so this
constituted a breaking change. This change restores the previous behaviour,
binding the same namespaces as was bound in 6.1.1.
To bind a reduced set of namespaces, the `bind_namespaces` parameter of
`rdflib.graph.Graph` or `rdflib.namespace.NamespaceManager` can be used.
- Closes <https://github.com/RDFLib/rdflib/issues/2103>.
|
|
|
|
|
|
|
| |
`get_target_namespace_elements` references the `args` global, which is not
defined if the function is called from outside the module. This commit fixes
that instead referencing the argument passed to the function.
- Closes <https://github.com/RDFLib/rdflib/issues/2072>.
|
|
|
|
|
|
|
|
|
|
| |
`Context.to_dict` is used in JSON-LD serialization, but it was not implemented.
This change adds the method.
- Closes <https://github.com/RDFLib/rdflib/issues/2138>.
---------
Co-authored-by: Marc-Antoine Parent <maparent@acm.org>
|
|
|
|
|
|
|
| |
The default prefix for `https://schema.org/` registered with
`rdflib.namespace.NamespaceManager` was inadvertently changed to `sdo` in 6.2.0,
this however constitutes a breaking change, as code that was using the `schema`
prefix would no longer have the same behaviour. This change changes the prefix
back to `schema`.
|
|
|
|
|
| |
SPARQL DESCRIBE was implemented in
<https://github.com/RDFLib/rdflib/pull/2221> so there should be no `xfail`
on it tests for it. This change removes one such `xfail` that was in place.
|
|
|
|
|
|
|
|
|
|
| |
A variable was only being initialized for string-valued inputs, but if a `dict`
input was passed the variable would still be accessed, resulting in a
`UnboundLocalError`.
This change initializes the variable always, instead of only when string-valued
input is used to construct a JSON-LD context.
- Closes <https://github.com/RDFLib/rdflib/issues/2303>.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The URI to IRI conversion was percentage-quoting characters that should not have
been quoted, like equals in the query string. It was also not quoting things
that should have been quoted, like the username and password components of a
URI.
This change improves the conversion by only quoting characters that are not
allowed in specific parts of the URI and quoting previously unquoted components.
The safe characters for each segment are taken from
[RFC3986](https://datatracker.ietf.org/doc/html/rfc3986).
The new behavior is heavily inspired by
[`werkzeug.urls.iri_to_uri`](https://github.com/pallets/werkzeug/blob/92c6380248c7272ee668e1f8bbd80447027ccce2/src/werkzeug/urls.py#L926-L931)
though there are some differences.
- Closes <https://github.com/RDFLib/rdflib/issues/2120>.
|
|
|
|
|
|
|
|
|
|
|
| |
This is being done so that it is easier for downstream packagers to run the test
suite without requiring internet access.
To run only tests that does not use the internet, run `pytest -m "not webtest"`.
The validation workflow validates that test run without internet access by
running the tests inside `firejail --net=none`.
- Closes <https://github.com/RDFLib/rdflib/issues/2293>.
|
|
|
|
|
|
| |
<https://github.com/RDFLib/rdflib/pull/1686> inadvertently removed the `wgs` prefix.
This change adds it back.
- Closes <https://github.com/RDFLib/rdflib/issues/2196>.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These methods were removed when `@total_ordering` was added, but
`@total_ordering` does not add them, so removing them essentially
removes functionality.
This change adds the methods back and adds tests to ensure they work
correctly.
All path related tests are also moved into one file.
- Closes <https://github.com/RDFLib/rdflib/issues/2281>.
- Closes <https://github.com/RDFLib/rdflib/issues/2242>.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
build(deps-dev): bump mypy from 1.0.1 to 1.1.1
Bumps [mypy](https://github.com/python/mypy) from 1.0.1 to 1.1.1.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v1.0.1...v1.1.1)
updated-dependencies:
- dependency-name: mypy
dependency-type: direct:development
update-type: version-update:semver-minor
Also added type ignores for newly detected type errors.
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
|
|
|
| |
docs: document available security measures
Several security measures can be used to mitigate risk when processing
potentially malicious input.
This change adds documentation about available security measures and
examples and tests that illustrate their usage.
|
|
|
|
|
|
|
| |
defined_namespace_creator (#2254)
FIxed a bug in defined_namespace_creator so that when a resource in an ontology is commented on multiple times (with e.g. rdfs:comment), it creates only a single class variable in the resulting DefinedNamespace where the Python comment is the concatenation of the rdfs:comments.
|
|
|
|
|
|
|
|
|
| |
I want to add examples for securing RDFLib network access using
`sys.addaudithook` and `urllib.request.install_opener`, but I want to
also validate the examples in our CI pipeline, so we can demonstrate
they work to our users.
This change adds validation for all examples, and the addition of the
security examples in a seperate PR will then also get validated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Here, adding type-hints to some of the SPARQL parser plugin code.
Includes a couple of small consequent changes:
1. Minor refactor of `prettify_parsetree()`, separating the public-facing callable from the internal code that does not need to be public-facing. That allows the public-facing callable to have more informative and restrictive type-hints for its arguments.
2. Added some test-coverage for `expandUnicodeEscapes()` - initially for my own understanding, but seems useful to leave it in place since I didn't see test-coverage for that function.
There should be no backwards-incompatible changes in this PR - at least, not intentionally.
---------
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
|
|
| |
Mainly so that users can use RDFLib in a safer way, and that we can make
safer changes to RDFLib in future.
There are also some accomodating type-hint related changes outside of `rdflib.util`.
This change does not have a runtime impact.
|
|
|
|
| |
This changes the test the code to avoid deprecation warnings that
occur. Mostly related to changes in python threading and pytest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I have added a bunch of tests for `InputSource` handling, checking
most kinds of input source with most parsers. During this, I detected
the following issues that I fixed:
- `rdflib.util._iri2uri()` was URL quoting the `netloc` parameter, but
this is wrong and the `idna` encoding already takes care of special
characters. I removed the URL quoting of `netloc`.
- HexTuple parsing was handling the input source in a way that would
only work for some input sources, and not raising errors for other
input sources. I changed the input source handling to be more generic.
- `rdflib.parser.create_input_source()` incorrectly used `file.buffer`
instead of `source.buffer` when dealing with IO stream sources.
Other changes with no runtime impact include:
- Changed the HTTP mocking stuff in test slightly to accommodate
serving arbitrary files, as I used this in the `InputSource` tests.
- Don't use Google in tests, as we keep getting
`urllib.error.HTTPError: HTTP Error 429: Too Many Requests`
from it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* build(deps-dev): bump mypy from 0.991 to 1.0.1
Bumps [mypy](https://github.com/python/mypy) from 0.991 to 1.0.1.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v0.991...v1.0.1)
---
updated-dependencies:
- dependency-name: mypy
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix type errors
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
|
|
|
| |
This adds an implementation for SPARQL DESCRIBE queries, using the built-in `cbd` method. I see there are several issues and PRs for DESCRIBE implementation. I believe this should close #479 and should resolve #1913, or at least pick up where it left off. It should also resolve #1096.
This implementation should support the full SPARQL specification for DESCRIBE; either explicit IRIs can be provided (with no WHERE clause), or variables projected from a WHERE clause can be provided, or variables projected from a WHERE clause AND explicit IRIs can be provided. If a WHERE clause is provided, it should be evaluated the same way as it would for a SELECT DISTINCT query (including dataset specifications).
The expected results for the test cases provided match the behaviour seen when running the same queries against the same data using ARQ.
A possible future extension would be to add a global option (similar to `rdflib.plugins.sparql.SPARQL_LOAD_GRAPHS`) to change the method used to describe resources instead of always using CBD.
|
|
|
| |
Altering order in which aggregate variable aliases are renamed to user-defined variable names to ensure that when defining a `VALUES` pattern outside of a `GROUP BY`, the variables in the query are correctly joined to those defined in the `VALUES` pattern.
|
|
|
|
|
|
|
| |
Fixing use of `SELECT *` in sub-select within `SELECT *` parent query as discovered in https://github.com/RDFLib/rdflib/issues/1722.
Now when an instance of `SELECT *` is encountered, the query tree/plan builder now correctly considers the projected variables of any sub-select statements when deciding which variables should be projected out.
Fixes <https://github.com/RDFLib/rdflib/issues/1722>.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
into pyproject.toml (#2187)
Replaced setuptools with poetry which provides simpler dependency management, dependency locking and virtual environment management.
This makes it easier for new developers to get a sane development environment (i.e. just `poetry install`) and also makes our CI pipeline more reproducable as now most development tools and libraries will only change through dependabot which will reduce the build failures on main and make breaking changes in dependencies more visible with dependabot PRs.
Other changes:
- moved isort, mypy and coverage config from setup.cfg to pyproject.toml
- Changes in the generated rdflib wheel:
- removed wheel extras related to development (i.e. `tests`, `docs`, `dev`) as poetry dependency groups should be used instead.
- Added an extra group `lxml` with a dependency on lxml.
- Added version ranges to dependencies, and tests to ensure these version ranges are valid.
- Changes to tests:
- Change pyparsing related tests so they work with older pyparsing.
- Made the html5lib related tests conditional on html5lib existing.
- Runtime changes:
- Use importlib.metadata to set `__version__`.
- Changed the devcontainer and related config to avoid contamination and interference with the host environment.
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
Co-authored-by: Edmond Chuc <edmond.chuc@outlook.com>
|
|
|
|
|
|
|
| |
(#2180)
The SPARQLConnector instance created by a SPARQLStore instance providing endpoint url + query string params using POST method did not append the query string to endpoint url.
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
| |
Since the names of empty graphs do not appear in the serialization, do not consider them for namespace issues.
|
|
|
| |
The the base URI passed to _prep_sources was being overwritten in anticipation of processing inner nestings, but this caused problems when processing multiple inputs. Changed the assignment to `base` to `new_base`.
|
|
|
|
|
|
| |
Copy kwargs dict to prevent POST headers to be used in GET request and vice versa.
Co-authored-by: Mark van der Pas <mark.van.der.pas@semaku.com>
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* compute_qname handle case where name could be unbound
In NamespaceManager.compute_qname it is possible for the variable name
to be unbound when a namespace is used as a predicate e.g. ns2: and
the namespace itself cannot be split any further.
Fixing this usually just kicks the can down the road because the only
reason to hit this condition is that you are trying to serialize a
predicate that has no valid NCName form into an xml format that
requires one, making it impossible to complete the conversion.
That said, this fix should make it easier to understand what is going
on by unmasking the real issue.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test_roundtrip.py and n3roundtrip suite add unbound name test
The new test graph will produce the UnboundLocalError with the old
code that is missing the name = '' assignment. The new code correctly
produces the xfail ValueError when trying to convert to xml.
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add the ability to pass a NamespaceManager to the n3 method on Paths just like for Terms
* chore: add some more type hints and ignores
Sadly the __mul__ dunder is not defined in a way that mypy can find it,
so there are some type errors that arise from using it. I added
`type: ignore` comments for those.
I also added some more type hints, which while not strictly necessary,
does contribute to more complete type hint coverage.
Co-authored-by: Iwan Aucamp <aucampia@gmail.com>
|
|
|
|
|
|
| |
New mypy version is reporting new errors.
In the long run we need to switch to poetry so we can better control
this.
|
|
|
|
|
| |
Add encoding "charset=UTF-8" to Content-Type header in `SPARQLConnector.update()` request.
Fixes #2095
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We are still using a somewhat outmoded name for our default branch, this
PR captures the decision to change it as a decision record and prepares
the rest of the repo for this change.
Once merged, the main branch of RDFLib will be renamed using the
procedure documented [here](https://github.com/github/renaming).
Other changes:
- Add area for decision records (this may be merged in another PR before
this one).
- Remove duplicated sections from `docs/upgrade4to5.rst` as they cause
problems when using `sphinx.ext.autosectionlabel` and serve no
purpose.
|
|
|
| |
Add an xfail test to reproduce issue #2011, confirmed with both pyoxigraph and Jena.
|
| |
|
|
|
|
|
|
|
| |
Add type hints to `rdflib.query` and result format implementations, also
add/adjust ignores and type hints in other modules to accommodate the
changes.
This does not include any runtime changes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The TriG serializer was only considering BNode references inside a single
graph and not counting the BNodes subjects as references when considering if a
BNode should be serialized as unlabeled blank nodes (i.e. `[ ]`), and as a
result it was serializing BNodes as unlabeled if they were in fact referencing
BNodes in other graphs.
One caveat of this change is that some RDF Datasets may be serialized
less succinctly in that unlabeled blank nodes would not be used nodes where it is
technically possible to use them. This can be trivially fixed, but a trivial fix
increases the computational complexity of serialization significantly.
Other changes:
- Removed the roundtrip xfail that this change fixed.
- Added another roundtrip test which has various combinations of BNode
references across graphs in a dataset, this test fails for JSON-LD
however, so while this change removes one xfail it also now adds
another.
- Set the default indent_size and style in `.editorconfig` as to avoid
relying on undefined system defaults.
|
|
|
|
|
|
|
|
|
| |
system (#2096)
This is so that it is simpler and easier to do something with all plugins without treating different plugins in different ways.
Also remove test code that tries to accommodate the plugin being present but not usable.
Fixes #1816
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
More or less complete type hints for `rdflib.plugins.sparql.algebra` and
`rdflib.plugins.sparql.operators`.
This does not change runtime behaviour.
Other changes:
- Fixed line endings of `test/test_issues/test_issue1043.py`
and `test/test_issues/test_issue910.py`.
- Removed a type hint comment that was present in rdflib/plugins/sparql/algebra.py
This is split-off from https://github.com/RDFLib/rdflib/pull/1850.
|
|
|
|
|
|
| |
Confirm various content type values are handled correctly with CONSTRUCT
queries and SPARQLStore.
This aims to confirm that #1195 is fixed.
|
|
|
| |
Fairly straight forward conversions with minimal changes.
|
|
|
|
|
|
|
|
|
|
|
|
| |
More or less complete type hints for the rdflib.graph module.
Other changes:
- Improved/simplified type hints in `rdflib.store` and store plugins.
- Add type ignores for various type errors that occur with the type
hints.
This is split-off from <https://github.com/RDFLib/rdflib/pull/1850>.
This PR does not change runtime behaviour.
|
|
|
|
|
|
|
|
| |
Fixed the generation of VALUES block for federated queries.
The values block was including non-variable values like BNodes which resulted
in invalid queries.
Closes #2079.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Check datatype against `None` instead of checking it's truthiness (i.e.
`if datatype is not None:` instead of `if datatype:`).
Checking truthiness instead of `is not None` causes a blank string to
be treated the same as None. The consequence of this was that
`Literal.datatype` could be a `str`, a `URIRef` or `None`, instead of
just a `URIRef` or `None` as was seemingly intended.
Other changes:
- Changed the type of `Literal.datatype` to be `Optional[URIRef]`
instead of `Optional[str]` now that `str` will always be converted to
`URIRef` even if it is a blank string.
- Changed `rdflib.util._coalesce` to make it easier and safer to use
with a non-`None` default value.
- Changed `rdflib.util` to avoid issues with circular imports.
|