| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
| |
xdiff is a dependency (from git core) and more properly belongs in the
'deps' directory. Move it there, and add a stub for cmake to resolve
xdiff from the system location in the future. (At present, bundled xdiff
remains hardcoded.)
|
|
|
|
|
| |
Add `OPENSSL_API_COMPAT=0x10100000L` since we use the now-deprecated
low-level hash functions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uninitialised memory
slide_hash knowingly reads (possibly) uninitialised memory, see comment
lower down about prev[n] potentially being garbage. In this case, the
result is never used - so we don't care about MSAN complaining about
this read.
By adding the no_sanitize("memory") attribute, clients of zlib won't
see this (unnecessary) error when building and running with
MemorySanitizer. An alternative approach is for clients to build zlib
with -fsanitize-ignorelist=... where the ignorelist contains something
like 'fun:slide_hash'. But that's more work and needs to be redone
for any and all CI systems running a given project with MSAN. Adding
this annotation to zlib's sources is overall more convenient - but
also won't affect non-MSAN builds.
This specific issue was found while running git's test suite, but has
also been reported by other clients, see e.g. #518.
|
|
|
|
|
| |
The `crc32_combine_gen64` missed a prototype in our define path.
Add one.
|
| |
|
|
|
|
|
|
| |
https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt
Modify guard to declare dummy function.
|
| |
|
|\
| |
| | |
Fix a long long that crept past
|
| |
| |
| |
| | |
This should be propagated to upstream.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Also applies to *_BINARY_DIR.
This effectively reverts 84083dcc8bd41332ccac9d7b537f3e254d79011c,
which broke all users of libgit2 that use it as a CMake subdirectory
(via `add_subdirectory()`). This is because CMAKE_SOURCE_DIR refers
to the root-most CMake directory, which in the case of
`add_subdirectory()` is a parent project to libgit2 and thus the paths
don't make any sense to the configuration files. Corollary,
CMAKE_SOURCE_DIR only makes sense if the CMake project is always the
root project - which can rarely be guaranteed.
In all honesty, CMake should deprecate and eventually remove
CMAKE_SOURCE_DIR and CMAKE_BINARY_DIR. It's been the source of headaches
and confusion for years, they're rarely useful over
CMAKE_CURRENT_(SOURCE|BINARY)_DIR or PROJECT_(SOURCE|BINARY)_DIR,
and they cause a lot of confusing configuration and source
code layouts to boot.
Any time they are used, they break `add_subdirectory()` almost 100% of
the time, cause confusing error messages, and hide subtle bugs.
|
| |
| |
| |
| |
| |
| | |
Instead of using the project-specific `libgit2_SOURCE_DIR` and
`libgit2_BINARY_DIR` variables, use `CMAKE_SOURCE_DIR` and
`CMAKE_BINARY_DIR`.
|
| | |
|
|/
|
|
|
| |
Ensure that we always use lowercase function names, and that we do not
have spaces preceding open parentheses, for consistency.
|
|
|
|
| |
The ntlmclient dependency can now dynamically load OpenSSL.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change builds libgit2 using Chromium's zlib implementation by
invoking cmake with `-DUSE_BUNDLED_ZLIB=ON -DUSE_CHROMIUM_ZLIB=ON`,
which is ~10% faster than the bundled zlib for the core::zstream suite.
This version of zlib has some optimizations:
a) Decompression (Intel+ARM): inflate_fast, adler32, crc32, etc.
b) Compression (Intel): fill_window, longest_match, hash function, etc.
Due to the introduction of SIMD optimizations, and to get the maximum
performance out of this fork of zlib, this requires an x86_64 processor
with SSE4.2 and CLMUL (anything Westmere or later, ~2010). The Chromium
zlib implementation also supports ARM with NEON, but it has not been
enabled in this patch.
Performance
===========
TL;DR: Running just `./libgit2_clar -score::zstream` 100 times in a loop
took 0:56.30 before and 0:50.67 after (~10% reduction!).
The bundled and system zlib implementations on an Ubuntu Focal system
perform relatively similar (the bundled one is marginally better due to
the compiler being able to inline some functions), so only the bundled
and Chromium zlibs were compared.
For a more balanced comparison (to ensure that nothing regressed
overall), `libgit2_clar` under `perf` was also run, and the zlib-related
functions were compared.
Bundled
-------
```shell
cmake \
-DUSE_BUNDLED_ZLIB=ON \
-DUSE_CHROMIUM_ZLIB=OFF \
-DCMAKE_BUILD_TYPE="RelWithDebInfo" \
-DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
-GNinja \
..
ninja
perf record --call-graph=dwarf ./libgit2_clar
perf report --children
```
```
Samples: 87K of event 'cycles', Event count (approx.): 75923450603
Children Self Command Shared Objec Symbol
+ 4.14% 0.01% libgit2_clar libgit2_clar [.] git_zstream_get_output_chunk
+ 2.91% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output
+ 0.69% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output (inlined)
0.17% 0.00% libgit2_clar libgit2_clar [.] git_zstream_init
0.02% 0.00% libgit2_clar libgit2_clar [.] git_zstream_reset
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_eos
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_done
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free (inlined)
Samples: 87K of event 'cycles', Event count (approx.): 75923450603
Children Self Command Shared Objec Symbol
+ 3.12% 0.01% libgit2_clar libgit2_clar [.] deflate
+ 2.65% 1.48% libgit2_clar libgit2_clar [.] deflate_slow
+ 1.60% 0.55% libgit2_clar libgit2_clar [.] inflate
+ 0.53% 0.00% libgit2_clar libgit2_clar [.] write_deflate
0.49% 0.36% libgit2_clar libgit2_clar [.] inflate_fast
0.46% 0.02% libgit2_clar libgit2_clar [.] deflate_fast
0.19% 0.19% libgit2_clar libgit2_clar [.] inflate_table
0.16% 0.01% libgit2_clar libgit2_clar [.] inflateInit_
0.15% 0.00% libgit2_clar libgit2_clar [.] inflateInit2_ (inlined)
0.10% 0.00% libgit2_clar libgit2_clar [.] deflateInit_
0.10% 0.00% libgit2_clar libgit2_clar [.] deflateInit2_
0.03% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateReset
0.02% 0.00% libgit2_clar libgit2_clar [.] inflateEnd
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateEnd
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateResetKeep
0.01% 0.01% libgit2_clar libgit2_clar [.] inflateReset2
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateResetKeep (inlined)
```
Chromium
--------
```shell
cmake \
-DUSE_BUNDLED_ZLIB=ON \
-DUSE_CHROMIUM_ZLIB=ON \
-DCMAKE_BUILD_TYPE="RelWithDebInfo" \
-DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
-GNinja \
..
ninja
perf record --call-graph=dwarf ./libgit2_clar
perf report --children
```
```
Samples: 97K of event 'cycles', Event count (approx.): 80862210917
Children Self Command Shared Objec Symbol
+ 3.31% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output_chunk
+ 2.27% 0.01% libgit2_clar libgit2_clar [.] git_zstream_get_output
+ 0.55% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output (inlined)
0.18% 0.00% libgit2_clar libgit2_clar [.] git_zstream_init
0.02% 0.00% libgit2_clar libgit2_clar [.] git_zstream_reset
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_done
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free
Samples: 97K of event 'cycles', Event count (approx.): 80862210917
Children Self Command Shared Objec Symbol
+ 2.55% 0.01% libgit2_clar libgit2_clar [.] deflate
+ 2.25% 1.41% libgit2_clar libgit2_clar [.] deflate_slow
+ 1.10% 0.52% libgit2_clar libgit2_clar [.] inflate
0.36% 0.00% libgit2_clar libgit2_clar [.] write_deflate
0.30% 0.03% libgit2_clar libgit2_clar [.] deflate_fast
0.28% 0.15% libgit2_clar libgit2_clar [.] inflate_fast_chunk_
0.19% 0.19% libgit2_clar libgit2_clar [.] inflate_table
0.17% 0.01% libgit2_clar libgit2_clar [.] inflateInit_
0.16% 0.00% libgit2_clar libgit2_clar [.] inflateInit2_ (inlined)
0.15% 0.00% libgit2_clar libgit2_clar [.] deflateInit_
0.15% 0.00% libgit2_clar libgit2_clar [.] deflateInit2_
0.11% 0.01% libgit2_clar libgit2_clar [.] adler32_z
0.09% 0.09% libgit2_clar libgit2_clar [.] adler32_simd_
0.05% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.05% 0.00% libgit2_clar libgit2_clar [.] deflate_read_buf
0.03% 0.00% libgit2_clar libgit2_clar [.] inflateEnd
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateReset
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateEnd
0.01% 0.01% libgit2_clar libgit2_clar [.] inflateReset2
0.01% 0.00% libgit2_clar libgit2_clar [.] inflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] adler32
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateResetKeep (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateResetKeep
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateStateCheck (inlined)
```
|
|
|
|
|
|
| |
Without this, mbedTLS installs in non-default install locations
that are otherwise found by the `FindmbedTLS.cmake` module are not
found by the C preprocessor at compile time.
|
|
|
|
|
| |
Update ntlm to include an htonll that is not dependent on system
libraries.
|
| |
|
| |
|
|
|
|
|
| |
We included their COPYING file, which was _not_ in fact their license.
Add the LICENSE file as well.
|
|
|
|
|
| |
We explicitly pass win32 & macOS, although some old version might not
have it.
|
|
|
|
| |
Use B_HOST_TO_BENDIAN_INT64 for that.
|
|\
| |
| | |
Make the tests pass cleanly with MemorySanitizer
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This change:
* Initializes a few variables that were being read before being
initialized.
* Includes https://github.com/madler/zlib/pull/393. As such,
it only works reliably with `-DUSE_BUNDLED_ZLIB=ON`.
|
|/
|
|
|
|
|
|
|
|
|
| |
This change makes the code pass the libgit2 tests cleanly when
MSan/UBSan are enabled. Notably:
* Changes malloc/memset combos into calloc for easier auditing.
* Makes `write_buf` return early if the buffer length is empty to avoid
arithmetic with NULL pointers (which UBSan does not like).
* Initializes a few arrays that were sometimes being read before being
written to.
|
|
|
|
|
|
|
|
|
|
|
| |
We currently use `FILE(GLOB ...)` in most places to find source and
header files. This is problematic in that the order of files returned
depends on the operating system's directory iteration order and may thus
not be deterministic. As a result, we link object files in unspecified
order, which may cause the linker to emit different code across runs.
Fix this issue by sorting all code used as input to the libgit2 library
to improve the reliability of reproducible builds.
|
| |
|
|
|
|
|
|
|
| |
We're currently doing unnecessary work to auto-detect backends even if
the functionality is disabled altogether. Let's fix this by removing the
extraneous FOO_BACKEND variables, instead letting auto-detection modify
the variable itself.
|
|
|
|
|
| |
The `CC_MD4()` function has been deprecated in macOS 10.15. Silence this
warning for now until we implement a proper fix.
|
|
|
|
|
|
|
|
|
|
|
|
| |
In commit 3828ea67b (deps: ntlmclient: fix missing htonll symbols on
FreeBSD and SunOS, 2020-02-21), we've fixed compilation on BSDs due to
missing `htonll` wrappers. While we are now using `htobe64` for both
Linux and OpenBSD, we decided to use `bswap64` on FreeBSD. While correct
on little endian systems, where we will swap from little- to big-endian,
we will also do the swap on big endian systems. As a result, we do not
use network byte order on such systems.
Fix the issue by using htobe64, as well.
|
|
|
|
|
|
|
|
|
| |
The ntlmclient dependency defines htonll on Linux-based systems, only.
As a result, non-Linux systems will run into compiler and/or linker
errors due to undefined symbols.
Fix this issue for FreeBSD, OpenBSD and SunOS/OpenSolaris by including
the proper headers and defining the symbol accordingly.
|
|
|
|
|
|
|
| |
Pull in commit
https://github.com/ethomson/ntlmclient/commit/e7b2583e1bc28c33c43854e7c318e859b4e83bef
to fix #5353.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The ntlmclient dependency has quite a lot of places with implicit
fallthroughs. As at least modern GCC has enabled warnings on
implicit fallthroughs by default, the developer is greeted with a
wall of warnings when compiling that dependency.
Disable implicit fallthrough warnings for ntlmclient to fix this
issue.
|
|
|
|
|
|
| |
When the end of the host is reached, and we're at the colon separating
the host with the port (ie, there is no numeric port) then do not error.
This is allowed by RFC 3986.
|
|
|
|
| |
Include https://github.com/ethomson/ntlmclient as a dependency.
|
|
|
|
|
| |
Our PCRE dependency has uncommented fallthroughs in switch statements.
Turn off warnings for those in the PCRE code.
|
|
|
|
| |
PCRE includes compatibility functions that may go unused. Don't warn.
|
|
|
|
|
|
|
| |
Use PCRE 8.42 as the builtin regex implementation, using its POSIX
compatibility layer. PCRE uses ASCII by default and the users locale
will not influence its behavior, so its `regcomp` implementation is
similar to `regcomp_l` with a C locale.
|
|
|
|
| |
It seems like MingW64's size_t is defined differently than in Linux.
|
|
|
|
|
|
|
|
| |
While our contribution guide tries to make clear the licenses that apply
to libgit2, it does not make clear that different licenses apply to our
bundled dependencies. Make this clear by listing each dependency
together with the licenses that they are governed by. Furthermore,
bundle the complete license texts next to the code they apply to.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GCC 7 has introduced new warnings for implicit fallthrough in switch
statements. Whenever there is no branch in a case block, GCC will watch
out for some heuristics which indicate that the implicit fallthrough is
intended, like a "fallthrough" comment. The third-party http-parser code
manages to trick this heuristic in one case, even though there is a
"FALLTHROUGH" comment. Fortunately, GCC has also added a strictness
level to the -Wimplicit-fallthrough diagnostic, such that we can loosen
this heuristic and make it more lax.
Set -Wimplicit-fallthrough=1 in http-parser's CMake build instructions,
which is the strictest level that gets rid of the warning. This level
will treat any kind of comment as a "fallthrough" comment, which
silences the warning.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current version of zlib bundled with libgit2 is version 1.2.8. This
version has several CVEs assigned:
- CVE-2016-9843
- CVE-2016-9841
- CVE-2016-9842
- CVE-2016-9840
Upgrade the bundled version to the current release 1.2.11, which has
these vulnerabilities fixes.
|
|
|
|
|
|
|
|
| |
Include the constants for `WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_1` and
`WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_2` so that they can be used by mingw.
This updates both the `deps/winhttp` framework (for classic mingw) and
adds the defines for mingw64, which does not use that framework.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Due to our split of CMake files into multiple modules, we had to replace
some uses of the `${CMAKE_CURRENT_SOURCE_DIR}` and
`${CMAKE_CURRENT_BINARY_DIR}` variables and replace them with
`${CMAKE_SOURCE_DIR}` and `${CMAKE_BINARY_DIR}`. This enabled us to
still be able to refer to top-level files when defining build
instructions inside of a subdirectory.
When replacing all variables, it was assumed that the absolute set of
variables is always relative to the current project. But in fact, this
is not the case, as these variables always point to the source and
binary directory as given by the top-levl project. So the change
actually broke the ability to include libgit2 directly as a subproject,
as source files cannot be found anymore.
Fix this by instead using project-specific source and binary directories
with `${libgit2_SOURCE_DIR}` and `${libgit2_BINARY_DIR}`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Our bundled deps are being built as simple static libraries which are
then linked into the libgit2 library via `TARGET_LINK_LIBRARIES`. While
this works for a dynamically built libgit2 library, using this function
to link two static libraries does not have the expected outcome of
merging those static libraries into one big library. This leads to
symbols of our bundled deps being undefined in the resulting libgit2
archive.
As we have bumped our minimum CMake version to 2.8.11, we can now easily
make use of object libraries for our bundled dependencies. So build
instructions are still self-contained inside of the dependency
directories and the resulting object libraries can just be added to the
LIBGIT2_OBJECTS list, which will cause them to be linked into the final
resulting static library. This fixes the issue of undefined symbols.
|
|
|
|
|
| |
Extract code required to build the zlib library into its own
CMakeLists.txt, which is included as required.
|