| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some http response header checks were removed such as checking for
Connection and Cache-Control. These headers are not guarenteed to be
received and depend on proxy and libmicrohttpd versions. Checking for
the existance of Content-Length and DEBUGINFOD-* headers is sufficient
since Content-Length is added upon creation of an MHD_Response object
and DEBUGINFOD-* are added manually.
(source on Content-Length being added:
https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html#
microhttpd_002dresponse-headers )
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
While checking PR27277 on some buildbots, greping would fail in
run-debuginfod-response-headers.sh. This was because querying the
debuginfod server occurs after checking if the responseh headers had
arrived, leaving the possibility to leave the querying loop before
outputting the headers which caused the grep failure. Querying now
occurs before checking if response headers have arrived, so that they
will certainly be printed and grep will find them.
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Allow users, with enough verbosity, to print the HTTP response headers
upon retrieving a file. These files may include several custome http
response headers such as X-DEBUGINFOD-FILE, X-DEBUGINFOD-SIZE, and
X-DEBUGINFOD-ARCHIVE. These headers are added from the daemon, in
debuginfod.cxx.
E.g output:
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 4095072
Cache-Control: public
Last-Modified: Thu, 09 Sep 2021 19:06:40 GMT
X-FILE: debuginfod
X-FILE-SIZE: 4095072
Content-Type: application/octet-stream
Date: Fri, 10 Sep 2021 16:38:06 GMT
https://sourceware.org/bugzilla/show_bug.cgi?id=27277
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, urls containing '/', so most urls, would escape '/' to %2F,
which is undesirable for use in other libraries which may escape
differently. This patch escapes the '/' and replaces all of them
ensuring there are no %2Fs sent.
Some inefficiencies within the code were fixed, such as changing constant
operations of a while loop within a for loop to a while loop outside of
a for loop. Also strlen is no longer used within the loop, simplifying
the interior operations to mere arithmetic.
https://sourceware.org/bugzilla/show_bug.cgi?id=28034
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
debuginfod's scan and groom operations (thread_main_scanner,
thread_main_fts_source_paths) are intended to be mutually exclusive,
as a substitute for more complicated sql transaction batching. (This
is because scanning / grooming involves inserting or deleting data
from multiple related tables.)
The workq class that governs this in debuginfod.cxx has a problem: if
the workq just becomes empty, its sole entry pulled by a scanner
thread in response to a wait_front(), an 'idler' groomer thread is
ALSO permitted to run, because there is no indication as to when the
scanner thread operation finishes, only when it starts.
Extending the workq with a counter ("fronters") to track any active
scanning activity (even if the workq is empty) lets us block idlers
groomers a little longer.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
|
|
|
|
|
|
| |
Without the endl the next log message will not start on its own line.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix some issues with the error reimplementation to make it match
the specification for error(3).
Flush stdout before printing to stderr. Also flush stderr afterwards,
which is not specified in the man page for error(3), but is what
bionic does.
error(3) prints strerror(errnum) if and only if errnum is nonzero,
but verr prints strerror(errno) unconditionaly. When errnum is nonzero
copy it to errno and use verr, and when it is not set use verrx that
doesn't print errno.
error(3) only exits if status is nonzero, but verr exits uncondtionally.
Use vwarn/vwarnx when status is zero, which don't exit.
Signed-off-by: Colin Cross <ccross@google.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are three "fake CUs" that are associated with .debug_loc,
.debug_loclist and .debug_addr. These fake CUs are used for "fake
attributes" to provide values that are stored in these sections
instead of in the .debug_info section. These fake CUs didn't have the
address size, offset size and DWARF version set. This meant that
values that depended on those properties might not be interpreted
correctly. One example was the value associated with a DW_OP_addrx
(which comes from the .debug_addr section).
Add a testcase using varlocs to test that addresses can correctly be
retrieved for gcc/clang, DWARF4/5 and 32/64 bits objects.
https://sourceware.org/bugzilla/show_bug.cgi?id=28220
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The set -o errtrace made run-debuginfod-fd-prefetch-caches.sh
fail. On some systems. Add set -o functrace to make it fail consistently.
The failure is because the grep -c for in the log file fails (it
returns zero). Fix this by using || true. But this is only a
workaround. It makes the test pass, but only because all values are
always zero. The test doesn't currently test anything.
Also make sure that err and cleanup are only executed once.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
| |
This fixes the following compilation warning:
findtextrel.c:184:1: warning: stack usage might be unbounded [-Wstack-usage=]
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There were still three tests that shared a cache between the servers
and client that queried those servers. Give them all separate caches.
Also the error handler for debuginfod tests wasn't called when a
command inside a function failed. Since testrun is a function, there
would be no metrics or error log files listed if the testrun command
failed. Making it hard to see what went wrong. Fix this by using
set -o errtrace
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
| |
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
|
|
|
|
|
|
|
| |
Similar to other x* functions, xasprintf is like asprintf except that
it dies in case of an error.
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
|
|
|
|
|
|
|
| |
Return values of functions returning "void *", e.g. calloc, malloc,
realloc, xcalloc, xmalloc, and xrealloc, do not need explicit casts.
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
|
|
|
|
|
|
|
|
|
|
| |
While -Wstack-usage= is already excluded from AM_CFLAGS for various
tools in src using *_no_Wstack_usage variables, this obviously does not
help when LTO is enabled, so add -Wno-error=stack-usage= to AM_LDFLAGS
for linking tools in src.
References: https://sourceware.org/bugzilla/show_bug.cgi?id=24498
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
|
|
|
|
|
|
|
|
|
| |
At startup the debuginfod server does a scan and groom cycle. Make
sure to wait for that before making any changes to the scan dirs. And
not just right before triggering a new one with SIGUSR1 for scan or
SIGURS2 for groom.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
| |
At startup the debuginfod server does a scan and groom cycle. Make
sure to wait for that before triggering a new one with SIGUSR1 or
SIGURST2.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
| |
A couple of test set DEBUGINFOD_URLS before starting a debuginfd
server causing the server to query itself or a nonexisting debuginfod
server as delegate. In most cases it should be set after, except for
the testcase that explicitly checks for errors when using an invalid
URL.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
| |
On error we would only print the metrics of one port (twice) because
of a typo. Also PORT1 and PORT2 could be equal because of a logic
error. Fix the typo and simplify the port selection by using
non-overlapping ranges to select PORT1 and PORT2.
|
| |
|
|
|
|
|
|
|
|
| |
Always set DEBUGINFOD_CACHE_PATH to an unique (new) directory
and make sure that each debuginfod invocation uses a new sqlite
database.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
| |
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
tests/run-debuginfod-find.sh was a massive test script with many broadly
varying tests. This caused the test script to fail when any number of
things went wrong and because of its intertwined nature, detecting the
source of a failure could be difficult. The size of the test script
also meant many unrelated tests were run making the testing process
unnecessarily lengthy.
This patch fractures tests/run-debuginfod-find.sh into smaller, more
manageable individual test script files. This ensures that when failure
occurs, a programmer can easily determine where their patch went
wrong. It also allows programmers to specify exactly which tests to
run, making testing more efficient. Redundancies are also reduced by
placing code in tests/debuginfod-subr.sh.
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is true that Clang does not support all gnu99 extensions [1],
but not all of them are used in the codebase and over time there
have been code cleanup efforts to improve Clang support.
For example after commit 779c57ea ("readelf: Pull advance_pc()
in file scope") there are no more nested function declarations
and elfutils now builds fine with Clang.
So in the interest of enabling Clang builds we remove the only
remaining blocker: the configure checks for nested functions and
variable length arrays which are also unused.
Considering mixed decls and code is also part of c99 standard,
the entire check becomes redundant and we can just replace
AC_PROG_CC -> AC_PROG_CC_C99.
[1] https://sourceware.org/bugzilla/show_bug.cgi?id=24964
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
debuginfod opens the database twice, once in read/wrote and once in
read-only mode. This means the magic ":memory:" in-memory database
cannot be used as is because the two connections don't really share
the underlying database. Fix this by turning ":memory:" into
":file::memory:?cache=shared" which makes the in-memory database
shared. See https://sqlite.org/inmemorydb.html
Document this in debuginfod.8 and make some tests use -d :memory:
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
| |
Sharing the database between the two debuginfod instances that forward
queries to each other causes issues. Make both debuginfod instances
use a new fresh database.
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If someone misconfigures a debuginfod federation to have loops, and
a nonexistent buildid lookup is attempted, bad things will happen,
as is documented.
This patch aims to reduce the risk by adding an option to debuginfod
that functions kind of like an IP packet's TTL: a limit on the length of
XFF: header that debuginfod is willing to process. If X-Forwarded-For:
exceeds N hops, it will not delegate a local lookup miss to upstream
debuginfods.
Commit ab38d167c40c99 causes federation loops for non-existent resources
to result in multiple temporary deadlocks, each lasting for
$DEBUGINFOD_TIMEOUT seconds. Since concurrent requests for each unique
resource are now serialized, federation loops can result in one server
thread waiting to acquire a lock while the server thread holding the
lock waits for the first thread to respond to an http request.
This PR can help protect against the above multiple temporary deadlocks
behaviour. Ex. if --forwarded-ttl-limit=0 then the timeout behaviour of
local loops should be avoided.
https://sourceware.org/bugzilla/show_bug.cgi?id=27917
Signed-off-by: Di Chen <dichen@redhat.com>
|
|
|
|
|
|
|
| |
This new file was supposed to be part of 4d6dd0e5a "lib: avoid potential
problems with `-fno-common`".
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
| |
This properly homes the fallback function into a translation unit rather
than trying to define an inline common definition for the fallback path.
The intent of the original approach was to actually simply avoid adding
a new source file that is used for the fallback path. However, that may
cause trouble with multiple definitions if the symbol does not get vague
linkage (which itself is not particularly great). This simplifies the
behaviour at the cost of an extra inode.
|
|
|
|
|
|
|
|
| |
This definition was in the fallback path, where `sys/cdefs.h` is not
available. Now that we have a single path through here, this macro gets
defined, though is unused. Remove the unused macro definition.
Signed-off-by: Saleem Abdulrasool <abdulras@google.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Introduce a configure time check for the presence of `error.h`. In the
case that `error.h` is not available, we can fall back to `err.h`.
Although `err.h` is not a C standard header (it is a BSD extension),
many libc implementations provide. If there are targets which do not
provide an implementation of `err.h`, it would be possible to further
extend the implementation to be more portable.
This resolves bug #21008.
Signed-off-by: Saleem Abdulrasool <abdulras@google.com>
|
|
|
|
|
|
|
|
|
| |
`error.h`'s inclusion was centralised into the `system.h` header. As
the implementation currently includes `system.h` already, the inclusion
of `error.h` is unnecessary. This prepares for a future portability
change to allow elfutil to build with alternate libc implementations.
Signed-off-by: Saleem Abdulrasool <abdulras@google.com>
|
|
|
|
|
|
|
|
|
|
| |
This header is a BSD header that is also available in glibc. However,
this is a not a standard C header and was used for `__CONCAT`. Because
this is not a standard header, not all libc implementations provide the
header. Remove the usage of the header and always use the previously
fallback path. This is needed in order to build with musl.
Signed-off-by: Saleem Abdulrasool <abdulras@google.com>
|
|
|
|
|
|
|
| |
The template rpm spec file needs to include the new .7 page in all the
subrpms whose man pages may .so it.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
The initial code for bug #27673 accidentally nuked all buildid service
concurrency, not just identical concurrent requests. Correct this
with one-liner patch. Observing the effect in the automated testsuite
is difficult, so hand-tested against large requests and short ones,
run in an interleaved way.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
|
|
|
|
|
|
|
| |
The debuginfod-client-config.7 shouldn't be included twice in
notrans_dist_man7_MANS.
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Create a new file, debuginfod-client-config.7, that holds all
environment variables and cache control files related info. Get rid of
repetitive definitions in three other files, instead, those files will
include the content of new file. Any future modification related to
environment variables and cache files will only require changes in one
file.
Signed-off-by: Alice Zhang <alizhang@redhat.com>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
glibc 2.34 calls pthread_kill from the raise function. Before raise
directly called the (tg)kill syscall. So allow pthread_kill to be the
first frame in a backtrace where raise is expected. Also change some
asserts to fprintf plus abort to make it more clear why the testcase
fails.
https://sourceware.org/bugzilla/show_bug.cgi?id=28190
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
DEBUGINFOD_TIMEOUT is a good way to catch servers that are too slow to
*start* transmitting a file. But we have no way of limiting total
download time or space. A user might prefer to have his debugger fetch
only quick & small files, and make do without the bigger ones. Some
transitive dependencies of e.g. gnome programs are huge: 3GB of LLVM
debuginfo, 1GB of webkitgtk, etc. etc.
DEBUGINFOD_MAXSIZE and DEBUGINFOD_MAXTIME were added to dictate the
max download size and time of a debuginfod client. DEBUGINFOD_MAXSIZE
is handled server-side and is sent using the http header:
X-DEBUGINFOD-MAXSIZE. The client side then checks to ensure this maxsize
has been respected.
https://sourceware.org/bugzilla/show_bug.cgi?id=27982
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
| |
Signed-off-by: Hayatsu Shunsuke <hayatsu.shunsuke@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Gazing at server logs, one sees a minority of clients who appear to have
duplicate query traffic coming in: the same URL, milliseconds apart.
Chances are the user accidentally doubled her $DEBUGINFOD_URLS somehow,
and the client library is dutifully asking the servers TWICE. Bug #27863
reduces the pain on the servers' CPU, but dupe network traffic is still
being paid. We should reject sending outright duplicate concurrent
traffic.
The urls are now simply removed upon finding a duplicate after url
construction.
https://sourceware.org/bugzilla/show_bug.cgi?id=27983
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
| |
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When requesting some source files, some URL-inconvenient chars
sometimes pop up. Example from f33 libstdc++:
/buildid/44d8485cb75512c2ca5c8f70afbd475cae30af4f/source/usr/src/debug/
gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/
libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/
condition_variable.cc
As this URL is passed into debuginfod's handler_cb, it appears that the
+ signs are helpfully unescaped to spaces by libmicrohttpd, which
'course breaks everything.
In order to ensure the server properly parses urls such as this one,
%-escape characters on the client side so that the correct url
is preserved and properly processed on the server side.
https://sourceware.org/bugzilla/show_bug.cgi?id=28034
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
| |
$PORT3's metrics are not reported on error and $PID4 was not properly
killed. This patch addresses both of those issues by reporting the
metrics of $PORT3 as $PORT1 and $PORT2 were in err() and waiting for
$PID4 to terminate before continuing with the test.
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
In most cases the last char of the sectio will be zero. Check that
first before calling memrchr. This is a minor optimization in normal
cases. But it helps asan a lot by removing the memrchr call in most
cases.
https://sourceware.org/bugzilla/show_bug.cgi?id=28101
Signed-off-by: Mark Wielaard <mark@klomp.org>
|
|
|
|
|
|
|
|
|
| |
Make advance_pc() a static function so we can get rid of another nested
function. Rename it to run_advance_pc() and use a local advance_pc()
macro to pass all the local variables. This is similar to what the
equivalent code in libdw/dwarf_getsrclines.c is doing.
Signed-off-by: Timm Bäder <tbaeder@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The debuginfod fdcache-prefetch logic has been observed to show some
degeneracies in operation. Since fdcache evictions are done
frequently, and freshly prefetched archive elements are put at the
back of lru[], each eviction round can summarily nuke things that
were just prefetched .... and are just going to be prefetched again.
It would be better to have two lru lists, or being able to insert
newly prefetched entries somewhere in the middle of the list rather
than at the very very end.
https://sourceware.org/bugzilla/show_bug.cgi?id=25978
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In debuginfod-client.c (debuginfod_query_server),insert a
goto statement for jumping back to the beginning of curl
handles set up if query fails and a non ENOENT error is returned.
Also introduced DEBUGINFOD_RETRY_LIMIT_ENV_VAR and default
DEBUGINFOD_RETRY_LIMIT(which is 2).
Correponding test has been added to tests/run-debuginfod-find.sh
Signed-off-by: Alice Zhang <alizhang@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The debuginfod -I/-X regexes operate during traversal to identify
those files in need of scanning. The regexes are not used during
grooming. This means that if from run to run, the regex changes so
that formerly indexed files are excluded from traversal, the data is
still retained in the index.
This is both good and bad. On one hand, if the underlying data is
still available, grooming will preserve the data, and let clients ask
for it. On the other hand, if the growing index size is a problem,
and one wishes to age no-longer-regex-matching index data out, there
is no way.
Let's add a debuginfod flag to use regexes during grooming.
Specifically, in groom(), where the stat() test exists, also check
for regex matching as in scan_source_paths(). Treat failure of the
regex the same way as though the file didn't exist.
Signed-off-by: Noah Sanci <nsanci@redhat.com>
|