summaryrefslogtreecommitdiff
path: root/src/libtracker-miner/tracker-miner-fs.c
Commit message (Collapse)AuthorAgeFilesLines
* libtracker-miner: Drop libtracker-miner as a public libraryCarlos Garnacho2019-12-121-2659/+0
| | | | | | | | | | | According to Debian code search, it's accumulated 0 users outside of tracker-miners. If everyone is relying on either tracker-miner-fs/rss or implementing their own minimal abstraction (TrackerMinerFS is a complex object, but the others are all fairly shallow), it does not make sense to drag this as public API anymore. This code moves into tracker-miners, and every user is expected to consume and insert data using the sparql library.
* libtracker-miner: Fix EVENT_QUEUE_ENABLE_TRACE featureSam Thursfield2019-09-211-118/+16
| | | | | I find this tracing useful sometimes when debugging test failures. However the existing code had bitrotted quite a lot. Now it works again.
* Prepare source files to documentationCorentin Noël2019-03-271-2/+2
|
* libtracker-miner: Port to G_ADD_PRIVATECarlos Garnacho2018-10-121-8/+5
| | | | | The priv pointer has been removed from object structs for all private types. For public types it's got to be kept there for ABI stability.
* libtracker-miner: Use ';' as separator for multiple update statementsCarlos Garnacho2018-08-301-2/+2
| | | | Let's stick to SPARQL1.1 correct syntax.
* libtracker-miner: Coalesce 2 CREATED eventsCarlos Garnacho2018-07-211-1/+2
| | | | | | | | | It is not even clear this is possible in real life cases, however the standalone tracker-file-notifier tests fall into this (due to IRI not being ever set, still this is an async op). In the case of 2 consecutive CREATED events on the same file, it would be dealt with in TrackerMinerFS as CREATED+ UPDATED. This was already harmless, but we can do better and swallow one of such events.
* libtracker-miner/tracker-miner-fs.c: Fix "Parent not indexed yet" errorsSam Thursfield2018-07-151-34/+65
| | | | | | | | | | | | | | | | | | | | | | | | | Commit cef502e668a640 ("Add TrackerMinerFS::move-file vmethod") introduced a regression which sometimes led to errors like this: Tracker-FATAL-WARNING: Parent 'file:///tmp/tracker-miner-fs-test-77E2LZ/recursive/4' not indexed yet This was causing tracker-miner-fs-test to fail in some cases. TrackerTaskPool assumes that there is only one task in the pool per GFile. When processing item_move() operations this wasn't true because we'd create one task for removing the existing dest_file, and another task for updating the URL of source_file to point to dest_file. Both tasks would be associated with dest_file. If the SPARQL buffer was flushed after the first task was created and before the second task was created, the second task would overwrite the first task in the ->priv->tasks hash table, so when the first task completed, the second task would be removed from the task pool without ever executing. This would mean that the URL of source_file never got updated to point at dest_file, which triggered the "Parent not indexed yet" error later on.
* libtracker-miner/tracker-miner-fs.c: Rename item_move() parameterSam Thursfield2018-07-151-8/+8
|
* libtracker-miner: Transfer attribute updates information through the eventCarlos Garnacho2018-06-231-29/+21
| | | | | | | | | | | First, seems cleaner to do it this way, as GObject data has undefined lifetime (yes, as long as the object lives, but the TrackerFileSystem may cache those). But this also fixes an unintended side effect that "attribute only" updates take precedence over full updates, events themselves may be coalesced away, but data would remain. Actually it's the other way around, if we get a full update and an attributes-only update, we may discard the second.
* libtracker-miner: Don't leak TrackerMinerFS:rootDebarshi Ray2018-02-071-0/+2
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=793061
* libtracker-miner: Remove double caching of file IRICarlos Garnacho2017-11-141-43/+11
| | | | | | This is doubly cached in the TrackerFileSystem and as GObject qdata (and obtained from the former in that case). Let's just rely on the former for all.
* libtracker*: Lower g_message()s to g_debug()Carlos Garnacho2017-11-141-2/+0
| | | | | | | | This is library code, so let's use g_debug() which obeys G_MESSAGES_DEBUG, instead of g_message() which shall be printed unless there is an special log handler that filters those out. This code may run on 3rd party code, where we can't trust we'll have a log handler that catches those from going to stdout.
* libtracker-miner: Remove unused quarksCarlos Garnacho2017-11-141-4/+0
|
* libtracker-miner: Refactor processing queuesCarlos Garnacho2017-11-141-440/+290
| | | | | | | | | | | | Replace all 4 queues for the different create/update/delete/move events with a single queue that contains generic QueueEvent structs. The GList node of the last event is stored as GFile qdata, in order to perform fast lookups when coalescing events. queue_event_coalesce() will attempt to convert any two events into less than that, it does rely on merging two events with no related events in between, those should be coalesced (or attempted to) when they arrive.
* libtracker-miner: Remove reentry counter and failed task reinsertionCarlos Garnacho2017-11-141-61/+15
| | | | | | | | | | | | | | | | TrackerFileNotifier guarantees that parent files are emitted before any children, the other usecase that this used to cover are explicit tracker_miner_fs_check_file() calls which used to also index parent directories, but it doesn't do that anymore since quite some time. So the only remaining case where we could end up with a file whose parent is neither being currently processed nor indexed is actual bugs. In that case, the bug likely won't go away by trying harder, which leads to logging for every child file, as they'll fail in cascade. Let's be less stubborn here, warn (once!) about the missing file and ditch all pending events happening on/inside it. People love filing bugs about tracker logging stuff, so I don't think bugs will go unnoticed anyway.
* libtracker-miner: Always fallback to URN query when queueing fileCarlos Garnacho2017-09-291-8/+7
| | | | | | | | | | | | Tracker may end up with nfo:FileDataObject prior to handling monitor events. Be it leftover data from previous bugs, explicit "tracker index" calls, or data from some other application. As we can't be really sure of the data consistence, always fallback to a URN query so we don't break nie:url UNIQUE constraint (inverse functional property in SPARQL parlance). https://bugzilla.gnome.org/show_bug.cgi?id=786132
* libtracker-miner: Process TrackerMinerFS elements in batchesCarlos Garnacho2017-07-091-13/+26
| | | | This prevents us from hitting too hard the main loop.
* libtracker-miner: Fix warnings if move ops happened during initial crawlingCarlos Garnacho2017-07-061-1/+10
| | | | | | | The checks to notify about indexing having finished on TrackerIndexingTree roots were mistaking ItemMovedData* with GFile*, which lead to warnings. This should be harmless, the signal might be possibly emitted before the move op is dispatched, that's all.
* libtracker-miner: Add missing ::move-file signalCarlos Garnacho2017-07-061-0/+9
| | | | | Fixes handling of moved files, since the subclass vmethod wouldn't be triggered.
* libtracker-miner: Remove writeback APICarlos Garnacho2017-07-031-392/+7
| | | | | | The only user that might ever care does already implement it itself. There is no need to provide this infrastructure that will be scarcely used in libtracker-miner API.
* libtracker-miner: Add filter_event vmethodCarlos Garnacho2017-07-021-0/+27
| | | | | | | | | This is not a signal because external users of a TrackerMiner have no business in modifying behavior at this level, this is reserved for subclasses that presumably know what they are doing. This vmethod is toggled for every event that gets received from the TrackerFileNotifier, before the file gets to hit any processing queue.
* libtracker-miner: Add connection property to TrackerFileNotifierCarlos Garnacho2017-06-291-1/+2
| | | | So it can be specified from the miner.
* libtracker-miner: Merge tracker_miner_fs_check_file[with_priority]Carlos Garnacho2017-06-291-27/+5
| | | | | | | Only check_file() remains, with an extra priority argument. The default G_PRIORITY_HIGH in the older check_file() was unintuitive, and is now explicitly specified in the org.freedesktop.Tracker1.Miner.Files.Index interface calls.
* libtracker-miner: Remove tracker_miner_fs_check_directory*()Carlos Garnacho2017-06-291-82/+0
| | | | | More unused API that is a thin wrapper to TrackerIndexingTree, just remove it.
* libtracker-miner: Remove TrackerMinerFS API to add/remove directoriesCarlos Garnacho2017-06-291-146/+0
| | | | | | | The whole set of tracker_miner_fs_add_directory_without_parent(), tracker_miner_fs_directory_add(), directory_remove() and directory_remove_full() are all covered by TrackerIndexingTree and basically unused, except for code in examples/.
* libtracker-miner: Remove tracker_miner_fs_[sg]et_initial_crawling()Carlos Garnacho2017-06-291-74/+1
| | | | It did nothing at the libtracker-miner level, and can be safely removed.
* libtracker-miner: Remove tracker_miner_fs_set_mtime_checking()Carlos Garnacho2017-06-291-75/+0
| | | | | | | | | | | This is just used to set the TRACKER_DIRECTORY_FLAG_CHECK_MTIME flag on the TrackerIndexingTree for all files. Given libtracker-miner has this fine grained switch and all use of it happens in src/miners/fs, just move the global toggle there and remove it from libtracker-miner API. The only usage of this flag inside libtracker-miner happened inside tracker_miner_fs_directory_add(), which was superseded by TrackerIndexingTree too and is scheduled for removal.
* tracker-miner-fs: Adopt TrackerThumbnailer from libtracker-minerCarlos Garnacho2017-06-291-93/+0
| | | | | | | | It makes no sense to have that at the library level, just move thumbnail handling to TrackerMinerFiles. Coincidentally, this removes further queries that required knowledge about the ontology in TrackerMinerFS.
* libtracker-miner: Remove tracker_miner_fs_force_recheck()Carlos Garnacho2017-06-291-31/+0
| | | | | It's unused and unneeded, just set the TRACKER_DIRECTORY_FLAG_CHECK_MTIME flag on the TrackerIndexingTree.
* libtracker-miner: Remove tracker_miner_fs_force_recheck()Carlos Garnacho2017-06-291-23/+0
| | | | It's been a no-op for years.
* libtracker-miner: Remove tracker_miner_fs_get_parent_urn()Carlos Garnacho2017-06-291-76/+1
| | | | | | | It is cached once to be used once. Besides, the parent GFile is obviously guaranteed to be a folder, and folders are (not so obviously) guaranteed to be cached. Thus looking up the URN should be fast enough.
* libtracker-miner: Add TrackerMinerFS::move-file vmethodCarlos Garnacho2017-06-291-130/+57
| | | | | We delegate the SPARQL generation to this vmethod, in order to keep TrackerMinerFS as agnostic of the ontology as possible.
* libtracker-miner: Split ::remove-file into two signalsCarlos Garnacho2017-06-291-84/+22
| | | | | | | | | | | | The only_children argument is a bit awkward as we emit ::remove-file on a file that is not removed at all. The TrackerSparqlBuilder argument has been also removed (the signals just have a gchar* return value containing the SPARQL for the delete op) so it's up to the caller to decide how to compose the SPARQL. This allows removing some more knowledge about specific ontologies from TrackerMinerFS, the ontology-dependent upper layers will know better how to delete the corresponding entities.
* libtracker-miner: Change API of ::process-file[-attributes]Carlos Garnacho2017-06-291-94/+54
| | | | | | | | | | | | | | | | | | | There's a few changes here: - The 2 vmethods are now given a GTask, its cancellable is to be used if the handling goes async. - tracker_miner_fs_file_notify() has changed into a more generic tracker_miner_fs_notify_finish() method, that takes such GTask and completes it. - The vmethods are no longer given a TrackerSparqlBuilder, instead they are expected to create the SPARQL through whatever mean is most fit. The sparql is given in the tracker_miner_fs_notify_finish() func. This opens the door to TrackerMinerFS implementations using TrackerResource. The intent is 1) Pass something to these vmethods that the user can't forge or mess with, as matching on GFile relies that it's the same pointer that it was given in the vmethods. And 2) Make the finish() function more generic to be fit to other methods going async.
* Remove ignore_next_update() APICarlos Garnacho2017-06-291-225/+1
| | | | | | | It's been deprecated for a long time, it stands in the middle of detaching TrackerMiner from DBus, and it's one less piece of ontology-dependent libtracker-miner code. Enough reasons to finally remove this.
* Remove libmediaart code entirelyCarlos Garnacho2017-06-291-8/+0
| | | | | The libmediaart dependency was disabled in commit 6a05068624bfa, it doesn't make sense to drag this code around.
* libtracker-miner: Shuffle refcount handling when queueing back a fileCarlos Garnacho2017-04-161-17/+7
| | | | | | | | | The file might or might not be inserted to the queue, which meant that the extra ref created outside the call might never dropped if the file didn't end up inserted again. Fix this by doing the refcount increase when actually inserting the file back in the queue. Reported by Jose M. Arroyo <jose.m.arroyo.se@gmail.com>.
* libtracker-miner: Add missing return after switch() caseCarlos Garnacho2016-12-141-0/+1
| | | | | | The QUEUE_UPDATED elements where being additionally checked against the QUEUE_WRITEBACK queue. This was harmless, but potentially confusing. Spotted through Coverity.
* miner-fs: Fix handling files moved soon after creatingMartin Kampas2016-06-051-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Noticed this when executing functional tests for write-back: (tracker-miner-fs:21288): Tracker-CRITICAL **: Could not execute sparql: Subject `(null)' is not in domain `nfo:FileDataObject' of property `nfo:fileName' This warning happens in item_move() when the source just didn't have time to be indexed. One example: copy ("file.txt", "temp_XYZ.file.txt") - received G_FILE_MONITOR_EVENT_CREATED ("temp.file.txt") - received G_FILE_MONITOR_EVENT_CHANGED ("temp.file.txt") - received G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT ("temp.file.txt") modify ("temp_XYZ.file.txt") - received G_FILE_MONITOR_EVENT_CHANGED ("temp.file.txt") - received G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT ("temp.file.txt") mv ("temp_XYZ.file.txt", "file.txt") - received G_FILE_MONITOR_EVENT_MOVED ("temp.file.txt", "file.txt") - emitted ITEM_MOVED ("temp.file.txt", "file.txt") It was already handled in item_move() in past, but removed with eef0e7f (libtracker-miner: Remove useless code) after previously misidentified as useless in scope of ee58e67 (libtracker-miner: Add compat layer for tracker_miner_fs_directory_*) The comment from ee58e67 """FIXME: This situation shouldn't happen from a TrackerFileNotifier event""" simply cannot be satisfied: no way to get "temp.file.txt" indexed before ITEM_MOVED is processed - the file disappears too fast. https://bugzilla.gnome.org/show_bug.cgi?id=678986
* libtracker-miner: cater for unbound nfo:belongsToContainer when moving itemsCarlos Garnacho2016-06-051-3/+3
| | | | | | | | Fixes warnings when moving indexing roots around. This query expects this property to be bound, resulting in no-op if that's not the case (e.g. indexing roots), later reinsertions of nie:url and other properties with max cardinality=1 trigger the whole update failure, because those weren't properly removed.
* libtracker-miner: Insert into the right graph when moving itemsCarlos Garnacho2016-06-041-2/+2
| | | | | The query to update parent-dependent data was using the source file urn as a graph urn.
* libtracker-miner: Do not recurse up the hierarchy for out-of-tree index requestsCarlos Garnacho2016-05-051-0/+1
| | | | | | | If we receive a tracker_miner_fs_check_file() request for a file out of indexing trees, it'd usually end up recursing until it ran out of parents (that is, up to file:///). This is quite pointless, if only one file was requested to be indexed.
* libtracker-miner: Avoid changing order of elements in the processing queueCarlos Garnacho2016-03-141-36/+16
| | | | | | | If should_wait() returns TRUE for an element, we end up putting the file in the queue again and incrementing its reentry counter. This situation should be deemed normal, so we can just peek the element, and only pop it if we should not wait.
* libtracker-miner: Cut some slack on the reentry counterCarlos Garnacho2016-03-141-1/+1
| | | | | REENTRY_MAX is 2 currently, so counter<2 is effectively 1, which doesn't make much for "reentry".
* libtracker-miner: Cancel ops in current buffers if a delete affects the fileCarlos Garnacho2016-02-241-0/+12
| | | | | | | If a folder being deleted affects operations currently in the currently issued tasks (eg. those we emitted ::process-file on) and writeback buffers, those operations would still attempt to proceed, with different degrees of success.
* libtracker-miner: Reset of reentry counter is not needed anymoreHaithem BEN GHORBAL2016-02-151-10/+0
| | | | Moreover, an infinite loop may occurs if process-file signal always fails.
* libtracker-miner: Invalidate files iri recursively in case of file removalHaithem BEN GHORBAL2016-02-151-1/+10
| | | | | | Directories must get all children invalidated, because already queued tasks might contain new instances of those same files, in which case they would still find the previous URN.
* libtracker-miner: Allow to invalidate files iri recursivelyHaithem BEN GHORBAL2016-02-151-1/+1
| | | | This will be useful for delete operations.
* libtracker-miner: Fix some memory leaks of TrackerTaskHaithem BEN GHORBAL2016-02-151-0/+4
| | | | | | | | These leaks had huge impact as each TrackerTask had a reference to a GFile, which prevent them to be removed from TrackerFileSystem when calling tracker_file_system_forget_files(). Due to this behavior, adding/removing/re-adding folders resulted in some folders/files not being indexed.
* libtracker-miner: Do not insert partial/empty sparql on errorCarlos Garnacho2016-01-281-5/+0
| | | | | | | | | | | | The check for these errors was done specifically so we could still insert (even if incomplete) data on tracker-extract failures, when we used to communicate with it directly from tracker-miner-fs. Nowadays, tracker-extract is a TrackerDecorator, and tracker-miner-fs should most likely receive only errors here on ENOENT and other errors that affect the file and its info as a whole. In these situations we end up with a task with a completely empty sparql string, which doesn't help much here.
View Lorry Controller Status