summaryrefslogtreecommitdiff
path: root/giscanner/cachestore.py
Commit message (Collapse)AuthorAgeFilesLines
* Always close filesChristoph Reiter2020-11-071-9/+11
| | | | | | | This means flushing changes and closing the fd. Otherwise this is done by the GC eventually.. Detected using PYTHONTRACEMALLOC=1 PYTHONDEVMODE=1
* Always pass an encoding to open() using text modeChristoph Reiter2020-11-071-2/+2
| | | | | | | | Otherwise we'll end up using the locale encoding. While it's usually utf-8, that's not the case on Windows. There is one place where a file with filenames is passed, not sure there so I left it and passed a explicit None.
* cachestore: handle cache getting deleted while loading it. Fixes #278Christoph Reiter2019-04-171-3/+7
| | | | | | | | In the unlucky event where the cache gets deleted after the os.path.exists() check but before we get its mtime things would error out. Instead of using os.path.exists() handle the exceptions of the operations which we expect to possibly fail.
* Merge branch 'drop-py-compat-code' into 'master'Christoph Reiter2018-12-081-10/+1
|\ | | | | | | | | Drop all Python 2 compat code See merge request GNOME/gobject-introspection!85
| * Drop all Python 2 compat codeChristoph Reiter2018-12-081-10/+1
| | | | | | | | We only support 3.4+ now.
* | cachestore: don't try to catch nonexisting pickle.BadPickleGet. Fixes #159Christoph Reiter2018-12-081-1/+1
|/ | | | | pickle.loads() can pretty much throw any kind of exception and we can't handle it besides ignoring it, so just catch all.
* When handling errors according to errno, catch both IOError and OSErrorSimon McVittie2017-06-191-14/+8
| | | | | | | | | | | | | | | | | | | | | | | | Different Python versions are not completely consistent about the error that is raised and its class hierarchy: Python 3.5.3rc1 (default, Jan 3 2017, 04:40:57) >>> try: open('/foo') ... except Exception as e: print(e.__class__.__mro__) (<class 'FileNotFoundError'>, <class 'OSError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>) Python 2.7.13 (default, Dec 18 2016, 20:19:42) >>> try: open('/foo') ... except Exception as e: print e.__class__.__mro (<type 'exceptions.IOError'>, <type 'exceptions.EnvironmentError'>, <type 'exceptions.StandardError'>, <type 'exceptions.Exception'>, <type 'exceptions.BaseException'>, <type 'object'>) This can lead to a race condition during cache cleaning, where two processes both try to delete the same file, and the one that loses the race fails. Signed-off-by: Simon McVittie <smcv@collabora.com> Reviewed-by: Iain Lane <laney@ubuntu.com> Reviewed-by: Colin Walters <walters@verbum.org> Bug: https://bugzilla.gnome.org/show_bug.cgi?id=772173
* giscanner: Encode sha1 input for Python 3 compatibilitySimon Feltman2015-09-291-2/+5
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=679438
* giscanner: Use pickle when cPickle is not availableSimon Feltman2015-09-291-6/+10
| | | | | | | | This adds compatibility with Python 3 which removed the cPickle module. Explicitly use binary files for reading and writing the cache. https://bugzilla.gnome.org/show_bug.cgi?id=679438
* giscanner: Use unicode literals in all Python filesSimon Feltman2015-09-291-0/+1
| | | | | | | | | | | | Add unicode_literals future import which turns any string literal into a unicode string. Return unicode strings from the Python C extension module. Force writing of annotations (g-ir-annotation-tool) to output utf8 encoded data to stdout. This is an initial pass at following the "unicode sandwich" model of programming (http://nedbatchelder.com/text/unipain.html) needed for supporting Python 3. https://bugzilla.gnome.org/show_bug.cgi?id=679438
* giscanner: Use print as a function for Python 3 compatibilitySimon Feltman2015-09-291-0/+1
| | | | | | | Use future import "print_function" and update relevant uses of print as a function call. See: PEP 3105 https://bugzilla.gnome.org/show_bug.cgi?id=679438
* giscanner: Enable "true division" for all Python filesSimon Feltman2015-09-291-0/+1
| | | | | | | | | | Import Python 3 compatible "true division" from the future (PEP 238). This changes the Python 2 classic division which uses floor division on integers to true division. Verfied we don't actually use the division operator anywhere in the code base so this a safety for supporting both Python 2 and 3. https://bugzilla.gnome.org/show_bug.cgi?id=679438
* giscanner: Use absolute_import for all Python filesSimon Feltman2015-09-291-0/+2
| | | | | | Use absolute_import to ensure Python 3 compatibility of the code base. https://bugzilla.gnome.org/show_bug.cgi?id=679438
* scanner: use open() as os.fdopen as context managersDieter Verfaillie2015-08-211-2/+4
| | | | | | Ensures files are correctly and immediately closed. https://bugzilla.gnome.org/show_bug.cgi?id=751926
* scanner: fix cachestore raceDieter Verfaillie2015-08-211-9/+8
| | | | | | | | Using tempfile.mkstemp should prevent the temp file from being cleaned while it might still be used by another process. https://bugzilla.gnome.org/show_bug.cgi?id=751926
* scanner: future-proof cachestore version hashingDieter Verfaillie2015-06-291-2/+1
| | | | | | | Python 3 does not write .pyc files like Python 2 used to do but instead put's them in a __pycache__ directory. Simply compute the version hash using the .py files instead.
* scanner: add "XDG Base Directory" functionsDieter Verfaillie2015-06-201-40/+11
| | | | | | | | | | | | Extract cache and data dir lookup code into documented get_user_cache_dir() and get_system_data_dirs() functions. Note that previously, the data dirs code did not fall back to '/usr/local/share:/usr/share' when the XDG_DATA_DIRS environment variable was either not set or empty, as required by the XDG Base Directory Specification. https://bugzilla.gnome.org/show_bug.cgi?id=747770
* scanner: honor XDG_CACHE_HOMEDieter Verfaillie2015-06-201-0/+5
| | | | | | | | We already use XDG_DATA_DIRS for .gir files lookup so we might as well honor XDG_CACHE_HOME instead of hardcoding ~/.cache. https://bugzilla.gnome.org/show_bug.cgi?id=747770
* giscanner/cachestore.py: Clean up a bitChun-wei Fan2014-08-011-4/+1
| | | | | | | Use the special os.path.expanduser('~'), as it it more portable, instead of quering the HOME or HOMEPATH envvar. https://bugzilla.gnome.org/show_bug.cgi?id=732668
* giscanner: Make _get_cachedir() Always Work on WindowsChun-wei Fan2014-07-041-1/+4
| | | | | | | | | | On Windows, checking for $(HOME) will work in a MSYS shell but not in cmd.exe (i.e. Visual Studio command prompt), so we need to check for HOMEPATH when we are building under a Visual Studio command prompt to get the users's home directory correctly. This will enable g-ir-doc-tool to work on Windows when run from cmd.exe. https://bugzilla.gnome.org/show_bug.cgi?id=732668
* giscanner: Use Python 3 compatible octal literal syntaxSimon Feltman2014-05-011-2/+2
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=679438
* cachestore: Use "write new, then rename" pattern, not "write in place"Colin Walters2014-04-011-1/+7
| | | | | | | This should fix race conditions when multiple processes attempt to access the cache concurrently. https://bugzilla.gnome.org/show_bug.cgi?id=724886
* tests: Update misc/pep8.py to 1.4.5Dieter Verfaillie2013-05-071-2/+3
| | | | | | | | | | | | | | | | | Version in our tree is a wee bit outdated. For example, later work will introduce an utf8 encoded python source file which our old pep8.py does not yet understand (yeah, it really was *that* ancient)... Updated from: https://raw.github.com/jcrocholl/pep8/1.4.5/pep8.py Takes 552c1f1525e37a30376790151c1ba437776682c5, f941537d1c0a40f0906490ed160db6c79af572d3, 5a4afe2a77d0ff7d9fea13dd93c3304a6ca993de and a17f157e19bd6792c00321c8020dca5e5a281f45 into account... https://bugzilla.gnome.org/show_bug.cgi?id=699535
* giscanner: prefer "except X as e" over "except X, e"Dieter Verfaillie2013-04-091-8/+8
| | | | | | It's more readable and as an added bonus Python 3 compatible. https://bugzilla.gnome.org/show_bug.cgi?id=697616
* Add GI_SCANNER_DISABLE_CACHE environment variableColin Walters2011-05-021-0/+2
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=649054
* scanner: Don't fail on mkdir if cachedir already existsGOBJECT_INTROSPECTION_0_10_2Colin Walters2011-02-031-1/+4
| | | | | Hit this race in practice with two g-ir-scanner invocations in Fedora's koji.
* Fix giscanner crash with no cache directoryRobert Ancell2010-08-121-0/+3
|
* [cachestore] Properly clean the cacheg-i-0.6Johan Dahlin2010-06-171-1/+1
| | | | | Propertly clear the cache after a version invalidation. Send in the full filename to unlink, not just the basename.
* [cachestore] Add versioningJohan Dahlin2010-06-101-1/+50
| | | | | | | | Version the cache by checking the SHA1 of the content of all python source files. If the SHA1 hash differs, just regenerate the cache. https://bugzilla.gnome.org/show_bug.cgi?id=568842
* Ignore errors caused by permissions in $HOMEJosselin Mouette2009-12-021-1/+4
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=596960
* Add missing self parameterJohan Dahlin2009-02-111-1/+1
| | | | This is really why you shouldn't catch TypeError.
* Catch AttributeError which seems to be occasionally raised for Record.Johan Dahlin2009-01-131-1/+1
| | | | | | | | | | 2009-01-13 Johan Dahlin <jdahlin@async.com.br> * giscanner/cachestore.py (CacheStore.load): Catch AttributeError which seems to be occasionally raised for Record. svn path=/trunk/; revision=1031
* Use shutil.move instead of os.rename, since it will automatically fallJohan Dahlin2008-12-091-2/+3
| | | | | | | | | | | | | 2008-12-09 Johan Dahlin <johan@async.com.br> * giscanner/cachestore.py (CacheStore.store): Use shutil.move instead of os.rename, since it will automatically fall back to copying+remove if the src directory is on a different partition from the dst directory. svn path=/trunk/; revision=989
* Don't require HOME to be set and if it set, don't require the directory toJohan Dahlin2008-12-081-1/+7
| | | | | | | | | | | 2008-12-08 Johan Dahlin <johan@async.com.br> * giscanner/cachestore.py (_get_cachedir): Don't require HOME to be set and if it set, don't require the directory to exist. svn path=/trunk/; revision=987
* yay for scripts checking number of newlinesJohan Dahlin2008-12-081-0/+1
| | | | svn path=/trunk/; revision=986
* Dump the cache file to a temporary file and rename it the expectedJohan Dahlin2008-12-081-7/+19
| | | | | | | | | | | | | 2008-12-08 Johan Dahlin <johan@async.com.br> * giscanner/cachestore.py (CacheStore.store): Dump the cache file to a temporary file and rename it the expected filename only when it's completely written. This should make the cache more roboust when run in parallel, and hopefully avoid triggering bugs in Python. svn path=/trunk/; revision=985
* Remove tabs, fixes make checkJohan Dahlin2008-12-011-1/+1
| | | | svn path=/trunk/; revision=980
* Bug 562289 – Race when removing invalid cacheJohan Dahlin2008-11-301-1/+4
| | | | | | | | | | | | 2008-11-30 Johan Dahlin <jdahlin@async.com.br> Bug 562289 – Race when removing invalid cache * giscanner/cachestore.py: ENOENT is mapped to a OSError, not IOError. svn path=/trunk/; revision=979
* Catch ValueError exceptions.Johan Dahlin2008-11-191-1/+1
| | | | | | | | | | 2008-11-19 Johan Dahlin <jdahlin@async.com.br> * giscanner/cachestore.py: Catch ValueError exceptions. svn path=/trunk/; revision=950
* Catch BadPickleGet exceptions as well.Johan Dahlin2008-11-181-1/+1
| | | | | | | | | | 2008-11-18 Johan Dahlin <jdahlin@async.com.br> * giscanner/cachestore.py: Catch BadPickleGet exceptions as well. svn path=/trunk/; revision=942
* Handle broken cache entries, write only filesystems and running out ofJohan Dahlin2008-11-071-3/+29
| | | | | | | | | | | 2008-11-07 Johan Dahlin <jdahlin@async.com.br> * giscanner/cachestore.py: Handle broken cache entries, write only filesystems and running out of diskspace. svn path=/trunk/; revision=867
* Relicense the giscanner library under LGPLv2+. This has been approved byJohan Dahlin2008-11-041-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2008-11-04 Johan Dahlin <jdahlin@async.com.br> * giscanner/__init__.py: * giscanner/ast.py: * giscanner/cachestore.py: * giscanner/cgobject.py: * giscanner/girparser.py: * giscanner/girwriter.py: * giscanner/giscannermodule.c: * giscanner/glibast.py: * giscanner/glibtransformer.py: * giscanner/libtoolimporter.py: * giscanner/minixpath.py: * giscanner/odict.py: * giscanner/sourcescanner.c: * giscanner/sourcescanner.h: * giscanner/sourcescanner.py: * giscanner/transformer.py: * giscanner/utils.py: * giscanner/xmlwriter.py: * COPYING: Relicense the giscanner library under LGPLv2+. This has been approved by all contributors. svn path=/trunk/; revision=862
* Remove arguments from the constructor, move them to separate accessors.Johan Dahlin2008-10-301-0/+87
2008-10-30 Johan Dahlin <jdahlin@async.com.br> * giscanner/girparser.py: Remove arguments from the constructor, move them to separate accessors. Add a new parse_tree method which takes an element tree instance. * tools/g-ir-scanner: Update callsite for this * giscanner/Makefile.am: * giscanner/cachestore.py: * giscanner/transformer.py: Cache the include parsing. Saves ~25% time when creating vte (which includes everything up to gtk+). svn path=/trunk/; revision=842
View Lorry Controller Status