summaryrefslogtreecommitdiff
path: root/src/atomic_ops
Commit message (Collapse)AuthorAgeFilesLines
* Fix test_atomic failure caused unaligned AO_double_t access on x86Ivan Maidanski2017-01-241-0/+5
| | | | | | | | | | | | | The failure is caused by violation of an assertion that checks AO_double_t variable is 8-byte aligned on x86. * doc/README.txt (AO_double_t): Add note about required alignment. * src/atomic_ops/sysdeps/standard_ao_double_t.h (AO_double_t): Add comment about alignment. * src/atomic_ops_stack.h (AO_stack_t): Likewise. * tests/test_atomic_include.template (test_atomicXX): Define old_w, w local variables as static (as otherwise, e.g., they could have 4-byte alignment on x86); add comment.
* Revert "Fix test_atomic fail caused unaligned AO_double_t access (VC/x86)"Ivan Maidanski2017-01-242-15/+0
| | | | | | | | | | This reverts commit 69c4010c1b3d43e2e179421e27379aaa5aeeb6d8. The failure is actually caused by the fact that default alignment of stack on x86 is 4 bytes. So, another solution should be proposed to fix the issue for other compilers (and, potentially, other targets). E.g. do not place AO_double_t volatile variables on stack.
* Implement char CAS and char/short add for msftc X86[_64] (VS 2013+)Ivan Maidanski2017-01-243-2/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>=1800] (_InterlockedCompareExchange8): Declare intrinsic. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>=1800 && !AO_PREFER_GENERALIZED && !_M_ARM] (_InterlockedExchangeAdd16, _InterlockedExchangeAdd8): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>=1800] (_InterlockedCompareExchange8): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>=1800 && !AO_PREFER_GENERALIZED] (_InterlockedExchangeAdd16, _InterlockedExchangeAdd8): Likewise. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>=1800] (AO_char_fetch_compare_and_swap_full): Implement using intrinsic. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>=1800 && !AO_PREFER_GENERALIZED && !_M_ARM] (AO_char_fetch_and_add_full, AO_short_fetch_and_add_full): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>=1800] (AO_char_fetch_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>=1800 && !AO_PREFER_GENERALIZED] (AO_char_fetch_and_add_full, AO_short_fetch_and_add_full): Likewise. * src/atomic_ops/sysdeps/msftc/x86.h [!AO_NO_ASM_XADD] (AO_char_fetch_and_add_full, AO_short_fetch_and_add_full): Do not define if AO_HAVE_char_fetch_and_add_full. * src/atomic_ops/sysdeps/msftc/x86_64.h [AO_ASM_X64_AVAILABLE] (AO_char_fetch_and_add_full, AO_short_fetch_and_add_full): Do not use asm-based implementation if _MSC_VER>=1800.
* Implement short inc/dec directly for msftc ARM and X86[_64]Ivan Maidanski2017-01-242-0/+36
| | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>1400 && !AO_PREFER_GENERALIZED] (_InterlockedIncrement16, _InterlockedDecrement16): Declare intrinsic. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>1400 && !AO_PREFER_GENERALIZED] (_InterlockedIncrement16, _InterlockedDecrement16): Likewise. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>1400 && !AO_PREFER_GENERALIZED] (AO_short_fetch_and_add1_full, AO_short_fetch_and_sub1_full): Implement using intrinsic. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>=1800 && !AO_PREFER_GENERALIZED] (AO_char_fetch_and_add_full, AO_short_fetch_and_add_full): Likewise.
* Really implement char and/or/xor and short CAS for msftc ARM (VS 2013+)Ivan Maidanski2017-01-241-1/+1
| | | | | | | | | | | | (fix commit 9182c62) * src/atomic_ops/sysdeps/msftc/common32_defs.h [_M_ARM] (_InterlockedAnd8, _InterlockedCompareExchange16, _InterlockedOr8, _InterlockedXor8): Declare intrinsic if _MSC_VER>=1800. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_M_ARM] (AO_char_and_full, AO_char_or_full, AO_char_xor_full, AO_short_fetch_compare_and_swap_full): Implement (using intrinsic) if _MSC_VER>=1800.
* Implement char and/or/xor and short CAS for msftc ARM and X86[_64]Ivan Maidanski2017-01-232-0/+79
| | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>1400 && !_M_ARM] (_InterlockedAnd8, _InterlockedCompareExchange16, _InterlockedOr8, _InterlockedXor8): Declare intrinsic. * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>1400] (_InterlockedAnd8, _InterlockedCompareExchange16, _InterlockedOr8, _InterlockedXor8): Likewise. * src/atomic_ops/sysdeps/msftc/common32_defs.h [_MSC_VER>1400 && !_M_ARM] (AO_char_and_full, AO_char_or_full, AO_char_xor_full, AO_short_fetch_compare_and_swap_full): Implement (using intrinsic). * src/atomic_ops/sysdeps/msftc/x86_64.h [_MSC_VER>1400] (AO_char_and_full, AO_char_or_full, AO_char_xor_full, AO_short_fetch_compare_and_swap_full): Likewise.
* Remove redundant include windows.h from headers (msftc/x86[_64])Ivan Maidanski2017-01-202-23/+20
| | | | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/msftc/common32_defs.h: Do not include windows.h if _MSC_VER > 1400 && AO_USE_INTERLOCKED_INTRINSICS && !_WIN32_WCE. * src/atomic_ops/sysdeps/msftc/common32_defs.h [!AO_PREFER_GENERALIZED || !AO_ASSUME_WINDOWS98] (AO_fetch_and_add_full, AO_fetch_and_add1_full, AO_fetch_and_sub1_full): Replace LONG to long; remove unnecessary cast to LONG. * src/atomic_ops/sysdeps/msftc/common32_defs.h [AO_ASSUME_WINDOWS98] (AO_fetch_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h (AO_int_fetch_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h [!AO_PREFER_GENERALIZED] (AO_int_fetch_and_add_full, AO_int_fetch_and_add1_full, AO_int_fetch_and_sub1_full): Likewise. * src/atomic_ops/sysdeps/msftc/common32_defs.h [AO_ASSUME_WINDOWS98] (AO_fetch_compare_and_swap_full): Replace PVOID to void*; remove unnecessary cast to AO_t. * src/atomic_ops/sysdeps/msftc/x86_64.h: Remove include windows.h. * src/atomic_ops/sysdeps/msftc/x86_64.h [!AO_PREFER_GENERALIZED] (AO_fetch_and_add_full, AO_fetch_and_add1_full, AO_fetch_and_sub1_full): Replace LONGLONG to __int64; remove unnecessary cast to LONGLONG. * src/atomic_ops/sysdeps/msftc/x86_64.h (AO_fetch_compare_and_swap_full): Likewise.
* Always define word-wide CAS for x86 (MS VC++ 8 or later)Ivan Maidanski2017-01-201-2/+3
| | | | | | | | | | | | | | | | | | | Visual Studio 2005 (MS VC++ 8.0) discontinued support of Windows 95. This patch deprecates AO_ASSUME_WINDOWS98 macro. * configure.ac (AO_ASSUME_WINDOWS98): Remove AH template. * doc/README.txt (AO_ASSUME_WINDOWS98): Remove documentation (as relates only to Win95). * doc/README_win32.txt (AO_ASSUME_WINDOWS98): Likwise. * doc/README_win32.txt (AO_ASSUME_VISTA): Update documentation. * src/Makefile.msft (CFLAGS): Remove "-D AO_ASSUME_WINDOWS98". * src/Makefile.msft (test_atomic_w95): Remove target. * src/Makefile.msft (check): Do not call test_atomic_w95; remove the corresponding printed message. * src/atomic_ops/sysdeps/msftc/x86.h: Remove AO_ASSUME_WINDOWS98 from comment. * src/atomic_ops/sysdeps/msftc/x86.h (AO_ASSUME_WINDOWS98): Define (implicitly) if _MSC_VER >= 1400.
* Fix test_atomic failure caused unaligned AO_double_t access on x86 (VC++)Ivan Maidanski2017-01-202-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Passing an unaligned AO_double_t pointer to AO double-wide primitives results in an undefined behavior of the latter when running on x86 (or violation of the corresponding assertion on the alignment). MS VC++/x86 aligns AO_double_t values on a 4-byte boundary by default. For the proper alignment, __declspec(align(8)) should be applied to variables defined in the client code (which uses double-wide AO primitives). Unfortunately, the attribute cannot be added to AO_double_t definition itself because the compiler does not allow the attribute for function arguments. This patch introduces AO_DOUBLE_ALIGN attribute for use by clients of the double-wide AO primitives (and, thus, AO_stack clients). Matters only Visual Studio compiler for X86. The inner clients (atomic_ops_malloc, test_atomic, test_stack) are updated to use this attribute. * doc/README_win32.txt [x86] (AO_DOUBLE_ALIGN): Document. * src/atomic_ops/sysdeps/generic_pthread.h (AO_DOUBLE_ALIGN): Define (as empty). * src/atomic_ops/sysdeps/standard_ao_double_t.h [!_WIN64 && _WIN32 && !__GNUC__ && _MSC_VER] (AO_DOUBLE_ALIGN): Define as declspec align(8); document it. * src/atomic_ops/sysdeps/standard_ao_double_t.h [!AO_DOUBLE_ALIGN] (AO_DOUBLE_ALIGN): Define as empty (otherwise). * src/atomic_ops_stack.h [AO_USE_ALMOST_LOCK_FREE && !AO_DOUBLE_ALIGN] (AO_DOUBLE_ALIGN): Likewise. * src/atomic_ops_malloc.c (AO_free_list): Use AO_DOUBLE_ALIGN attribute. * tests/test_stack.c (the_list): Likewise. * src/atomic_ops_stack.h [!AO_USE_ALMOST_LOCK_FREE] (AO_stack_t): Document AO_DOUBLE_ALIGN usage (by clients). * tests/test_atomic_include.template (test_atomicXX): Use AO_DOUBLE_ALIGN attribute for old_w and w double-wide local variables (to avoid alignment assertion violation or AO primitives undefined behavior on x86 if the test code is compiled by VC++).
* Add assertion that double-wide CAS target is aligned (msftc/x86[_64])Ivan Maidanski2017-01-202-0/+3
| | | | | | | | * src/atomic_ops/sysdeps/msftc/x86.h [AO_ASSUME_VISTA] (AO_double_compare_and_swap_full): Add assert() that addr pointer is aligned according to the size of AO_double_t. * src/atomic_ops/sysdeps/msftc/x86_64.h [AO_CMPXCHG16B_AVAILABLE && _MSC_VER>=1500] (AO_compare_double_and_swap_double_full): Likewise.
* Add asm-based char/short/int CAS implementation for gcc/x86[_64]Ivan Maidanski2017-01-171-1/+63
| | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/x86.h [!AO_GCC_ATOMIC_TEST_AND_SET] (AO_char_and_full, AO_char_or_full, AO_char_xor_full, AO_short_and_full, AO_short_or_full, AO_short_xor_full): Do not define if AO_PREFER_GENERALIZED. * src/atomic_ops/sysdeps/gcc/x86.h [!AO_GCC_ATOMIC_TEST_AND_SET] (AO_char_fetch_compare_and_swap_full, AO_short_fetch_compare_and_swap_full): Implement. * src/atomic_ops/sysdeps/gcc/x86.h [!AO_GCC_ATOMIC_TEST_AND_SET && __x86_64__ && !__ILP32__] (AO_int_fetch_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/gcc/x86.h [!AO_GCC_ATOMIC_TEST_AND_SET && __x86_64__ && !__ILP32__] (AO_int_fetch_and_add_full, AO_int_and_full, AO_int_or_full, AO_int_xor_full): Do not define if AO_PREFER_GENERALIZED.
* Implement int CAS/inc/dec for msftc/x86_64Ivan Maidanski2017-01-171-1/+28
| | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/msftc/x86_64.h (_InterlockedExchangeAdd): Declare (as intrinsic) only unless AO_PREFER_GENERALIZED. * src/atomic_ops/sysdeps/msftc/x86_64.h (_InterlockedCompareExchange): Declare intrinsic. * src/atomic_ops/sysdeps/msftc/x86_64.h [!AO_PREFER_GENERALIZED] (_InterlockedIncrement, _InterlockedDecrement): Likewise. * src/atomic_ops/sysdeps/msftc/x86_64.h (AO_int_fetch_compare_and_swap_full): Define inline function (using _InterlockedCompareExchange). * src/atomic_ops/sysdeps/msftc/x86_64.h (AO_int_fetch_and_add_full): Do not define if AO_PREFER_GENERALIZED. * src/atomic_ops/sysdeps/msftc/x86_64.h [!AO_PREFER_GENERALIZED] (AO_int_fetch_and_add1_full, AO_int_fetch_and_sub1_full): Define inline function _InterlockedIncrement/Decrement).
* Remove outdated FIXME in msftc/arm.h about _InterlockedOps memory barrierIvan Maidanski2017-01-161-2/+0
| | | | | * src/atomic_ops/sysdeps/msftc/arm.h: Remove FIXME about memory barrier of _InterlockedOps (they have the correct one according to the MSDN).
* Fix 'atomic_load/store symbols undefined' error on OS X 10.8 i386Ivan Maidanski2017-01-121-0/+15
| | | | | | | | | | | | | Issue #23 (libatomic_ops). * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && !__x86_64__ && __APPLE__ && __MACH__]: Include AvailabilityMacros.h unless MAC_OS_X_VERSION_MIN_REQUIRED already defined. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && !__x86_64__ && __APPLE__ && __MACH__ && MAC_OS_X_VERSION_MIN_REQUIRED < 1090] (AO_SKIPATOMIC_double_load, AO_SKIPATOMIC_double_load_acquire, AO_SKIPATOMIC_double_store, AO_SKIPATOMIC_double_store_release): Define.
* Fix AO_stack build failure on OS X 10.7 because of missing CAS (x86)Ivan Maidanski2016-12-291-1/+4
| | | | | | | | | | (fix commit 7a6a7cc) Issue #23 (libatomic_ops). * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __APPLE_CC__] (AO_GCC_FORCE_HAVE_CAS): Define also for arch i386; refine comment.
* Fix AO_stack build failure on OS X 10.7 because of missing CAS (x86_64)Ivan Maidanski2016-12-281-0/+3
| | | | | | | | | Issue #23 (libatomic_ops). * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __APPLE_CC__ && __x86_64__] (AO_GCC_FORCE_HAVE_CAS): Define (to workaround missing __GCC_HAVE_SYNC_COMPARE_AND_SWAP_<n> predefined macro in Apple clang-425/x86_64); add comment.
* Workaround a bug in double-wide intrinsics of Clang/x64 with ASan enabledIvan Maidanski2016-11-241-2/+6
| | | | | | | | | | | | | | | As of clang-3.8, double-wide arguments are incorrectly passed to atomic intrinsic operations for x64 target if Address Sanitizer is enabled. * src/atomic_ops.h [__has_feature(address_sanitizer)] (AO_ADDRESS_SANITIZER): New internal macro. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __clang__ && __x86_64__ && !__ILP32__] (AO_SKIPATOMIC_double_compare_and_swap_ANY, AO_SKIPATOMIC_double_load, AO_SKIPATOMIC_double_load_acquire, AO_SKIPATOMIC_double_store, AO_SKIPATOMIC_double_store_release): Define also if AO_ADDRESS_SANITIZER; update comment.
* Code refactoring of Clang double-wide intrinsic workarounds (gcc/x86)Ivan Maidanski2016-11-242-28/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | (fix commits 9803386, f6a1b68, eb6d4fd) * src/atomic_ops/sysdeps/gcc/generic.h [AO_HAVE_DOUBLE_PTR_STORAGE && !AO_HAVE_double_store] (AO_double_store): Do not define if AO_SKIPATOMIC_double_store. * src/atomic_ops/sysdeps/gcc/generic.h [AO_HAVE_DOUBLE_PTR_STORAGE && !AO_HAVE_double_store_release] (AO_double_store_release): Do not define if AO_SKIPATOMIC_double_load. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __clang__ && !__x86_64__ && !__APPLE_CC__ && !__CYGWIN__ && !AO_PREFER_BUILTIN_ATOMICS] (AO_SKIPATOMIC_double_load, AO_SKIPATOMIC_double_load_acquire, AO_SKIPATOMIC_double_store, AO_SKIPATOMIC_double_store_release): Define internal macro (undefined at the end of file). * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && (__clang__ && !__x86_64__ && !__APPLE_CC__ && !__CYGWIN__ && !AO_PREFER_BUILTIN_ATOMICS) || (__APPLE_CC__ && __x86_64__))] (AO_SKIPATOMIC_double_compare_and_swap_ANY): Move definition here (remove at other places in this file); undefine at the end of the file. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET]: Check AO_SKIPATOMIC_double_compare_and_swap_ANY instead of checking multiple macros to detect whether a workaround for the CAS (and others) double-wide operations is needed.
* Undefine internal macros that are defined solely for generic.hIvan Maidanski2016-11-232-0/+14
| | | | | | | | | | | | | (fix commits 03a0465f, 3c52619c, 66f7a4d3) * src/atomic_ops/sysdeps/gcc/aarch64.h (AO_SKIPATOMIC_double_load, AO_SKIPATOMIC_double_load_acquire): Undefine at the end of file. * src/atomic_ops/sysdeps/gcc/arm.h (AO_SKIPATOMIC_ANY_and_ANY, AO_SKIPATOMIC_ANY_or_ANY, AO_SKIPATOMIC_ANY_xor_ANY, AO_SKIPATOMIC_char_store, AO_SKIPATOMIC_char_store_release, AO_SKIPATOMIC_int_store, AO_SKIPATOMIC_int_store_release, AO_SKIPATOMIC_short_store, AO_SKIPATOMIC_short_store_release, AO_SKIPATOMIC_store, AO_SKIPATOMIC_store_release): Likewise.
* Refine AO_nop_write comment for ARM big.LITTLE architectureIvan Maidanski2016-11-181-1/+2
| | | | | | * src/atomic_ops/sysdeps/gcc/arm.h [!AO_UNIPROCESSOR && AO_ARM_HAVE_DMB && !AO_PREFER_BUILTIN_ATOMICS] (AO_nop_write): Refine comment about "dmb ishst" for big.LITTLE architecture.
* Fix indentation of #endif in s390.hIvan Maidanski2016-11-121-1/+1
| | | | | | | (fix commit c2edeab) * src/atomic_ops/sysdeps/gcc/s390.h (AO_fetch_compare_and_swap_full): Adjust indentation of #endif.
* Implement fetch-CAS for s390[x] (gcc)Tobias Leich2016-11-101-1/+18
| | | | | | | * src/atomic_ops/sysdeps/gcc/s390.h (AO_compare_and_swap_full): Do not define if AO_GENERALIZE_ASM_BOOL_CAS. * src/atomic_ops/sysdeps/gcc/s390.h (AO_fetch_compare_and_swap_full): Implement; remove TODO item.
* Remove spaces at EOLn in asm codeIvan Maidanski2016-11-082-32/+32
| | | | | | | | | | * src/atomic_ops/sysdeps/gcc/hexagon.h (AO_test_and_set, AO_compare_and_swap, AO_fetch_compare_and_swap): Remove spaces preceding "\n". * src/atomic_ops/sysdeps/gcc/mips.h (AO_nop_full, AO_fetch_and_add, AO_test_and_set, AO_compare_and_swap): Likewise. * src/atomic_ops/sysdeps/gcc/hexagon.h (AO_fetch_and_add, AO_test_and_set, AO_fetch_compare_and_swap): Align comments.
* Fix missing double_compare_and_swap_dd_acquire_readIvan Maidanski2016-11-011-0/+16
| | | | | | | | | | This matters only if AO_double_compare_and_swap[_acquire_read] actually defined in generalize.h too. * src/atomic_ops/generalize.h [!AO_HAVE_double_compare_and_swap_dd_acquire_read] (AO_double_compare_and_swap_dd_acquire_read): Define (same as definition as auto-generated in generalize-small.h).
* Suppress 'ISO C does not support __int128 type' GCC/Clang pedantic warningIvan Maidanski2016-10-251-1/+15
| | | | | | * src/atomic_ops/sysdeps/standard_ao_double_t.h [!__ILP32__ && (__GNUC__ >= 4.7 || __clang__ >= 3.6)] (double_ptr_storage): Turn off pedantic warnings reporting (for this typedef only).
* Support AIX/ppc (gcc)Jean Girardet2016-10-101-16/+34
| | | | | | | | | | * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_PPC_L, AO_PPC_BR_A): New internal macros (undefined at the end of file); definition depends on _AIX macro presence. * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_load_acquire, AO_test_and_set, AO_compare_and_swap, AO_fetch_compare_and_swap, AO_fetch_and_add): Use AO_PPC_L for label definitions; use AO_PPC_BR_A to specify jump target.
* Code refactoring of gcc/powerpc.h to avoid code duplicationIvan Maidanski2016-10-071-96/+51
| | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_PPC_LD, AO_PPC_LxARX, AO_PPC_CMPx, AO_PPC_STxCXd, AO_PPC_LOAD_CLOBBER): New internal macros (undefined at the end of the file). * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_T_IS_INT): Move definition upper (to have single check of __ppc64__). * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_load_acquire): Use AO_PPC_LD, AO_PPC_LOAD_CLOBBER (and eliminate code duplication). * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_test_and_set, AO_compare_and_swap, AO_fetch_compare_and_swap, AO_fetch_and_add): Use AO_PPC_LxARX, AO_PPC_CMPx, AO_PPC_STxCXd (and eliminate code duplication).
* Fix 'visibility' of some CPU-specific internal macrosIvan Maidanski2016-10-078-0/+32
| | | | | | | | | | | | | | | | | | | | | Undefine internal macros that are defined and used solely in a single CPU-specific file. * src/atomic_ops/sysdeps/gcc/arm.h (AO_BR_ALIGN, AO_MASK_PTR, AO_THUMB_GO_ARM, AO_THUMB_RESTORE_MODE, AO_THUMB_SWITCH_CLOBBERS): Undefine at the end of file. * src/atomic_ops/sysdeps/gcc/hppa.h (AO_PA_LDCW_ALIGNMENT, AO_ldcw, AO_ldcw_align): Likewise. * src/atomic_ops/sysdeps/gcc/ia64.h (AO_IN_ADDR, AO_LEN, AO_MASK, AO_OUT_ADDR, AO_SWIZZLE): Likewise. * src/atomic_ops/sysdeps/gcc/mips.h (AO_MIPS_LL, AO_MIPS_LL_1, AO_MIPS_SC, AO_MIPS_SET_ISA): Likewise. * src/atomic_ops/sysdeps/hpc/hppa.h (AO_PA_LDCW_ALIGNMENT, AO_ldcw, AO_ldcw_align): Likewise. * src/atomic_ops/sysdeps/hpc/ia64.h (AO_T_FASIZE, AO_T_SIZE): Likewise. * src/atomic_ops/sysdeps/icc/ia64.h (AO_INTEL_PTR_t): Likewise. * src/atomic_ops/sysdeps/test_and_set_t_is_char.h (AO_TS_SET_TRUEVAL): Likewise.
* Workaround '#error' cppcheck error messagesIvan Maidanski2016-10-053-5/+8
| | | | | | | | | * src/atomic_ops.h [AO_REQUIRE_CAS && !AO_CAN_EMUL_CAS]: Skip #error pragma if CPPCHECK. * src/atomic_ops/generalize.h [AO_HAVE_nop_acquire]: Likewise. * src/atomic_ops/generalize.h [AO_HAVE_nop_release]: Likewise. * src/atomic_ops/sysdeps/armcc/arm_v6.h [__TARGET_ARCH_ARM < 6]: Likewise.
* Fix typos in 'error' pragma messagesIvan Maidanski2016-10-052-3/+3
| | | | | | | | | | | | | | * src/atomic_ops.h [_HPUX_SOURCE && !__ia64] (AO_barrier_dummy): Fix typo ("do not") in comment. * src/atomic_ops/generalize.h [AO_HAVE_nop_acquire]: Fix typo ("do not") in #error message. * src/atomic_ops/generalize.h [AO_HAVE_nop_release]: Likewise. * src/atomic_ops/sysdeps/armcc/arm_v6.h [__TARGET_ARCH_ARM < 6]: Likewise. * src/atomic_ops/sysdeps/armcc/arm_v6.h [__TARGET_ARCH_ARM < 6]: Add #error pragma. * src/atomic_ops_stack.c [AO_HAVE_compare_and_swap_double]: Fix typo ("does not") in #error message.
* Workaround a bug in double-wide CAS intrinsic of Apple clang-600/x64Ivan Maidanski2016-08-201-1/+7
| | | | | | | | | | | | | The bug causes test_stack failure. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __APPLE_CC__ && __x86_64__]: Fall-back to asm-based implementation of double-wide CAS (AO_compare_double_and_swap_double_full); add comment explaining the reason. * src/atomic_ops/sysdeps/gcc/x86.h [__GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 && AO_GCC_ATOMIC_TEST_AND_SET && __APPLE_CC__] (AO_SKIPATOMIC_double_compare_and_swap_ANY): Define.
* Use C11 double-wide atomic intrinsics for Clang/x86 on Cygwin and OS XIvan Maidanski2016-08-201-1/+2
| | | | | | | | Others (e.g., NDK clang/i686) still might need -latomic linker option. * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET && __clang__]: Do not fallback to non-intrinsic implementation if __APPLE_CC__ or __CYGWIN__.
* Code refactoring of int-wide primitives in gcc/x86.hIvan Maidanski2016-08-201-88/+86
| | | | | | * src/atomic_ops/sysdeps/gcc/x86.h (AO_int_fetch_and_add_full, AO_int_and_full, AO_int_or_full, AO_int_xor_full, AO_T_IS_INT): Move to separate 'ifdef' block (inside AO_DISABLE_GCC_ATOMICS case).
* Avoid extra nop_full in stack_pop_acquire if atomic intrinsics used (x86)Ivan Maidanski2016-08-191-0/+34
| | | | | | | | | * src/atomic_ops/generalize.h [AO_HAVE_double_compare_and_swap_acquire] (AO_compare_double_and_swap_double_acquire): Implement (by redirecting to AO_double_compare_and_swap_acquire). * src/atomic_ops/generalize.h [AO_HAVE_double_compare_and_swap_release] (AO_compare_double_and_swap_double_release): Implement (by redirecting to AO_double_compare_and_swap_release).
* Add asm-based and/or/xor implementation for char/short/int (gcc/x86)Ivan Maidanski2016-08-181-0/+81
| | | | | | | | | | * src/atomic_ops/sysdeps/gcc/x86.h [AO_DISABLE_GCC_ATOMICS] (AO_char_and_full, AO_char_or_full, AO_char_xor_full, AO_short_and_full, AO_short_or_full, AO_short_xor_full): New inline function. * src/atomic_ops/sysdeps/gcc/x86.h [AO_DISABLE_GCC_ATOMICS && !AO_T_IS_INT] (AO_int_and_full, AO_int_or_full, AO_int_xor_full): Likewise.
* Fix (delete) comment for AO_and_full (x86)Ivan Maidanski2016-08-172-2/+0
| | | | | | | | * src/atomic_ops/sysdeps/gcc/x86.h [!AO_PREFER_GENERALIZED] (AO_and_full): Remove incorrect comment (the code should work even for i386). * src/atomic_ops/sysdeps/sunc/x86.h [!AO_PREFER_GENERALIZED] (AO_and_full): Likewise.
* gcc/generic-small.h: Add TODO itemsHans Boehm2016-05-122-5/+60
| | | | | | | * src/atomic_ops/sysdeps/gcc/generic-small.h: Regenerate. * src/atomic_ops/sysdeps/gcc/generic-small.template: Add TODO items for XSIZE_load_read, XSIZE_load_full, XSIZE_load_dd_acquire_read, XSIZE_store_write.
* Basic support of TILE-Gx and TILEPro CPUsChris Metcalf2016-05-111-0/+50
| | | | | | | | | This patch is an updated version of the CentOS 6 patch that we have been carrying as part of our own CentOS-like distribution since 2012. * src/Makefile.am (nobase_private_HEADERS): Add tile.h. * src/atomic_ops.h [__tile__]: Include tile.h file. * src/atomic_ops/sysdeps/gcc/tile.h: New file.
* Use GCC atomic intrinsics for x86 and x64 (gcc 4.8+ and clang 3.4+)Ivan Maidanski2016-04-253-22/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/generic.h (AO_GCC_HAVE_double_SYNC_CAS): Do not define if AO_SKIPATOMIC_double_compare_and_swap_ANY. * src/atomic_ops/sysdeps/gcc/x86.h (AO_GCC_ATOMIC_TEST_AND_SET): Define if gcc4.8+ (but not Intel compiler) or clang3.4+ (if not AO_DISABLE_GCC_ATOMICS). * src/atomic_ops/sysdeps/gcc/x86.h [AO_GCC_ATOMIC_TEST_AND_SET]: Do not include all_aligned_atomic_load_store.h, test_and_set_t_is_char.h, ordered_except_wr.h. * src/atomic_ops/sysdeps/gcc/x86.h (AO_nop_full, AO_fetch_and_add_full, AO_char_fetch_and_add_full, AO_short_fetch_and_add_full, AO_and_full, AO_or_full, AO_xor_full, AO_test_and_set_full, AO_compare_and_swap_full) [AO_GCC_ATOMIC_TEST_AND_SET]: Do not define. * src/atomic_ops/sysdeps/gcc/x86.h: Include standard_ao_double_t.h if AO_PREFER_BUILTIN_ATOMICS, or AO_GCC_ATOMIC_TEST_AND_SET and not clang/x86. * src/atomic_ops/sysdeps/gcc/x86.h (AO_double_load_acquire, AO_double_store_release, AO_SKIPATOMIC_double_compare_and_swap_ANY): Define if AO_GCC_ATOMIC_TEST_AND_SET and clang/x86 (unless AO_PREFER_BUILTIN_ATOMICS). * src/atomic_ops/sysdeps/gcc/x86.h (AO_compare_double_and_swap_double_full) [x86]: Define also for clang/x86 if AO_GCC_ATOMIC_TEST_AND_SET (unless AO_PREFER_BUILTIN_ATOMICS). * src/atomic_ops/sysdeps/gcc/x86.h (AO_compare_double_and_swap_double_full) [x86_64]: Remove TODO item; check __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 to detect presence of cmpxchg16b. * src/atomic_ops/sysdeps/gcc/x86.h: Include generic.h if AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/standard_ao_double_t.h (double_ptr_storage): Use unsigned __int128 instead of __m128 for GCC x86_64 atomic intrinsics.
* Do not force GCC atomic intrinsics usage if AO_PREFER_BUILTIN_ATOMICSIvan Maidanski2016-04-212-6/+2
| | | | | | | | | | The macro should only prevent usage of assembly completely but only if at least some of GCC atomic intrinsics are available. * src/atomic_ops/sysdeps/gcc/arm.h (AO_PREFER_BUILTIN_ATOMICS): Remove from the condition whether to include generic.h. * src/atomic_ops/sysdeps/gcc/mips.h (AO_PREFER_BUILTIN_ATOMICS): Likewise.
* Remove AO_T_IS_INT definition for GCC atomic intrinsics caseIvan Maidanski2016-04-183-9/+4
| | | | | | | | | | | | (Code refactoring) In case of GCC/C11 atomic intrinsics are used, AO_int_* are defined in generic-arithm.h and generic-small.h files. * src/atomic_ops/sysdeps/gcc/arm.h (AO_T_IS_INT): Do not define if AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/gcc/mips.h (AO_T_IS_INT): Likewise. * src/atomic_ops/sysdeps/gcc/nios2.h (AO_T_IS_INT): Do not define.
* Fix AO_TS_VAL_t used for __atomic_test_and_set (gcc)Ivan Maidanski2016-04-156-21/+8
| | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/aarch64.h (AO_GCC_ATOMIC_TEST_AND_SET): Remove. * src/atomic_ops/sysdeps/gcc/mips.h [!AO_DISABLE_GCC_ATOMICS] (AO_GCC_ATOMIC_TEST_AND_SET): Likewise. * src/atomic_ops/sysdeps/gcc/nios2.h (AO_GCC_ATOMIC_TEST_AND_SET): Likewise. * src/atomic_ops/sysdeps/test_and_set_t_is_ao_t.h (AO_TS_SET_TRUEVAL): Likewise. * src/atomic_ops/sysdeps/gcc/aarch64.h: Remove include test_and_set_t_is_ao_t.h file. * src/atomic_ops/sysdeps/gcc/mips.h [!AO_DISABLE_GCC_ATOMICS]: Likewise. * src/atomic_ops/sysdeps/gcc/nios2.h: Likewise. * src/atomic_ops/sysdeps/gcc/arm.h: Include test_and_set_t_is_ao_t.h only if not AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/gcc/generic.h (AO_GCC_ATOMIC_TEST_AND_SET): Define macro. * src/atomic_ops/sysdeps/gcc/generic.h: Include test_and_set_t_is_char.h file. * src/atomic_ops/sysdeps/test_and_set_t_is_ao_t.h (AO_TS_val): Define AO_TS_set to 1; update comment.
* Fix typo in gcc/arm.h (ARMv6)Ivan Maidanski2016-04-151-1/+1
| | | | | * src/atomic_ops/sysdeps/gcc/arm.h (AO_SKIPATOMIC_ANY_and_ANY): Fix CPU architecture version in comment (ARMv6).
* Avoid __atomic_and/or/xor_fetch if unsupported (clang/arm-v5te)atomics-expIvan Maidanski2016-04-143-338/+448
| | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/arm.h (AO_SKIPATOMIC_ANY_and_ANY, AO_SKIPATOMIC_ANY_or_ANY, AO_SKIPATOMIC_ANY_xor_ANY): Define new internal macro (before include "generic.h") if AO_GCC_ATOMIC_TEST_AND_SET and __clang__, and not AO_ARM_HAVE_LDREX. * src/atomic_ops/sysdeps/gcc/generic-arithm.h: Regenerate. * src/atomic_ops/sysdeps/gcc/generic-arithm.template (AO_XSIZE_and_XBAR): Do not define if AO_SKIPATOMIC_ANY_and_ANY. * src/atomic_ops/sysdeps/gcc/generic-arithm.template (AO_XSIZE_or_XBAR): Do not define if AO_SKIPATOMIC_ANY_or_ANY. * src/atomic_ops/sysdeps/gcc/generic-arithm.template (AO_XSIZE_xor_XBAR): Do not define if AO_SKIPATOMIC_ANY_xor_ANY.
* Use GCC atomic intrinsics for ARM (gcc 4.8+ and clang 3.5+)Ivan Maidanski2016-04-123-85/+151
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/arm.h (AO_GCC_ATOMIC_TEST_AND_SET): Define if gcc 4.8+ or clang 3.5+ or AO_PREFER_BUILTIN_ATOMICS unless AO_DISABLE_GCC_ATOMICS. * src/atomic_ops/sysdeps/gcc/arm.h (AO_nop_write, AO_store, AO_char_store, AO_short_store): Do not define (in assembly code) if AO_PREFER_BUILTIN_ATOMICS. * src/atomic_ops/sysdeps/gcc/arm.h (AO_SKIPATOMIC_store, AO_SKIPATOMIC_store_release, AO_SKIPATOMIC_char_store, AO_SKIPATOMIC_char_store_release, AO_SKIPATOMIC_short_store, AO_SKIPATOMIC_short_store_release, AO_SKIPATOMIC_int_store, AO_SKIPATOMIC_int_store_release): Define if AO_BROKEN_TASKSWITCH_CLREX. * src/atomic_ops/sysdeps/gcc/arm.h (AO_nop_full, AO_test_and_set, AO_fetch_and_add, AO_fetch_and_add1, AO_fetch_and_sub1, AO_and, AO_or, AO_xor, AO_char_fetch_and_add, AO_short_fetch_and_add, AO_compare_and_swap, AO_fetch_compare_and_swap, AO_double_load, AO_double_store, AO_double_compare_and_swap, AO_test_and_set_full): Do not define (in assembly code) if AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/gcc/arm.h: Do not include atomic_store.h, all_atomic_only_load.h, char_atomic_store.h, short_atomic_store.h, all_aligned_atomic_load_store.h if AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/gcc/arm.h: Include generic.h (at the end of the file) if AO_GCC_ATOMIC_TEST_AND_SET. * src/atomic_ops/sysdeps/gcc/generic-small.h: Regenerate. * src/atomic_ops/sysdeps/gcc/generic-small.template (AO_XSIZE_store): Do not define if AO_SKIPATOMIC_XSIZE_store. * src/atomic_ops/sysdeps/gcc/generic-small.template (AO_XSIZE_store_release): Do not define if AO_SKIPATOMIC_XSIZE_store_release.
* Use GCC atomic intrinsics for MIPS (GCC 4.9+ and clang 3.5+)Ivan Maidanski2016-04-081-14/+38
| | | | | | | | | | * src/atomic_ops/sysdeps/gcc/mips.h: Include generic.h and do not include all_aligned_atomic_load_store.h if AO_PREFER_BUILTIN_ATOMICS or GCC 4.9+ or clang 3.5+ unless AO_DISABLE_GCC_ATOMICS. * src/atomic_ops/sysdeps/gcc/mips.h (AO_MIPS_SET_ISA, AO_MIPS_LL_1, AO_MIPS_SC, AO_MIPS_LL, AO_nop_full, AO_fetch_and_add, AO_test_and_set, AO_compare_and_swap, AO_fetch_compare_and_swap): Do not define if generic.h included.
* Implement CAS_acquire/release/full using __atomic_compare_exchange_n (gcc)Ivan Maidanski2016-04-083-31/+404
| | | | | | | | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/generic-small.h: Regenerate. * src/atomic_ops/sysdeps/gcc/generic-small.template (AO_XSIZE_fetch_compare_and_swap, AO_XSIZE_compare_and_swap): Use __atomic_compare_exchange_n(RELAXED) instead of __sync_val/bool_compare_and_swap. * src/atomic_ops/sysdeps/gcc/generic.h (AO_double_compare_and_swap): Likewise. * src/atomic_ops/sysdeps/gcc/generic-small.template [AO_GCC_HAVE_XSIZE_SYNC_CAS] (AO_XSIZE_fetch_compare_and_swap_acquire, AO_XSIZE_fetch_compare_and_swap_release, AO_XSIZE_fetch_compare_and_swap_full): New inline function; remove TODO item. * src/atomic_ops/sysdeps/gcc/generic-small.template [AO_GCC_HAVE_XSIZE_SYNC_CAS && !AO_GENERALIZE_ASM_BOOL_CAS] (AO_XSIZE_compare_and_swap_acquire, AO_XSIZE_compare_and_swap_release, AO_XSIZE_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/gcc/generic.h [AO_GCC_HAVE_double_SYNC_CAS] (AO_double_compare_and_swap_acquire, AO_double_compare_and_swap_release, AO_double_compare_and_swap_full): Likewise. * src/atomic_ops/sysdeps/gcc/generic.h (AO_compare_and_swap_acquire): Reformat code.
* Avoid atomic_compare_exchange_n if no __GCC_HAVE_SYNC_COMPARE_AND_SWAP_nIvan Maidanski2016-04-074-96/+144
| | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/aarch64.h (AO_GCC_FORCE_HAVE_CAS, AO_GCC_HAVE_double_SYNC_CAS): Define macro before include generic.h if __clang__ (workaround). * src/atomic_ops/sysdeps/gcc/generic-small.h: Regenerate. * src/atomic_ops/sysdeps/gcc/generic-small.template (AO_XSIZE_fetch_compare_and_swap, AO_XSIZE_compare_and_swap): Do not define unless AO_GCC_HAVE_XSIZE_SYNC_CAS. * src/atomic_ops/sysdeps/gcc/generic.h (AO_GCC_HAVE_char_SYNC_CAS, AO_GCC_HAVE_short_SYNC_CAS, AO_GCC_HAVE_int_SYNC_CAS, AO_GCC_HAVE_SYNC_CAS): New internal macro (based on __GCC_HAVE_SYNC_COMPARE_AND_SWAP_<n> or AO_GCC_FORCE_HAVE_CAS presence). * src/atomic_ops/sysdeps/gcc/generic.h (AO_GCC_HAVE_double_SYNC_CAS): New internal macro if AO_HAVE_DOUBLE_PTR_STORAGE. * src/atomic_ops/sysdeps/gcc/generic.h (AO_double_compare_and_swap): Check AO_GCC_HAVE_double_SYNC_CAS instead of AO_HAVE_DOUBLE_PTR_STORAGE.
* Eliminate redundant lwsync 2nd call in CAS_full on fail (gcc/PowerPC)Ivan Maidanski2016-04-061-2/+4
| | | | | | * src/atomic_ops/sysdeps/gcc/powerpc.h (AO_compare_and_swap_full, AO_fetch_compare_and_swap_full): Do not call AO_lwsync 2nd time if CAS is not successful.
* Define AO_TS_SET as __GCC_ATOMIC_TEST_AND_SET_TRUEVAL if applicableIvan Maidanski2016-04-044-6/+28
| | | | | | | | | | | | | | | | | | | * src/atomic_ops/sysdeps/gcc/aarch64.h (AO_GCC_ATOMIC_TEST_AND_SET): Define (before include test_and_set_t_is_ao_t.h). * src/atomic_ops/sysdeps/gcc/nios2.h (AO_GCC_ATOMIC_TEST_AND_SET): Likewise. * src/atomic_ops/sysdeps/test_and_set_t_is_ao_t.h: Refine comment. * src/atomic_ops/sysdeps/test_and_set_t_is_char.h: Likewise. * src/atomic_ops/sysdeps/test_and_set_t_is_ao_t.h (AO_TS_SET_TRUEVAL): New internal macro (defined via __GCC_ATOMIC_TEST_AND_SET_TRUEVAL). * src/atomic_ops/sysdeps/test_and_set_t_is_char.h (AO_TS_SET_TRUEVAL): Likewise. * src/atomic_ops/sysdeps/test_and_set_t_is_ao_t.h (AO_TS_val): Use AO_TS_SET_TRUEVAL instead of 1. * src/atomic_ops/sysdeps/test_and_set_t_is_char.h (AO_BYTE_TS_val): Use AO_TS_SET_TRUEVAL instead of 0xff; reformat code. * src/atomic_ops/sysdeps/test_and_set_t_is_char.h (AO_TS_t): Move down (to be after AO_TS_SET).
View Lorry Controller Status