Changeset 205921 in webkit


Ignore:
Timestamp:
Sep 14, 2016 11:51:26 AM (8 years ago)
Author:
jfbastien@apple.com
Message:

Atomics on ARM don't require full-system fencing, and other minutiae
https://bugs.webkit.org/show_bug.cgi?id=161928

Reviewed by Geoffrey Garen.

Add cmpxchg versions with both success and failure memory
ordering. In some interesting cases we can craft code which needs
barriers which aren't as strong.

weakCompareAndSwap is super dubious, its 3 uses seem
questionable... but for now I'm just adding debug asserts.

Rename armv7_dmb* functions to arm_dmb* because they apply to v7
and v8 (or more precisely; to ARMv7's ARM and Thumb2, as well as
ARMv8's aarch32 A32/T32 and aarch64).

Use inner-shareability domain for ARM barriers instead of
full-system. This is what C++ uses.

The default case for barriers simply used a compiler barrier. This
is generally wrong, e.g. for MIPS.

  • wtf/Atomics.h:

(WTF::Atomic::compareExchangeWeak): offer two-order version
(WTF::Atomic::compareExchangeStrong): offer two-order version
(WTF::weakCompareAndSwap): a few assertions
(WTF::arm_dmb): rename since it applies to ARMv7 and v8; make it innser-shareable
(WTF::arm_dmb_st): rename since it applies to ARMv7 and v8; make it innser-shareable
(WTF::loadLoadFence): incorrect generally
(WTF::loadStoreFence): incorrect generally
(WTF::storeLoadFence): incorrect generally
(WTF::storeStoreFence): incorrect generally
(WTF::memoryBarrierAfterLock): incorrect generally
(WTF::memoryBarrierBeforeUnlock): incorrect generally
(WTF::armV7_dmb): Deleted.
(WTF::armV7_dmb_st): Deleted.

Location:
trunk/Source/WTF
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WTF/ChangeLog

    r205914 r205921  
     12016-09-14  JF Bastien  <jfbastien@apple.com>
     2
     3        Atomics on ARM don't require full-system fencing, and other minutiae
     4        https://bugs.webkit.org/show_bug.cgi?id=161928
     5
     6        Reviewed by Geoffrey Garen.
     7
     8        Add cmpxchg versions with both success and failure memory
     9        ordering. In some interesting cases we can craft code which needs
     10        barriers which aren't as strong.
     11
     12        weakCompareAndSwap is super dubious, its 3 uses seem
     13        questionable... but for now I'm just adding debug asserts.
     14
     15        Rename armv7_dmb* functions to arm_dmb* because they apply to v7
     16        and v8 (or more precisely; to ARMv7's ARM and Thumb2, as well as
     17        ARMv8's aarch32 A32/T32 and aarch64).
     18
     19        Use inner-shareability domain for ARM barriers instead of
     20        full-system. This is what C++ uses.
     21
     22        The default case for barriers simply used a compiler barrier. This
     23        is generally wrong, e.g. for MIPS.
     24
     25        * wtf/Atomics.h:
     26        (WTF::Atomic::compareExchangeWeak): offer two-order version
     27        (WTF::Atomic::compareExchangeStrong): offer two-order version
     28        (WTF::weakCompareAndSwap): a few assertions
     29        (WTF::arm_dmb): rename since it applies to ARMv7 and v8; make it innser-shareable
     30        (WTF::arm_dmb_st): rename since it applies to ARMv7 and v8; make it innser-shareable
     31        (WTF::loadLoadFence): incorrect generally
     32        (WTF::loadStoreFence): incorrect generally
     33        (WTF::storeLoadFence): incorrect generally
     34        (WTF::storeStoreFence): incorrect generally
     35        (WTF::memoryBarrierAfterLock): incorrect generally
     36        (WTF::memoryBarrierBeforeUnlock): incorrect generally
     37        (WTF::armV7_dmb): Deleted.
     38        (WTF::armV7_dmb_st): Deleted.
     39
    1402016-09-14  JF Bastien  <jfbastien@apple.com>
    241
  • trunk/Source/WTF/wtf/Atomics.h

    r205914 r205921  
    6868    }
    6969
     70    ALWAYS_INLINE bool compareExchangeWeak(T expected, T desired, std::memory_order order_success, std::memory_order order_failure)
     71    {
     72#if OS(WINDOWS)
     73        // Windows makes strange assertions about the argument to compare_exchange_weak, and anyway,
     74        // Windows is X86 so seq_cst is cheap.
     75        order_success = std::memory_order_seq_cst;
     76        order_failure = std::memory_order_seq_cst;
     77#endif
     78        T expectedOrActual = expected;
     79        return value.compare_exchange_weak(expectedOrActual, desired, order_success, order_failure);
     80    }
     81
    7082    ALWAYS_INLINE bool compareExchangeStrong(T expected, T desired, std::memory_order order = std::memory_order_seq_cst)
    7183    {
     
    7789        return value.compare_exchange_strong(expectedOrActual, desired, order);
    7890    }
    79    
     91
     92    ALWAYS_INLINE bool compareExchangeStrong(T expected, T desired, std::memory_order order_success, std::memory_order order_failure)
     93    {
     94#if OS(WINDOWS)
     95        // See above.
     96        order_success = std::memory_order_seq_cst;
     97        order_failure = std::memory_order_seq_cst;
     98#endif
     99        T expectedOrActual = expected;
     100        return value.compare_exchange_strong(expectedOrActual, desired, order_success, order_failure);
     101    }
     102
    80103    template<typename U>
    81104    ALWAYS_INLINE T exchangeAndAdd(U addend, std::memory_order order = std::memory_order_seq_cst)
     
    104127inline bool weakCompareAndSwap(volatile T* location, T expected, T newValue)
    105128{
     129    ASSERT(isPointerTypeAlignmentOkay(location) && "natural alignment required");
     130    ASSERT(bitwise_cast<std::atomic<T>*>(location)->is_lock_free() && "expected lock-free type");
    106131    return bitwise_cast<Atomic<T>*>(location)->compareExchangeWeak(expected, newValue, std::memory_order_relaxed);
    107132}
     
    123148// Full memory fence. No accesses will float above this, and no accesses will sink
    124149// below it.
    125 inline void armV7_dmb()
    126 {
    127     asm volatile("dmb sy" ::: "memory");
     150inline void arm_dmb()
     151{
     152    asm volatile("dmb ish" ::: "memory");
    128153}
    129154
    130155// Like the above, but only affects stores.
    131 inline void armV7_dmb_st()
    132 {
    133     asm volatile("dmb st" ::: "memory");
    134 }
    135 
    136 inline void loadLoadFence() { armV7_dmb(); }
    137 inline void loadStoreFence() { armV7_dmb(); }
    138 inline void storeLoadFence() { armV7_dmb(); }
    139 inline void storeStoreFence() { armV7_dmb_st(); }
    140 inline void memoryBarrierAfterLock() { armV7_dmb(); }
    141 inline void memoryBarrierBeforeUnlock() { armV7_dmb(); }
     156inline void arm_dmb_st()
     157{
     158    asm volatile("dmb ishst" ::: "memory");
     159}
     160
     161inline void loadLoadFence() { arm_dmb(); }
     162inline void loadStoreFence() { arm_dmb(); }
     163inline void storeLoadFence() { arm_dmb(); }
     164inline void storeStoreFence() { arm_dmb_st(); }
     165inline void memoryBarrierAfterLock() { arm_dmb(); }
     166inline void memoryBarrierBeforeUnlock() { arm_dmb(); }
    142167
    143168#elif CPU(X86) || CPU(X86_64)
     
    165190#else
    166191
    167 inline void loadLoadFence() { compilerFence(); }
    168 inline void loadStoreFence() { compilerFence(); }
    169 inline void storeLoadFence() { compilerFence(); }
    170 inline void storeStoreFence() { compilerFence(); }
    171 inline void memoryBarrierAfterLock() { compilerFence(); }
    172 inline void memoryBarrierBeforeUnlock() { compilerFence(); }
     192inline void loadLoadFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
     193inline void loadStoreFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
     194inline void storeLoadFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
     195inline void storeStoreFence() { std::atomic_thread_fence(std::memory_order_seq_cst); }
     196inline void memoryBarrierAfterLock() { std::atomic_thread_fence(std::memory_order_seq_cst); }
     197inline void memoryBarrierBeforeUnlock() { std::atomic_thread_fence(std::memory_order_seq_cst); }
    173198
    174199#endif
Note: See TracChangeset for help on using the changeset viewer.