Changeset 220807 in webkit


Ignore:
Timestamp:
Aug 16, 2017 1:38:57 PM (7 years ago)
Author:
mark.lam@apple.com
Message:

Enhance MacroAssembler::probe() to support an initializeStackFunction callback.
​https://bugs.webkit.org/show_bug.cgi?id=175617
<rdar://problem/33912104>

Reviewed by JF Bastien.

This patch adds a new feature to MacroAssembler::probe() where the probe function
can provide a ProbeFunction callback to fill in stack values after the stack
pointer has been adjusted. The probe function can use this feature as follows:

  1. Set the new sp value in the ProbeContext's CPUState.
  1. Set the ProbeContext's initializeStackFunction to a ProbeFunction callback which will do the work of filling in the stack values after the probe trampoline has adjusted the machine stack pointer.
  1. Set the ProbeContext's initializeStackArgs to any value that the client wants to pass to the initializeStackFunction callback.
  1. Return from the probe function.

Upon returning from the probe function, the probe trampoline will adjust the
the stack pointer based on the sp value in CPUState. If initializeStackFunction
is not set, the probe trampoline will restore registers and return to its caller.

If initializeStackFunction is set, the trampoline will move the ProbeContext
beyond the range of the stack pointer i.e. it will place the new ProbeContext at
an address lower than where CPUState.sp() points. This ensures that the
ProbeContext will not be trashed by the initializeStackFunction when it writes to
the stack. Then, the trampoline will call back to the initializeStackFunction
ProbeFunction to let it fill in the stack values as desired. The
initializeStackFunction ProbeFunction will be passed the moved ProbeContext at
the new location.

initializeStackFunction may now write to the stack at addresses greater or
equal to CPUState.sp(), but not below that. initializeStackFunction is also
not allowed to change CPUState.sp(). If the initializeStackFunction does not
abide by these rules, then behavior is undefined, and bad things may happen.

For future reference, some implementation details that this patch needed to
be mindful of:

  1. When the probe trampoline allocates stack space for the ProbeContext, it should include OUT_SIZE as well. This ensures that it doesn't have to move the ProbeContext on exit if the probe function didn't change the sp.
  1. If the trampoline has to move the ProbeContext, it needs to point the machine sp to new ProbeContext first before copying over the ProbeContext data. This protects the new ProbeContext from possibly being trashed by interrupts.
  1. When computing the new address of ProbeContext to move to, we need to make sure that it is properly aligned in accordance with stack ABI requirements (just like we did when we allocated the ProbeContext on entry to the probe trampoline).
  1. When copying the ProbeContext to its new location, the trampoline should always copy words from low addresses to high addresses. This is because if we're moving the ProbeContext, we'll always be moving it to a lower address.
  • assembler/MacroAssembler.h:
  • assembler/MacroAssemblerARM.cpp:
  • assembler/MacroAssemblerARM64.cpp:
  • assembler/MacroAssemblerARMv7.cpp:
  • assembler/MacroAssemblerX86Common.cpp:
  • assembler/testmasm.cpp:

(JSC::testProbePreservesGPRS):
(JSC::testProbeModifiesStackPointer):
(JSC::fillStack):
(JSC::testProbeModifiesStackWithCallback):
(JSC::run):

Location:
trunk/Source/JavaScriptCore
Files:
7 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ChangeLog

    r220791 r220807  
     12017-08-16  Mark Lam  <mark.lam@apple.com>
     2
     3        Enhance MacroAssembler::probe() to support an initializeStackFunction callback.
     4        https://bugs.webkit.org/show_bug.cgi?id=175617
     5        <rdar://problem/33912104>
     6
     7        Reviewed by JF Bastien.
     8
     9        This patch adds a new feature to MacroAssembler::probe() where the probe function
     10        can provide a ProbeFunction callback to fill in stack values after the stack
     11        pointer has been adjusted.  The probe function can use this feature as follows:
     12
     13        1. Set the new sp value in the ProbeContext's CPUState.
     14
     15        2. Set the ProbeContext's initializeStackFunction to a ProbeFunction callback
     16           which will do the work of filling in the stack values after the probe
     17           trampoline has adjusted the machine stack pointer.
     18
     19        3. Set the ProbeContext's initializeStackArgs to any value that the client wants
     20           to pass to the initializeStackFunction callback.
     21
     22        4. Return from the probe function.
     23
     24        Upon returning from the probe function, the probe trampoline will adjust the
     25        the stack pointer based on the sp value in CPUState.  If initializeStackFunction
     26        is not set, the probe trampoline will restore registers and return to its caller.
     27
     28        If initializeStackFunction is set, the trampoline will move the ProbeContext
     29        beyond the range of the stack pointer i.e. it will place the new ProbeContext at
     30        an address lower than where CPUState.sp() points.  This ensures that the
     31        ProbeContext will not be trashed by the initializeStackFunction when it writes to
     32        the stack.  Then, the trampoline will call back to the initializeStackFunction
     33        ProbeFunction to let it fill in the stack values as desired.  The
     34        initializeStackFunction ProbeFunction will be passed the moved ProbeContext at
     35        the new location.
     36
     37        initializeStackFunction may now write to the stack at addresses greater or
     38        equal to CPUState.sp(), but not below that.  initializeStackFunction is also
     39        not allowed to change CPUState.sp().  If the initializeStackFunction does not
     40        abide by these rules, then behavior is undefined, and bad things may happen.
     41
     42        For future reference, some implementation details that this patch needed to
     43        be mindful of:
     44
     45        1. When the probe trampoline allocates stack space for the ProbeContext, it
     46           should include OUT_SIZE as well.  This ensures that it doesn't have to move
     47           the ProbeContext on exit if the probe function didn't change the sp.
     48
     49        2. If the trampoline has to move the ProbeContext, it needs to point the machine
     50           sp to new ProbeContext first before copying over the ProbeContext data.  This
     51           protects the new ProbeContext from possibly being trashed by interrupts.
     52
     53        3. When computing the new address of ProbeContext to move to, we need to make
     54           sure that it is properly aligned in accordance with stack ABI requirements
     55           (just like we did when we allocated the ProbeContext on entry to the
     56           probe trampoline).
     57
     58        4. When copying the ProbeContext to its new location, the trampoline should
     59           always copy words from low addresses to high addresses.  This is because if
     60           we're moving the ProbeContext, we'll always be moving it to a lower address.
     61
     62        * assembler/MacroAssembler.h:
     63        * assembler/MacroAssemblerARM.cpp:
     64        * assembler/MacroAssemblerARM64.cpp:
     65        * assembler/MacroAssemblerARMv7.cpp:
     66        * assembler/MacroAssemblerX86Common.cpp:
     67        * assembler/testmasm.cpp:
     68        (JSC::testProbePreservesGPRS):
     69        (JSC::testProbeModifiesStackPointer):
     70        (JSC::fillStack):
     71        (JSC::testProbeModifiesStackWithCallback):
     72        (JSC::run):
     73
    1742017-08-16  Csaba Osztrogonác  <ossy@webkit.org>
    275

  • trunk/Source/JavaScriptCore/assembler/MacroAssembler.h

    r220720 r220807  
    18431843    // of the call to the user probe function.
    18441844    //
     1845    // The probe function may choose to move the stack pointer (in any direction).
     1846    // To do this, the probe function needs to set the new sp value in the CPUState.
     1847    //
     1848    // The probe function may also choose to fill stack space with some values.
     1849    // To do this, the probe function must first:
     1850    // 1. Set the new sp value in the ProbeContext's CPUState.
     1851    // 2. Set the ProbeContext's initializeStackFunction to a ProbeFunction callback
     1852    //    which will do the work of filling in the stack values after the probe
     1853    //    trampoline has adjusted the machine stack pointer.
     1854    // 3. Set the ProbeContext's initializeStackArgs to any value that the client wants
     1855    //    to pass to the initializeStackFunction callback.
     1856    // 4. Return from the probe function.
     1857    //
     1858    // Upon returning from the probe function, the probe trampoline will adjust the
     1859    // the stack pointer based on the sp value in CPUState. If initializeStackFunction
     1860    // is not set, the probe trampoline will restore registers and return to its caller.
     1861    //
     1862    // If initializeStackFunction is set, the trampoline will move the ProbeContext
     1863    // beyond the range of the stack pointer i.e. it will place the new ProbeContext at
     1864    // an address lower than where CPUState.sp() points. This ensures that the
     1865    // ProbeContext will not be trashed by the initializeStackFunction when it writes to
     1866    // the stack. Then, the trampoline will call back to the initializeStackFunction
     1867    // ProbeFunction to let it fill in the stack values as desired. The
     1868    // initializeStackFunction ProbeFunction will be passed the moved ProbeContext at
     1869    // the new location.
     1870    //
     1871    // initializeStackFunction may now write to the stack at addresses greater or
     1872    // equal to CPUState.sp(), but not below that. initializeStackFunction is also
     1873    // not allowed to change CPUState.sp(). If the initializeStackFunction does not
     1874    // abide by these rules, then behavior is undefined, and bad things may happen.
     1875    //
    18451876    // Note: this version of probe() should be implemented by the target specific
    18461877    // MacroAssembler.
    … …  
    19962027    ProbeFunction probeFunction;
    19972028    void* arg;
     2029    ProbeFunction initializeStackFunction;
     2030    void* initializeStackArg;
    19982031    CPUState cpu;
    19992032

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM.cpp

    r220629 r220807  
    100100
    101101#if COMPILER(GCC_OR_CLANG)
    102    
     102
    103103// The following are offsets for ProbeContext fields accessed
    104104// by the ctiMasmProbeTrampoline stub.
    … …  
    107107#define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE)
    108108#define PROBE_ARG_OFFSET (1 * PTR_SIZE)
    109 
    110 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE)
     109#define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE)
     110#define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE)
     111
     112#define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE)
    111113
    112114#define GPREG_SIZE 4
    … …  
    168170
    169171#define PROBE_SIZE (PROBE_FIRST_FPREG_OFFSET + (32 * FPREG_SIZE))
    170 #define PROBE_ALIGNED_SIZE (PROBE_SIZE)
     172
     173#define OUT_SIZE GPREG_SIZE
    171174
    172175// These ASSERTs remind you that if you change the layout of ProbeContext,
    … …  
    175178COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline);
    176179COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline);
     180COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline);
     181COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline);
    177182
    178183COMPILE_ASSERT(!(PROBE_CPU_R0_OFFSET & 0x3), ProbeContext_cpu_r0_offset_should_be_4_byte_aligned);
    … …  
    198203COMPILE_ASSERT(PROBE_OFFSETOF(cpu.sprs[ARMRegisters::fpscr]) == PROBE_CPU_FPSCR_OFFSET, ProbeContext_cpu_fpscr_offset_matches_ctiMasmProbeTrampoline);
    199204
    200 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0xf), ProbeContext_cpu_d0_offset_should_be_16_byte_aligned);
     205COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x7), ProbeContext_cpu_d0_offset_should_be_8_byte_aligned);
    201206
    202207COMPILE_ASSERT(PROBE_OFFSETOF(cpu.fprs[ARMRegisters::d0]) == PROBE_CPU_D0_OFFSET, ProbeContext_cpu_d0_offset_matches_ctiMasmProbeTrampoline);
    … …  
    234239
    235240COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline);
    236 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0xf), ProbeContext_aligned_size_offset_should_be_16_byte_aligned);
    237241#undef PROBE_OFFSETOF
    238242
    … …  
    255259    "mov       ip, sp" "\n"
    256260    "mov       r3, sp" "\n"
    257     "sub       r3, r3, #" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) "\n"
     261    "sub       r3, r3, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
    258262
    259263    // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    260264    "bic       r3, r3, #0xf" "\n"
    261     "mov       sp, r3" "\n"
     265    "mov       sp, r3" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it.
    262266
    263267    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    … …  
    285289
    286290    "add       ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_D0_OFFSET) "\n"
    287     "vstmia.64 ip, { d0-d15 }" "\n"
    288     "vstmia.64 ip, { d16-d31 }" "\n"
     291    "vstmia.64 ip!, { d0-d15 }" "\n"
     292    "vstmia.64 ip!, { d16-d31 }" "\n"
    289293
    290294    "mov       fp, sp" "\n" // Save the ProbeContext*.
     295
     296    // Initialize ProbeContext::initializeStackFunction to zero.
     297    "mov       r0, #0" "\n"
     298    "str       r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    291299
    292300    "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
    293301    "mov       r0, sp" "\n" // the ProbeContext* arg.
    294302    "blx       ip" "\n"
     303
     304    // Make sure the ProbeContext is entirely below the result stack pointer so
     305    // that register values are still preserved when we call the initializeStack
     306    // function.
     307    "ldr       r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
     308    "add       r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
     309    "cmp       r1, r2" "\n"
     310    "bge     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n"
     311
     312    // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext.
     313    "sub       r1, r1, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
     314    "bic       r1, r1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned.
     315    "mov       sp, r1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext.
     316
     317    // Copy the ProbeContext to the safe place.
     318    // Note: we have to copy from low address to higher address because we're moving the
     319    // ProbeContext to a lower address.
     320    "mov       r5, fp" "\n"
     321    "mov       r6, r1" "\n"
     322    "add       r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
     323
     324    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
     325    "ldr       r3, [r5], #4" "\n"
     326    "ldr       r4, [r5], #4" "\n"
     327    "str       r3, [r6], #4" "\n"
     328    "str       r4, [r6], #4" "\n"
     329    "cmp       r5, r7" "\n"
     330    "blt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
     331
     332    "mov       fp, r1" "\n"
     333
     334    // Call initializeStackFunction if present.
     335    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n"
     336    "ldr       r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     337    "cbz       r2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
     338
     339    "mov       r0, fp" "\n" // Set the ProbeContext* arg.
     340    "blx       r2" "\n" // Call the initializeStackFunction (loaded into r2 above).
     341
     342    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    295343
    296344    "mov       sp, fp" "\n"
    … …  
    308356
    309357    // There are 5 more registers left to restore: ip, sp, lr, pc, and apsr.
    310     // There are 2 issues that complicate the restoration of these last few
    311     // registers:
    312     //
    313     // 1. Normal ARM calling convention relies on moving lr to pc to return to
    314     //    the caller. In our case, the address to return to is specified by
    315     //    ProbeContext.cpu.pc. And at that moment, we won't have any available
    316     //    scratch registers to hold the return address (lr needs to hold
    317     //    ProbeContext.cpu.lr, not the return address).
    318     //
    319     //    The solution is to store the return address on the stack and load the
    320     //     pc from there.
    321     //
    322     // 2. Issue 1 means we will need to write to the stack location at
    323     //    ProbeContext.cpu.gprs[sp] - PTR_SIZE. But if the user probe function had
    324     //    modified the value of ProbeContext.cpu.gprs[sp] to point in the range between
    325     //    &ProbeContext.cpu.gprs[ip] thru &ProbeContext.cpu.sprs[aspr], then the action
    326     //    for Issue 1 may trash the values to be restored before we can restore them.
    327     //
    328     //    The solution is to check if ProbeContext.cpu.gprs[sp] contains a value in
    329     //    the undesirable range. If so, we copy the remaining ProbeContext
    330     //    register data to a safe area first, and restore the remaining register
    331     //    from this new safe area.
    332 
    333     // The restore area for the pc will be located at 1 word below the resultant sp.
    334     // All restore values are located at offset <= PROBE_CPU_APSR_OFFSET. Hence,
    335     // we need to make sure that resultant sp > offset of apsr + 1.
    336     "add       ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET + PTR_SIZE) "\n"
     358
     359    // Set up the restore area for sp and pc.
    337360    "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    338     "cmp       lr, ip" "\n"
    339     "bgt     " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"
    340 
    341     // Getting here means that the restore area will overlap the ProbeContext data
    342     // that we will need to get the restoration values from. So, let's move that
    343     // data to a safe place before we start writing into the restore area.
    344     // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the
    345     // restore area. This ensures that:
    346     // 1. The safe area does not overlap the restore area.
    347     // 2. The safe area does not overlap the ProbeContext.
    348     //    This makes it so that we can use memcpy (does not require memmove) semantics
    349     //    to copy the restore values to the safe area.
    350    
    351     // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].
    352     "sub       lr, lr, #(2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ")" "\n"
    353 
    354     "mov       ip, sp" "\n" // Save the original ProbeContext*.
    355 
    356     // Make sure the stack pointer points to the safe area. This ensures that the
    357     // safe area is protected from interrupt handlers overwriting it.
    358     "mov       sp, lr" "\n" // sp now points to the new ProbeContext in the safe area.
    359 
    360     "mov       lr, ip" "\n" // Use lr as the old ProbeContext*.
    361 
    362     // Copy the restore data to the new ProbeContext*.
    363     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    364     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    365     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    366     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    367     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    368     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    369     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    370     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    371     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
    372     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
    373 
    374     // ctiMasmProbeTrampolineEnd expects lr to contain the sp value to be restored.
    375     // Since we used it as scratch above, let's restore it.
    376     "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    377 
    378     SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"
    379 
    380     // Set up the restore area for sp and pc.
    381     // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].
    382361
    383362    // Push the pc on to the restore area.
    384363    "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    385     "sub       lr, lr, #" STRINGIZE_VALUE_OF(PTR_SIZE) "\n"
     364    "sub       lr, lr, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n"
    386365    "str       ip, [lr]" "\n"
    387366    // Point sp to the restore area.

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.cpp

    r220630 r220807  
    4444#define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE)
    4545#define PROBE_ARG_OFFSET (1 * PTR_SIZE)
    46 
    47 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE)
     46#define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE)
     47#define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE)
     48
     49#define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE)
    4850
    4951#define GPREG_SIZE 8
    … …  
    132134COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline);
    133135COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline);
     136COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline);
     137COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline);
    134138
    135139COMPILE_ASSERT(!(PROBE_CPU_X0_OFFSET & 0x7), ProbeContext_cpu_r0_offset_should_be_8_byte_aligned);
    … …  
    213217// Conditions for using ldp and stp.
    214218static_assert(PROBE_CPU_PC_OFFSET == PROBE_CPU_SP_OFFSET + PTR_SIZE, "PROBE_CPU_SP_OFFSET and PROBE_CPU_PC_OFFSET must be adjacent");
     219static_assert(!(PROBE_SIZE_PLUS_EXTRAS & 0xf), "PROBE_SIZE_PLUS_EXTRAS should be 16 byte aligned"); // the ProbeContext copying code relies on this.
    215220
    216221#undef PROBE_OFFSETOF
    … …  
    270275static_assert(OUT_LR_OFFSET == offsetof(OutgoingProbeRecord, lr), "OUT_LR_OFFSET is incorrect");
    271276static_assert(OUT_SIZE == sizeof(OutgoingProbeRecord), "OUT_SIZE is incorrect");
    272 static_assert(!(sizeof(OutgoingProbeRecord) & 0xf), "IncomingProbeStack must be 16-byte aligned");
     277static_assert(!(sizeof(OutgoingProbeRecord) & 0xf), "OutgoingProbeStack must be 16-byte aligned");
    273278
    274279#define STATE_PC_NOT_CHANGED 0
    … …  
    291296    "mov       x27, sp" "\n"
    292297
    293     "sub       x27, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS) "\n"
     298    "sub       x27, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n"
    294299    "bic       x27, x27, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    295     "mov       sp, x27" "\n" // Make sure interrupts don't over-write our data on the stack.
     300    "mov       sp, x27" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it.
    296301
    297302    "stp       x0, x1, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n"
    … …  
    351356    "mov       x27, sp" "\n" // Save the ProbeContext* in a callee saved register.
    352357
     358    // Initialize ProbeContext::initializeStackFunction to zero.
     359    "str       xzr, [x27, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     360
    353361    // Note: we haven't changed the value of fp. Hence, it is still pointing to the frame of
    354362    // the caller of the probe (which is what we want in order to play nice with debuggers e.g. lldb).
    355363    "mov       x0, sp" "\n" // Set the ProbeContext* arg.
    356364    "blr       x2" "\n" // Call the probe handler function (loaded into x2 above).
     365
     366    // Make sure the ProbeContext is entirely below the result stack pointer so
     367    // that register values are still preserved when we call the initializeStack
     368    // function.
     369    "ldr       x1, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
     370    "add       x2, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n" // End of ProbeContext + buffer.
     371    "cmp       x1, x2" "\n"
     372    "bge     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n"
     373
     374    // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext.
     375    "sub       x1, x1, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n"
     376    "bic       x1, x1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned.
     377    "mov       sp, x1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext.
     378
     379    // Copy the ProbeContext to the safe place.
     380    // Note: we have to copy from low address to higher address because we're moving the
     381    // ProbeContext to a lower address.
     382    "mov       x5, x27" "\n"
     383    "mov       x6, x1" "\n"
     384    "add       x7, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS) "\n"
     385
     386    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
     387    "ldp       x3, x4, [x5], #16" "\n"
     388    "stp       x3, x4, [x6], #16" "\n"
     389    "cmp       x5, x7" "\n"
     390    "blt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
     391
     392    "mov       x27, x1" "\n"
     393
     394    // Call initializeStackFunction if present.
     395    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n"
     396    "ldr       x2, [x27, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     397    "cbz       x2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
     398
     399    "mov       x0, x27" "\n" // Set the ProbeContext* arg.
     400    "blr       x2" "\n" // Call the initializeStackFunction (loaded into x2 above).
     401
     402    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    357403
    358404    "mov       sp, x27" "\n"
    … …  
    429475    "ldr       x29, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n" // Preload the probe return site pc.
    430476
    431     // The probe handler may have moved the sp. For the return process, we may need
    432     // space for 2 OutgoingProbeRecords below the final sp value. We need to make
    433     // sure that the space for these 2 OutgoingProbeRecords do not overlap the
    434     // restore values of the registers.
    435 
    436     // All restore values are located at offset <= PROBE_CPU_FPSR_OFFSET. Hence,
    437     // we need to make sure that resultant sp > offset of fpsr + 2 * sizeof(OutgoingProbeRecord).
    438 
    439     "add       x27, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FPSR_OFFSET + 2 * OUT_SIZE) "\n"
    440     "ldr       x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    441     "cmp       x28, x27" "\n"
    442     "bgt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineFillOutgoingProbeRecords) "\n"
    443 
    444     // There is overlap. We need to copy the ProbeContext to a safe area first.
    445     // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the OutgoingProbeRecords are.
    446     // This ensures that:
    447     // 1. The safe area does not overlap the OutgoingProbeRecords.
    448     // 2. The safe area does not overlap the ProbeContext.
    449 
    450     // x28 already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].
    451     "sub       x28, x28, #" STRINGIZE_VALUE_OF(2 * PROBE_SIZE) "\n"
    452 
    453     "mov       x27, sp" "\n" // Save the original ProbeContext*.
    454 
    455     // Make sure the stack pointer points to the safe area. This ensures that the
    456     // safe area is protected from interrupt handlers overwriting it.
    457     "mov       sp, x28" "\n" // sp now points to the new ProbeContext in the safe area.
    458 
    459     // Copy the relevant restore data to the new ProbeContext*.
    460     "str       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n" // Stash the pc changed state away so that we can use lr.
    461 
    462     "ldp       x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_X27_OFFSET) "]" "\n" // copy x27 and x28.
    463     "stp       x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X27_OFFSET) "]" "\n"
    464     "ldp       x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_FP_OFFSET) "]" "\n" // copy fp and lr.
    465     "stp       x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FP_OFFSET) "]" "\n"
    466     "ldp       x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // copy sp and pc.
    467     "stp       x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    468     "ldp       x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_NZCV_OFFSET) "]" "\n" // copy nzcv and fpsr.
    469     "stp       x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_NZCV_OFFSET) "]" "\n"
    470 
    471     "ldr       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n" // Retrieve the stashed the pc changed state.
    472 
    473477    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineFillOutgoingProbeRecords) ":" "\n"
    474478

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARMv7.cpp

    r220579 r220807  
    4343#define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE)
    4444#define PROBE_ARG_OFFSET (1 * PTR_SIZE)
    45 
    46 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE)
     45#define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE)
     46#define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE)
     47
     48#define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE)
    4749
    4850#define GPREG_SIZE 4
    … …  
    102104#define PROBE_CPU_D30_OFFSET (PROBE_FIRST_FPREG_OFFSET + (30 * FPREG_SIZE))
    103105#define PROBE_CPU_D31_OFFSET (PROBE_FIRST_FPREG_OFFSET + (31 * FPREG_SIZE))
     106
    104107#define PROBE_SIZE (PROBE_FIRST_FPREG_OFFSET + (32 * FPREG_SIZE))
    105 #define PROBE_ALIGNED_SIZE (PROBE_SIZE)
     108
     109#define OUT_SIZE GPREG_SIZE
    106110
    107111// These ASSERTs remind you that if you change the layout of ProbeContext,
    … …  
    110114COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline);
    111115COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline);
     116COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline);
     117COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline);
    112118
    113119COMPILE_ASSERT(!(PROBE_CPU_R0_OFFSET & 0x3), ProbeContext_cpu_r0_offset_should_be_4_byte_aligned);
    … …  
    133139COMPILE_ASSERT(PROBE_OFFSETOF(cpu.sprs[ARMRegisters::fpscr]) == PROBE_CPU_FPSCR_OFFSET, ProbeContext_cpu_fpscr_offset_matches_ctiMasmProbeTrampoline);
    134140
    135 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0xf), ProbeContext_cpu_d0_offset_should_be_16_byte_aligned);
     141COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x7), ProbeContext_cpu_d0_offset_should_be_8_byte_aligned);
    136142
    137143COMPILE_ASSERT(PROBE_OFFSETOF(cpu.fprs[ARMRegisters::d0]) == PROBE_CPU_D0_OFFSET, ProbeContext_cpu_d0_offset_matches_ctiMasmProbeTrampoline);
    … …  
    170176
    171177COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline);
    172 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0xf), ProbeContext_aligned_size_offset_should_be_16_byte_aligned);
    173 
    174178#undef PROBE_OFFSETOF
    175179   
    … …  
    194198    "mov       ip, sp" "\n"
    195199    "mov       r0, sp" "\n"
    196     "sub       r0, r0, #" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) "\n"
     200    "sub       r0, r0, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
    197201
    198202    // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    199203    "bic       r0, r0, #0xf" "\n"
    200     "mov       sp, r0" "\n"
     204    "mov       sp, r0" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it.
    201205
    202206    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    … …  
    229233    "mov       fp, sp" "\n" // Save the ProbeContext*.
    230234
     235    // Initialize ProbeContext::initializeStackFunction to zero.
     236    "mov       r0, #0" "\n"
     237    "str       r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     238
    231239    "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
    232240    "mov       r0, sp" "\n" // the ProbeContext* arg.
    233241    "blx       ip" "\n"
     242
     243    // Make sure the ProbeContext is entirely below the result stack pointer so
     244    // that register values are still preserved when we call the initializeStack
     245    // function.
     246    "ldr       r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
     247    "add       r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
     248    "cmp       r1, r2" "\n"
     249    "it        ge" "\n"
     250    "bge     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n"
     251
     252    // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext.
     253    "sub       r1, r1, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
     254    "bic       r1, r1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned.
     255    "mov       sp, r1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext.
     256
     257    // Copy the ProbeContext to the safe place.
     258    // Note: we have to copy from low address to higher address because we're moving the
     259    // ProbeContext to a lower address.
     260    "mov       r5, fp" "\n"
     261    "mov       r6, r1" "\n"
     262    "add       r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
     263
     264    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
     265    "ldr       r3, [r5], #4" "\n"
     266    "ldr       r4, [r5], #4" "\n"
     267    "str       r3, [r6], #4" "\n"
     268    "str       r4, [r6], #4" "\n"
     269    "cmp       r5, r7" "\n"
     270    "it        lt" "\n"
     271    "blt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
     272
     273    "mov       fp, r1" "\n"
     274
     275    // Call initializeStackFunction if present.
     276    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n"
     277    "ldr       r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     278    "cbz       r2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
     279
     280    "mov       r0, fp" "\n" // Set the ProbeContext* arg.
     281    "blx       r2" "\n" // Call the initializeStackFunction (loaded into r2 above).
     282
     283    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    234284
    235285    "mov       sp, fp" "\n"
    … …  
    248298
    249299    // There are 5 more registers left to restore: ip, sp, lr, pc, and apsr.
    250     // There are 2 issues that complicate the restoration of these last few
    251     // registers:
    252     //
    253     // 1. Normal ARM calling convention relies on moving lr to pc to return to
    254     //    the caller. In our case, the address to return to is specified by
    255     //    ProbeContext.cpu.gprs[pc]. And at that moment, we won't have any available
    256     //    scratch registers to hold the return address (lr needs to hold
    257     //    ProbeContext.cpu.gprs[lr], not the return address).
    258     //
    259     //    The solution is to store the return address on the stack and load the
    260     //    pc from there.
    261     //
    262     // 2. Issue 1 means we will need to write to the stack location at
    263     //    ProbeContext.cpu.gprs[sp] - PTR_SIZE. But if the user probe function had
    264     //    modified the value of ProbeContext.cpu.gprs[sp] to point in the range between
    265     //    &ProbeContext.cpu.gprs[ip] thru &ProbeContext.cpu.sprs[aspr], then the action
    266     //    for Issue 1 may trash the values to be restored before we can restore them.
    267     //
    268     //    The solution is to check if ProbeContext.cpu.gprs[sp] contains a value in
    269     //    the undesirable range. If so, we copy the remaining ProbeContext
    270     //    register data to a safe area first, and restore the remaining register
    271     //    from this new safe area.
    272 
    273     // The restore area for the pc will be located at 1 word below the resultant sp.
    274     // All restore values are located at offset <= PROBE_CPU_APSR_OFFSET. Hence,
    275     // we need to make sure that resultant sp > offset of apsr + 1.
    276     "add       ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET + PTR_SIZE) "\n"
     300
     301    // Set up the restore area for sp and pc.
    277302    "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    278     "cmp       lr, ip" "\n"
    279     "it        gt" "\n"
    280     "bgt     " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"
    281 
    282     // Getting here means that the restore area will overlap the ProbeContext data
    283     // that we will need to get the restoration values from. So, let's move that
    284     // data to a safe place before we start writing into the restore area.
    285     // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the
    286     // restore area. This ensures that:
    287     // 1. The safe area does not overlap the restore area.
    288     // 2. The safe area does not overlap the ProbeContext.
    289     //    This makes it so that we can use memcpy (does not require memmove) semantics
    290     //    to copy the restore values to the safe area.
    291 
    292     // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].
    293     "sub       lr, lr, #(2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ")" "\n"
    294    
    295     "mov       ip, sp" "\n" // Save the original ProbeContext*.
    296    
    297     // Make sure the stack pointer points to the safe area. This ensures that the
    298     // safe area is protected from interrupt handlers overwriting it.
    299     "mov       sp, lr" "\n" // sp now points to the new ProbeContext in the safe area.
    300    
    301     "mov       lr, ip" "\n" // Use lr as the old ProbeContext*.
    302    
    303     // Copy the restore data to the new ProbeContext*.
    304     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    305     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    306     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    307     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    308     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    309     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    310     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    311     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    312     "ldr       ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
    313     "str       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
    314 
    315     // ctiMasmProbeTrampolineEnd expects lr to contain the sp value to be restored.
    316     // Since we used it as scratch above, let's restore it.
    317     "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    318 
    319     ".thumb_func " THUMB_FUNC_PARAM(ctiMasmProbeTrampolineEnd) "\n"
    320     SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"
    321 
    322     // Set up the restore area for sp and pc.
    323     // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].
    324303
    325304    // Push the pc on to the restore area.
    326305    "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    327     "sub       lr, lr, #" STRINGIZE_VALUE_OF(PTR_SIZE) "\n"
     306    "sub       lr, lr, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n"
    328307    "str       ip, [lr]" "\n"
    329308    // Point sp to the restore area.

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.cpp

    r220701 r220807  
    4747#define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE)
    4848#define PROBE_ARG_OFFSET (1 * PTR_SIZE)
    49 
    50 #define PROBE_FIRST_GPR_OFFSET (2 * PTR_SIZE)
     49#define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE)
     50#define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE)
     51
     52#define PROBE_FIRST_GPR_OFFSET (4 * PTR_SIZE)
    5153#define PROBE_CPU_EAX_OFFSET (PROBE_FIRST_GPR_OFFSET + (0 * PTR_SIZE))
    5254#define PROBE_CPU_ECX_OFFSET (PROBE_FIRST_GPR_OFFSET + (1 * PTR_SIZE))
    … …  
    8890#if CPU(X86)
    8991#define PROBE_SIZE (PROBE_CPU_XMM7_OFFSET + XMM_SIZE)
    90 #define PROBE_ALIGNED_SIZE (PROBE_SIZE + (2 * XMM_SIZE))
    9192#else // CPU(X86_64)
    9293#define PROBE_CPU_XMM8_OFFSET (PROBE_FIRST_XMM_OFFSET + (8 * XMM_SIZE))
    … …  
    99100#define PROBE_CPU_XMM15_OFFSET (PROBE_FIRST_XMM_OFFSET + (15 * XMM_SIZE))
    100101#define PROBE_SIZE (PROBE_CPU_XMM15_OFFSET + XMM_SIZE)
    101 #define PROBE_ALIGNED_SIZE (PROBE_SIZE + (4 * XMM_SIZE))
    102102#endif // CPU(X86_64)
     103
     104// The outgoing record to be popped off the stack at the end consists of:
     105// eflags, eax, ecx, ebp, eip.
     106#define OUT_SIZE        (5 * PTR_SIZE)
    103107
    104108// These ASSERTs remind you that if you change the layout of ProbeContext,
    … …  
    107111COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline);
    108112COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline);
     113COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline);
     114COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline);
    109115
    110116COMPILE_ASSERT(PROBE_OFFSETOF(cpu.gprs[X86Registers::eax]) == PROBE_CPU_EAX_OFFSET, ProbeContext_cpu_eax_offset_matches_ctiMasmProbeTrampoline);
    … …  
    153159
    154160COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline);
    155 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0x1f), ProbeContext_aligned_size_offset_should_be_32_byte_aligned);
    156161
    157162#undef PROBE_OFFSETOF
    … …  
    176181
    177182    "movl %esp, %eax" "\n"
    178     "subl $" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %esp" "\n"
    179 
    180     // The X86_64 ABI specifies that the worse case stack alignment requirement
    181     // is 32 bytes.
     183    "subl $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %esp" "\n"
     184
     185    // The X86_64 ABI specifies that the worse case stack alignment requirement is 32 bytes.
    182186    "andl $~0x1f, %esp" "\n"
    183187
    … …  
    213217    "movq %xmm7, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM7_OFFSET) "(%ebp)" "\n"
    214218
     219    "xorl %eax, %eax" "\n"
     220    "movl %eax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%ebp)" "\n"
     221
    215222    // Reserve stack space for the arg while maintaining the required stack
    216223    // pointer 32 byte alignment:
    … …  
    219226
    220227    "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%ebp)" "\n"
     228
     229    // Make sure the ProbeContext is entirely below the result stack pointer so
     230    // that register values are still preserved when we call the initializeStack
     231    // function.
     232    "movl $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %ecx" "\n"
     233    "movl %ebp, %eax" "\n"
     234    "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %edx" "\n"
     235    "addl %ecx, %eax" "\n"
     236    "cmpl %eax, %edx" "\n"
     237    "jge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n"
     238
     239    // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext.
     240    "subl %ecx, %edx" "\n"
     241    "andl $~0x1f, %edx" "\n" // Keep the stack pointer 32 bytes aligned.
     242    "xorl %eax, %eax" "\n"
     243    "movl %edx, %esp" "\n"
     244
     245    "movl $" STRINGIZE_VALUE_OF(PROBE_SIZE) ", %ecx" "\n"
     246
     247    // Copy the ProbeContext to the safe place.
     248    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
     249    "movl (%ebp, %eax), %edx" "\n"
     250    "movl %edx, (%esp, %eax)" "\n"
     251    "addl $" STRINGIZE_VALUE_OF(PTR_SIZE) ", %eax" "\n"
     252    "cmpl %eax, %ecx" "\n"
     253    "jg " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
     254
     255    "movl %esp, %ebp" "\n"
     256
     257    // Call initializeStackFunction if present.
     258    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n"
     259    "xorl %ecx, %ecx" "\n"
     260    "addl " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%ebp), %ecx" "\n"
     261    "je " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
     262
     263    // Reserve stack space for the arg while maintaining the required stack
     264    // pointer 32 byte alignment:
     265    "subl $0x20, %esp" "\n"
     266    "movl %ebp, 0(%esp)" "\n" // the ProbeContext* arg.
     267    "call *%ecx" "\n"
     268
     269    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    221270
    222271    // To enable probes to modify register state, we copy all registers
    … …  
    250299    // ecx now points to the restore area.
    251300
    252     // Before we copy values from the ProbeContext to the restore area, we need to
    253     // make sure that the restore area does not overlap any of the values that we'll
    254     // be copying from in the ProbeContext. All the restore values to be copied from
    255     // comes from offset <= PROBE_CPU_EFLAGS_OFFSET in the ProbeContext.
    256     "movl %ebp, %eax" "\n"
    257     "addl $" STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) ", %eax" "\n"
    258     "cmpl %eax, %ecx" "\n"
    259     "jg " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"
    260 
    261     // Getting here means that the restore area will overlap the ProbeContext data
    262     // that we will need to get the restoration values from. So, let's move that
    263     // data to a safe place before we start writing into the restore area.
    264     // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the
    265     // restore area. This ensures that:
    266     // 1. The safe area does not overlap the restore area.
    267     // 2. The safe area does not overlap the ProbeContext.
    268     //    This makes it so that we can use memcpy (does not require memmove) semantics
    269     //    to copy the restore values to the safe area.
    270     // Note: the safe area does not have to 32-byte align it because we're not using
    271     // it to store any xmm regs.
    272     "movl %ecx, %eax" "\n"
    273     "subl $2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %eax" "\n"
    274 
    275     // eax now points to the safe area.
    276 
    277     // Make sure the stack pointer points to the safe area. This ensures that the
    278     // safe area is protected from interrupt handlers overwriting it.
    279     "movl %eax, %esp" "\n"
    280 
    281     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%ebp), %ecx" "\n"
    282     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%eax)" "\n"
    283     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%ebp), %ecx" "\n"
    284     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%eax)" "\n"
    285     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%ebp), %ecx" "\n"
    286     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%eax)" "\n"
    287     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %ecx" "\n"
    288     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%eax)" "\n"
    289     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%ebp), %ecx" "\n"
    290     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%eax)" "\n"
    291     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%ebp), %ecx" "\n"
    292     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%eax)" "\n"
    293     "movl %eax, %ebp" "\n"
    294 
    295     // We used ecx above as scratch register. Let's restore it to points to the
    296     // restore area.
    297     "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %ecx" "\n"
    298     "subl $5 * " STRINGIZE_VALUE_OF(PTR_SIZE) ", %ecx" "\n"
    299 
    300     // ecx now points to the restore area.
    301 
    302     SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"
    303 
    304301    // Copy remaining restore values from the ProbeContext to the restore area.
     302    // Note: We already ensured above that the ProbeContext is in a safe location before
     303    // calling the initializeStackFunction. The initializeStackFunction is not allowed to
     304    // change the stack pointer again.
    305305    "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%ebp), %eax" "\n"
    306306    "movl %eax, 0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%ecx)" "\n"
    … …  
    343343
    344344    "movq %rsp, %rax" "\n"
    345     "subq $" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %rsp" "\n"
    346 
    347     // The X86_64 ABI specifies that the worse case stack alignment requirement
    348     // is 32 bytes.
     345    "subq $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %rsp" "\n"
     346
     347    // The X86_64 ABI specifies that the worse case stack alignment requirement is 32 bytes.
    349348    "andq $~0x1f, %rsp" "\n"
     349    // Since sp points to the ProbeContext, we've ensured that it's protected from interrupts before we initialize it.
    350350
    351351    "movq %rbp, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rsp)" "\n"
    … …  
    397397    "movq %xmm15, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM15_OFFSET) "(%rbp)" "\n"
    398398
     399    "xorq %rax, %rax" "\n"
     400    "movq %rax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%rbp)" "\n"
     401
    399402    "movq %rbp, %rdi" "\n" // the ProbeContext* arg.
    400403    "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%rbp)" "\n"
     404
     405    // Make sure the ProbeContext is entirely below the result stack pointer so
     406    // that register values are still preserved when we call the initializeStack
     407    // function.
     408    "movq $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %rcx" "\n"
     409    "movq %rbp, %rax" "\n"
     410    "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rdx" "\n"
     411    "addq %rcx, %rax" "\n"
     412    "cmpq %rax, %rdx" "\n"
     413    "jge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n"
     414 
     415    // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext.
     416    "subq %rcx, %rdx" "\n"
     417    "andq $~0x1f, %rdx" "\n" // Keep the stack pointer 32 bytes aligned.
     418    "xorq %rax, %rax" "\n"
     419    "movq %rdx, %rsp" "\n"
     420
     421    "movq $" STRINGIZE_VALUE_OF(PROBE_SIZE) ", %rcx" "\n"
     422
     423    // Copy the ProbeContext to the safe place.
     424    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
     425    "movq (%rbp, %rax), %rdx" "\n"
     426    "movq %rdx, (%rsp, %rax)" "\n"
     427    "addq $" STRINGIZE_VALUE_OF(PTR_SIZE) ", %rax" "\n"
     428    "cmpq %rax, %rcx" "\n"
     429    "jg " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
     430
     431    "movq %rsp, %rbp" "\n"
     432
     433    // Call initializeStackFunction if present.
     434    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n"
     435    "xorq %rcx, %rcx" "\n"
     436    "addq " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%rbp), %rcx" "\n"
     437    "je " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
     438
     439    "movq %rbp, %rdi" "\n" // the ProbeContext* arg.
     440    "call *%rcx" "\n"
     441
     442    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    401443
    402444    // To enable probes to modify register state, we copy all registers
    … …  
    447489    // rcx now points to the restore area.
    448490
    449     // Before we copy values from the ProbeContext to the restore area, we need to
    450     // make sure that the restore area does not overlap any of the values that we'll
    451     // be copying from in the ProbeContext. All the restore values to be copied from
    452     // comes from offset <= PROBE_CPU_EFLAGS_OFFSET in the ProbeContext.
    453     "movq %rbp, %rax" "\n"
    454     "addq $" STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) ", %rax" "\n"
    455     "cmpq %rax, %rcx" "\n"
    456     "jg " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"
    457 
    458     // Getting here means that the restore area will overlap the ProbeContext data
    459     // that we will need to get the restoration values from. So, let's move that
    460     // data to a safe place before we start writing into the restore area.
    461     // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the
    462     // restore area. This ensures that:
    463     // 1. The safe area does not overlap the restore area.
    464     // 2. The safe area does not overlap the ProbeContext.
    465     //    This makes it so that we can use memcpy (does not require memmove) semantics
    466     //    to copy the restore values to the safe area.
    467     // Note: the safe area does not have to 32-byte align it because we're not using
    468     // it to store any xmm regs.
    469     "movq %rcx, %rax" "\n"
    470     "subq $2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %rax" "\n"
    471 
    472     // rax now points to the safe area.
    473 
    474     // Make sure the stack pointer points to the safe area. This ensures that the
    475     // safe area is protected from interrupt handlers overwriting it.
    476     "movq %rax, %rsp" "\n"
    477 
    478     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%rbp), %rcx" "\n"
    479     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%rax)" "\n"
    480     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rbp), %rcx" "\n"
    481     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rax)" "\n"
    482     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rbp), %rcx" "\n"
    483     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rax)" "\n"
    484     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rcx" "\n"
    485     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rax)" "\n"
    486     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%rbp), %rcx" "\n"
    487     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%rax)" "\n"
    488     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rbp), %rcx" "\n"
    489     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rax)" "\n"
    490     "movq %rax, %rbp" "\n"
    491 
    492     // We used rcx above as scratch register. Let's restore it to points to the
    493     // restore area.
    494     "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rcx" "\n"
    495     "subq $5 * " STRINGIZE_VALUE_OF(PTR_SIZE) ", %rcx" "\n"
    496 
    497     // rcx now points to the restore area.
    498 
    499     SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"
    500 
    501491    // Copy remaining restore values from the ProbeContext to the restore area.
     492    // Note: We already ensured above that the ProbeContext is in a safe location before
     493    // calling the initializeStackFunction. The initializeStackFunction is not allowed to
     494    // change the stack pointer again.
    502495    "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rbp), %rax" "\n"
    503496    "movq %rax, 0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rcx)" "\n"

  • trunk/Source/JavaScriptCore/assembler/testmasm.cpp

    r220744 r220807  
    6464namespace {
    6565
     66using CPUState = MacroAssembler::CPUState;
     67
    6668StaticLock crashLock;
    6769
    … …  
    268270    // to validate that the probe preserves register values.
    269271    unsigned probeCallCount = 0;
    270     MacroAssembler::CPUState originalState;
     272    CPUState originalState;
    271273
    272274    compileAndRun<void>([&] (CCallHelpers& jit) {
    … …  
    348350{
    349351    unsigned probeCallCount = 0;
    350     MacroAssembler::CPUState originalState;
     352    CPUState originalState;
    351353    void* originalSP { nullptr };
    352354    void* modifiedSP { nullptr };
    … …  
    509511}
    510512
     513struct FillStackData {
     514    CPUState originalState;
     515    void* originalSP { nullptr };
     516    void* newSP { nullptr };
     517    uintptr_t modifiedFlags { 0 };
     518    MacroAssembler::SPRegisterID flagsSPR;
     519};
     520
     521static void fillStack(ProbeContext* context)
     522{
     523    auto& cpu = context->cpu;
     524
     525    FillStackData& data = *reinterpret_cast<FillStackData*>(context->initializeStackArg);
     526    CPUState& originalState = data.originalState;
     527    void*& originalSP = data.originalSP;
     528    void*& newSP = data.newSP;
     529    uintptr_t& modifiedFlags = data.modifiedFlags;
     530    MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR;
     531
     532    CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), reinterpret_cast<void*>(fillStack));
     533    CHECK_EQ(cpu.sp(), newSP);
     534
     535    // Verify that the probe has put the ProbeContext out of harm's way.
     536    CHECK_EQ((reinterpret_cast<void*>(context + 1) <= cpu.sp()), true);
     537
     538    // Verify the CPU state.
     539    for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
     540        if (isFP(id)) {
     541            CHECK_EQ(cpu.gpr(id), originalState.gpr(id));
     542            continue;
     543        }
     544        if (isSpecialGPR(id))
     545            continue;
     546        CHECK_EQ(cpu.gpr(id), testWord(id));
     547    }
     548    for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
     549        CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id));
     550    CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags);
     551
     552    // Fill the stack with values.
     553    uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP);
     554    int count = 0;
     555    while (p < reinterpret_cast<uintptr_t*>(originalSP))
     556        *p++ = testWord(count++);
     557};
     558
     559void testProbeModifiesStackWithCallback()
     560{
     561    unsigned probeCallCount = 0;
     562    FillStackData data;
     563    CPUState& originalState = data.originalState;
     564    void*& originalSP = data.originalSP;
     565    void*& newSP = data.newSP;
     566    uintptr_t& modifiedFlags = data.modifiedFlags;
     567    size_t numberOfExtraEntriesToWrite = 10; // ARM64 requires that this be 2 word aligned.
     568    MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR;
     569
     570#if CPU(X86) || CPU(X86_64)
     571    flagsSPR = X86Registers::eflags;
     572    uintptr_t flagsMask = 0xc5;
     573#elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
     574    flagsSPR = ARMRegisters::apsr;
     575    uintptr_t flagsMask = 0xf0000000;
     576#elif CPU(ARM64)
     577    flagsSPR = ARM64Registers::nzcv;
     578    uintptr_t flagsMask = 0xf0000000;
     579#endif
     580
     581    compileAndRun<void>([&] (CCallHelpers& jit) {
     582        jit.emitFunctionPrologue();
     583
     584        // Write expected values into the registers.
     585        jit.probe([&] (ProbeContext* context) {
     586            auto& cpu = context->cpu;
     587            probeCallCount++;
     588
     589            // Preserve the original CPU state.
     590            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
     591                originalState.gpr(id) = cpu.gpr(id);
     592                if (isSpecialGPR(id))
     593                    continue;
     594                cpu.gpr(id) = testWord(static_cast<int>(id));
     595            }
     596            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id)) {
     597                originalState.fpr(id) = cpu.fpr(id);
     598                cpu.fpr(id) = bitwise_cast<double>(testWord64(id));
     599            }
     600            originalState.spr(flagsSPR) = cpu.spr(flagsSPR);
     601            modifiedFlags = originalState.spr(flagsSPR) ^ flagsMask;
     602            cpu.spr(flagsSPR) = modifiedFlags;
     603
     604            CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), 0);
     605
     606            // Prepare for initializeStack callback.
     607            context->initializeStackFunction = fillStack;
     608            context->initializeStackArg = &data;
     609
     610            // Ensure that we'll be writing over the regions of the stack where the ProbeContext is.
     611            originalSP = cpu.sp();
     612            newSP = reinterpret_cast<uintptr_t*>(context) - numberOfExtraEntriesToWrite;
     613            cpu.sp() = newSP;
     614        });
     615
     616        // Validate that the registers and stack have the expected values.
     617        jit.probe([&] (ProbeContext* context) {
     618            auto& cpu = context->cpu;
     619            probeCallCount++;
     620
     621            // Validate the register values.
     622            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
     623                if (isFP(id)) {
     624                    CHECK_EQ(cpu.gpr(id), originalState.gpr(id));
     625                    continue;
     626                }
     627                if (isSpecialGPR(id))
     628                    continue;
     629                CHECK_EQ(cpu.gpr(id), testWord(id));
     630            }
     631            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
     632                CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id));
     633            CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags);
     634            CHECK_EQ(cpu.sp(), newSP);
     635
     636            // Validate the stack with values.
     637            uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP);
     638            int count = 0;
     639            while (p < reinterpret_cast<uintptr_t*>(originalSP))
     640                CHECK_EQ(*p++, testWord(count++));
     641        });
     642
     643        // Restore the original state.
     644        jit.probe([&] (ProbeContext* context) {
     645            auto& cpu = context->cpu;
     646            probeCallCount++;
     647            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
     648                if (isSpecialGPR(id))
     649                    continue;
     650                cpu.gpr(id) = originalState.gpr(id);
     651            }
     652            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
     653                cpu.fpr(id) = originalState.fpr(id);
     654            cpu.spr(flagsSPR) = originalState.spr(flagsSPR);
     655            cpu.sp() = originalSP;
     656        });
     657
     658        jit.emitFunctionEpilogue();
     659        jit.ret();
     660    });
     661
     662    CHECK_EQ(probeCallCount, 3);
     663}
     664
    511665#define RUN(test) do {                          \
    512666        if (!shouldRun(#test))                  \
    … …  
    541695    RUN(testProbeModifiesStackPointerToNBytesBelowSP());
    542696    RUN(testProbeModifiesProgramCounter());
     697    RUN(testProbeModifiesStackWithCallback());
    543698
    544699    if (tasks.isEmpty())
Note: See TracChangeset for help on using the changeset viewer.