Changeset 220807 in webkit
- Timestamp:
- Aug 16, 2017 1:38:57 PM (7 years ago)
- Location:
- trunk/Source/JavaScriptCore
- Files:
-
- 7 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/JavaScriptCore/ChangeLog
r220791 r220807 1 2017-08-16 Mark Lam <mark.lam@apple.com> 2 3 Enhance MacroAssembler::probe() to support an initializeStackFunction callback. 4 https://bugs.webkit.org/show_bug.cgi?id=175617 5 <rdar://problem/33912104> 6 7 Reviewed by JF Bastien. 8 9 This patch adds a new feature to MacroAssembler::probe() where the probe function 10 can provide a ProbeFunction callback to fill in stack values after the stack 11 pointer has been adjusted. The probe function can use this feature as follows: 12 13 1. Set the new sp value in the ProbeContext's CPUState. 14 15 2. Set the ProbeContext's initializeStackFunction to a ProbeFunction callback 16 which will do the work of filling in the stack values after the probe 17 trampoline has adjusted the machine stack pointer. 18 19 3. Set the ProbeContext's initializeStackArgs to any value that the client wants 20 to pass to the initializeStackFunction callback. 21 22 4. Return from the probe function. 23 24 Upon returning from the probe function, the probe trampoline will adjust the 25 the stack pointer based on the sp value in CPUState. If initializeStackFunction 26 is not set, the probe trampoline will restore registers and return to its caller. 27 28 If initializeStackFunction is set, the trampoline will move the ProbeContext 29 beyond the range of the stack pointer i.e. it will place the new ProbeContext at 30 an address lower than where CPUState.sp() points. This ensures that the 31 ProbeContext will not be trashed by the initializeStackFunction when it writes to 32 the stack. Then, the trampoline will call back to the initializeStackFunction 33 ProbeFunction to let it fill in the stack values as desired. The 34 initializeStackFunction ProbeFunction will be passed the moved ProbeContext at 35 the new location. 36 37 initializeStackFunction may now write to the stack at addresses greater or 38 equal to CPUState.sp(), but not below that. initializeStackFunction is also 39 not allowed to change CPUState.sp(). If the initializeStackFunction does not 40 abide by these rules, then behavior is undefined, and bad things may happen. 41 42 For future reference, some implementation details that this patch needed to 43 be mindful of: 44 45 1. When the probe trampoline allocates stack space for the ProbeContext, it 46 should include OUT_SIZE as well. This ensures that it doesn't have to move 47 the ProbeContext on exit if the probe function didn't change the sp. 48 49 2. If the trampoline has to move the ProbeContext, it needs to point the machine 50 sp to new ProbeContext first before copying over the ProbeContext data. This 51 protects the new ProbeContext from possibly being trashed by interrupts. 52 53 3. When computing the new address of ProbeContext to move to, we need to make 54 sure that it is properly aligned in accordance with stack ABI requirements 55 (just like we did when we allocated the ProbeContext on entry to the 56 probe trampoline). 57 58 4. When copying the ProbeContext to its new location, the trampoline should 59 always copy words from low addresses to high addresses. This is because if 60 we're moving the ProbeContext, we'll always be moving it to a lower address. 61 62 * assembler/MacroAssembler.h: 63 * assembler/MacroAssemblerARM.cpp: 64 * assembler/MacroAssemblerARM64.cpp: 65 * assembler/MacroAssemblerARMv7.cpp: 66 * assembler/MacroAssemblerX86Common.cpp: 67 * assembler/testmasm.cpp: 68 (JSC::testProbePreservesGPRS): 69 (JSC::testProbeModifiesStackPointer): 70 (JSC::fillStack): 71 (JSC::testProbeModifiesStackWithCallback): 72 (JSC::run): 73 1 74 2017-08-16 Csaba Osztrogonác <ossy@webkit.org> 2 75 -
trunk/Source/JavaScriptCore/assembler/MacroAssembler.h
r220720 r220807 1843 1843 // of the call to the user probe function. 1844 1844 // 1845 // The probe function may choose to move the stack pointer (in any direction). 1846 // To do this, the probe function needs to set the new sp value in the CPUState. 1847 // 1848 // The probe function may also choose to fill stack space with some values. 1849 // To do this, the probe function must first: 1850 // 1. Set the new sp value in the ProbeContext's CPUState. 1851 // 2. Set the ProbeContext's initializeStackFunction to a ProbeFunction callback 1852 // which will do the work of filling in the stack values after the probe 1853 // trampoline has adjusted the machine stack pointer. 1854 // 3. Set the ProbeContext's initializeStackArgs to any value that the client wants 1855 // to pass to the initializeStackFunction callback. 1856 // 4. Return from the probe function. 1857 // 1858 // Upon returning from the probe function, the probe trampoline will adjust the 1859 // the stack pointer based on the sp value in CPUState. If initializeStackFunction 1860 // is not set, the probe trampoline will restore registers and return to its caller. 1861 // 1862 // If initializeStackFunction is set, the trampoline will move the ProbeContext 1863 // beyond the range of the stack pointer i.e. it will place the new ProbeContext at 1864 // an address lower than where CPUState.sp() points. This ensures that the 1865 // ProbeContext will not be trashed by the initializeStackFunction when it writes to 1866 // the stack. Then, the trampoline will call back to the initializeStackFunction 1867 // ProbeFunction to let it fill in the stack values as desired. The 1868 // initializeStackFunction ProbeFunction will be passed the moved ProbeContext at 1869 // the new location. 1870 // 1871 // initializeStackFunction may now write to the stack at addresses greater or 1872 // equal to CPUState.sp(), but not below that. initializeStackFunction is also 1873 // not allowed to change CPUState.sp(). If the initializeStackFunction does not 1874 // abide by these rules, then behavior is undefined, and bad things may happen. 1875 // 1845 1876 // Note: this version of probe() should be implemented by the target specific 1846 1877 // MacroAssembler. … … 1996 2027 ProbeFunction probeFunction; 1997 2028 void* arg; 2029 ProbeFunction initializeStackFunction; 2030 void* initializeStackArg; 1998 2031 CPUState cpu; 1999 2032 -
trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM.cpp
r220629 r220807 100 100 101 101 #if COMPILER(GCC_OR_CLANG) 102 102 103 103 // The following are offsets for ProbeContext fields accessed 104 104 // by the ctiMasmProbeTrampoline stub. … … 107 107 #define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE) 108 108 #define PROBE_ARG_OFFSET (1 * PTR_SIZE) 109 110 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE) 109 #define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE) 110 #define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE) 111 112 #define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE) 111 113 112 114 #define GPREG_SIZE 4 … … 168 170 169 171 #define PROBE_SIZE (PROBE_FIRST_FPREG_OFFSET + (32 * FPREG_SIZE)) 170 #define PROBE_ALIGNED_SIZE (PROBE_SIZE) 172 173 #define OUT_SIZE GPREG_SIZE 171 174 172 175 // These ASSERTs remind you that if you change the layout of ProbeContext, … … 175 178 COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline); 176 179 COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline); 180 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline); 181 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline); 177 182 178 183 COMPILE_ASSERT(!(PROBE_CPU_R0_OFFSET & 0x3), ProbeContext_cpu_r0_offset_should_be_4_byte_aligned); … … 198 203 COMPILE_ASSERT(PROBE_OFFSETOF(cpu.sprs[ARMRegisters::fpscr]) == PROBE_CPU_FPSCR_OFFSET, ProbeContext_cpu_fpscr_offset_matches_ctiMasmProbeTrampoline); 199 204 200 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x f), ProbeContext_cpu_d0_offset_should_be_16_byte_aligned);205 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x7), ProbeContext_cpu_d0_offset_should_be_8_byte_aligned); 201 206 202 207 COMPILE_ASSERT(PROBE_OFFSETOF(cpu.fprs[ARMRegisters::d0]) == PROBE_CPU_D0_OFFSET, ProbeContext_cpu_d0_offset_matches_ctiMasmProbeTrampoline); … … 234 239 235 240 COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline); 236 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0xf), ProbeContext_aligned_size_offset_should_be_16_byte_aligned);237 241 #undef PROBE_OFFSETOF 238 242 … … 255 259 "mov ip, sp" "\n" 256 260 "mov r3, sp" "\n" 257 "sub r3, r3, #" STRINGIZE_VALUE_OF(PROBE_ ALIGNED_SIZE) "\n"261 "sub r3, r3, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" 258 262 259 263 // The ARM EABI specifies that the stack needs to be 16 byte aligned. 260 264 "bic r3, r3, #0xf" "\n" 261 "mov sp, r3" "\n" 265 "mov sp, r3" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it. 262 266 263 267 "str lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" … … 285 289 286 290 "add ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_D0_OFFSET) "\n" 287 "vstmia.64 ip , { d0-d15 }" "\n"288 "vstmia.64 ip , { d16-d31 }" "\n"291 "vstmia.64 ip!, { d0-d15 }" "\n" 292 "vstmia.64 ip!, { d16-d31 }" "\n" 289 293 290 294 "mov fp, sp" "\n" // Save the ProbeContext*. 295 296 // Initialize ProbeContext::initializeStackFunction to zero. 297 "mov r0, #0" "\n" 298 "str r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 291 299 292 300 "ldr ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n" 293 301 "mov r0, sp" "\n" // the ProbeContext* arg. 294 302 "blx ip" "\n" 303 304 // Make sure the ProbeContext is entirely below the result stack pointer so 305 // that register values are still preserved when we call the initializeStack 306 // function. 307 "ldr r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp. 308 "add r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer. 309 "cmp r1, r2" "\n" 310 "bge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n" 311 312 // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext. 313 "sub r1, r1, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" 314 "bic r1, r1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned. 315 "mov sp, r1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext. 316 317 // Copy the ProbeContext to the safe place. 318 // Note: we have to copy from low address to higher address because we're moving the 319 // ProbeContext to a lower address. 320 "mov r5, fp" "\n" 321 "mov r6, r1" "\n" 322 "add r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n" 323 324 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n" 325 "ldr r3, [r5], #4" "\n" 326 "ldr r4, [r5], #4" "\n" 327 "str r3, [r6], #4" "\n" 328 "str r4, [r6], #4" "\n" 329 "cmp r5, r7" "\n" 330 "blt " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n" 331 332 "mov fp, r1" "\n" 333 334 // Call initializeStackFunction if present. 335 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n" 336 "ldr r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 337 "cbz r2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n" 338 339 "mov r0, fp" "\n" // Set the ProbeContext* arg. 340 "blx r2" "\n" // Call the initializeStackFunction (loaded into r2 above). 341 342 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n" 295 343 296 344 "mov sp, fp" "\n" … … 308 356 309 357 // There are 5 more registers left to restore: ip, sp, lr, pc, and apsr. 310 // There are 2 issues that complicate the restoration of these last few 311 // registers: 312 // 313 // 1. Normal ARM calling convention relies on moving lr to pc to return to 314 // the caller. In our case, the address to return to is specified by 315 // ProbeContext.cpu.pc. And at that moment, we won't have any available 316 // scratch registers to hold the return address (lr needs to hold 317 // ProbeContext.cpu.lr, not the return address). 318 // 319 // The solution is to store the return address on the stack and load the 320 // pc from there. 321 // 322 // 2. Issue 1 means we will need to write to the stack location at 323 // ProbeContext.cpu.gprs[sp] - PTR_SIZE. But if the user probe function had 324 // modified the value of ProbeContext.cpu.gprs[sp] to point in the range between 325 // &ProbeContext.cpu.gprs[ip] thru &ProbeContext.cpu.sprs[aspr], then the action 326 // for Issue 1 may trash the values to be restored before we can restore them. 327 // 328 // The solution is to check if ProbeContext.cpu.gprs[sp] contains a value in 329 // the undesirable range. If so, we copy the remaining ProbeContext 330 // register data to a safe area first, and restore the remaining register 331 // from this new safe area. 332 333 // The restore area for the pc will be located at 1 word below the resultant sp. 334 // All restore values are located at offset <= PROBE_CPU_APSR_OFFSET. Hence, 335 // we need to make sure that resultant sp > offset of apsr + 1. 336 "add ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET + PTR_SIZE) "\n" 358 359 // Set up the restore area for sp and pc. 337 360 "ldr lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" 338 "cmp lr, ip" "\n"339 "bgt " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"340 341 // Getting here means that the restore area will overlap the ProbeContext data342 // that we will need to get the restoration values from. So, let's move that343 // data to a safe place before we start writing into the restore area.344 // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the345 // restore area. This ensures that:346 // 1. The safe area does not overlap the restore area.347 // 2. The safe area does not overlap the ProbeContext.348 // This makes it so that we can use memcpy (does not require memmove) semantics349 // to copy the restore values to the safe area.350 351 // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].352 "sub lr, lr, #(2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ")" "\n"353 354 "mov ip, sp" "\n" // Save the original ProbeContext*.355 356 // Make sure the stack pointer points to the safe area. This ensures that the357 // safe area is protected from interrupt handlers overwriting it.358 "mov sp, lr" "\n" // sp now points to the new ProbeContext in the safe area.359 360 "mov lr, ip" "\n" // Use lr as the old ProbeContext*.361 362 // Copy the restore data to the new ProbeContext*.363 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"364 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"365 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"366 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"367 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"368 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"369 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"370 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"371 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"372 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"373 374 // ctiMasmProbeTrampolineEnd expects lr to contain the sp value to be restored.375 // Since we used it as scratch above, let's restore it.376 "ldr lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"377 378 SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"379 380 // Set up the restore area for sp and pc.381 // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].382 361 383 362 // Push the pc on to the restore area. 384 363 "ldr ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" 385 "sub lr, lr, #" STRINGIZE_VALUE_OF( PTR_SIZE) "\n"364 "sub lr, lr, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n" 386 365 "str ip, [lr]" "\n" 387 366 // Point sp to the restore area. -
trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.cpp
r220630 r220807 44 44 #define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE) 45 45 #define PROBE_ARG_OFFSET (1 * PTR_SIZE) 46 47 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE) 46 #define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE) 47 #define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE) 48 49 #define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE) 48 50 49 51 #define GPREG_SIZE 8 … … 132 134 COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline); 133 135 COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline); 136 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline); 137 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline); 134 138 135 139 COMPILE_ASSERT(!(PROBE_CPU_X0_OFFSET & 0x7), ProbeContext_cpu_r0_offset_should_be_8_byte_aligned); … … 213 217 // Conditions for using ldp and stp. 214 218 static_assert(PROBE_CPU_PC_OFFSET == PROBE_CPU_SP_OFFSET + PTR_SIZE, "PROBE_CPU_SP_OFFSET and PROBE_CPU_PC_OFFSET must be adjacent"); 219 static_assert(!(PROBE_SIZE_PLUS_EXTRAS & 0xf), "PROBE_SIZE_PLUS_EXTRAS should be 16 byte aligned"); // the ProbeContext copying code relies on this. 215 220 216 221 #undef PROBE_OFFSETOF … … 270 275 static_assert(OUT_LR_OFFSET == offsetof(OutgoingProbeRecord, lr), "OUT_LR_OFFSET is incorrect"); 271 276 static_assert(OUT_SIZE == sizeof(OutgoingProbeRecord), "OUT_SIZE is incorrect"); 272 static_assert(!(sizeof(OutgoingProbeRecord) & 0xf), " IncomingProbeStack must be 16-byte aligned");277 static_assert(!(sizeof(OutgoingProbeRecord) & 0xf), "OutgoingProbeStack must be 16-byte aligned"); 273 278 274 279 #define STATE_PC_NOT_CHANGED 0 … … 291 296 "mov x27, sp" "\n" 292 297 293 "sub x27, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS ) "\n"298 "sub x27, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n" 294 299 "bic x27, x27, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned. 295 "mov sp, x27" "\n" // Make sure interrupts don't over-write our data on the stack.300 "mov sp, x27" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it. 296 301 297 302 "stp x0, x1, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n" … … 351 356 "mov x27, sp" "\n" // Save the ProbeContext* in a callee saved register. 352 357 358 // Initialize ProbeContext::initializeStackFunction to zero. 359 "str xzr, [x27, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 360 353 361 // Note: we haven't changed the value of fp. Hence, it is still pointing to the frame of 354 362 // the caller of the probe (which is what we want in order to play nice with debuggers e.g. lldb). 355 363 "mov x0, sp" "\n" // Set the ProbeContext* arg. 356 364 "blr x2" "\n" // Call the probe handler function (loaded into x2 above). 365 366 // Make sure the ProbeContext is entirely below the result stack pointer so 367 // that register values are still preserved when we call the initializeStack 368 // function. 369 "ldr x1, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp. 370 "add x2, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n" // End of ProbeContext + buffer. 371 "cmp x1, x2" "\n" 372 "bge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n" 373 374 // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext. 375 "sub x1, x1, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS + OUT_SIZE) "\n" 376 "bic x1, x1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned. 377 "mov sp, x1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext. 378 379 // Copy the ProbeContext to the safe place. 380 // Note: we have to copy from low address to higher address because we're moving the 381 // ProbeContext to a lower address. 382 "mov x5, x27" "\n" 383 "mov x6, x1" "\n" 384 "add x7, x27, #" STRINGIZE_VALUE_OF(PROBE_SIZE_PLUS_EXTRAS) "\n" 385 386 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n" 387 "ldp x3, x4, [x5], #16" "\n" 388 "stp x3, x4, [x6], #16" "\n" 389 "cmp x5, x7" "\n" 390 "blt " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n" 391 392 "mov x27, x1" "\n" 393 394 // Call initializeStackFunction if present. 395 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n" 396 "ldr x2, [x27, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 397 "cbz x2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n" 398 399 "mov x0, x27" "\n" // Set the ProbeContext* arg. 400 "blr x2" "\n" // Call the initializeStackFunction (loaded into x2 above). 401 402 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n" 357 403 358 404 "mov sp, x27" "\n" … … 429 475 "ldr x29, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n" // Preload the probe return site pc. 430 476 431 // The probe handler may have moved the sp. For the return process, we may need432 // space for 2 OutgoingProbeRecords below the final sp value. We need to make433 // sure that the space for these 2 OutgoingProbeRecords do not overlap the434 // restore values of the registers.435 436 // All restore values are located at offset <= PROBE_CPU_FPSR_OFFSET. Hence,437 // we need to make sure that resultant sp > offset of fpsr + 2 * sizeof(OutgoingProbeRecord).438 439 "add x27, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FPSR_OFFSET + 2 * OUT_SIZE) "\n"440 "ldr x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"441 "cmp x28, x27" "\n"442 "bgt " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineFillOutgoingProbeRecords) "\n"443 444 // There is overlap. We need to copy the ProbeContext to a safe area first.445 // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the OutgoingProbeRecords are.446 // This ensures that:447 // 1. The safe area does not overlap the OutgoingProbeRecords.448 // 2. The safe area does not overlap the ProbeContext.449 450 // x28 already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].451 "sub x28, x28, #" STRINGIZE_VALUE_OF(2 * PROBE_SIZE) "\n"452 453 "mov x27, sp" "\n" // Save the original ProbeContext*.454 455 // Make sure the stack pointer points to the safe area. This ensures that the456 // safe area is protected from interrupt handlers overwriting it.457 "mov sp, x28" "\n" // sp now points to the new ProbeContext in the safe area.458 459 // Copy the relevant restore data to the new ProbeContext*.460 "str x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n" // Stash the pc changed state away so that we can use lr.461 462 "ldp x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_X27_OFFSET) "]" "\n" // copy x27 and x28.463 "stp x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X27_OFFSET) "]" "\n"464 "ldp x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_FP_OFFSET) "]" "\n" // copy fp and lr.465 "stp x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FP_OFFSET) "]" "\n"466 "ldp x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // copy sp and pc.467 "stp x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"468 "ldp x28, x30, [x27, #" STRINGIZE_VALUE_OF(PROBE_CPU_NZCV_OFFSET) "]" "\n" // copy nzcv and fpsr.469 "stp x28, x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_NZCV_OFFSET) "]" "\n"470 471 "ldr x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n" // Retrieve the stashed the pc changed state.472 473 477 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineFillOutgoingProbeRecords) ":" "\n" 474 478 -
trunk/Source/JavaScriptCore/assembler/MacroAssemblerARMv7.cpp
r220579 r220807 43 43 #define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE) 44 44 #define PROBE_ARG_OFFSET (1 * PTR_SIZE) 45 46 #define PROBE_FIRST_GPREG_OFFSET (2 * PTR_SIZE) 45 #define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE) 46 #define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE) 47 48 #define PROBE_FIRST_GPREG_OFFSET (4 * PTR_SIZE) 47 49 48 50 #define GPREG_SIZE 4 … … 102 104 #define PROBE_CPU_D30_OFFSET (PROBE_FIRST_FPREG_OFFSET + (30 * FPREG_SIZE)) 103 105 #define PROBE_CPU_D31_OFFSET (PROBE_FIRST_FPREG_OFFSET + (31 * FPREG_SIZE)) 106 104 107 #define PROBE_SIZE (PROBE_FIRST_FPREG_OFFSET + (32 * FPREG_SIZE)) 105 #define PROBE_ALIGNED_SIZE (PROBE_SIZE) 108 109 #define OUT_SIZE GPREG_SIZE 106 110 107 111 // These ASSERTs remind you that if you change the layout of ProbeContext, … … 110 114 COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline); 111 115 COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline); 116 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline); 117 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline); 112 118 113 119 COMPILE_ASSERT(!(PROBE_CPU_R0_OFFSET & 0x3), ProbeContext_cpu_r0_offset_should_be_4_byte_aligned); … … 133 139 COMPILE_ASSERT(PROBE_OFFSETOF(cpu.sprs[ARMRegisters::fpscr]) == PROBE_CPU_FPSCR_OFFSET, ProbeContext_cpu_fpscr_offset_matches_ctiMasmProbeTrampoline); 134 140 135 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x f), ProbeContext_cpu_d0_offset_should_be_16_byte_aligned);141 COMPILE_ASSERT(!(PROBE_CPU_D0_OFFSET & 0x7), ProbeContext_cpu_d0_offset_should_be_8_byte_aligned); 136 142 137 143 COMPILE_ASSERT(PROBE_OFFSETOF(cpu.fprs[ARMRegisters::d0]) == PROBE_CPU_D0_OFFSET, ProbeContext_cpu_d0_offset_matches_ctiMasmProbeTrampoline); … … 170 176 171 177 COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline); 172 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0xf), ProbeContext_aligned_size_offset_should_be_16_byte_aligned);173 174 178 #undef PROBE_OFFSETOF 175 179 … … 194 198 "mov ip, sp" "\n" 195 199 "mov r0, sp" "\n" 196 "sub r0, r0, #" STRINGIZE_VALUE_OF(PROBE_ ALIGNED_SIZE) "\n"200 "sub r0, r0, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" 197 201 198 202 // The ARM EABI specifies that the stack needs to be 16 byte aligned. 199 203 "bic r0, r0, #0xf" "\n" 200 "mov sp, r0" "\n" 204 "mov sp, r0" "\n" // Set the sp to protect the ProbeContext from interrupts before we initialize it. 201 205 202 206 "str lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" … … 229 233 "mov fp, sp" "\n" // Save the ProbeContext*. 230 234 235 // Initialize ProbeContext::initializeStackFunction to zero. 236 "mov r0, #0" "\n" 237 "str r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 238 231 239 "ldr ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n" 232 240 "mov r0, sp" "\n" // the ProbeContext* arg. 233 241 "blx ip" "\n" 242 243 // Make sure the ProbeContext is entirely below the result stack pointer so 244 // that register values are still preserved when we call the initializeStack 245 // function. 246 "ldr r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp. 247 "add r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer. 248 "cmp r1, r2" "\n" 249 "it ge" "\n" 250 "bge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n" 251 252 // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext. 253 "sub r1, r1, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" 254 "bic r1, r1, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned. 255 "mov sp, r1" "\n" // Set the new sp to protect that memory from interrupts before we copy the ProbeContext. 256 257 // Copy the ProbeContext to the safe place. 258 // Note: we have to copy from low address to higher address because we're moving the 259 // ProbeContext to a lower address. 260 "mov r5, fp" "\n" 261 "mov r6, r1" "\n" 262 "add r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n" 263 264 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n" 265 "ldr r3, [r5], #4" "\n" 266 "ldr r4, [r5], #4" "\n" 267 "str r3, [r6], #4" "\n" 268 "str r4, [r6], #4" "\n" 269 "cmp r5, r7" "\n" 270 "it lt" "\n" 271 "blt " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n" 272 273 "mov fp, r1" "\n" 274 275 // Call initializeStackFunction if present. 276 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n" 277 "ldr r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n" 278 "cbz r2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n" 279 280 "mov r0, fp" "\n" // Set the ProbeContext* arg. 281 "blx r2" "\n" // Call the initializeStackFunction (loaded into r2 above). 282 283 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n" 234 284 235 285 "mov sp, fp" "\n" … … 248 298 249 299 // There are 5 more registers left to restore: ip, sp, lr, pc, and apsr. 250 // There are 2 issues that complicate the restoration of these last few 251 // registers: 252 // 253 // 1. Normal ARM calling convention relies on moving lr to pc to return to 254 // the caller. In our case, the address to return to is specified by 255 // ProbeContext.cpu.gprs[pc]. And at that moment, we won't have any available 256 // scratch registers to hold the return address (lr needs to hold 257 // ProbeContext.cpu.gprs[lr], not the return address). 258 // 259 // The solution is to store the return address on the stack and load the 260 // pc from there. 261 // 262 // 2. Issue 1 means we will need to write to the stack location at 263 // ProbeContext.cpu.gprs[sp] - PTR_SIZE. But if the user probe function had 264 // modified the value of ProbeContext.cpu.gprs[sp] to point in the range between 265 // &ProbeContext.cpu.gprs[ip] thru &ProbeContext.cpu.sprs[aspr], then the action 266 // for Issue 1 may trash the values to be restored before we can restore them. 267 // 268 // The solution is to check if ProbeContext.cpu.gprs[sp] contains a value in 269 // the undesirable range. If so, we copy the remaining ProbeContext 270 // register data to a safe area first, and restore the remaining register 271 // from this new safe area. 272 273 // The restore area for the pc will be located at 1 word below the resultant sp. 274 // All restore values are located at offset <= PROBE_CPU_APSR_OFFSET. Hence, 275 // we need to make sure that resultant sp > offset of apsr + 1. 276 "add ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET + PTR_SIZE) "\n" 300 301 // Set up the restore area for sp and pc. 277 302 "ldr lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" 278 "cmp lr, ip" "\n"279 "it gt" "\n"280 "bgt " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"281 282 // Getting here means that the restore area will overlap the ProbeContext data283 // that we will need to get the restoration values from. So, let's move that284 // data to a safe place before we start writing into the restore area.285 // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the286 // restore area. This ensures that:287 // 1. The safe area does not overlap the restore area.288 // 2. The safe area does not overlap the ProbeContext.289 // This makes it so that we can use memcpy (does not require memmove) semantics290 // to copy the restore values to the safe area.291 292 // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].293 "sub lr, lr, #(2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ")" "\n"294 295 "mov ip, sp" "\n" // Save the original ProbeContext*.296 297 // Make sure the stack pointer points to the safe area. This ensures that the298 // safe area is protected from interrupt handlers overwriting it.299 "mov sp, lr" "\n" // sp now points to the new ProbeContext in the safe area.300 301 "mov lr, ip" "\n" // Use lr as the old ProbeContext*.302 303 // Copy the restore data to the new ProbeContext*.304 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"305 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"306 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"307 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"308 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"309 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"310 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"311 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"312 "ldr ip, [lr, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"313 "str ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"314 315 // ctiMasmProbeTrampolineEnd expects lr to contain the sp value to be restored.316 // Since we used it as scratch above, let's restore it.317 "ldr lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"318 319 ".thumb_func " THUMB_FUNC_PARAM(ctiMasmProbeTrampolineEnd) "\n"320 SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"321 322 // Set up the restore area for sp and pc.323 // lr already contains [sp, #STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET)].324 303 325 304 // Push the pc on to the restore area. 326 305 "ldr ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" 327 "sub lr, lr, #" STRINGIZE_VALUE_OF( PTR_SIZE) "\n"306 "sub lr, lr, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n" 328 307 "str ip, [lr]" "\n" 329 308 // Point sp to the restore area. -
trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.cpp
r220701 r220807 47 47 #define PROBE_PROBE_FUNCTION_OFFSET (0 * PTR_SIZE) 48 48 #define PROBE_ARG_OFFSET (1 * PTR_SIZE) 49 50 #define PROBE_FIRST_GPR_OFFSET (2 * PTR_SIZE) 49 #define PROBE_INIT_STACK_FUNCTION_OFFSET (2 * PTR_SIZE) 50 #define PROBE_INIT_STACK_ARG_OFFSET (3 * PTR_SIZE) 51 52 #define PROBE_FIRST_GPR_OFFSET (4 * PTR_SIZE) 51 53 #define PROBE_CPU_EAX_OFFSET (PROBE_FIRST_GPR_OFFSET + (0 * PTR_SIZE)) 52 54 #define PROBE_CPU_ECX_OFFSET (PROBE_FIRST_GPR_OFFSET + (1 * PTR_SIZE)) … … 88 90 #if CPU(X86) 89 91 #define PROBE_SIZE (PROBE_CPU_XMM7_OFFSET + XMM_SIZE) 90 #define PROBE_ALIGNED_SIZE (PROBE_SIZE + (2 * XMM_SIZE))91 92 #else // CPU(X86_64) 92 93 #define PROBE_CPU_XMM8_OFFSET (PROBE_FIRST_XMM_OFFSET + (8 * XMM_SIZE)) … … 99 100 #define PROBE_CPU_XMM15_OFFSET (PROBE_FIRST_XMM_OFFSET + (15 * XMM_SIZE)) 100 101 #define PROBE_SIZE (PROBE_CPU_XMM15_OFFSET + XMM_SIZE) 101 #define PROBE_ALIGNED_SIZE (PROBE_SIZE + (4 * XMM_SIZE))102 102 #endif // CPU(X86_64) 103 104 // The outgoing record to be popped off the stack at the end consists of: 105 // eflags, eax, ecx, ebp, eip. 106 #define OUT_SIZE (5 * PTR_SIZE) 103 107 104 108 // These ASSERTs remind you that if you change the layout of ProbeContext, … … 107 111 COMPILE_ASSERT(PROBE_OFFSETOF(probeFunction) == PROBE_PROBE_FUNCTION_OFFSET, ProbeContext_probeFunction_offset_matches_ctiMasmProbeTrampoline); 108 112 COMPILE_ASSERT(PROBE_OFFSETOF(arg) == PROBE_ARG_OFFSET, ProbeContext_arg_offset_matches_ctiMasmProbeTrampoline); 113 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackFunction) == PROBE_INIT_STACK_FUNCTION_OFFSET, ProbeContext_initializeStackFunction_offset_matches_ctiMasmProbeTrampoline); 114 COMPILE_ASSERT(PROBE_OFFSETOF(initializeStackArg) == PROBE_INIT_STACK_ARG_OFFSET, ProbeContext_initializeStackArg_offset_matches_ctiMasmProbeTrampoline); 109 115 110 116 COMPILE_ASSERT(PROBE_OFFSETOF(cpu.gprs[X86Registers::eax]) == PROBE_CPU_EAX_OFFSET, ProbeContext_cpu_eax_offset_matches_ctiMasmProbeTrampoline); … … 153 159 154 160 COMPILE_ASSERT(sizeof(ProbeContext) == PROBE_SIZE, ProbeContext_size_matches_ctiMasmProbeTrampoline); 155 COMPILE_ASSERT(!(PROBE_ALIGNED_SIZE & 0x1f), ProbeContext_aligned_size_offset_should_be_32_byte_aligned);156 161 157 162 #undef PROBE_OFFSETOF … … 176 181 177 182 "movl %esp, %eax" "\n" 178 "subl $" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %esp" "\n" 179 180 // The X86_64 ABI specifies that the worse case stack alignment requirement 181 // is 32 bytes. 183 "subl $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %esp" "\n" 184 185 // The X86_64 ABI specifies that the worse case stack alignment requirement is 32 bytes. 182 186 "andl $~0x1f, %esp" "\n" 183 187 … … 213 217 "movq %xmm7, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM7_OFFSET) "(%ebp)" "\n" 214 218 219 "xorl %eax, %eax" "\n" 220 "movl %eax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%ebp)" "\n" 221 215 222 // Reserve stack space for the arg while maintaining the required stack 216 223 // pointer 32 byte alignment: … … 219 226 220 227 "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%ebp)" "\n" 228 229 // Make sure the ProbeContext is entirely below the result stack pointer so 230 // that register values are still preserved when we call the initializeStack 231 // function. 232 "movl $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %ecx" "\n" 233 "movl %ebp, %eax" "\n" 234 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %edx" "\n" 235 "addl %ecx, %eax" "\n" 236 "cmpl %eax, %edx" "\n" 237 "jge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n" 238 239 // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext. 240 "subl %ecx, %edx" "\n" 241 "andl $~0x1f, %edx" "\n" // Keep the stack pointer 32 bytes aligned. 242 "xorl %eax, %eax" "\n" 243 "movl %edx, %esp" "\n" 244 245 "movl $" STRINGIZE_VALUE_OF(PROBE_SIZE) ", %ecx" "\n" 246 247 // Copy the ProbeContext to the safe place. 248 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n" 249 "movl (%ebp, %eax), %edx" "\n" 250 "movl %edx, (%esp, %eax)" "\n" 251 "addl $" STRINGIZE_VALUE_OF(PTR_SIZE) ", %eax" "\n" 252 "cmpl %eax, %ecx" "\n" 253 "jg " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n" 254 255 "movl %esp, %ebp" "\n" 256 257 // Call initializeStackFunction if present. 258 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n" 259 "xorl %ecx, %ecx" "\n" 260 "addl " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%ebp), %ecx" "\n" 261 "je " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n" 262 263 // Reserve stack space for the arg while maintaining the required stack 264 // pointer 32 byte alignment: 265 "subl $0x20, %esp" "\n" 266 "movl %ebp, 0(%esp)" "\n" // the ProbeContext* arg. 267 "call *%ecx" "\n" 268 269 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n" 221 270 222 271 // To enable probes to modify register state, we copy all registers … … 250 299 // ecx now points to the restore area. 251 300 252 // Before we copy values from the ProbeContext to the restore area, we need to253 // make sure that the restore area does not overlap any of the values that we'll254 // be copying from in the ProbeContext. All the restore values to be copied from255 // comes from offset <= PROBE_CPU_EFLAGS_OFFSET in the ProbeContext.256 "movl %ebp, %eax" "\n"257 "addl $" STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) ", %eax" "\n"258 "cmpl %eax, %ecx" "\n"259 "jg " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"260 261 // Getting here means that the restore area will overlap the ProbeContext data262 // that we will need to get the restoration values from. So, let's move that263 // data to a safe place before we start writing into the restore area.264 // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the265 // restore area. This ensures that:266 // 1. The safe area does not overlap the restore area.267 // 2. The safe area does not overlap the ProbeContext.268 // This makes it so that we can use memcpy (does not require memmove) semantics269 // to copy the restore values to the safe area.270 // Note: the safe area does not have to 32-byte align it because we're not using271 // it to store any xmm regs.272 "movl %ecx, %eax" "\n"273 "subl $2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %eax" "\n"274 275 // eax now points to the safe area.276 277 // Make sure the stack pointer points to the safe area. This ensures that the278 // safe area is protected from interrupt handlers overwriting it.279 "movl %eax, %esp" "\n"280 281 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%ebp), %ecx" "\n"282 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%eax)" "\n"283 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%ebp), %ecx" "\n"284 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%eax)" "\n"285 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%ebp), %ecx" "\n"286 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%eax)" "\n"287 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %ecx" "\n"288 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%eax)" "\n"289 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%ebp), %ecx" "\n"290 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%eax)" "\n"291 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%ebp), %ecx" "\n"292 "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%eax)" "\n"293 "movl %eax, %ebp" "\n"294 295 // We used ecx above as scratch register. Let's restore it to points to the296 // restore area.297 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp), %ecx" "\n"298 "subl $5 * " STRINGIZE_VALUE_OF(PTR_SIZE) ", %ecx" "\n"299 300 // ecx now points to the restore area.301 302 SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"303 304 301 // Copy remaining restore values from the ProbeContext to the restore area. 302 // Note: We already ensured above that the ProbeContext is in a safe location before 303 // calling the initializeStackFunction. The initializeStackFunction is not allowed to 304 // change the stack pointer again. 305 305 "movl " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%ebp), %eax" "\n" 306 306 "movl %eax, 0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%ecx)" "\n" … … 343 343 344 344 "movq %rsp, %rax" "\n" 345 "subq $" STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %rsp" "\n" 346 347 // The X86_64 ABI specifies that the worse case stack alignment requirement 348 // is 32 bytes. 345 "subq $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %rsp" "\n" 346 347 // The X86_64 ABI specifies that the worse case stack alignment requirement is 32 bytes. 349 348 "andq $~0x1f, %rsp" "\n" 349 // Since sp points to the ProbeContext, we've ensured that it's protected from interrupts before we initialize it. 350 350 351 351 "movq %rbp, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rsp)" "\n" … … 397 397 "movq %xmm15, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM15_OFFSET) "(%rbp)" "\n" 398 398 399 "xorq %rax, %rax" "\n" 400 "movq %rax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%rbp)" "\n" 401 399 402 "movq %rbp, %rdi" "\n" // the ProbeContext* arg. 400 403 "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%rbp)" "\n" 404 405 // Make sure the ProbeContext is entirely below the result stack pointer so 406 // that register values are still preserved when we call the initializeStack 407 // function. 408 "movq $" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) ", %rcx" "\n" 409 "movq %rbp, %rax" "\n" 410 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rdx" "\n" 411 "addq %rcx, %rax" "\n" 412 "cmpq %rax, %rdx" "\n" 413 "jge " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) "\n" 414 415 // Allocate a safe place on the stack below the result stack pointer to stash the ProbeContext. 416 "subq %rcx, %rdx" "\n" 417 "andq $~0x1f, %rdx" "\n" // Keep the stack pointer 32 bytes aligned. 418 "xorq %rax, %rax" "\n" 419 "movq %rdx, %rsp" "\n" 420 421 "movq $" STRINGIZE_VALUE_OF(PROBE_SIZE) ", %rcx" "\n" 422 423 // Copy the ProbeContext to the safe place. 424 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n" 425 "movq (%rbp, %rax), %rdx" "\n" 426 "movq %rdx, (%rsp, %rax)" "\n" 427 "addq $" STRINGIZE_VALUE_OF(PTR_SIZE) ", %rax" "\n" 428 "cmpq %rax, %rcx" "\n" 429 "jg " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n" 430 431 "movq %rsp, %rbp" "\n" 432 433 // Call initializeStackFunction if present. 434 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeContextIsSafe) ":" "\n" 435 "xorq %rcx, %rcx" "\n" 436 "addq " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%rbp), %rcx" "\n" 437 "je " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n" 438 439 "movq %rbp, %rdi" "\n" // the ProbeContext* arg. 440 "call *%rcx" "\n" 441 442 LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n" 401 443 402 444 // To enable probes to modify register state, we copy all registers … … 447 489 // rcx now points to the restore area. 448 490 449 // Before we copy values from the ProbeContext to the restore area, we need to450 // make sure that the restore area does not overlap any of the values that we'll451 // be copying from in the ProbeContext. All the restore values to be copied from452 // comes from offset <= PROBE_CPU_EFLAGS_OFFSET in the ProbeContext.453 "movq %rbp, %rax" "\n"454 "addq $" STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) ", %rax" "\n"455 "cmpq %rax, %rcx" "\n"456 "jg " SYMBOL_STRING(ctiMasmProbeTrampolineEnd) "\n"457 458 // Getting here means that the restore area will overlap the ProbeContext data459 // that we will need to get the restoration values from. So, let's move that460 // data to a safe place before we start writing into the restore area.461 // Let's locate the "safe area" at 2x sizeof(ProbeContext) below where the462 // restore area. This ensures that:463 // 1. The safe area does not overlap the restore area.464 // 2. The safe area does not overlap the ProbeContext.465 // This makes it so that we can use memcpy (does not require memmove) semantics466 // to copy the restore values to the safe area.467 // Note: the safe area does not have to 32-byte align it because we're not using468 // it to store any xmm regs.469 "movq %rcx, %rax" "\n"470 "subq $2 * " STRINGIZE_VALUE_OF(PROBE_ALIGNED_SIZE) ", %rax" "\n"471 472 // rax now points to the safe area.473 474 // Make sure the stack pointer points to the safe area. This ensures that the475 // safe area is protected from interrupt handlers overwriting it.476 "movq %rax, %rsp" "\n"477 478 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%rbp), %rcx" "\n"479 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%rax)" "\n"480 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rbp), %rcx" "\n"481 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rax)" "\n"482 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rbp), %rcx" "\n"483 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBP_OFFSET) "(%rax)" "\n"484 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rcx" "\n"485 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rax)" "\n"486 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%rbp), %rcx" "\n"487 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%rax)" "\n"488 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rbp), %rcx" "\n"489 "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rax)" "\n"490 "movq %rax, %rbp" "\n"491 492 // We used rcx above as scratch register. Let's restore it to points to the493 // restore area.494 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp), %rcx" "\n"495 "subq $5 * " STRINGIZE_VALUE_OF(PTR_SIZE) ", %rcx" "\n"496 497 // rcx now points to the restore area.498 499 SYMBOL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"500 501 491 // Copy remaining restore values from the ProbeContext to the restore area. 492 // Note: We already ensured above that the ProbeContext is in a safe location before 493 // calling the initializeStackFunction. The initializeStackFunction is not allowed to 494 // change the stack pointer again. 502 495 "movq " STRINGIZE_VALUE_OF(PROBE_CPU_EFLAGS_OFFSET) "(%rbp), %rax" "\n" 503 496 "movq %rax, 0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rcx)" "\n" -
trunk/Source/JavaScriptCore/assembler/testmasm.cpp
r220744 r220807 64 64 namespace { 65 65 66 using CPUState = MacroAssembler::CPUState; 67 66 68 StaticLock crashLock; 67 69 … … 268 270 // to validate that the probe preserves register values. 269 271 unsigned probeCallCount = 0; 270 MacroAssembler::CPUState originalState;272 CPUState originalState; 271 273 272 274 compileAndRun<void>([&] (CCallHelpers& jit) { … … 348 350 { 349 351 unsigned probeCallCount = 0; 350 MacroAssembler::CPUState originalState;352 CPUState originalState; 351 353 void* originalSP { nullptr }; 352 354 void* modifiedSP { nullptr }; … … 509 511 } 510 512 513 struct FillStackData { 514 CPUState originalState; 515 void* originalSP { nullptr }; 516 void* newSP { nullptr }; 517 uintptr_t modifiedFlags { 0 }; 518 MacroAssembler::SPRegisterID flagsSPR; 519 }; 520 521 static void fillStack(ProbeContext* context) 522 { 523 auto& cpu = context->cpu; 524 525 FillStackData& data = *reinterpret_cast<FillStackData*>(context->initializeStackArg); 526 CPUState& originalState = data.originalState; 527 void*& originalSP = data.originalSP; 528 void*& newSP = data.newSP; 529 uintptr_t& modifiedFlags = data.modifiedFlags; 530 MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR; 531 532 CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), reinterpret_cast<void*>(fillStack)); 533 CHECK_EQ(cpu.sp(), newSP); 534 535 // Verify that the probe has put the ProbeContext out of harm's way. 536 CHECK_EQ((reinterpret_cast<void*>(context + 1) <= cpu.sp()), true); 537 538 // Verify the CPU state. 539 for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) { 540 if (isFP(id)) { 541 CHECK_EQ(cpu.gpr(id), originalState.gpr(id)); 542 continue; 543 } 544 if (isSpecialGPR(id)) 545 continue; 546 CHECK_EQ(cpu.gpr(id), testWord(id)); 547 } 548 for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id)) 549 CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id)); 550 CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags); 551 552 // Fill the stack with values. 553 uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP); 554 int count = 0; 555 while (p < reinterpret_cast<uintptr_t*>(originalSP)) 556 *p++ = testWord(count++); 557 }; 558 559 void testProbeModifiesStackWithCallback() 560 { 561 unsigned probeCallCount = 0; 562 FillStackData data; 563 CPUState& originalState = data.originalState; 564 void*& originalSP = data.originalSP; 565 void*& newSP = data.newSP; 566 uintptr_t& modifiedFlags = data.modifiedFlags; 567 size_t numberOfExtraEntriesToWrite = 10; // ARM64 requires that this be 2 word aligned. 568 MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR; 569 570 #if CPU(X86) || CPU(X86_64) 571 flagsSPR = X86Registers::eflags; 572 uintptr_t flagsMask = 0xc5; 573 #elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL) 574 flagsSPR = ARMRegisters::apsr; 575 uintptr_t flagsMask = 0xf0000000; 576 #elif CPU(ARM64) 577 flagsSPR = ARM64Registers::nzcv; 578 uintptr_t flagsMask = 0xf0000000; 579 #endif 580 581 compileAndRun<void>([&] (CCallHelpers& jit) { 582 jit.emitFunctionPrologue(); 583 584 // Write expected values into the registers. 585 jit.probe([&] (ProbeContext* context) { 586 auto& cpu = context->cpu; 587 probeCallCount++; 588 589 // Preserve the original CPU state. 590 for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) { 591 originalState.gpr(id) = cpu.gpr(id); 592 if (isSpecialGPR(id)) 593 continue; 594 cpu.gpr(id) = testWord(static_cast<int>(id)); 595 } 596 for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id)) { 597 originalState.fpr(id) = cpu.fpr(id); 598 cpu.fpr(id) = bitwise_cast<double>(testWord64(id)); 599 } 600 originalState.spr(flagsSPR) = cpu.spr(flagsSPR); 601 modifiedFlags = originalState.spr(flagsSPR) ^ flagsMask; 602 cpu.spr(flagsSPR) = modifiedFlags; 603 604 CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), 0); 605 606 // Prepare for initializeStack callback. 607 context->initializeStackFunction = fillStack; 608 context->initializeStackArg = &data; 609 610 // Ensure that we'll be writing over the regions of the stack where the ProbeContext is. 611 originalSP = cpu.sp(); 612 newSP = reinterpret_cast<uintptr_t*>(context) - numberOfExtraEntriesToWrite; 613 cpu.sp() = newSP; 614 }); 615 616 // Validate that the registers and stack have the expected values. 617 jit.probe([&] (ProbeContext* context) { 618 auto& cpu = context->cpu; 619 probeCallCount++; 620 621 // Validate the register values. 622 for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) { 623 if (isFP(id)) { 624 CHECK_EQ(cpu.gpr(id), originalState.gpr(id)); 625 continue; 626 } 627 if (isSpecialGPR(id)) 628 continue; 629 CHECK_EQ(cpu.gpr(id), testWord(id)); 630 } 631 for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id)) 632 CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id)); 633 CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags); 634 CHECK_EQ(cpu.sp(), newSP); 635 636 // Validate the stack with values. 637 uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP); 638 int count = 0; 639 while (p < reinterpret_cast<uintptr_t*>(originalSP)) 640 CHECK_EQ(*p++, testWord(count++)); 641 }); 642 643 // Restore the original state. 644 jit.probe([&] (ProbeContext* context) { 645 auto& cpu = context->cpu; 646 probeCallCount++; 647 for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) { 648 if (isSpecialGPR(id)) 649 continue; 650 cpu.gpr(id) = originalState.gpr(id); 651 } 652 for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id)) 653 cpu.fpr(id) = originalState.fpr(id); 654 cpu.spr(flagsSPR) = originalState.spr(flagsSPR); 655 cpu.sp() = originalSP; 656 }); 657 658 jit.emitFunctionEpilogue(); 659 jit.ret(); 660 }); 661 662 CHECK_EQ(probeCallCount, 3); 663 } 664 511 665 #define RUN(test) do { \ 512 666 if (!shouldRun(#test)) \ … … 541 695 RUN(testProbeModifiesStackPointerToNBytesBelowSP()); 542 696 RUN(testProbeModifiesProgramCounter()); 697 RUN(testProbeModifiesStackWithCallback()); 543 698 544 699 if (tasks.isEmpty())
Note: See TracChangeset
for help on using the changeset viewer.