Changeset 243886 in webkit

Timestamp:

Apr 4, 2019, 11:54:16 AM (7 years ago)

Author:

sbarati@apple.com

Message:

Teach Call ICs how to call Wasm
​https://bugs.webkit.org/show_bug.cgi?id=196387

Reviewed by Filip Pizlo.

JSTests:

• wasm/function-tests/stack-trace.js:

Source/JavaScriptCore:

This patch teaches JS to call Wasm without going through the native thunk.
Currently, we emit a JIT "JS" callee stub which marshals arguments from
JS to Wasm. Like the native version of this, this thunk is responsible
for saving and restoring the VM's current Wasm context. Instead of emitting
an exception handler, we also teach the unwinder how to read the previous
wasm context to restore it as it unwindws past this frame.

This patch is straight forward, and leaves some areas for perf improvement:

• We can teach the DFG/FTL to directly use the Wasm calling convention when it knows it's calling a single Wasm function. This way we don't shuffle registers to the stack and then back into registers.
• We bail out to the slow path for mismatched arity. I opened a bug to fix optimize arity check failures: ​https://bugs.webkit.org/show_bug.cgi?id=196564
• We bail out to the slow path Double JSValues flowing into i32 arguments. We should teach this thunk how to do that conversion directly.

This patch also refactors the code to explicitly have a single pinned size register.
We used pretend in some places that we could have more than one pinned size register.
However, there was other code that just asserted the size was one. This patch just rips
out this code since we never moved to having more than one pinned size register. Doing
this refactoring cleans up the various places where we set up the size register.

This patch is a 50-60% progression on JetStream 2's richards-wasm.

• JavaScriptCore.xcodeproj/project.pbxproj:
• Sources.txt:
• assembler/MacroAssemblerCodeRef.h:

(JSC::MacroAssemblerCodeRef::operator=):
(JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef):

• interpreter/Interpreter.cpp:

(JSC::UnwindFunctor::operator() const):
(JSC::UnwindFunctor::copyCalleeSavesToEntryFrameCalleeSavesBuffer const):

• interpreter/StackVisitor.cpp:

(JSC::StackVisitor::Frame::calleeSaveRegistersForUnwinding):
(JSC::StackVisitor::Frame::calleeSaveRegisters): Deleted.

• interpreter/StackVisitor.h:
• jit/JITOperations.cpp:
• jit/RegisterSet.cpp:

(JSC::RegisterSet::runtimeTagRegisters):
(JSC::RegisterSet::specialRegisters):
(JSC::RegisterSet::runtimeRegisters): Deleted.

• jit/RegisterSet.h:
• jit/Repatch.cpp:

(JSC::linkPolymorphicCall):

• runtime/JSFunction.cpp:

(JSC::getCalculatedDisplayName):

• runtime/JSGlobalObject.cpp:

(JSC::JSGlobalObject::init):
(JSC::JSGlobalObject::visitChildren):

• runtime/JSGlobalObject.h:

(JSC::JSGlobalObject::jsToWasmICCalleeStructure const):

• runtime/VM.cpp:

(JSC::VM::VM):

• runtime/VM.h:
• wasm/WasmAirIRGenerator.cpp:

(JSC::Wasm::AirIRGenerator::AirIRGenerator):
(JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::AirIRGenerator::addCallIndirect):

• wasm/WasmB3IRGenerator.cpp:

(JSC::Wasm::B3IRGenerator::B3IRGenerator):
(JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::B3IRGenerator::addCallIndirect):

• wasm/WasmBinding.cpp:

(JSC::Wasm::wasmToWasm):

• wasm/WasmContext.h:

(JSC::Wasm::Context::pointerToInstance):

• wasm/WasmContextInlines.h:

(JSC::Wasm::Context::store):

• wasm/WasmMemoryInformation.cpp:

(JSC::Wasm::getPinnedRegisters):
(JSC::Wasm::PinnedRegisterInfo::get):
(JSC::Wasm::PinnedRegisterInfo::PinnedRegisterInfo):

• wasm/WasmMemoryInformation.h:

(JSC::Wasm::PinnedRegisterInfo::toSave const):

• wasm/WasmOMGPlan.cpp:

(JSC::Wasm::OMGPlan::work):

• wasm/js/JSToWasm.cpp:

(JSC::Wasm::createJSToWasmWrapper):

• wasm/js/JSToWasmICCallee.cpp: Added.

(JSC::JSToWasmICCallee::create):
(JSC::JSToWasmICCallee::createStructure):
(JSC::JSToWasmICCallee::visitChildren):

• wasm/js/JSToWasmICCallee.h: Added.

(JSC::JSToWasmICCallee::function):
(JSC::JSToWasmICCallee::JSToWasmICCallee):

• wasm/js/WebAssemblyFunction.cpp:

(JSC::WebAssemblyFunction::useTagRegisters const):
(JSC::WebAssemblyFunction::calleeSaves const):
(JSC::WebAssemblyFunction::usedCalleeSaveRegisters const):
(JSC::WebAssemblyFunction::previousInstanceOffset const):
(JSC::WebAssemblyFunction::previousInstance):
(JSC::WebAssemblyFunction::jsCallEntrypointSlow):
(JSC::WebAssemblyFunction::visitChildren):
(JSC::WebAssemblyFunction::destroy):

• wasm/js/WebAssemblyFunction.h:
• wasm/js/WebAssemblyFunctionHeapCellType.cpp: Added.

(JSC::WebAssemblyFunctionDestroyFunc::operator() const):
(JSC::WebAssemblyFunctionHeapCellType::WebAssemblyFunctionHeapCellType):
(JSC::WebAssemblyFunctionHeapCellType::~WebAssemblyFunctionHeapCellType):
(JSC::WebAssemblyFunctionHeapCellType::finishSweep):
(JSC::WebAssemblyFunctionHeapCellType::destroy):

• wasm/js/WebAssemblyFunctionHeapCellType.h: Added.
• wasm/js/WebAssemblyPrototype.h:

Location:

trunk

Files:

• JSTests/ChangeLog (modified) (1 diff)
• JSTests/wasm/function-tests/stack-trace.js (modified) (1 diff)
• Source/JavaScriptCore/CMakeLists.txt (modified) (1 diff)
• Source/JavaScriptCore/ChangeLog (modified) (1 diff)
• Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj (modified) (8 diffs)
• Source/JavaScriptCore/Sources.txt (modified) (2 diffs)
• Source/JavaScriptCore/assembler/MacroAssemblerCodeRef.h (modified) (2 diffs)
• Source/JavaScriptCore/interpreter/Interpreter.cpp (modified) (2 diffs)
• Source/JavaScriptCore/interpreter/StackVisitor.cpp (modified) (3 diffs)
• Source/JavaScriptCore/interpreter/StackVisitor.h (modified) (1 diff)
• Source/JavaScriptCore/jit/JITOperations.cpp (modified) (3 diffs)
• Source/JavaScriptCore/jit/RegisterSet.cpp (modified) (2 diffs)
• Source/JavaScriptCore/jit/RegisterSet.h (modified) (1 diff)
• Source/JavaScriptCore/jit/Repatch.cpp (modified) (3 diffs)
• Source/JavaScriptCore/jit/Repatch.h (modified) (1 diff)
• Source/JavaScriptCore/runtime/JSFunction.cpp (modified) (2 diffs)
• Source/JavaScriptCore/runtime/JSGlobalObject.cpp (modified) (2 diffs)
• Source/JavaScriptCore/runtime/JSGlobalObject.h (modified) (2 diffs)
• Source/JavaScriptCore/runtime/VM.cpp (modified) (3 diffs)
• Source/JavaScriptCore/runtime/VM.h (modified) (2 diffs)
• Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp (modified) (5 diffs)
• Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp (modified) (5 diffs)
• Source/JavaScriptCore/wasm/WasmBinding.cpp (modified) (2 diffs)
• Source/JavaScriptCore/wasm/WasmContext.h (modified) (1 diff)
• Source/JavaScriptCore/wasm/WasmContextInlines.h (modified) (1 diff)
• Source/JavaScriptCore/wasm/WasmMemoryInformation.cpp (modified) (3 diffs)
• Source/JavaScriptCore/wasm/WasmMemoryInformation.h (modified) (2 diffs)
• Source/JavaScriptCore/wasm/WasmOMGPlan.cpp (modified) (1 diff)
• Source/JavaScriptCore/wasm/js/JSToWasm.cpp (modified) (1 diff)
• Source/JavaScriptCore/wasm/js/JSToWasmICCallee.cpp (copied) (copied from trunk/Source/JavaScriptCore/wasm/js/WebAssemblyPrototype.h ) (2 diffs)
• Source/JavaScriptCore/wasm/js/JSToWasmICCallee.h (copied) (copied from trunk/Source/JavaScriptCore/wasm/WasmContext.h ) (2 diffs)
• Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp (modified) (3 diffs)
• Source/JavaScriptCore/wasm/js/WebAssemblyFunction.h (modified) (4 diffs)
• Source/JavaScriptCore/wasm/js/WebAssemblyFunctionHeapCellType.cpp (copied) (copied from trunk/Source/JavaScriptCore/wasm/WasmContextInlines.h ) (2 diffs)
• Source/JavaScriptCore/wasm/js/WebAssemblyFunctionHeapCellType.h (copied) (copied from trunk/Source/JavaScriptCore/wasm/WasmContext.h ) (3 diffs)
• Source/JavaScriptCore/wasm/js/WebAssemblyPrototype.h (modified) (1 diff)

trunk/JSTests/ChangeLog

@@ +1 @@
+2019-04-04  Saam barati  <sbarati@apple.com>
+
+        Teach Call ICs how to call Wasm
+        https://bugs.webkit.org/show_bug.cgi?id=196387
+
+        Reviewed by Filip Pizlo.
+
+        * wasm/function-tests/stack-trace.js:
+
 2019-04-04  Caio Lima  <ticaiolima@gmail.com>
 

trunk/JSTests/wasm/function-tests/stack-trace.js

@@ -44 +44 @@
     assert.truthy(stacktrace[0].indexOf("imp") !== -1); // the arrow function import named "imp".
     assert.eq(stacktrace[1], "wasm-stub@[wasm code]"); // the wasm->js stub
-    assert.eq(stacktrace[2], "<?>.wasm-function[4]@[wasm code]");
-    assert.eq(stacktrace[3], "<?>.wasm-function[2]@[wasm code]");
-    assert.eq(stacktrace[4], "<?>.wasm-function[3]@[wasm code]");
-    assert.eq(stacktrace[5], "<?>.wasm-function[1]@[wasm code]");
-    assert.eq(stacktrace[6], "wasm-stub@[wasm code]"); // wasm entry
-
+    let found = false;
+    for (let i = 0; i < stacktrace.length; ++i) {
+        let str = stacktrace[i];
+        if (str !== "<?>.wasm-function[4]@[wasm code]")
+            continue;
+        found = true;
+        assert.eq(stacktrace[i + 1], "<?>.wasm-function[2]@[wasm code]");
+        assert.eq(stacktrace[i + 2], "<?>.wasm-function[3]@[wasm code]");
+        assert.eq(stacktrace[i + 3], "<?>.wasm-function[1]@[wasm code]");
+    }
+    assert.truthy(found);
     stacktrace = null;
 }

trunk/Source/JavaScriptCore/CMakeLists.txt

@@ -101 +101 @@
 
     wasm/js/JSWebAssembly.cpp
+    wasm/js/JSToWasmICCallee.cpp
     wasm/js/WebAssemblyCompileErrorConstructor.cpp
     wasm/js/WebAssemblyCompileErrorPrototype.cpp
+    wasm/js/WebAssemblyFunctionHeapCellType.cpp
     wasm/js/WebAssemblyInstanceConstructor.cpp
     wasm/js/WebAssemblyInstancePrototype.cpp

trunk/Source/JavaScriptCore/ChangeLog

@@ +1 @@
+2019-04-04  Saam barati  <sbarati@apple.com>
+
+        Teach Call ICs how to call Wasm
+        https://bugs.webkit.org/show_bug.cgi?id=196387
+
+        Reviewed by Filip Pizlo.
+
+        This patch teaches JS to call Wasm without going through the native thunk.
+        Currently, we emit a JIT "JS" callee stub which marshals arguments from
+        JS to Wasm. Like the native version of this, this thunk is responsible
+        for saving and restoring the VM's current Wasm context. Instead of emitting
+        an exception handler, we also teach the unwinder how to read the previous
+        wasm context to restore it as it unwindws past this frame.
+        
+        This patch is straight forward, and leaves some areas for perf improvement:
+        - We can teach the DFG/FTL to directly use the Wasm calling convention when
+          it knows it's calling a single Wasm function. This way we don't shuffle
+          registers to the stack and then back into registers.
+        - We bail out to the slow path for mismatched arity. I opened a bug to fix
+          optimize arity check failures: https://bugs.webkit.org/show_bug.cgi?id=196564
+        - We bail out to the slow path Double JSValues flowing into i32 arguments.
+          We should teach this thunk how to do that conversion directly.
+        
+        This patch also refactors the code to explicitly have a single pinned size register.
+        We used pretend in some places that we could have more than one pinned size register.
+        However, there was other code that just asserted the size was one. This patch just rips
+        out this code since we never moved to having more than one pinned size register. Doing
+        this refactoring cleans up the various places where we set up the size register.
+        
+        This patch is a 50-60% progression on JetStream 2's richards-wasm.
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * Sources.txt:
+        * assembler/MacroAssemblerCodeRef.h:
+        (JSC::MacroAssemblerCodeRef::operator=):
+        (JSC::MacroAssemblerCodeRef::MacroAssemblerCodeRef):
+        * interpreter/Interpreter.cpp:
+        (JSC::UnwindFunctor::operator() const):
+        (JSC::UnwindFunctor::copyCalleeSavesToEntryFrameCalleeSavesBuffer const):
+        * interpreter/StackVisitor.cpp:
+        (JSC::StackVisitor::Frame::calleeSaveRegistersForUnwinding):
+        (JSC::StackVisitor::Frame::calleeSaveRegisters): Deleted.
+        * interpreter/StackVisitor.h:
+        * jit/JITOperations.cpp:
+        * jit/RegisterSet.cpp:
+        (JSC::RegisterSet::runtimeTagRegisters):
+        (JSC::RegisterSet::specialRegisters):
+        (JSC::RegisterSet::runtimeRegisters): Deleted.
+        * jit/RegisterSet.h:
+        * jit/Repatch.cpp:
+        (JSC::linkPolymorphicCall):
+        * runtime/JSFunction.cpp:
+        (JSC::getCalculatedDisplayName):
+        * runtime/JSGlobalObject.cpp:
+        (JSC::JSGlobalObject::init):
+        (JSC::JSGlobalObject::visitChildren):
+        * runtime/JSGlobalObject.h:
+        (JSC::JSGlobalObject::jsToWasmICCalleeStructure const):
+        * runtime/VM.cpp:
+        (JSC::VM::VM):
+        * runtime/VM.h:
+        * wasm/WasmAirIRGenerator.cpp:
+        (JSC::Wasm::AirIRGenerator::AirIRGenerator):
+        (JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
+        (JSC::Wasm::AirIRGenerator::addCallIndirect):
+        * wasm/WasmB3IRGenerator.cpp:
+        (JSC::Wasm::B3IRGenerator::B3IRGenerator):
+        (JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
+        (JSC::Wasm::B3IRGenerator::addCallIndirect):
+        * wasm/WasmBinding.cpp:
+        (JSC::Wasm::wasmToWasm):
+        * wasm/WasmContext.h:
+        (JSC::Wasm::Context::pointerToInstance):
+        * wasm/WasmContextInlines.h:
+        (JSC::Wasm::Context::store):
+        * wasm/WasmMemoryInformation.cpp:
+        (JSC::Wasm::getPinnedRegisters):
+        (JSC::Wasm::PinnedRegisterInfo::get):
+        (JSC::Wasm::PinnedRegisterInfo::PinnedRegisterInfo):
+        * wasm/WasmMemoryInformation.h:
+        (JSC::Wasm::PinnedRegisterInfo::toSave const):
+        * wasm/WasmOMGPlan.cpp:
+        (JSC::Wasm::OMGPlan::work):
+        * wasm/js/JSToWasm.cpp:
+        (JSC::Wasm::createJSToWasmWrapper):
+        * wasm/js/JSToWasmICCallee.cpp: Added.
+        (JSC::JSToWasmICCallee::create):
+        (JSC::JSToWasmICCallee::createStructure):
+        (JSC::JSToWasmICCallee::visitChildren):
+        * wasm/js/JSToWasmICCallee.h: Added.
+        (JSC::JSToWasmICCallee::function):
+        (JSC::JSToWasmICCallee::JSToWasmICCallee):
+        * wasm/js/WebAssemblyFunction.cpp:
+        (JSC::WebAssemblyFunction::useTagRegisters const):
+        (JSC::WebAssemblyFunction::calleeSaves const):
+        (JSC::WebAssemblyFunction::usedCalleeSaveRegisters const):
+        (JSC::WebAssemblyFunction::previousInstanceOffset const):
+        (JSC::WebAssemblyFunction::previousInstance):
+        (JSC::WebAssemblyFunction::jsCallEntrypointSlow):
+        (JSC::WebAssemblyFunction::visitChildren):
+        (JSC::WebAssemblyFunction::destroy):
+        * wasm/js/WebAssemblyFunction.h:
+        * wasm/js/WebAssemblyFunctionHeapCellType.cpp: Added.
+        (JSC::WebAssemblyFunctionDestroyFunc::operator() const):
+        (JSC::WebAssemblyFunctionHeapCellType::WebAssemblyFunctionHeapCellType):
+        (JSC::WebAssemblyFunctionHeapCellType::~WebAssemblyFunctionHeapCellType):
+        (JSC::WebAssemblyFunctionHeapCellType::finishSweep):
+        (JSC::WebAssemblyFunctionHeapCellType::destroy):
+        * wasm/js/WebAssemblyFunctionHeapCellType.h: Added.
+        * wasm/js/WebAssemblyPrototype.h:
+
 2019-04-04  Yusuke Suzuki  <ysuzuki@apple.com>
 

trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -874 +874 @@
                 521131F71F82BF14007CCEEE /* PolyProtoAccessChain.h in Headers */ = {isa = PBXBuildFile; fileRef = 521131F61F82BF11007CCEEE /* PolyProtoAccessChain.h */; settings = {ATTRIBUTES = (Private, ); }; };
                 521322461ECBCE8200F65615 /* WebAssemblyFunctionBase.h in Headers */ = {isa = PBXBuildFile; fileRef = 521322441ECBCE8200F65615 /* WebAssemblyFunctionBase.h */; };
+                523FD88E225566C9003B3DCC /* WebAssemblyFunctionHeapCellType.h in Headers */ = {isa = PBXBuildFile; fileRef = 523FD88C225566C3003B3DCC /* WebAssemblyFunctionHeapCellType.h */; };
                 524E9D7322092B5200A6BEEE /* AirAllocateRegistersAndStackAndGenerateCode.h in Headers */ = {isa = PBXBuildFile; fileRef = 524E9D7222092B4600A6BEEE /* AirAllocateRegistersAndStackAndGenerateCode.h */; };
                 5250D2D21E8DA05A0029A932 /* WasmThunks.h in Headers */ = {isa = PBXBuildFile; fileRef = 5250D2D01E8DA05A0029A932 /* WasmThunks.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -881 +882 @@
                 52678F911A04177C006A306D /* ControlFlowProfiler.h in Headers */ = {isa = PBXBuildFile; fileRef = 52678F901A04177C006A306D /* ControlFlowProfiler.h */; settings = {ATTRIBUTES = (Private, ); }; };
                 526AC4B71E977C5D003500E1 /* WasmCodeBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 526AC4B51E977C5D003500E1 /* WasmCodeBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
+                527CE35422555FE500C6F382 /* JSToWasmICCallee.h in Headers */ = {isa = PBXBuildFile; fileRef = 527CE35322555FDD00C6F382 /* JSToWasmICCallee.h */; };
                 52847ADC21FFB8690061A9DB /* WasmAirIRGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = 52847ADA21FFB8630061A9DB /* WasmAirIRGenerator.h */; };
                 52B310FB1974AE610080857C /* FunctionHasExecutedCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 52B310FA1974AE610080857C /* FunctionHasExecutedCache.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -3378 +3380 @@
                 521322431ECBCE8200F65615 /* WebAssemblyFunctionBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = WebAssemblyFunctionBase.cpp; path = js/WebAssemblyFunctionBase.cpp; sourceTree = "<group>"; };
                 521322441ECBCE8200F65615 /* WebAssemblyFunctionBase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WebAssemblyFunctionBase.h; path = js/WebAssemblyFunctionBase.h; sourceTree = "<group>"; };
+                523FD88C225566C3003B3DCC /* WebAssemblyFunctionHeapCellType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = WebAssemblyFunctionHeapCellType.h; path = js/WebAssemblyFunctionHeapCellType.h; sourceTree = "<group>"; };
+                523FD88D225566C4003B3DCC /* WebAssemblyFunctionHeapCellType.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = WebAssemblyFunctionHeapCellType.cpp; path = js/WebAssemblyFunctionHeapCellType.cpp; sourceTree = "<group>"; };
                 524E9D7122092B4500A6BEEE /* AirAllocateRegistersAndStackAndGenerateCode.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = AirAllocateRegistersAndStackAndGenerateCode.cpp; path = b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp; sourceTree = "<group>"; };
                 524E9D7222092B4600A6BEEE /* AirAllocateRegistersAndStackAndGenerateCode.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = AirAllocateRegistersAndStackAndGenerateCode.h; path = b3/air/AirAllocateRegistersAndStackAndGenerateCode.h; sourceTree = "<group>"; };
@@ -3390 +3394 @@
                 526AC4B51E977C5D003500E1 /* WasmCodeBlock.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WasmCodeBlock.h; sourceTree = "<group>"; };
                 527773DD1AAF83AC00BDE7E8 /* RuntimeType.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RuntimeType.cpp; sourceTree = "<group>"; };
+                527CE35222555FDD00C6F382 /* JSToWasmICCallee.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = JSToWasmICCallee.cpp; path = js/JSToWasmICCallee.cpp; sourceTree = "<group>"; };
+                527CE35322555FDD00C6F382 /* JSToWasmICCallee.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = JSToWasmICCallee.h; path = js/JSToWasmICCallee.h; sourceTree = "<group>"; };
                 52847AD921FFB8630061A9DB /* WasmAirIRGenerator.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmAirIRGenerator.cpp; sourceTree = "<group>"; };
                 52847ADA21FFB8630061A9DB /* WasmAirIRGenerator.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmAirIRGenerator.h; sourceTree = "<group>"; };
@@ -8269 +8275 @@
                                 AD8DD6D01F6708A30004EB52 /* JSToWasm.cpp */,
                                 AD8DD6CF1F67089F0004EB52 /* JSToWasm.h */,
+                                527CE35222555FDD00C6F382 /* JSToWasmICCallee.cpp */,
+                                527CE35322555FDD00C6F382 /* JSToWasmICCallee.h */,
                                 ADD09AF21F624829001313C2 /* JSWebAssembly.cpp */,
                                 ADD09AF31F62482E001313C2 /* JSWebAssembly.h */,
@@ -8298 +8306 @@
                                 AD4937C91DDD27340077C807 /* WebAssemblyFunction.cpp */,
                                 AD4937CA1DDD27340077C807 /* WebAssemblyFunction.h */,
+                                523FD88D225566C4003B3DCC /* WebAssemblyFunctionHeapCellType.cpp */,
+                                523FD88C225566C3003B3DCC /* WebAssemblyFunctionHeapCellType.h */,
                                 521322431ECBCE8200F65615 /* WebAssemblyFunctionBase.cpp */,
                                 521322441ECBCE8200F65615 /* WebAssemblyFunctionBase.h */,
@@ -8615 +8625 @@
                                 DC69B99D1D15F914002E3C00 /* B3InferSwitches.h in Headers */,
                                 0FEC85BA1BE1462F0080FF74 /* B3InsertionSet.h in Headers */,
+                                523FD88E225566C9003B3DCC /* WebAssemblyFunctionHeapCellType.h in Headers */,
                                 0FEC85BB1BE1462F0080FF74 /* B3InsertionSetInlines.h in Headers */,
                                 0FDF67D21D9C6D27001B9825 /* B3Kind.h in Headers */,
@@ -8807 +8818 @@
                                 0FBB73BB1DEF8645002C009E /* DeleteAllCodeEffort.h in Headers */,
                                 0F96303C1D4192CD005609D9 /* DestructionMode.h in Headers */,
+                                527CE35422555FE500C6F382 /* JSToWasmICCallee.h in Headers */,
                                 A77A423E17A0BBFD00A8DB81 /* DFGAbstractHeap.h in Headers */,
                                 A704D90317A0BAA8006BA554 /* DFGAbstractInterpreter.h in Headers */,

trunk/Source/JavaScriptCore/Sources.txt

@@ -1015 +1015 @@
 wasm/js/JSToWasm.cpp
 wasm/js/JSToWasm.h
+wasm/js/JSToWasmICCallee.cpp
 wasm/js/JSWebAssembly.cpp
 wasm/js/JSWebAssemblyCodeBlock.cpp
@@ -1031 +1032 @@
 wasm/js/WebAssemblyFunction.cpp
 wasm/js/WebAssemblyFunctionBase.cpp
+wasm/js/WebAssemblyFunctionHeapCellType.cpp
 wasm/js/WebAssemblyInstanceConstructor.cpp
 wasm/js/WebAssemblyInstancePrototype.cpp

trunk/Source/JavaScriptCore/assembler/MacroAssemblerCodeRef.h

@@ -381 +381 @@
         ASSERT(m_codePtr);
     }
+
+    template<PtrTag otherTag>
+    MacroAssemblerCodeRef& operator=(const MacroAssemblerCodeRef<otherTag>& otherCodeRef)
+    {
+        m_codePtr = MacroAssemblerCodePtr<tag>::createFromExecutableAddress(otherCodeRef.code().template retaggedExecutableAddress<tag>());
+        m_executableMemory = otherCodeRef.m_executableMemory;
+        return *this;
+    }
     
     // Use this only when you know that the codePtr refers to code that is
@@ -444 +452 @@
     template<PtrTag otherTag>
     MacroAssemblerCodeRef(const MacroAssemblerCodeRef<otherTag>& otherCodeRef)
-        : m_codePtr(MacroAssemblerCodePtr<tag>::createFromExecutableAddress(otherCodeRef.code().template retaggedExecutableAddress<tag>()))
-        , m_executableMemory(otherCodeRef.m_executableMemory)
-    { }
+    {
+        *this = otherCodeRef;
+    }
 
     MacroAssemblerCodePtr<tag> m_codePtr;

trunk/Source/JavaScriptCore/interpreter/Interpreter.cpp

@@ -550 +550 @@
         }
 
+#if ENABLE(WEBASSEMBLY)
+        if (visitor->callee().isCell()) {
+            if (auto* jsToWasmICCallee = jsDynamicCast<JSToWasmICCallee*>(m_vm, visitor->callee().asCell()))
+                m_vm.wasmContext.store(jsToWasmICCallee->function()->previousInstance(m_callFrame), m_vm.softStackLimit());
+        }
+#endif
+
         notifyDebuggerOfUnwinding(m_vm, m_callFrame);
 
@@ -565 +572 @@
     {
 #if !ENABLE(C_LOOP) && NUMBER_OF_CALLEE_SAVES_REGISTERS > 0
-        const RegisterAtOffsetList* currentCalleeSaves = visitor->calleeSaveRegisters();
+        Optional<RegisterAtOffsetList> currentCalleeSaves = visitor->calleeSaveRegistersForUnwinding();
 
         if (!currentCalleeSaves)

trunk/Source/JavaScriptCore/interpreter/StackVisitor.cpp

@@ -34 +34 @@
 #include "WasmCallee.h"
 #include "WasmIndexOrName.h"
+#include "WebAssemblyFunction.h"
 #include <wtf/text/StringBuilder.h>
 
@@ -253 +254 @@
 }
 
-const RegisterAtOffsetList* StackVisitor::Frame::calleeSaveRegisters()
+Optional<RegisterAtOffsetList> StackVisitor::Frame::calleeSaveRegistersForUnwinding()
 {
     if (isInlinedFrame())
-        return nullptr;
+        return WTF::nullopt;
 
 #if !ENABLE(C_LOOP) && NUMBER_OF_CALLEE_SAVES_REGISTERS > 0
@@ -264 +265 @@
         if (callee().isCell()) {
             RELEASE_ASSERT(isWebAssemblyToJSCallee(callee().asCell()));
-            return nullptr;
+            return WTF::nullopt;
         }
         Wasm::Callee* wasmCallee = callee().asWasmCallee();
-        return wasmCallee->calleeSaveRegisters();
+        return *wasmCallee->calleeSaveRegisters();
+    }
+
+    if (callee().isCell()) {
+        if (auto* jsToWasmICCallee = jsDynamicCast<JSToWasmICCallee*>(*callee().asCell()->vm(), callee().asCell()))
+            return jsToWasmICCallee->function()->usedCalleeSaveRegisters();
     }
 #endif // ENABLE(WEBASSEMBLY)
 
     if (CodeBlock* codeBlock = this->codeBlock())
-        return codeBlock->calleeSaveRegisters();
+        return *codeBlock->calleeSaveRegisters();
 
 #endif // !ENABLE(C_LOOP) && NUMBER_OF_CALLEE_SAVES_REGISTERS > 0
 
-    return nullptr;
+    return WTF::nullopt;
 }
 

trunk/Source/JavaScriptCore/interpreter/StackVisitor.h

@@ -95 +95 @@
         JS_EXPORT_PRIVATE void computeLineAndColumn(unsigned& line, unsigned& column) const;
 
-        const RegisterAtOffsetList* calleeSaveRegisters();
+        Optional<RegisterAtOffsetList> calleeSaveRegistersForUnwinding();
 
         ClonedArguments* createArguments();

trunk/Source/JavaScriptCore/jit/JITOperations.cpp

@@ -77 +77 @@
 #include "TypeProfilerLog.h"
 #include "VMInlines.h"
+#include "WebAssemblyFunction.h"
 #include <wtf/InlineASM.h>
 
@@ -1037 +1038 @@
     MacroAssemblerCodePtr<JSEntryPtrTag> codePtr;
     CodeBlock* codeBlock = nullptr;
-    if (executable->isHostFunction())
-        codePtr = executable->entrypointFor(kind, MustCheckArity);
-    else {
+    if (executable->isHostFunction()) {
+        codePtr = jsToWasmICCodePtr(*vm, kind, callee);
+        if (!codePtr)
+            codePtr = executable->entrypointFor(kind, MustCheckArity);
+    } else {
         FunctionExecutable* functionExecutable = static_cast<FunctionExecutable*>(executable);
 
@@ -1065 +1068 @@
         codePtr = functionExecutable->entrypointFor(kind, arity);
     }
+
     if (!callLinkInfo->seenOnce())
         callLinkInfo->setSeen();

trunk/Source/JavaScriptCore/jit/RegisterSet.cpp

@@ -59 +59 @@
 }
 
-RegisterSet RegisterSet::runtimeRegisters()
+RegisterSet RegisterSet::runtimeTagRegisters()
 {
 #if USE(JSVALUE64)
@@ -71 +71 @@
 {
     return RegisterSet(
-        stackRegisters(), reservedHardwareRegisters(), runtimeRegisters());
+        stackRegisters(), reservedHardwareRegisters(), runtimeTagRegisters());
 }
 

trunk/Source/JavaScriptCore/jit/RegisterSet.h

@@ -51 +51 @@
     JS_EXPORT_PRIVATE static RegisterSet stackRegisters();
     JS_EXPORT_PRIVATE static RegisterSet reservedHardwareRegisters();
-    static RegisterSet runtimeRegisters();
+    static RegisterSet runtimeTagRegisters();
     static RegisterSet specialRegisters(); // The union of stack, reserved hardware, and runtime registers.
     JS_EXPORT_PRIVATE static RegisterSet calleeSaveRegisters();

trunk/Source/JavaScriptCore/jit/Repatch.cpp

@@ -976 +976 @@
         list = stub->variants();
     else if (JSObject* oldCallee = callLinkInfo.callee())
-        list = CallVariantList{ CallVariant(oldCallee) };
+        list = CallVariantList { CallVariant(oldCallee) };
     
     list = variantListWithVariant(list, newVariant);
@@ -1138 +1138 @@
         if (variant.executable()) {
             ASSERT(variant.executable()->hasJITCodeForCall());
-            codePtr = variant.executable()->generatedJITCodeForCall()->addressForCall(ArityCheckNotRequired);
+            
+            codePtr = jsToWasmICCodePtr(vm, callLinkInfo.specializationKind(), variant.function());
+            if (!codePtr)
+                codePtr = variant.executable()->generatedJITCodeForCall()->addressForCall(ArityCheckNotRequired);
         } else {
             ASSERT(variant.internalFunction());
@@ -1278 +1281 @@
 }
 
+MacroAssemblerCodePtr<JSEntryPtrTag> jsToWasmICCodePtr(VM& vm, CodeSpecializationKind kind, JSObject* callee)
+{
+#if ENABLE(WEBASSEMBLY)
+    if (!callee)
+        return nullptr;
+    if (kind != CodeForCall)
+        return nullptr;
+    if (auto* wasmFunction = jsDynamicCast<WebAssemblyFunction*>(vm, callee))
+        return wasmFunction->jsCallEntrypoint();
+#else
+    UNUSED_PARAM(vm);
+    UNUSED_PARAM(kind);
+    UNUSED_PARAM(callee);
+#endif
+    return nullptr;
+}
+
 } // namespace JSC
 

trunk/Source/JavaScriptCore/jit/Repatch.h

@@ -58 +58 @@
 void resetInstanceOf(StructureStubInfo&);
 void ftlThunkAwareRepatchCall(CodeBlock*, CodeLocationCall<JSInternalPtrTag>, FunctionPtr<CFunctionPtrTag> newCalleeFunction);
+MacroAssemblerCodePtr<JSEntryPtrTag> jsToWasmICCodePtr(VM&, CodeSpecializationKind, JSObject* callee);
+
 
 } // namespace JSC

trunk/Source/JavaScriptCore/runtime/JSFunction.cpp

@@ -49 +49 @@
 #include "PropertyNameArray.h"
 #include "StackVisitor.h"
+#include "WebAssemblyFunction.h"
 
 namespace JSC {
@@ -628 +629 @@
 String getCalculatedDisplayName(VM& vm, JSObject* object)
 {
+#if ENABLE(WEBASSEMBLY)
+    if (jsDynamicCast<JSToWasmICCallee*>(vm, object))
+        return "wasm-stub"_s;
+#endif
+
     if (!jsDynamicCast<JSFunction*>(vm, object) && !jsDynamicCast<InternalFunction*>(vm, object))
         return emptyString();
-
 
     Structure* structure = object->structure(vm);

trunk/Source/JavaScriptCore/runtime/JSGlobalObject.cpp

@@ -1054 +1054 @@
                 init.set(WebAssemblyFunction::createStructure(init.vm, init.owner, init.owner->m_functionPrototype.get()));
             });
+        m_jsToWasmICCalleeStructure.initLater(
+            [] (const Initializer<Structure>& init) {
+                init.set(JSToWasmICCallee::createStructure(init.vm, init.owner, JSValue()));
+            });
         m_webAssemblyWrapperFunctionStructure.initLater(
             [] (const Initializer<Structure>& init) {
@@ -1752 +1756 @@
     thisObject->m_webAssemblyModuleRecordStructure.visit(visitor);
     thisObject->m_webAssemblyFunctionStructure.visit(visitor);
+    thisObject->m_jsToWasmICCalleeStructure.visit(visitor);
     thisObject->m_webAssemblyWrapperFunctionStructure.visit(visitor);
     thisObject->m_webAssemblyToJSCalleeStructure.visit(visitor);

trunk/Source/JavaScriptCore/runtime/JSGlobalObject.h

@@ -401 +401 @@
     LazyProperty<JSGlobalObject, Structure> m_webAssemblyModuleRecordStructure;
     LazyProperty<JSGlobalObject, Structure> m_webAssemblyFunctionStructure;
+    LazyProperty<JSGlobalObject, Structure> m_jsToWasmICCalleeStructure;
     LazyProperty<JSGlobalObject, Structure> m_webAssemblyWrapperFunctionStructure;
     LazyProperty<JSGlobalObject, Structure> m_webAssemblyToJSCalleeStructure;
@@ -765 +766 @@
     Structure* webAssemblyModuleRecordStructure() const { return m_webAssemblyModuleRecordStructure.get(this); }
     Structure* webAssemblyFunctionStructure() const { return m_webAssemblyFunctionStructure.get(this); }
+    Structure* jsToWasmICCalleeStructure() const { return m_jsToWasmICCalleeStructure.get(this); }
     Structure* webAssemblyWrapperFunctionStructure() const { return m_webAssemblyWrapperFunctionStructure.get(this); }
     Structure* webAssemblyToJSCalleeStructure() const { return m_webAssemblyToJSCalleeStructure.get(this); }

trunk/Source/JavaScriptCore/runtime/VM.cpp

@@ -149 +149 @@
 #include "WeakGCMapInlines.h"
 #include "WebAssemblyFunction.h"
+#include "WebAssemblyFunctionHeapCellType.h"
 #include "WebAssemblyWrapperFunction.h"
 #include <wtf/ProcessID.h>
@@ -281 +282 @@
 #if ENABLE(WEBASSEMBLY)
     , webAssemblyCodeBlockHeapCellType(std::make_unique<JSWebAssemblyCodeBlockHeapCellType>())
+    , webAssemblyFunctionHeapCellType(std::make_unique<WebAssemblyFunctionHeapCellType>())
 #endif
     , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, auxiliaryHeapCellType.get(), primitiveGigacageAllocator.get())
@@ -1265 +1267 @@
 #if ENABLE(WEBASSEMBLY)
 DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyCodeBlockSpace, webAssemblyCodeBlockHeapCellType.get(), JSWebAssemblyCodeBlock)
-DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyFunctionSpace, cellHeapCellType.get(), WebAssemblyFunction)
+DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyFunctionSpace, webAssemblyFunctionHeapCellType.get(), WebAssemblyFunction)
 DYNAMIC_ISO_SUBSPACE_DEFINE_MEMBER_SLOW(webAssemblyWrapperFunctionSpace, cellHeapCellType.get(), WebAssemblyWrapperFunction)
 #endif

trunk/Source/JavaScriptCore/runtime/VM.h

@@ -159 +159 @@
 class Watchpoint;
 class WatchpointSet;
+class WebAssemblyFunctionHeapCellType;
 
 #if ENABLE(FTL_JIT)
@@ -338 +339 @@
 #if ENABLE(WEBASSEMBLY)
     std::unique_ptr<JSWebAssemblyCodeBlockHeapCellType> webAssemblyCodeBlockHeapCellType;
+    std::unique_ptr<WebAssemblyFunctionHeapCellType> webAssemblyFunctionHeapCellType;
 #endif
     

trunk/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp

@@ -681 +681 @@
 
     if (mode != MemoryMode::Signaling) {
-        ASSERT(!pinnedRegs.sizeRegisters[0].sizeOffset);
-        m_memorySizeGPR = pinnedRegs.sizeRegisters[0].sizeRegister;
-        for (const PinnedSizeRegisterInfo& regInfo : pinnedRegs.sizeRegisters)
-            m_code.pinRegister(regInfo.sizeRegister);
+        m_memorySizeGPR = pinnedRegs.sizeRegister;
+        m_code.pinRegister(m_memorySizeGPR);
     }
 
@@ -824 +822 @@
         RegisterSet clobbers;
         clobbers.set(pinnedRegs->baseMemoryPointer);
-        for (auto info : pinnedRegs->sizeRegisters)
-            clobbers.set(info.sizeRegister);
+        clobbers.set(pinnedRegs->sizeRegister);
 
         auto* patchpoint = addPatchpoint(B3::Void);
@@ -835 +832 @@
 
         patchpoint->setGenerator([pinnedRegs] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
-            GPRReg baseMemory = pinnedRegs->baseMemoryPointer;
-            const auto& sizeRegs = pinnedRegs->sizeRegisters;
-            ASSERT(sizeRegs.size() >= 1);
-            ASSERT(!sizeRegs[0].sizeOffset); // The following code assumes we start at 0, and calculates subsequent size registers relative to 0.
-            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister);
-            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), baseMemory);
-            for (unsigned i = 1; i < sizeRegs.size(); ++i)
-                jit.add64(CCallHelpers::TrustedImm32(-sizeRegs[i].sizeOffset), sizeRegs[0].sizeRegister, sizeRegs[i].sizeRegister);
+            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
+            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), pinnedRegs->baseMemoryPointer);
         });
 
@@ -1856 +1847 @@
             GPRReg oldContextInstance = params[1].gpr();
             const PinnedRegisterInfo& pinnedRegs = PinnedRegisterInfo::get();
-            const auto& sizeRegs = pinnedRegs.sizeRegisters;
             GPRReg baseMemory = pinnedRegs.baseMemoryPointer;
             ASSERT(newContextInstance != baseMemory);
@@ -1862 +1852 @@
             jit.storePtr(baseMemory, CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedStackLimit()));
             jit.storeWasmContextInstance(newContextInstance);
-            ASSERT(sizeRegs[0].sizeRegister != baseMemory);
             // FIXME: We should support more than one memory size register
             //   see: https://bugs.webkit.org/show_bug.cgi?id=162952
-            ASSERT(sizeRegs.size() == 1);
-            ASSERT(sizeRegs[0].sizeRegister != newContextInstance);
-            ASSERT(!sizeRegs[0].sizeOffset);
-            jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister); // Memory size.
+            ASSERT(pinnedRegs.sizeRegister != newContextInstance);
+            jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
         });

trunk/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp

@@ -347 +347 @@
 
     if (mode != MemoryMode::Signaling) {
-        ASSERT(!pinnedRegs.sizeRegisters[0].sizeOffset);
-        m_memorySizeGPR = pinnedRegs.sizeRegisters[0].sizeRegister;
-        for (const PinnedSizeRegisterInfo& regInfo : pinnedRegs.sizeRegisters)
-            m_proc.pinRegister(regInfo.sizeRegister);
+        m_memorySizeGPR = pinnedRegs.sizeRegister;
+        m_proc.pinRegister(m_memorySizeGPR);
     }
 
@@ -470 +468 @@
         RegisterSet clobbers;
         clobbers.set(pinnedRegs->baseMemoryPointer);
-        for (auto info : pinnedRegs->sizeRegisters)
-            clobbers.set(info.sizeRegister);
+        clobbers.set(pinnedRegs->sizeRegister);
 
         B3::PatchpointValue* patchpoint = block->appendNew<B3::PatchpointValue>(proc, B3::Void, origin());
@@ -484 +481 @@
         patchpoint->setGenerator([pinnedRegs] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
             GPRReg baseMemory = pinnedRegs->baseMemoryPointer;
-            const auto& sizeRegs = pinnedRegs->sizeRegisters;
-            ASSERT(sizeRegs.size() >= 1);
-            ASSERT(!sizeRegs[0].sizeOffset); // The following code assumes we start at 0, and calculates subsequent size registers relative to 0.
-            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister);
+            jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemorySize()), pinnedRegs->sizeRegister);
             jit.loadPtr(CCallHelpers::Address(params[0].gpr(), Instance::offsetOfCachedMemory()), baseMemory);
-            for (unsigned i = 1; i < sizeRegs.size(); ++i)
-                jit.add64(CCallHelpers::TrustedImm32(-sizeRegs[i].sizeOffset), sizeRegs[0].sizeRegister, sizeRegs[i].sizeRegister);
         });
     }
@@ -1283 +1275 @@
             GPRReg oldContextInstance = params[1].gpr();
             const PinnedRegisterInfo& pinnedRegs = PinnedRegisterInfo::get();
-            const auto& sizeRegs = pinnedRegs.sizeRegisters;
             GPRReg baseMemory = pinnedRegs.baseMemoryPointer;
             ASSERT(newContextInstance != baseMemory);
@@ -1289 +1280 @@
             jit.storePtr(baseMemory, CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedStackLimit()));
             jit.storeWasmContextInstance(newContextInstance);
-            ASSERT(sizeRegs[0].sizeRegister != baseMemory);
+            ASSERT(pinnedRegs.sizeRegister != baseMemory);
             // FIXME: We should support more than one memory size register
             //   see: https://bugs.webkit.org/show_bug.cgi?id=162952
-            ASSERT(sizeRegs.size() == 1);
-            ASSERT(sizeRegs[0].sizeRegister != newContextInstance);
-            ASSERT(!sizeRegs[0].sizeOffset);
-            jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister); // Memory size.
+            ASSERT(pinnedRegs.sizeRegister != newContextInstance);
+            jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
             jit.loadPtr(CCallHelpers::Address(newContextInstance, Instance::offsetOfCachedMemory()), baseMemory); // Memory::void*.
         });

trunk/Source/JavaScriptCore/wasm/WasmBinding.cpp

@@ -49 +49 @@
     GPRReg baseMemory = pinnedRegs.baseMemoryPointer;
     ASSERT(baseMemory != scratch);
-    const auto& sizeRegs = pinnedRegs.sizeRegisters;
-    ASSERT(sizeRegs.size() >= 1);
-    ASSERT(sizeRegs[0].sizeRegister != baseMemory);
-    ASSERT(sizeRegs[0].sizeRegister != scratch);
-    GPRReg sizeRegAsScratch = sizeRegs[0].sizeRegister;
+    ASSERT(pinnedRegs.sizeRegister != baseMemory);
+    ASSERT(pinnedRegs.sizeRegister != scratch);
+    GPRReg sizeRegAsScratch = pinnedRegs.sizeRegister;
 
     // B3's call codegen ensures that the JSCell is a WebAssemblyFunction.
@@ -68 +66 @@
     // FIXME the following code assumes that all Wasm::Instance have the same pinned registers. https://bugs.webkit.org/show_bug.cgi?id=162952
     // Set up the callee's baseMemory register as well as the memory size registers.
-    ASSERT(!sizeRegs[0].sizeOffset); // The following code assumes we start at 0, and calculates subsequent size registers relative to 0.
-    jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister); // Memory size.
+    jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister); // Memory size.
     jit.loadPtr(JIT::Address(baseMemory, Wasm::Instance::offsetOfCachedMemory()), baseMemory); // Wasm::Memory::void*.
-    for (unsigned i = 1; i < sizeRegs.size(); ++i) {
-        ASSERT(sizeRegs[i].sizeRegister != baseMemory);
-        ASSERT(sizeRegs[i].sizeRegister != scratch);
-        jit.add64(JIT::TrustedImm32(-sizeRegs[i].sizeOffset), sizeRegs[0].sizeRegister, sizeRegs[i].sizeRegister);
-    }
 
     // Tail call into the callee WebAssembly function.

trunk/Source/JavaScriptCore/wasm/WasmContext.h

@@ -33 +33 @@
 
 struct Context {
-    Instance* instance { nullptr };
-
     Instance* load() const;
     void store(Instance*, void* softStackLimit);
 
     static bool useFastTLS();
+
+    Instance** pointerToInstance()
+    {
+        ASSERT(!useFastTLS());
+        return &instance;
+    }
+
+private:
+    Instance* instance { nullptr };
 };
 

trunk/Source/JavaScriptCore/wasm/WasmContextInlines.h

@@ -56 +56 @@
 inline void Context::store(Instance* inst, void* softStackLimit)
 {
+    if (inst)
+        inst->setCachedStackLimit(softStackLimit);
+
 #if ENABLE(FAST_TLS_JIT)
     if (useFastTLS())
         _pthread_setspecific_direct(WTF_WASM_CONTEXT_KEY, bitwise_cast<void*>(inst));
+    else
 #endif
-    instance = inst;
-    if (instance)
-        instance->setCachedStackLimit(softStackLimit);
+        instance = inst;
 }
 

trunk/Source/JavaScriptCore/wasm/WasmMemoryInformation.cpp

@@ -45 +45 @@
         if (!remainingPinnedRegisters || RegisterSet::stackRegisters().get(reg))
             return;
+        if (RegisterSet::runtimeTagRegisters().get(reg)) {
+            // Since we don't need to, we currently don't pick from the tag registers to allow
+            // JS->Wasm stubs to freely use these registers.
+            return;
+        }
         --remainingPinnedRegisters;
         registers.append(gpr);
@@ -56 +61 @@
     static std::once_flag staticPinnedRegisterInfoFlag;
     std::call_once(staticPinnedRegisterInfoFlag, [] () {
-        Vector<PinnedSizeRegisterInfo> sizeRegisters;
-        GPRReg baseMemoryPointer = InvalidGPRReg;
-        GPRReg wasmContextInstancePointer = InvalidGPRReg;
-
-        // FIXME: We should support more than one memory size register, and we should allow different
-        //        WebAssembly.Instance to have different pins. Right now we take a vector with only one entry.
-        //        If we have more than one size register, we can have one for each load size class.
-        //        see: https://bugs.webkit.org/show_bug.cgi?id=162952
-        Vector<unsigned> pinnedSizes = { 0 };
-        unsigned numberOfPinnedRegisters = pinnedSizes.size() + 1;
+        unsigned numberOfPinnedRegisters = 2;
         if (!Context::useFastTLS())
             ++numberOfPinnedRegisters;
         Vector<GPRReg> pinnedRegs = getPinnedRegisters(numberOfPinnedRegisters);
 
-        baseMemoryPointer = pinnedRegs.takeLast();
+        GPRReg baseMemoryPointer = pinnedRegs.takeLast();
+        GPRReg sizeRegister = pinnedRegs.takeLast();
+        GPRReg wasmContextInstancePointer = InvalidGPRReg;
         if (!Context::useFastTLS())
             wasmContextInstancePointer = pinnedRegs.takeLast();
 
-        ASSERT(pinnedSizes.size() == pinnedRegs.size());
-        for (unsigned i = 0; i < pinnedSizes.size(); ++i)
-            sizeRegisters.append({ pinnedRegs[i], pinnedSizes[i] });
-        staticPinnedRegisterInfo.construct(WTFMove(sizeRegisters), baseMemoryPointer, wasmContextInstancePointer);
+        staticPinnedRegisterInfo.construct(sizeRegister, baseMemoryPointer, wasmContextInstancePointer);
     });
 
@@ -83 +78 @@
 }
 
-PinnedRegisterInfo::PinnedRegisterInfo(Vector<PinnedSizeRegisterInfo>&& sizeRegisters, GPRReg baseMemoryPointer, GPRReg wasmContextInstancePointer)
-    : sizeRegisters(WTFMove(sizeRegisters))
+PinnedRegisterInfo::PinnedRegisterInfo(GPRReg sizeRegister, GPRReg baseMemoryPointer, GPRReg wasmContextInstancePointer)
+    : sizeRegister(sizeRegister)
     , baseMemoryPointer(baseMemoryPointer)
     , wasmContextInstancePointer(wasmContextInstancePointer)

trunk/Source/JavaScriptCore/wasm/WasmMemoryInformation.h

@@ -46 +46 @@
 class PinnedRegisterInfo {
 public:
-    PinnedRegisterInfo(Vector<PinnedSizeRegisterInfo>&&, GPRReg, GPRReg);
+    PinnedRegisterInfo(GPRReg, GPRReg, GPRReg);
 
     static const PinnedRegisterInfo& get();
@@ -56 +56 @@
         if (wasmContextInstancePointer != InvalidGPRReg)
             result.set(wasmContextInstancePointer);
-        if (mode != MemoryMode::Signaling) {
-            for (const auto& info : sizeRegisters)
-                result.set(info.sizeRegister);
-        }
+        if (mode != MemoryMode::Signaling)
+            result.set(sizeRegister);
         return result;
     }
 
-    Vector<PinnedSizeRegisterInfo> sizeRegisters;
+    GPRReg sizeRegister;
     GPRReg baseMemoryPointer;
     GPRReg wasmContextInstancePointer;

trunk/Source/JavaScriptCore/wasm/WasmOMGPlan.cpp

@@ -139 +139 @@
         LockHolder holder(m_codeBlock->m_lock);
 
-        auto repatchCalls = [&] (const Vector<UnlinkedWasmToWasmCall>&  callsites) {
+        auto repatchCalls = [&] (const Vector<UnlinkedWasmToWasmCall>& callsites) {
             for (auto& call : callsites) {
                 dataLogLnIf(WasmOMGPlanInternal::verbose, "Considering repatching call at: ", RawPointer(call.callLocation.dataLocation()), " that targets ", call.functionIndexSpace);

trunk/Source/JavaScriptCore/wasm/js/JSToWasm.cpp

@@ -214 +214 @@
 
         GPRReg currentInstanceGPR = Context::useFastTLS() ? baseMemory : wasmContextInstanceGPR;
-        if (mode != MemoryMode::Signaling) {
-            const auto& sizeRegs = pinnedRegs.sizeRegisters;
-            ASSERT(sizeRegs.size() >= 1);
-            ASSERT(!sizeRegs[0].sizeOffset); // The following code assumes we start at 0, and calculates subsequent size registers relative to 0.
-            jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemorySize()), sizeRegs[0].sizeRegister);
-            for (unsigned i = 1; i < sizeRegs.size(); ++i)
-                jit.add64(CCallHelpers::TrustedImm32(-sizeRegs[i].sizeOffset), sizeRegs[0].sizeRegister, sizeRegs[i].sizeRegister);
-        }
+        if (mode != MemoryMode::Signaling)
+            jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister);
 
         jit.loadPtr(CCallHelpers::Address(currentInstanceGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);

trunk/Source/JavaScriptCore/wasm/js/JSToWasmICCallee.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
- * Copyright (C) 2018 Oleksandr Skachkov <gskachkov@gmail.com>.
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -25 +24 @@
  */
 
-#pragma once
+#include "config.h"
+#include "JSToWasmICCallee.h"
 
 #if ENABLE(WEBASSEMBLY)
 
-#include "JSDestructibleObject.h"
-#include "JSObject.h"
+#include "WebAssemblyFunction.h"
 
 namespace JSC {
 
-class WebAssemblyPrototype final : public JSNonFinalObject {
-public:
-    typedef JSNonFinalObject Base;
-    static const unsigned StructureFlags = Base::StructureFlags | HasStaticPropertyTable;
+const ClassInfo JSToWasmICCallee::s_info = { "JSToWasmICCallee", &Base::s_info, nullptr, nullptr, CREATE_METHOD_TABLE(JSToWasmICCallee) };
 
-    static WebAssemblyPrototype* create(VM&, JSGlobalObject*, Structure*);
-    static Structure* createStructure(VM&, JSGlobalObject*, JSValue);
-    JS_EXPORT_PRIVATE static void webAssemblyModuleValidateAsync(ExecState*, JSPromiseDeferred*, Vector<uint8_t>&&);
-    JS_EXPORT_PRIVATE static void webAssemblyModuleInstantinateAsync(ExecState*, JSPromiseDeferred*, Vector<uint8_t>&&, JSObject*);
+JSToWasmICCallee* JSToWasmICCallee::create(VM& vm, JSGlobalObject* globalObject, WebAssemblyFunction* function)
+{
+    auto* structure = globalObject->jsToWasmICCalleeStructure();
+    JSToWasmICCallee* result = new (NotNull, allocateCell<JSToWasmICCallee>(vm.heap)) JSToWasmICCallee(vm, globalObject, structure);
+    result->finishCreation(vm);
+    result->m_function.set(vm, result, function);
+    return result;
+}
 
-    DECLARE_INFO;
+Structure* JSToWasmICCallee::createStructure(VM& vm, JSGlobalObject* globalObject, JSValue prototype)
+{
+    return Structure::create(vm, globalObject, prototype, TypeInfo(JSCalleeType, StructureFlags), info());
+}
 
-    static JSValue instantiate(ExecState*, JSPromiseDeferred*, const Identifier&, JSValue);
+void JSToWasmICCallee::visitChildren(JSCell* cell, SlotVisitor& visitor)
+{
+    JSToWasmICCallee* thisObject = jsCast<JSToWasmICCallee*>(cell);
+    ASSERT_GC_OBJECT_INHERITS(thisObject, info());
 
-protected:
-    void finishCreation(VM&, JSGlobalObject*);
-
-private:
-    WebAssemblyPrototype(VM&, Structure*);
-};
-
-EncodedJSValue JSC_HOST_CALL webAssemblyCompileStreamingInternal(ExecState*);
-EncodedJSValue JSC_HOST_CALL webAssemblyInstantiateStreamingInternal(ExecState*);
+    Base::visitChildren(thisObject, visitor);
+    visitor.append(thisObject->m_function);
+}
 
 } // namespace JSC

trunk/Source/JavaScriptCore/wasm/js/JSToWasmICCallee.h

@@ -1 +1 @@
 /*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -26 +26 @@
 #pragma once
 
+#include "JSCallee.h"
+
+namespace JSC {
+
 #if ENABLE(WEBASSEMBLY)
 
-namespace JSC { namespace Wasm {
+class WebAssemblyFunction;
 
-class Instance;
+class JSToWasmICCallee : public JSCallee {
+    using Base = JSCallee;
+public:
+    DECLARE_INFO;
 
-struct Context {
-    Instance* instance { nullptr };
+    static JSToWasmICCallee* create(VM&, JSGlobalObject*, WebAssemblyFunction*);
+    static Structure* createStructure(VM&, JSGlobalObject*, JSValue);
 
-    Instance* load() const;
-    void store(Instance*, void* softStackLimit);
+    WebAssemblyFunction* function() { return m_function.get(); }
 
-    static bool useFastTLS();
+private:
+    JSToWasmICCallee(VM& vm, JSGlobalObject* globalObject, Structure* structure)
+        : Base(vm, globalObject, structure)
+    { }
+    static void visitChildren(JSCell*, SlotVisitor&);
+
+    WriteBarrier<WebAssemblyFunction> m_function;
 };
 
-} } // namespace JSC::Wasm
+#endif // ENABLE(WEBASSEMBLY)
 
-#endif // ENABLE(WEBASSEMBLY)
+} // namespace JSC

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp

@@ -37 +37 @@
 #include "JSWebAssemblyRuntimeError.h"
 #include "LLIntThunks.h"
+#include "LinkBuffer.h"
 #include "ProtoCallFrame.h"
 #include "VM.h"
 #include "WasmCallee.h"
+#include "WasmCallingConvention.h"
 #include "WasmContextInlines.h"
 #include "WasmFormat.h"
 #include "WasmMemory.h"
+#include "WasmMemoryInformation.h"
+#include "WasmModuleInformation.h"
 #include "WasmSignatureInlines.h"
 #include <wtf/FastTLS.h>
@@ -149 +153 @@
 }
 
+bool WebAssemblyFunction::useTagRegisters() const
+{
+    const auto& signature = Wasm::SignatureInformation::get(signatureIndex());
+    return signature.argumentCount() || signature.returnType() != Wasm::Void;
+}
+
+RegisterSet WebAssemblyFunction::calleeSaves() const
+{
+    RegisterSet toSave = Wasm::PinnedRegisterInfo::get().toSave(instance()->memoryMode());
+    if (useTagRegisters()) {
+        RegisterSet tagRegisters = RegisterSet::runtimeTagRegisters();
+        // We rely on these being disjoint sets.
+#if !ASSERT_DISABLED
+        for (Reg reg : tagRegisters)
+            ASSERT(!toSave.contains(reg));
+#endif
+        toSave.merge(tagRegisters);
+    }
+    return toSave;
+}
+
+RegisterAtOffsetList WebAssemblyFunction::usedCalleeSaveRegisters() const
+{
+    return RegisterAtOffsetList { calleeSaves(), RegisterAtOffsetList::OffsetBaseType::FramePointerBased };
+}
+
+ptrdiff_t WebAssemblyFunction::previousInstanceOffset() const
+{
+    ptrdiff_t result = calleeSaves().numberOfSetRegisters() * sizeof(CPURegister);
+    result = -result - sizeof(CPURegister);
+#if !ASSERT_DISABLED
+    ptrdiff_t minOffset = 1;
+    for (const RegisterAtOffset& regAtOffset : usedCalleeSaveRegisters()) {
+        ptrdiff_t offset = regAtOffset.offset();
+        ASSERT(offset < 0);
+        minOffset = std::min(offset, minOffset);
+    }
+    ASSERT(minOffset - static_cast<ptrdiff_t>(sizeof(CPURegister)) == result);
+#endif
+    return result;
+}
+
+Wasm::Instance* WebAssemblyFunction::previousInstance(CallFrame* callFrame)
+{
+    ASSERT(callFrame->callee().rawPtr() == m_jsToWasmICCallee.get());
+    auto* result = *bitwise_cast<Wasm::Instance**>(bitwise_cast<char*>(callFrame) + previousInstanceOffset());
+    return result;
+}
+
+MacroAssemblerCodePtr<JSEntryPtrTag> WebAssemblyFunction::jsCallEntrypointSlow()
+{
+    VM& vm = *this->vm();
+    CCallHelpers jit;
+
+    const auto& signature = Wasm::SignatureInformation::get(signatureIndex());
+    const auto& pinnedRegs = Wasm::PinnedRegisterInfo::get();
+    RegisterAtOffsetList registersToSpill = usedCalleeSaveRegisters();
+
+    auto& moduleInformation = instance()->instance().module().moduleInformation();
+
+    unsigned totalFrameSize = registersToSpill.size() * sizeof(CPURegister);
+    totalFrameSize += sizeof(CPURegister); // Slot for the VM's previous wasm instance.
+    totalFrameSize += Wasm::WasmCallingConvention::headerSizeInBytes();
+    totalFrameSize -= sizeof(CallerFrameAndPC);
+
+    unsigned numGPRs = 0;
+    unsigned numFPRs = 0;
+    bool argumentsIncludeI64 = false;
+    for (unsigned i = 0; i < signature.argumentCount(); i++) {
+        switch (signature.argument(i)) {
+        case Wasm::I64:
+            argumentsIncludeI64 = true;
+            break;
+        case Wasm::I32:
+            if (numGPRs >= Wasm::wasmCallingConvention().m_gprArgs.size())
+                totalFrameSize += sizeof(CPURegister);
+            ++numGPRs;
+            break;
+        case Wasm::F32:
+        case Wasm::F64:
+            if (numFPRs >= Wasm::wasmCallingConvention().m_fprArgs.size())
+                totalFrameSize += sizeof(CPURegister);
+            ++numFPRs;
+            break;
+        default:
+            RELEASE_ASSERT_NOT_REACHED();
+        }
+    }
+
+    if (argumentsIncludeI64)
+        return nullptr;
+
+    totalFrameSize = WTF::roundUpToMultipleOf(stackAlignmentBytes(), totalFrameSize);
+
+    jit.emitFunctionPrologue();
+    jit.subPtr(MacroAssembler::TrustedImm32(totalFrameSize), MacroAssembler::stackPointerRegister);
+    jit.store64(CCallHelpers::TrustedImm64(0), CCallHelpers::addressFor(CallFrameSlot::codeBlock));
+
+    for (const RegisterAtOffset& regAtOffset : registersToSpill) {
+        GPRReg reg = regAtOffset.reg().gpr();
+        ptrdiff_t offset = regAtOffset.offset();
+        jit.storePtr(reg, CCallHelpers::Address(GPRInfo::callFrameRegister, offset));
+    }
+
+    GPRReg scratchGPR = Wasm::wasmCallingConventionAir().prologueScratch(1);
+    GPRReg scratch2GPR = Wasm::wasmCallingConventionAir().prologueScratch(0);
+    jit.loadPtr(vm.addressOfSoftStackLimit(), scratch2GPR);
+
+    CCallHelpers::JumpList slowPath;
+    slowPath.append(jit.branchPtr(CCallHelpers::Above, MacroAssembler::stackPointerRegister, GPRInfo::callFrameRegister));
+    slowPath.append(jit.branchPtr(CCallHelpers::Below, MacroAssembler::stackPointerRegister, scratch2GPR));
+
+    // Ensure:
+    // argCountPlusThis - 1 >= signature.argumentCount()
+    // argCountPlusThis >= signature.argumentCount() + 1
+    // FIXME: We should handle mismatched arity
+    // https://bugs.webkit.org/show_bug.cgi?id=196564
+    slowPath.append(jit.branch32(CCallHelpers::Below,
+        CCallHelpers::payloadFor(CallFrameSlot::argumentCount), CCallHelpers::TrustedImm32(signature.argumentCount() + 1)));
+
+    if (useTagRegisters())
+        jit.emitMaterializeTagCheckRegisters();
+
+    // First we do stack slots for FPRs so we can use FPR argument registers as scratch.
+    // After that, we handle FPR argument registers.
+    // We also handle all GPR types here as we have GPR scratch registers.
+    {
+        CCallHelpers::Address calleeFrame = CCallHelpers::Address(MacroAssembler::stackPointerRegister, -static_cast<ptrdiff_t>(sizeof(CallerFrameAndPC)));
+        numGPRs = 0;
+        numFPRs = 0;
+        FPRReg scratchFPR = Wasm::wasmCallingConvention().m_fprArgs[0].fpr();
+
+        ptrdiff_t jsOffset = CallFrameSlot::firstArgument * sizeof(EncodedJSValue);
+
+        ptrdiff_t wasmOffset = CallFrame::headerSizeInRegisters * sizeof(CPURegister);
+        for (unsigned i = 0; i < signature.argumentCount(); i++) {
+            switch (signature.argument(i)) {
+            case Wasm::I32:
+                jit.load64(CCallHelpers::Address(GPRInfo::callFrameRegister, jsOffset), scratchGPR);
+                slowPath.append(jit.branchIfNotInt32(scratchGPR));
+                if (numGPRs >= Wasm::wasmCallingConvention().m_gprArgs.size()) {
+                    jit.store32(scratchGPR, calleeFrame.withOffset(wasmOffset));
+                    wasmOffset += sizeof(CPURegister);
+                } else {
+                    jit.zeroExtend32ToPtr(scratchGPR, Wasm::wasmCallingConvention().m_gprArgs[numGPRs].gpr());
+                    ++numGPRs;
+                }
+                break;
+            case Wasm::F32:
+            case Wasm::F64:
+                if (numFPRs >= Wasm::wasmCallingConvention().m_fprArgs.size()) {
+                    jit.load64(CCallHelpers::Address(GPRInfo::callFrameRegister, jsOffset), scratchGPR);
+                    slowPath.append(jit.branchIfNotNumber(scratchGPR));
+                    auto isInt32 = jit.branchIfInt32(scratchGPR);
+                    if (signature.argument(i) == Wasm::F32) {
+                        jit.unboxDouble(scratchGPR, scratchGPR, scratchFPR);
+                        jit.convertDoubleToFloat(scratchFPR, scratchFPR);
+                        jit.storeFloat(scratchFPR, calleeFrame.withOffset(wasmOffset));
+                    } else {
+                        jit.add64(GPRInfo::tagTypeNumberRegister, scratchGPR, scratchGPR);
+                        jit.store64(scratchGPR, calleeFrame.withOffset(wasmOffset));
+                    }
+                    auto done = jit.jump();
+
+                    isInt32.link(&jit);
+                    if (signature.argument(i) == Wasm::F32) {
+                        jit.convertInt32ToFloat(scratchGPR, scratchFPR);
+                        jit.storeFloat(scratchFPR, calleeFrame.withOffset(wasmOffset));
+                    } else {
+                        jit.convertInt32ToDouble(scratchGPR, scratchFPR);
+                        jit.storeDouble(scratchFPR, calleeFrame.withOffset(wasmOffset));
+                    }
+                    done.link(&jit);
+
+                    wasmOffset += sizeof(CPURegister);
+                } else
+                    ++numFPRs;
+                break;
+            default:
+                RELEASE_ASSERT_NOT_REACHED();
+            }
+
+            jsOffset += sizeof(EncodedJSValue);
+        }
+    }
+
+    // Now handle FPR arguments in registers.
+    {
+        numFPRs = 0;
+        ptrdiff_t jsOffset = CallFrameSlot::firstArgument * sizeof(EncodedJSValue);
+        for (unsigned i = 0; i < signature.argumentCount(); i++) {
+            switch (signature.argument(i)) {
+            case Wasm::F32:
+            case Wasm::F64:
+                if (numFPRs < Wasm::wasmCallingConvention().m_fprArgs.size()) {
+                    FPRReg argFPR = Wasm::wasmCallingConvention().m_fprArgs[numFPRs].fpr();
+                    jit.load64(CCallHelpers::Address(GPRInfo::callFrameRegister, jsOffset), scratchGPR);
+                    slowPath.append(jit.branchIfNotNumber(scratchGPR));
+                    auto isInt32 = jit.branchIfInt32(scratchGPR);
+                    jit.unboxDouble(scratchGPR, scratchGPR, argFPR);
+                    if (signature.argument(i) == Wasm::F32)
+                        jit.convertDoubleToFloat(argFPR, argFPR);
+                    auto done = jit.jump();
+
+                    isInt32.link(&jit);
+                    if (signature.argument(i) == Wasm::F32)
+                        jit.convertInt32ToFloat(scratchGPR, argFPR);
+                    else
+                        jit.convertInt32ToDouble(scratchGPR, argFPR);
+
+                    done.link(&jit);
+                    ++numFPRs;
+                }
+                break;
+            default:
+                break;
+            }
+
+            jsOffset += sizeof(EncodedJSValue);
+        }
+    }
+
+    // At this point, we're committed to doing a fast call.
+
+    if (Wasm::Context::useFastTLS()) 
+        jit.loadWasmContextInstance(scratchGPR);
+    else
+        jit.loadPtr(vm.wasmContext.pointerToInstance(), scratchGPR);
+    ptrdiff_t previousInstanceOffset = this->previousInstanceOffset();
+    jit.storePtr(scratchGPR, CCallHelpers::Address(GPRInfo::callFrameRegister, previousInstanceOffset));
+
+    jit.move(CCallHelpers::TrustedImmPtr(&instance()->instance()), scratchGPR);
+    if (Wasm::Context::useFastTLS()) 
+        jit.storeWasmContextInstance(scratchGPR);
+    else {
+        jit.move(scratchGPR, pinnedRegs.wasmContextInstancePointer);
+        jit.storePtr(scratchGPR, vm.wasmContext.pointerToInstance());
+    }
+    // This contains the cached stack limit still.
+    jit.storePtr(scratch2GPR, CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedStackLimit()));
+
+    if (!!moduleInformation.memory) {
+        GPRReg baseMemory = pinnedRegs.baseMemoryPointer;
+
+        if (instance()->memoryMode() != Wasm::MemoryMode::Signaling) {
+            ASSERT(pinnedRegs.sizeRegister != scratchGPR);
+            jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemorySize()), pinnedRegs.sizeRegister);
+        }
+
+        jit.loadPtr(CCallHelpers::Address(scratchGPR, Wasm::Instance::offsetOfCachedMemory()), baseMemory);
+    }
+
+    // We use this callee to indicate how to unwind past these types of frames:
+    // 1. We need to know where to get callee saves.
+    // 2. We need to know to restore the previous wasm context.
+    if (!m_jsToWasmICCallee)
+        m_jsToWasmICCallee.set(vm, this, JSToWasmICCallee::create(vm, globalObject(), this));
+    jit.storePtr(CCallHelpers::TrustedImmPtr(m_jsToWasmICCallee.get()), CCallHelpers::addressFor(CallFrameSlot::callee));
+
+    {
+        // FIXME: Currently we just do an indirect jump. But we should teach the Module
+        // how to repatch us:
+        // https://bugs.webkit.org/show_bug.cgi?id=196570
+        jit.loadPtr(entrypointLoadLocation(), scratchGPR);
+        jit.call(scratchGPR, WasmEntryPtrTag);
+    }
+
+    ASSERT(!RegisterSet::runtimeTagRegisters().contains(GPRInfo::nonPreservedNonReturnGPR));
+    jit.loadPtr(CCallHelpers::Address(GPRInfo::callFrameRegister, previousInstanceOffset), GPRInfo::nonPreservedNonReturnGPR);
+    if (Wasm::Context::useFastTLS())
+        jit.storeWasmContextInstance(GPRInfo::nonPreservedNonReturnGPR);
+    else
+        jit.storePtr(GPRInfo::nonPreservedNonReturnGPR, vm.wasmContext.pointerToInstance());
+
+    switch (signature.returnType()) {
+    case Wasm::Void:
+        jit.moveTrustedValue(jsUndefined(), JSValueRegs { GPRInfo::returnValueGPR });
+        break;
+    case Wasm::I32:
+        jit.zeroExtend32ToPtr(GPRInfo::returnValueGPR, GPRInfo::returnValueGPR);
+        jit.boxInt32(GPRInfo::returnValueGPR, JSValueRegs { GPRInfo::returnValueGPR });
+        break;
+    case Wasm::F32:
+        jit.convertFloatToDouble(FPRInfo::returnValueFPR, FPRInfo::returnValueFPR);
+        FALLTHROUGH;
+    case Wasm::F64: {
+        jit.moveTrustedValue(jsNumber(pureNaN()), JSValueRegs { GPRInfo::returnValueGPR });
+        auto isNaN = jit.branchIfNaN(FPRInfo::returnValueFPR);
+        jit.boxDouble(FPRInfo::returnValueFPR, JSValueRegs { GPRInfo::returnValueGPR });
+        isNaN.link(&jit);
+        break;
+    }
+    case Wasm::I64:
+    case Wasm::Func:
+    case Wasm::Anyfunc:
+        return nullptr;
+    default:
+        break;
+    }
+
+    auto emitRestoreCalleeSaves = [&] {
+        for (const RegisterAtOffset& regAtOffset : registersToSpill) {
+            GPRReg reg = regAtOffset.reg().gpr();
+            ASSERT(reg != GPRInfo::returnValueGPR);
+            ptrdiff_t offset = regAtOffset.offset();
+            jit.loadPtr(CCallHelpers::Address(GPRInfo::callFrameRegister, offset), reg);
+        }
+    };
+
+    emitRestoreCalleeSaves();
+
+    jit.emitFunctionEpilogue();
+    jit.ret();
+
+    slowPath.link(&jit);
+    emitRestoreCalleeSaves();
+    jit.move(CCallHelpers::TrustedImmPtr(this), GPRInfo::regT0);
+    jit.emitFunctionEpilogue();
+    auto jumpToHostCallThunk = jit.jump();
+
+    LinkBuffer linkBuffer(jit, nullptr, JITCompilationCanFail);
+    if (UNLIKELY(linkBuffer.didFailToAllocate()))
+        return nullptr;
+
+    linkBuffer.link(jumpToHostCallThunk, CodeLocationLabel<JSEntryPtrTag>(executable()->entrypointFor(CodeForCall, MustCheckArity).executableAddress()));
+    m_jsCallEntrypoint = FINALIZE_CODE(linkBuffer, WasmEntryPtrTag, "JS->Wasm IC");
+    return m_jsCallEntrypoint.code();
+}
+
 WebAssemblyFunction* WebAssemblyFunction::create(VM& vm, JSGlobalObject* globalObject, Structure* structure, unsigned length, const String& name, JSWebAssemblyInstance* instance, Wasm::Callee& jsEntrypoint, Wasm::WasmToWasmImportableFunction::LoadLocation wasmToWasmEntrypointLoadLocation, Wasm::SignatureIndex signatureIndex)
 {
@@ -170 +503 @@
 { }
 
+void WebAssemblyFunction::visitChildren(JSCell* cell, SlotVisitor& visitor)
+{
+    WebAssemblyFunction* thisObject = jsCast<WebAssemblyFunction*>(cell);
+    ASSERT_GC_OBJECT_INHERITS(thisObject, info());
+
+    Base::visitChildren(thisObject, visitor);
+    visitor.append(thisObject->m_jsToWasmICCallee);
+}
+
+void WebAssemblyFunction::destroy(JSCell* cell)
+{
+    static_cast<WebAssemblyFunction*>(cell)->WebAssemblyFunction::~WebAssemblyFunction();
+}
+
 } // namespace JSC
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.h

@@ -28 +28 @@
 #if ENABLE(WEBASSEMBLY)
 
+#include "JSToWasmICCallee.h"
 #include "MacroAssemblerCodeRef.h"
 #include "WasmCallee.h"
@@ -49 +50 @@
 
     const static unsigned StructureFlags = Base::StructureFlags;
+
+    static const bool needsDestruction = true;
+    static void destroy(JSCell*);
 
     template<typename CellType, SubspaceAccess mode>
@@ -75 +79 @@
     static ptrdiff_t offsetOfEntrypointLoadLocation() { return OBJECT_OFFSETOF(WebAssemblyFunction, m_importableFunction) + WasmToWasmImportableFunction::offsetOfEntrypointLoadLocation(); }
 
+    MacroAssemblerCodePtr<JSEntryPtrTag> jsCallEntrypoint()
+    {
+        if (m_jsCallEntrypoint)
+            return m_jsCallEntrypoint.code();
+        return jsCallEntrypointSlow();
+    }
+
+    RegisterAtOffsetList usedCalleeSaveRegisters() const;
+    Wasm::Instance* previousInstance(CallFrame*);
+
 private:
+    static void visitChildren(JSCell*, SlotVisitor&);
     WebAssemblyFunction(VM&, JSGlobalObject*, Structure*, Wasm::Callee& jsEntrypoint, WasmToWasmImportableFunction::LoadLocation entrypointLoadLocation, Wasm::SignatureIndex);
+
+    MacroAssemblerCodePtr<JSEntryPtrTag> jsCallEntrypointSlow();
+    ptrdiff_t previousInstanceOffset() const;
+    bool useTagRegisters() const;
+
+    RegisterSet calleeSaves() const;
 
     // It's safe to just hold the raw WasmToWasmImportableFunction/jsEntrypoint because we have a reference
@@ -83 +104 @@
     MacroAssemblerCodePtr<WasmEntryPtrTag> m_jsEntrypoint;
     WasmToWasmImportableFunction m_importableFunction;
+    WriteBarrier<JSToWasmICCallee> m_jsToWasmICCallee;
+    // Used for JS calling into Wasm.
+    MacroAssemblerCodeRef<JSEntryPtrTag> m_jsCallEntrypoint;
 };
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyFunctionHeapCellType.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -21 +21 @@
  * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
  */
 
-#pragma once
+#include "config.h"
+#include "WebAssemblyFunctionHeapCellType.h"
 
 #if ENABLE(WEBASSEMBLY)
 
-#include "Options.h"
-#include "WasmContext.h"
-#include "WasmInstance.h"
-#include <mutex>
-#include <wtf/FastTLS.h>
+#include "JSCInlines.h"
+#include "MarkedBlockInlines.h"
+#include "WebAssemblyFunction.h"
 
-namespace JSC { namespace Wasm {
+namespace JSC {
 
-inline bool Context::useFastTLS()
+struct WebAssemblyFunctionDestroyFunc {
+    ALWAYS_INLINE void operator()(VM&, JSCell* cell) const
+    {
+        static_assert(std::is_final<WebAssemblyFunction>::value, "Otherwise, this code would not be correct.");
+        WebAssemblyFunction::info()->methodTable.destroy(cell);
+    }
+};
+
+WebAssemblyFunctionHeapCellType::WebAssemblyFunctionHeapCellType()
+    : HeapCellType(CellAttributes(NeedsDestruction, HeapCell::JSCell))
 {
-#if ENABLE(FAST_TLS_JIT)
-    return Options::useFastTLSForWasmContext();
-#else
-    return false;
-#endif
 }
 
-inline Instance* Context::load() const
+WebAssemblyFunctionHeapCellType::~WebAssemblyFunctionHeapCellType()
 {
-#if ENABLE(FAST_TLS_JIT)
-    if (useFastTLS())
-        return bitwise_cast<Instance*>(_pthread_getspecific_direct(WTF_WASM_CONTEXT_KEY));
-#endif
-    return instance;
 }
 
-inline void Context::store(Instance* inst, void* softStackLimit)
+void WebAssemblyFunctionHeapCellType::finishSweep(MarkedBlock::Handle& handle, FreeList* freeList)
 {
-#if ENABLE(FAST_TLS_JIT)
-    if (useFastTLS())
-        _pthread_setspecific_direct(WTF_WASM_CONTEXT_KEY, bitwise_cast<void*>(inst));
-#endif
-    instance = inst;
-    if (instance)
-        instance->setCachedStackLimit(softStackLimit);
+    handle.finishSweepKnowingHeapCellType(freeList, WebAssemblyFunctionDestroyFunc());
 }
 
-} } // namespace JSC::Wasm
+void WebAssemblyFunctionHeapCellType::destroy(VM& vm, JSCell* cell)
+{
+    WebAssemblyFunctionDestroyFunc()(vm, cell);
+}
+
+} // namespace JSC
 
 #endif // ENABLE(WEBASSEMBLY)

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyFunctionHeapCellType.h

@@ -1 +1 @@
 /*
- * Copyright (C) 2017 Apple Inc. All rights reserved.
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -21 +21 @@
  * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
  */
 
@@ -28 +28 @@
 #if ENABLE(WEBASSEMBLY)
 
-namespace JSC { namespace Wasm {
+#include "HeapCellType.h"
 
-class Instance;
+namespace JSC {
 
-struct Context {
-    Instance* instance { nullptr };
-
-    Instance* load() const;
-    void store(Instance*, void* softStackLimit);
-
-    static bool useFastTLS();
+class WebAssemblyFunctionHeapCellType : public HeapCellType {
+public:
+    WebAssemblyFunctionHeapCellType();
+    virtual ~WebAssemblyFunctionHeapCellType();
+    
+    void finishSweep(MarkedBlock::Handle&, FreeList*) override;
+    void destroy(VM&, JSCell*) override;
 };
 
-} } // namespace JSC::Wasm
+} // namespace JSC
 
 #endif // ENABLE(WEBASSEMBLY)

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyPrototype.h

@@ -34 +34 @@
 namespace JSC {
 
+class JSPromiseDeferred;
+
 class WebAssemblyPrototype final : public JSNonFinalObject {
 public: