Changeset 213386 in webkit

Timestamp:

Mar 3, 2017 2:24:21 PM (7 years ago)

Author:

keith_miller@apple.com

Message:

WASM should support faster loads.
​https://bugs.webkit.org/show_bug.cgi?id=162693

Reviewed by Saam Barati.

Source/JavaScriptCore:

This patch adds support for WebAssembly using a 32-bit address
space for memory (along with some extra space for offset
overflow). With a 32-bit address space (we call them
Signaling/fast memories), we reserve the virtual address space for
2^{32 + offset bytes of memory and only mark the usable section as
read/write. If wasm code would read/write out of bounds we use a
custom signal handler to catch the SIGBUS. The signal handler then
checks if the faulting instruction is wasm code and tells the
thread to resume executing from the wasm exception
handler. Otherwise, the signal handler crashes the process, as
usual.}

All of the allocations of these memories are managed by the
Wasm::Memory class. In order to avoid TLB churn in the OS we cache
old Signaling memories that are no longer in use. Since getting
the wrong memory can cause recompiles, we try to reserve a memory
for modules that do not import a memory. If a module does import a
memory, we try to guess the type of memory we are going to get
based on the last one allocated.

This patch also changes how the wasm JS-api manages objects. Since
we can compile different versions of code, this patch adds a new
JSWebAssemblyCodeBlock class that holds all the information
specific to running a module in a particular bounds checking
mode. Additionally, the Wasm::Memory object is now a reference
counted class that is shared between the JSWebAssemblyMemory
object and the ArrayBuffer that also views it.

• JavaScriptCore.xcodeproj/project.pbxproj:
• jit/JITThunks.cpp:

(JSC::JITThunks::existingCTIStub):

• jit/JITThunks.h:
• jsc.cpp:

(jscmain):

• runtime/Options.h:
• runtime/VM.cpp:

(JSC::VM::VM):

• runtime/VM.h:
• wasm/JSWebAssemblyCodeBlock.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyModule.h.

(JSC::JSWebAssemblyCodeBlock::create):
(JSC::JSWebAssemblyCodeBlock::createStructure):
(JSC::JSWebAssemblyCodeBlock::functionImportCount):
(JSC::JSWebAssemblyCodeBlock::mode):
(JSC::JSWebAssemblyCodeBlock::module):
(JSC::JSWebAssemblyCodeBlock::jsEntrypointCalleeFromFunctionIndexSpace):
(JSC::JSWebAssemblyCodeBlock::wasmEntrypointCalleeFromFunctionIndexSpace):
(JSC::JSWebAssemblyCodeBlock::setJSEntrypointCallee):
(JSC::JSWebAssemblyCodeBlock::setWasmEntrypointCallee):
(JSC::JSWebAssemblyCodeBlock::callees):
(JSC::JSWebAssemblyCodeBlock::offsetOfCallees):
(JSC::JSWebAssemblyCodeBlock::allocationSize):

• wasm/WasmB3IRGenerator.cpp:

(JSC::Wasm::B3IRGenerator::B3IRGenerator):
(JSC::Wasm::getMemoryBaseAndSize):
(JSC::Wasm::B3IRGenerator::emitCheckAndPreparePointer):
(JSC::Wasm::B3IRGenerator::emitLoadOp):
(JSC::Wasm::B3IRGenerator::emitStoreOp):

• wasm/WasmCallingConvention.h:
• wasm/WasmFaultSignalHandler.cpp: Added.

(JSC::Wasm::trapHandler):
(JSC::Wasm::registerCode):
(JSC::Wasm::unregisterCode):
(JSC::Wasm::fastMemoryEnabled):
(JSC::Wasm::enableFastMemory):

• wasm/WasmFaultSignalHandler.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCallee.cpp.
• wasm/WasmFormat.h:

(JSC::Wasm::ModuleInformation::importFunctionCount):
(JSC::Wasm::ModuleInformation::hasMemory): Deleted.

• wasm/WasmMemory.cpp:

(JSC::Wasm::mmapBytes):
(JSC::Wasm::Memory::lastAllocatedMode):
(JSC::Wasm::availableFastMemories):
(JSC::Wasm::tryGetFastMemory):
(JSC::Wasm::releaseFastMemory):
(JSC::Wasm::Memory::Memory):
(JSC::Wasm::Memory::createImpl):
(JSC::Wasm::Memory::create):
(JSC::Wasm::Memory::~Memory):
(JSC::Wasm::Memory::grow):
(JSC::Wasm::Memory::dump):
(JSC::Wasm::Memory::makeString):

• wasm/WasmMemory.h:

(JSC::Wasm::Memory::operator bool):
(JSC::Wasm::Memory::size):
(JSC::Wasm::Memory::check):
(JSC::Wasm::Memory::Memory): Deleted.
(JSC::Wasm::Memory::offsetOfMemory): Deleted.
(JSC::Wasm::Memory::offsetOfSize): Deleted.

• wasm/WasmMemoryInformation.cpp:

(JSC::Wasm::MemoryInformation::MemoryInformation):

• wasm/WasmMemoryInformation.h:

(JSC::Wasm::MemoryInformation::hasReservedMemory):
(JSC::Wasm::MemoryInformation::takeReservedMemory):
(JSC::Wasm::MemoryInformation::mode):

• wasm/WasmModuleParser.cpp:
• wasm/WasmModuleParser.h:

(JSC::Wasm::ModuleParser::ModuleParser):

• wasm/WasmPlan.cpp:

(JSC::Wasm::Plan::parseAndValidateModule):
(JSC::Wasm::Plan::run):

• wasm/WasmPlan.h:

(JSC::Wasm::Plan::mode):

• wasm/js/JSWebAssemblyCallee.cpp:

(JSC::JSWebAssemblyCallee::finishCreation):
(JSC::JSWebAssemblyCallee::destroy):

• wasm/js/JSWebAssemblyCodeBlock.cpp: Added.

(JSC::JSWebAssemblyCodeBlock::JSWebAssemblyCodeBlock):
(JSC::JSWebAssemblyCodeBlock::destroy):
(JSC::JSWebAssemblyCodeBlock::isSafeToRun):
(JSC::JSWebAssemblyCodeBlock::visitChildren):
(JSC::JSWebAssemblyCodeBlock::UnconditionalFinalizer::finalizeUnconditionally):

• wasm/js/JSWebAssemblyInstance.cpp:

(JSC::JSWebAssemblyInstance::setMemory):
(JSC::JSWebAssemblyInstance::finishCreation):
(JSC::JSWebAssemblyInstance::visitChildren):

• wasm/js/JSWebAssemblyInstance.h:

(JSC::JSWebAssemblyInstance::module):
(JSC::JSWebAssemblyInstance::codeBlock):
(JSC::JSWebAssemblyInstance::memoryMode):
(JSC::JSWebAssemblyInstance::setMemory): Deleted.

• wasm/js/JSWebAssemblyMemory.cpp:

(JSC::JSWebAssemblyMemory::create):
(JSC::JSWebAssemblyMemory::JSWebAssemblyMemory):
(JSC::JSWebAssemblyMemory::buffer):
(JSC::JSWebAssemblyMemory::grow):
(JSC::JSWebAssemblyMemory::destroy):

• wasm/js/JSWebAssemblyMemory.h:

(JSC::JSWebAssemblyMemory::memory):
(JSC::JSWebAssemblyMemory::offsetOfMemory):
(JSC::JSWebAssemblyMemory::offsetOfSize):

• wasm/js/JSWebAssemblyModule.cpp:

(JSC::JSWebAssemblyModule::buildCodeBlock):
(JSC::JSWebAssemblyModule::create):
(JSC::JSWebAssemblyModule::JSWebAssemblyModule):
(JSC::JSWebAssemblyModule::codeBlock):
(JSC::JSWebAssemblyModule::finishCreation):
(JSC::JSWebAssemblyModule::visitChildren):
(JSC::JSWebAssemblyModule::UnconditionalFinalizer::finalizeUnconditionally): Deleted.

• wasm/js/JSWebAssemblyModule.h:

(JSC::JSWebAssemblyModule::takeReservedMemory):
(JSC::JSWebAssemblyModule::signatureIndexFromFunctionIndexSpace):
(JSC::JSWebAssemblyModule::codeBlock):
(JSC::JSWebAssemblyModule::functionImportCount): Deleted.
(JSC::JSWebAssemblyModule::jsEntrypointCalleeFromFunctionIndexSpace): Deleted.
(JSC::JSWebAssemblyModule::wasmEntrypointCalleeFromFunctionIndexSpace): Deleted.
(JSC::JSWebAssemblyModule::setJSEntrypointCallee): Deleted.
(JSC::JSWebAssemblyModule::setWasmEntrypointCallee): Deleted.
(JSC::JSWebAssemblyModule::callees): Deleted.
(JSC::JSWebAssemblyModule::offsetOfCallees): Deleted.
(JSC::JSWebAssemblyModule::allocationSize): Deleted.

• wasm/js/WebAssemblyFunction.cpp:

(JSC::callWebAssemblyFunction):

• wasm/js/WebAssemblyInstanceConstructor.cpp:

(JSC::constructJSWebAssemblyInstance):

• wasm/js/WebAssemblyMemoryConstructor.cpp:

(JSC::constructJSWebAssemblyMemory):

• wasm/js/WebAssemblyModuleConstructor.cpp:

(JSC::WebAssemblyModuleConstructor::createModule):

• wasm/js/WebAssemblyModuleRecord.cpp:

(JSC::WebAssemblyModuleRecord::link):
(JSC::WebAssemblyModuleRecord::evaluate):

Source/WTF:

Add new forms of dataLog that take a boolean which describes if the log should happen. This makes cases where we have a static const bool for printing nicer since you can do:

dataLogIf(verbose, things, to, print);

instead of:

if (verbose)

dataLog(things, to, print);

Also, add a operator! to Ref that has the same semantics as C++ refs.

• wtf/DataLog.h:

(WTF::dataLogLn):
(WTF::dataLogIf):
(WTF::dataLogLnIf):

• wtf/Ref.h:

(WTF::Ref::operator!):

Location:

trunk/Source

Files:

• JavaScriptCore/ChangeLog (modified) (1 diff)
• JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj (modified) (8 diffs)
• JavaScriptCore/jit/JITThunks.cpp (modified) (1 diff)
• JavaScriptCore/jit/JITThunks.h (modified) (1 diff)
• JavaScriptCore/jsc.cpp (modified) (2 diffs)
• JavaScriptCore/runtime/Options.h (modified) (1 diff)
• JavaScriptCore/runtime/VM.cpp (modified) (1 diff)
• JavaScriptCore/runtime/VM.h (modified) (1 diff)
• JavaScriptCore/wasm/JSWebAssemblyCodeBlock.h (copied) (copied from trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyModule.h) (5 diffs)
• JavaScriptCore/wasm/WasmB3IRGenerator.cpp (modified) (9 diffs)
• JavaScriptCore/wasm/WasmCallingConvention.h (modified) (1 diff)
• JavaScriptCore/wasm/WasmFaultSignalHandler.cpp (added)
• JavaScriptCore/wasm/WasmFaultSignalHandler.h (copied) (copied from trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyCallee.cpp) (2 diffs)
• JavaScriptCore/wasm/WasmFormat.h (modified) (1 diff)
• JavaScriptCore/wasm/WasmMemory.cpp (modified) (4 diffs)
• JavaScriptCore/wasm/WasmMemory.h (modified) (4 diffs)
• JavaScriptCore/wasm/WasmMemoryInformation.cpp (modified) (4 diffs)
• JavaScriptCore/wasm/WasmMemoryInformation.h (modified) (3 diffs)
• JavaScriptCore/wasm/WasmModuleParser.cpp (modified) (1 diff)
• JavaScriptCore/wasm/WasmModuleParser.h (modified) (3 diffs)
• JavaScriptCore/wasm/WasmPlan.cpp (modified) (4 diffs)
• JavaScriptCore/wasm/WasmPlan.h (modified) (2 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyCallee.cpp (modified) (3 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyCodeBlock.cpp (added)
• JavaScriptCore/wasm/js/JSWebAssemblyInstance.cpp (modified) (4 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyInstance.h (modified) (4 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyMemory.cpp (modified) (7 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyMemory.h (modified) (1 diff)
• JavaScriptCore/wasm/js/JSWebAssemblyModule.cpp (modified) (4 diffs)
• JavaScriptCore/wasm/js/JSWebAssemblyModule.h (modified) (6 diffs)
• JavaScriptCore/wasm/js/WebAssemblyFunction.cpp (modified) (1 diff)
• JavaScriptCore/wasm/js/WebAssemblyInstanceConstructor.cpp (modified) (5 diffs)
• JavaScriptCore/wasm/js/WebAssemblyMemoryConstructor.cpp (modified) (1 diff)
• JavaScriptCore/wasm/js/WebAssemblyModuleConstructor.cpp (modified) (1 diff)
• JavaScriptCore/wasm/js/WebAssemblyModuleRecord.cpp (modified) (6 diffs)
• WTF/ChangeLog (modified) (1 diff)
• WTF/wtf/DataLog.h (modified) (2 diffs)
• WTF/wtf/Ref.h (modified) (1 diff)

trunk/Source/JavaScriptCore/ChangeLog

@@ +1 @@
+2017-03-03  Keith Miller  <keith_miller@apple.com>
+
+        WASM should support faster loads.
+        https://bugs.webkit.org/show_bug.cgi?id=162693
+
+        Reviewed by Saam Barati.
+
+        This patch adds support for WebAssembly using a 32-bit address
+        space for memory (along with some extra space for offset
+        overflow). With a 32-bit address space (we call them
+        Signaling/fast memories), we reserve the virtual address space for
+        2^32 + offset bytes of memory and only mark the usable section as
+        read/write. If wasm code would read/write out of bounds we use a
+        custom signal handler to catch the SIGBUS. The signal handler then
+        checks if the faulting instruction is wasm code and tells the
+        thread to resume executing from the wasm exception
+        handler. Otherwise, the signal handler crashes the process, as
+        usual.
+
+        All of the allocations of these memories are managed by the
+        Wasm::Memory class. In order to avoid TLB churn in the OS we cache
+        old Signaling memories that are no longer in use. Since getting
+        the wrong memory can cause recompiles, we try to reserve a memory
+        for modules that do not import a memory. If a module does import a
+        memory, we try to guess the type of memory we are going to get
+        based on the last one allocated.
+
+        This patch also changes how the wasm JS-api manages objects. Since
+        we can compile different versions of code, this patch adds a new
+        JSWebAssemblyCodeBlock class that holds all the information
+        specific to running a module in a particular bounds checking
+        mode. Additionally, the Wasm::Memory object is now a reference
+        counted class that is shared between the JSWebAssemblyMemory
+        object and the ArrayBuffer that also views it.
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * jit/JITThunks.cpp:
+        (JSC::JITThunks::existingCTIStub):
+        * jit/JITThunks.h:
+        * jsc.cpp:
+        (jscmain):
+        * runtime/Options.h:
+        * runtime/VM.cpp:
+        (JSC::VM::VM):
+        * runtime/VM.h:
+        * wasm/JSWebAssemblyCodeBlock.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyModule.h.
+        (JSC::JSWebAssemblyCodeBlock::create):
+        (JSC::JSWebAssemblyCodeBlock::createStructure):
+        (JSC::JSWebAssemblyCodeBlock::functionImportCount):
+        (JSC::JSWebAssemblyCodeBlock::mode):
+        (JSC::JSWebAssemblyCodeBlock::module):
+        (JSC::JSWebAssemblyCodeBlock::jsEntrypointCalleeFromFunctionIndexSpace):
+        (JSC::JSWebAssemblyCodeBlock::wasmEntrypointCalleeFromFunctionIndexSpace):
+        (JSC::JSWebAssemblyCodeBlock::setJSEntrypointCallee):
+        (JSC::JSWebAssemblyCodeBlock::setWasmEntrypointCallee):
+        (JSC::JSWebAssemblyCodeBlock::callees):
+        (JSC::JSWebAssemblyCodeBlock::offsetOfCallees):
+        (JSC::JSWebAssemblyCodeBlock::allocationSize):
+        * wasm/WasmB3IRGenerator.cpp:
+        (JSC::Wasm::B3IRGenerator::B3IRGenerator):
+        (JSC::Wasm::getMemoryBaseAndSize):
+        (JSC::Wasm::B3IRGenerator::emitCheckAndPreparePointer):
+        (JSC::Wasm::B3IRGenerator::emitLoadOp):
+        (JSC::Wasm::B3IRGenerator::emitStoreOp):
+        * wasm/WasmCallingConvention.h:
+        * wasm/WasmFaultSignalHandler.cpp: Added.
+        (JSC::Wasm::trapHandler):
+        (JSC::Wasm::registerCode):
+        (JSC::Wasm::unregisterCode):
+        (JSC::Wasm::fastMemoryEnabled):
+        (JSC::Wasm::enableFastMemory):
+        * wasm/WasmFaultSignalHandler.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCallee.cpp.
+        * wasm/WasmFormat.h:
+        (JSC::Wasm::ModuleInformation::importFunctionCount):
+        (JSC::Wasm::ModuleInformation::hasMemory): Deleted.
+        * wasm/WasmMemory.cpp:
+        (JSC::Wasm::mmapBytes):
+        (JSC::Wasm::Memory::lastAllocatedMode):
+        (JSC::Wasm::availableFastMemories):
+        (JSC::Wasm::tryGetFastMemory):
+        (JSC::Wasm::releaseFastMemory):
+        (JSC::Wasm::Memory::Memory):
+        (JSC::Wasm::Memory::createImpl):
+        (JSC::Wasm::Memory::create):
+        (JSC::Wasm::Memory::~Memory):
+        (JSC::Wasm::Memory::grow):
+        (JSC::Wasm::Memory::dump):
+        (JSC::Wasm::Memory::makeString):
+        * wasm/WasmMemory.h:
+        (JSC::Wasm::Memory::operator bool):
+        (JSC::Wasm::Memory::size):
+        (JSC::Wasm::Memory::check):
+        (JSC::Wasm::Memory::Memory): Deleted.
+        (JSC::Wasm::Memory::offsetOfMemory): Deleted.
+        (JSC::Wasm::Memory::offsetOfSize): Deleted.
+        * wasm/WasmMemoryInformation.cpp:
+        (JSC::Wasm::MemoryInformation::MemoryInformation):
+        * wasm/WasmMemoryInformation.h:
+        (JSC::Wasm::MemoryInformation::hasReservedMemory):
+        (JSC::Wasm::MemoryInformation::takeReservedMemory):
+        (JSC::Wasm::MemoryInformation::mode):
+        * wasm/WasmModuleParser.cpp:
+        * wasm/WasmModuleParser.h:
+        (JSC::Wasm::ModuleParser::ModuleParser):
+        * wasm/WasmPlan.cpp:
+        (JSC::Wasm::Plan::parseAndValidateModule):
+        (JSC::Wasm::Plan::run):
+        * wasm/WasmPlan.h:
+        (JSC::Wasm::Plan::mode):
+        * wasm/js/JSWebAssemblyCallee.cpp:
+        (JSC::JSWebAssemblyCallee::finishCreation):
+        (JSC::JSWebAssemblyCallee::destroy):
+        * wasm/js/JSWebAssemblyCodeBlock.cpp: Added.
+        (JSC::JSWebAssemblyCodeBlock::JSWebAssemblyCodeBlock):
+        (JSC::JSWebAssemblyCodeBlock::destroy):
+        (JSC::JSWebAssemblyCodeBlock::isSafeToRun):
+        (JSC::JSWebAssemblyCodeBlock::visitChildren):
+        (JSC::JSWebAssemblyCodeBlock::UnconditionalFinalizer::finalizeUnconditionally):
+        * wasm/js/JSWebAssemblyInstance.cpp:
+        (JSC::JSWebAssemblyInstance::setMemory):
+        (JSC::JSWebAssemblyInstance::finishCreation):
+        (JSC::JSWebAssemblyInstance::visitChildren):
+        * wasm/js/JSWebAssemblyInstance.h:
+        (JSC::JSWebAssemblyInstance::module):
+        (JSC::JSWebAssemblyInstance::codeBlock):
+        (JSC::JSWebAssemblyInstance::memoryMode):
+        (JSC::JSWebAssemblyInstance::setMemory): Deleted.
+        * wasm/js/JSWebAssemblyMemory.cpp:
+        (JSC::JSWebAssemblyMemory::create):
+        (JSC::JSWebAssemblyMemory::JSWebAssemblyMemory):
+        (JSC::JSWebAssemblyMemory::buffer):
+        (JSC::JSWebAssemblyMemory::grow):
+        (JSC::JSWebAssemblyMemory::destroy):
+        * wasm/js/JSWebAssemblyMemory.h:
+        (JSC::JSWebAssemblyMemory::memory):
+        (JSC::JSWebAssemblyMemory::offsetOfMemory):
+        (JSC::JSWebAssemblyMemory::offsetOfSize):
+        * wasm/js/JSWebAssemblyModule.cpp:
+        (JSC::JSWebAssemblyModule::buildCodeBlock):
+        (JSC::JSWebAssemblyModule::create):
+        (JSC::JSWebAssemblyModule::JSWebAssemblyModule):
+        (JSC::JSWebAssemblyModule::codeBlock):
+        (JSC::JSWebAssemblyModule::finishCreation):
+        (JSC::JSWebAssemblyModule::visitChildren):
+        (JSC::JSWebAssemblyModule::UnconditionalFinalizer::finalizeUnconditionally): Deleted.
+        * wasm/js/JSWebAssemblyModule.h:
+        (JSC::JSWebAssemblyModule::takeReservedMemory):
+        (JSC::JSWebAssemblyModule::signatureIndexFromFunctionIndexSpace):
+        (JSC::JSWebAssemblyModule::codeBlock):
+        (JSC::JSWebAssemblyModule::functionImportCount): Deleted.
+        (JSC::JSWebAssemblyModule::jsEntrypointCalleeFromFunctionIndexSpace): Deleted.
+        (JSC::JSWebAssemblyModule::wasmEntrypointCalleeFromFunctionIndexSpace): Deleted.
+        (JSC::JSWebAssemblyModule::setJSEntrypointCallee): Deleted.
+        (JSC::JSWebAssemblyModule::setWasmEntrypointCallee): Deleted.
+        (JSC::JSWebAssemblyModule::callees): Deleted.
+        (JSC::JSWebAssemblyModule::offsetOfCallees): Deleted.
+        (JSC::JSWebAssemblyModule::allocationSize): Deleted.
+        * wasm/js/WebAssemblyFunction.cpp:
+        (JSC::callWebAssemblyFunction):
+        * wasm/js/WebAssemblyInstanceConstructor.cpp:
+        (JSC::constructJSWebAssemblyInstance):
+        * wasm/js/WebAssemblyMemoryConstructor.cpp:
+        (JSC::constructJSWebAssemblyMemory):
+        * wasm/js/WebAssemblyModuleConstructor.cpp:
+        (JSC::WebAssemblyModuleConstructor::createModule):
+        * wasm/js/WebAssemblyModuleRecord.cpp:
+        (JSC::WebAssemblyModuleRecord::link):
+        (JSC::WebAssemblyModuleRecord::evaluate):
+
 2017-03-03  Mark Lam  <mark.lam@apple.com>
 

trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -1325 +1325 @@
                 5370B4F51BF26202005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 5370B4F31BF25EA2005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.cpp */; };
                 5370B4F61BF26205005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.h in Headers */ = {isa = PBXBuildFile; fileRef = 5370B4F41BF25EA2005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.h */; };
+                5381B9371E60E9660090F794 /* WasmFaultSignalHandler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 5381B9361E60E9660090F794 /* WasmFaultSignalHandler.cpp */; };
+                5381B9391E60E97D0090F794 /* WasmFaultSignalHandler.h in Headers */ = {isa = PBXBuildFile; fileRef = 5381B9381E60E97D0090F794 /* WasmFaultSignalHandler.h */; };
+                5381B9401E65DFEB0090F794 /* JSWebAssemblyCodeBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 5381B93F1E65DFEB0090F794 /* JSWebAssemblyCodeBlock.h */; };
+                5383AA301E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 5383AA2F1E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp */; };
                 53917E7B1B7906FA000EBD33 /* JSGenericTypedArrayViewPrototypeFunctions.h in Headers */ = {isa = PBXBuildFile; fileRef = 53917E7A1B7906E4000EBD33 /* JSGenericTypedArrayViewPrototypeFunctions.h */; };
                 539FB8BA1C99DA7C00940FA1 /* JSArrayInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 539FB8B91C99DA7C00940FA1 /* JSArrayInlines.h */; };
@@ -3792 +3796 @@
                 5370B4F31BF25EA2005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AdaptiveInferredPropertyValueWatchpointBase.cpp; sourceTree = "<group>"; };
                 5370B4F41BF25EA2005C40FC /* AdaptiveInferredPropertyValueWatchpointBase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AdaptiveInferredPropertyValueWatchpointBase.h; sourceTree = "<group>"; };
+                5381B9361E60E9660090F794 /* WasmFaultSignalHandler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WasmFaultSignalHandler.cpp; sourceTree = "<group>"; };
+                5381B9381E60E97D0090F794 /* WasmFaultSignalHandler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WasmFaultSignalHandler.h; sourceTree = "<group>"; };
+                5381B93F1E65DFEB0090F794 /* JSWebAssemblyCodeBlock.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSWebAssemblyCodeBlock.h; sourceTree = "<group>"; };
+                5383AA2F1E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = JSWebAssemblyCodeBlock.cpp; path = js/JSWebAssemblyCodeBlock.cpp; sourceTree = "<group>"; };
                 53917E7A1B7906E4000EBD33 /* JSGenericTypedArrayViewPrototypeFunctions.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSGenericTypedArrayViewPrototypeFunctions.h; sourceTree = "<group>"; };
                 53917E7C1B791106000EBD33 /* JSTypedArrayViewPrototype.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSTypedArrayViewPrototype.h; sourceTree = "<group>"; };
@@ -6228 +6236 @@
                                 53F40E941D5A7AEF0099A1B6 /* WasmModuleParser.h */,
                                 ADB6F67C1E15D7500082F384 /* WasmPageCount.cpp */,
+                                5381B9361E60E9660090F794 /* WasmFaultSignalHandler.cpp */,
+                                5381B9381E60E97D0090F794 /* WasmFaultSignalHandler.h */,
                                 79B759731DFA4C600052174C /* WasmPageCount.h */,
                                 53F40E8C1D5901F20099A1B6 /* WasmParser.h */,
@@ -7822 +7832 @@
                                 79E423E01DEE65320078D355 /* JSWebAssemblyCallee.cpp */,
                                 79E423E11DEE65320078D355 /* JSWebAssemblyCallee.h */,
+                                5383AA2F1E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp */,
+                                5381B93F1E65DFEB0090F794 /* JSWebAssemblyCodeBlock.h */,
                                 AD2FCBA61DB58DA400B3E736 /* JSWebAssemblyCompileError.cpp */,
                                 AD2FCBA71DB58DA400B3E736 /* JSWebAssemblyCompileError.h */,
@@ -7962 +7974 @@
                                 0FB3878E1BFBC44D00E3AB1E /* AirBlockWorklist.h in Headers */,
                                 0F61832A1C45BF070072450B /* AirCCallingConvention.h in Headers */,
+                                5381B9401E65DFEB0090F794 /* JSWebAssemblyCodeBlock.h in Headers */,
                                 0FEC85741BDACDC70080FF74 /* AirCCallSpecial.h in Headers */,
                                 0FEC85761BDACDC70080FF74 /* AirCode.h in Headers */,
@@ -8281 +8294 @@
                                 0FC3CCFC19ADA410006AC72A /* DFGBlockMap.h in Headers */,
                                 0FC3CCFD19ADA410006AC72A /* DFGBlockMapInlines.h in Headers */,
+                                5381B9391E60E97D0090F794 /* WasmFaultSignalHandler.h in Headers */,
                                 0FC3CCFE19ADA410006AC72A /* DFGBlockSet.h in Headers */,
                                 0FBF158D19B7A53100695DD0 /* DFGBlockSetInlines.h in Headers */,
@@ -10078 +10092 @@
                                 A704D90517A0BAA8006BA554 /* DFGInPlaceAbstractState.cpp in Sources */,
                                 0F3BD1B71B896A0700598AA6 /* DFGInsertionSet.cpp in Sources */,
+                                5381B9371E60E9660090F794 /* WasmFaultSignalHandler.cpp in Sources */,
                                 0F300B7B18AB1B1400A6D72E /* DFGIntegerCheckCombiningPhase.cpp in Sources */,
                                 0F898F311B27689F0083A33C /* DFGIntegerRangeOptimizationPhase.cpp in Sources */,
@@ -10481 +10496 @@
                                 FEB137571BB11EF900CD5100 /* MacroAssemblerARM64.cpp in Sources */,
                                 A729009C17976C6000317298 /* MacroAssemblerARMv7.cpp in Sources */,
+                                5383AA301E65E8A100A532FC /* JSWebAssemblyCodeBlock.cpp in Sources */,
                                 0F6DB7EC1D617D1100CDBF8E /* MacroAssemblerCodeRef.cpp in Sources */,
                                 FE68C6381B90DE0B0042BCB3 /* MacroAssemblerPrinter.cpp in Sources */,

trunk/Source/JavaScriptCore/jit/JITThunks.cpp

@@ -83 +83 @@
 }
 
+MacroAssemblerCodeRef JITThunks::existingCTIStub(ThunkGenerator generator)
+{
+    LockHolder locker(m_lock);
+    CTIStubMap::iterator entry = m_ctiStubMap.find(generator);
+    if (entry == m_ctiStubMap.end())
+        return MacroAssemblerCodeRef();
+    return entry->value;
+}
+
 void JITThunks::finalize(Handle<Unknown> handle, void*)
 {

trunk/Source/JavaScriptCore/jit/JITThunks.h

@@ -59 +59 @@
 
     MacroAssemblerCodeRef ctiStub(VM*, ThunkGenerator);
+    MacroAssemblerCodeRef existingCTIStub(ThunkGenerator);
 
     NativeExecutable* hostFunctionStub(VM*, NativeFunction, NativeFunction constructor, const String& name);

trunk/Source/JavaScriptCore/jsc.cpp

@@ -73 +73 @@
 #include "TestRunnerUtils.h"
 #include "TypeProfilerLog.h"
+#include "WasmFaultSignalHandler.h"
 #include "WasmPlan.h"
 #include "WasmMemory.h"
@@ -3774 +3775 @@
     JSC::initializeThreading();
     startTimeoutThreadIfNeeded();
+#if ENABLE(WEBASSEMBLY)
+    JSC::Wasm::enableFastMemory();
+#endif
 
     int result;

trunk/Source/JavaScriptCore/runtime/Options.h

@@ -431 +431 @@
     v(bool, useWebAssembly, true, Normal, "Expose the WebAssembly global object.") \
     v(bool, simulateWebAssemblyLowMemory, false, Normal, "If true, the Memory object won't mmap the full 'maximum' range and instead will allocate the minimum required amount.") \
+    v(bool, useWebAssemblyFastMemory, true, Normal, "If true, we will try to use a 32-bit address space with a signal handler to bounds check wasm memory.")
+
 
 enum OptionEquivalence {

trunk/Source/JavaScriptCore/runtime/VM.cpp

@@ -242 +242 @@
     webAssemblyCalleeStructure.set(*this, JSWebAssemblyCallee::createStructure(*this, 0, jsNull()));
     webAssemblyToJSCalleeStructure.set(*this, WebAssemblyToJSCallee::createStructure(*this, 0, jsNull()));
+    webAssemblyCodeBlockStructure.set(*this, JSWebAssemblyCodeBlock::createStructure(*this, 0, jsNull()));
     webAssemblyToJSCallee.set(*this, WebAssemblyToJSCallee::create(*this, webAssemblyToJSCalleeStructure.get()));
 #endif

trunk/Source/JavaScriptCore/runtime/VM.h

@@ -335 +335 @@
     Strong<Structure> webAssemblyCalleeStructure;
     Strong<Structure> webAssemblyToJSCalleeStructure;
+    Strong<Structure> webAssemblyCodeBlockStructure;
     Strong<JSCell> webAssemblyToJSCallee;
 #endif

trunk/Source/JavaScriptCore/wasm/JSWebAssemblyCodeBlock.h

@@ -1 +1 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -28 +28 @@
 #if ENABLE(WEBASSEMBLY)
 
-#include "JSDestructibleObject.h"
-#include "JSObject.h"
+#include "JSCell.h"
 #include "JSWebAssemblyCallee.h"
 #include "UnconditionalFinalizer.h"
@@ -38 +37 @@
 namespace JSC {
 
-class SymbolTable;
+class JSWebAssemblyModule;
+class JSWebAssemblyMemory;
 
-class JSWebAssemblyModule : public JSDestructibleObject {
+class JSWebAssemblyCodeBlock : public JSCell {
 public:
-    typedef JSDestructibleObject Base;
+    typedef JSCell Base;
+    static const unsigned StructureFlags = Base::StructureFlags | StructureIsImmortal;
 
-    static JSWebAssemblyModule* create(VM&, Structure*, std::unique_ptr<Wasm::ModuleInformation>&&, Bag<CallLinkInfo>&&, Vector<Wasm::WasmExitStubs>&&, SymbolTable*, unsigned);
-    static Structure* createStructure(VM&, JSGlobalObject*, JSValue);
+    static JSWebAssemblyCodeBlock* create(VM& vm, JSWebAssemblyModule* owner, Bag<CallLinkInfo>&& callLinkInfos, Vector<Wasm::WasmExitStubs>&& exitStubs, Wasm::Memory::Mode mode, unsigned calleeCount)
+    {
+        auto* result = new (NotNull, allocateCell<JSWebAssemblyCodeBlock>(vm.heap, allocationSize(calleeCount))) JSWebAssemblyCodeBlock(vm, owner, std::forward<Bag<CallLinkInfo>>(callLinkInfos), std::forward<Vector<Wasm::WasmExitStubs>>(exitStubs), mode, calleeCount);
+        result->finishCreation(vm);
+        return result;
+    }
 
-    DECLARE_INFO;
+    static Structure* createStructure(VM& vm, JSGlobalObject* globalObject, JSValue prototype)
+    {
+        return Structure::create(vm, globalObject, prototype, TypeInfo(CellType, StructureFlags), info());
+    }
 
-    const Wasm::ModuleInformation& moduleInformation() const { return *m_moduleInformation.get(); }
-    SymbolTable* exportSymbolTable() const { return m_exportSymbolTable.get(); }
-    Wasm::SignatureIndex signatureIndexFromFunctionIndexSpace(unsigned functionIndexSpace) const
-    {
-        return m_moduleInformation->signatureIndexFromFunctionIndexSpace(functionIndexSpace);
-    }
     unsigned functionImportCount() const { return m_wasmExitStubs.size(); }
+    Wasm::Memory::Mode mode() const { return m_mode; }
+    JSWebAssemblyModule* module() const { return m_module.get(); }
+    bool isSafeToRun(JSWebAssemblyMemory*);
 
     JSWebAssemblyCallee* jsEntrypointCalleeFromFunctionIndexSpace(unsigned functionIndexSpace)
@@ -90 +95 @@
     }
 
-protected:
-    JSWebAssemblyModule(VM&, Structure*, std::unique_ptr<Wasm::ModuleInformation>&&, Bag<CallLinkInfo>&&, Vector<Wasm::WasmExitStubs>&&, unsigned calleeCount);
-    void finishCreation(VM&, SymbolTable*);
+private:
+    JSWebAssemblyCodeBlock(VM&, JSWebAssemblyModule*, Bag<CallLinkInfo>&&, Vector<Wasm::WasmExitStubs>&&, Wasm::Memory::Mode, unsigned calleeCount);
+    DECLARE_EXPORT_INFO;
+    static const bool needsDestruction = true;
     static void destroy(JSCell*);
     static void visitChildren(JSCell*, SlotVisitor&);
 
-private:
     static size_t offsetOfCallees()
     {
-        return WTF::roundUpToMultipleOf<sizeof(WriteBarrier<JSWebAssemblyCallee>)>(sizeof(JSWebAssemblyModule));
+        return WTF::roundUpToMultipleOf<sizeof(WriteBarrier<JSWebAssemblyCallee>)>(sizeof(JSWebAssemblyCodeBlock));
     }
 
@@ -107 +112 @@
     }
 
-    class UnconditionalFinalizer : public JSC::UnconditionalFinalizer { 
+    class UnconditionalFinalizer : public JSC::UnconditionalFinalizer {
         void finalizeUnconditionally() override;
     };
 
+    WriteBarrier<JSWebAssemblyModule> m_module;
     UnconditionalFinalizer m_unconditionalFinalizer;
-    std::unique_ptr<Wasm::ModuleInformation> m_moduleInformation;
     Bag<CallLinkInfo> m_callLinkInfos;
-    WriteBarrier<SymbolTable> m_exportSymbolTable;
     Vector<Wasm::WasmExitStubs> m_wasmExitStubs;
+    Wasm::Memory::Mode m_mode;
     unsigned m_calleeCount;
 };

trunk/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp

@@ -208 +208 @@
 private:
     ExpressionType emitCheckAndPreparePointer(ExpressionType pointer, uint32_t offset, uint32_t sizeOfOp);
+    B3::Kind memoryKind(B3::Opcode memoryOp);
     ExpressionType emitLoadOp(LoadOpType, Origin, ExpressionType pointer, uint32_t offset);
     void emitStoreOp(StoreOpType, Origin, ExpressionType pointer, ExpressionType value, uint32_t offset);
@@ -260 +261 @@
         m_proc.pinRegister(regInfo.sizeRegister);
 
-    if (info.hasMemory()) {
+    if (info.memory) {
         m_proc.setWasmBoundsCheckGenerator([=] (CCallHelpers& jit, GPRReg pinnedGPR, unsigned) {
             AllowMacroScratchRegisterUsage allowScratch(jit);
@@ -283 +284 @@
     Value* memoryObject = block->appendNew<MemoryValue>(proc, Load, pointerType(), Origin(), instance, JSWebAssemblyInstance::offsetOfMemory());
 
-    static_assert(sizeof(decltype(vm.topJSWebAssemblyInstance->memory()->memory()->memory())) == sizeof(void*), "codegen relies on this size");
-    static_assert(sizeof(decltype(vm.topJSWebAssemblyInstance->memory()->memory()->size())) == sizeof(uint64_t), "codegen relies on this size");
+    static_assert(sizeof(decltype(vm.topJSWebAssemblyInstance->memory()->memory().memory())) == sizeof(void*), "codegen relies on this size");
+    static_assert(sizeof(decltype(vm.topJSWebAssemblyInstance->memory()->memory().size())) == sizeof(uint64_t), "codegen relies on this size");
     MemoryBaseAndSize result;
     result.base = block->appendNew<MemoryValue>(proc, Load, pointerType(), Origin(), memoryObject, JSWebAssemblyMemory::offsetOfMemory());
@@ -455 +456 @@
 inline Value* B3IRGenerator::emitCheckAndPreparePointer(ExpressionType pointer, uint32_t offset, uint32_t sizeOfOperation)
 {
-    ASSERT(m_memoryBaseGPR && m_memorySizeGPR);
-    ASSERT(sizeOfOperation + offset > offset);
-    m_currentBlock->appendNew<WasmBoundsCheckValue>(m_proc, Origin(), pointer, m_memorySizeGPR, sizeOfOperation + offset - 1);
+    ASSERT(m_memoryBaseGPR);
+    if (m_info.memory.mode() == Memory::Mode::BoundsChecking) {
+        ASSERT(m_memorySizeGPR);
+        ASSERT(sizeOfOperation + offset > offset);
+        m_currentBlock->appendNew<WasmBoundsCheckValue>(m_proc, Origin(), pointer, m_memorySizeGPR, sizeOfOperation + offset - 1);
+    }
     pointer = m_currentBlock->appendNew<Value>(m_proc, ZExt32, Origin(), pointer);
     return m_currentBlock->appendNew<WasmAddressValue>(m_proc, Origin(), pointer, m_memoryBaseGPR);
@@ -487 +491 @@
 }
 
+inline B3::Kind B3IRGenerator::memoryKind(B3::Opcode memoryOp)
+{
+    if (m_info.memory.mode() == Memory::Signaling)
+        return trapping(memoryOp);
+    return memoryOp;
+}
+
 inline Value* B3IRGenerator::emitLoadOp(LoadOpType op, Origin origin, ExpressionType pointer, uint32_t offset)
 {
     switch (op) {
     case LoadOpType::I32Load8S: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load8S, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load8S), origin, pointer, offset);
     }
 
     case LoadOpType::I64Load8S: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load8S, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load8S), origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, SExt32, origin, value);
     }
 
     case LoadOpType::I32Load8U: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load8Z, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load8Z), origin, pointer, offset);
     }
 
     case LoadOpType::I64Load8U: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load8Z, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load8Z), origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, ZExt32, origin, value);
     }
 
     case LoadOpType::I32Load16S: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load16S, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load16S), origin, pointer, offset);
     }
     case LoadOpType::I64Load16S: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load16S, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load16S), origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, SExt32, origin, value);
     }
 
     case LoadOpType::I32Load: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Int32, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Int32, origin, pointer, offset);
     }
 
     case LoadOpType::I64Load32U: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Int32, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Int32, origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, ZExt32, origin, value);
     }
 
     case LoadOpType::I64Load32S: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Int32, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Int32, origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, SExt32, origin, value);
     }
 
     case LoadOpType::I64Load: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Int64, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Int64, origin, pointer, offset);
     }
 
     case LoadOpType::F32Load: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Float, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Float, origin, pointer, offset);
     }
 
     case LoadOpType::F64Load: {
-        return m_currentBlock->appendNew<MemoryValue>(m_proc, Load, Double, origin, pointer, offset);
+        return m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load), Double, origin, pointer, offset);
     }
 
@@ -545 +556 @@
     // it's added. https://bugs.webkit.org/show_bug.cgi?id=165884
     case LoadOpType::I32Load16U: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load16S, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load16S), origin, pointer, offset);
         return m_currentBlock->appendNew<Value>(m_proc, BitAnd, Origin(), value,
                 m_currentBlock->appendNew<Const32Value>(m_proc, Origin(), 0x0000ffff));
     }
     case LoadOpType::I64Load16U: {
-        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, Load16S, origin, pointer, offset);
+        Value* value = m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Load16S), origin, pointer, offset);
         Value* partialResult = m_currentBlock->appendNew<Value>(m_proc, BitAnd, Origin(), value,
                 m_currentBlock->appendNew<Const32Value>(m_proc, Origin(), 0x0000ffff));
@@ -632 +643 @@
 
     case StoreOpType::I32Store8:
-        m_currentBlock->appendNew<MemoryValue>(m_proc, Store8, origin, value, pointer, offset);
+        m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Store8), origin, value, pointer, offset);
         return;
 
@@ -640 +651 @@
 
     case StoreOpType::I32Store16:
-        m_currentBlock->appendNew<MemoryValue>(m_proc, Store16, origin, value, pointer, offset);
+        m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Store16), origin, value, pointer, offset);
         return;
 
@@ -651 +662 @@
     case StoreOpType::F32Store:
     case StoreOpType::F64Store:
-        m_currentBlock->appendNew<MemoryValue>(m_proc, Store, origin, value, pointer, offset);
+        m_currentBlock->appendNew<MemoryValue>(m_proc, memoryKind(Store), origin, value, pointer, offset);
         return;
     }

trunk/Source/JavaScriptCore/wasm/WasmCallingConvention.h

@@ -41 +41 @@
 #include "RegisterSet.h"
 #include "WasmFormat.h"
+#include "WasmSignature.h"
 
 namespace JSC { namespace Wasm {

trunk/Source/JavaScriptCore/wasm/WasmFaultSignalHandler.h

@@ -1 +1 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -24 +24 @@
  */
 
-#include "config.h"
-#include "JSWebAssemblyCallee.h"
-
 #if ENABLE(WEBASSEMBLY)
-
-#include "JSCInlines.h"
 
 namespace JSC {
 
-const ClassInfo JSWebAssemblyCallee::s_info = { "WebAssemblyCallee", nullptr, 0, CREATE_METHOD_TABLE(JSWebAssemblyCallee) };
+class VM;
 
-JSWebAssemblyCallee::JSWebAssemblyCallee(VM& vm)
-    : Base(vm, vm.webAssemblyCalleeStructure.get())
-{ }
+namespace Wasm {
 
-void JSWebAssemblyCallee::finishCreation(VM& vm, Wasm::Entrypoint&& entrypoint)
-{
-    Base::finishCreation(vm);
+void registerCode(VM&, void* start, void* end);
+void unregisterCode(VM&, void* start, void* end);
 
-    m_entrypoint = WTFMove(entrypoint);
-}
+bool fastMemoryEnabled();
+JS_EXPORT_PRIVATE void enableFastMemory();
 
-void JSWebAssemblyCallee::destroy(JSCell* cell)
-{
-    JSWebAssemblyCallee* thisObject = static_cast<JSWebAssemblyCallee*>(cell);
-    thisObject->JSWebAssemblyCallee::~JSWebAssemblyCallee();
-}
-
-} // namespace JSC
+} } // namespace JSC::Wasm
 
 #endif // ENABLE(WEBASSEMBLY)

trunk/Source/JavaScriptCore/wasm/WasmFormat.h

@@ -257 +257 @@
 
     uint32_t importFunctionCount() const { return importFunctionSignatureIndices.size(); }
-    bool hasMemory() const { return !!memory; }
 
     ~ModuleInformation();

trunk/Source/JavaScriptCore/wasm/WasmMemory.cpp

@@ -29 +29 @@
 #if ENABLE(WEBASSEMBLY)
 
+#include "VM.h"
+#include "WasmFaultSignalHandler.h"
+
 #include <wtf/HexNumber.h>
+#include <wtf/NeverDestroyed.h>
 #include <wtf/PrintStream.h>
 #include <wtf/text/WTFString.h>
@@ -39 +43 @@
 }
 
-void Memory::dump(PrintStream& out) const
-{
-    String memoryHex;
-    WTF::appendUnsigned64AsHex((uint64_t)(uintptr_t)m_memory, memoryHex);
-    out.print("Memory at 0x", memoryHex, ", size ", m_size, "B capacity ", m_mappedCapacity, "B, initial ", m_initial, " maximum ", m_maximum, " mode ", makeString(m_mode));
-}
-
-const char* Memory::makeString(Mode mode) const
-{
-    switch (mode) {
-    case Mode::BoundsChecking: return "BoundsChecking";
-    }
-    RELEASE_ASSERT_NOT_REACHED();
-    return "";
-}
-
-static_assert(sizeof(uint64_t) == sizeof(size_t), "We rely on allowing the maximum size of Memory we map to be 2^32 which is larger than fits in a 32-bit integer that we'd pass to mprotect if this didn't hold.");
-
-Memory::Memory(PageCount initial, PageCount maximum, bool& failed)
-    : m_size(initial.bytes())
+inline bool mmapBytes(size_t bytes, void*& memory)
+{
+    dataLogIf(verbose, "Attempting to mmap ", bytes, " bytes: ");
+    // FIXME: It would be nice if we had a VM tag for wasm memory. https://bugs.webkit.org/show_bug.cgi?id=163600
+    void* result = mmap(nullptr, bytes, PROT_NONE, MAP_PRIVATE | MAP_ANON, -1, 0);
+    if (result == MAP_FAILED) {
+        dataLogLnIf(verbose, "failed");
+        return false;
+    }
+    dataLogLnIf(verbose, "succeeded");
+    memory = result;
+    return true;
+}
+
+// We use this as a heuristic to guess what mode a memory import will be. Most of the time we expect users to
+// allocate the memory they are going to pass to all their modules right before compilation.
+static Memory::Mode lastAllocatedMemoryMode { Memory::Mode::Signaling };
+
+Memory::Mode Memory::lastAllocatedMode()
+{
+    return lastAllocatedMemoryMode;
+}
+
+static_assert(sizeof(uint64_t) == sizeof(size_t), "We rely on allowing the maximum size of Memory we map to be 2^33 which is larger than fits in a 32-bit integer that we'd pass to mprotect if this didn't hold.");
+
+static const size_t fastMemoryMappedBytes = (static_cast<size_t>(std::numeric_limits<uint32_t>::max()) + 1) * 2; // pointer max + offset max. This is all we need since a load straddling readable memory will trap.
+static const unsigned maxFastMemories = 4;
+static unsigned allocatedFastMemories { 0 };
+static StaticLock memoryLock;
+inline Deque<void*, maxFastMemories>& availableFastMemories(const LockHolder&)
+{
+    static NeverDestroyed<Deque<void*, maxFastMemories>> availableFastMemories;
+    return availableFastMemories;
+}
+
+inline bool tryGetFastMemory(VM& vm, void*& memory, size_t& mappedCapacity, Memory::Mode& mode)
+{
+    // We might GC here so we should be holding the API lock.
+    // FIXME: We should be able to syncronously trigger the GC from another thread.
+    ASSERT(vm.currentThreadIsHoldingAPILock());
+    if (!fastMemoryEnabled())
+        return false;
+
+    // We need to be sure we have a stub prior to running code.
+    if (!vm.getCTIStub(throwExceptionFromWasmThunkGenerator).size())
+        return false;
+
+    auto dequeFastMemory = [&] () -> bool {
+        // FIXME: We should eventually return these to the OS if we go some number of GCs
+        // without using them.
+        LockHolder locker(memoryLock);
+        if (!availableFastMemories(locker).isEmpty()) {
+            memory = availableFastMemories(locker).takeFirst();
+            mappedCapacity = fastMemoryMappedBytes;
+            mode = Memory::Signaling;
+            return true;
+        }
+        return false;
+    };
+
+    ASSERT(allocatedFastMemories <= maxFastMemories);
+    if (dequeFastMemory())
+        return true;
+
+    // If we have allocated all the fast memories... too bad.
+    if (allocatedFastMemories == maxFastMemories) {
+        // There is a reasonable chance that another module has died but has not been collected yet. Don't lose hope yet!
+        vm.heap.collectSync();
+        return dequeFastMemory();
+    }
+
+    if (mmapBytes(fastMemoryMappedBytes, memory)) {
+        mappedCapacity = fastMemoryMappedBytes;
+        mode = Memory::Signaling;
+        allocatedFastMemories++;
+    }
+    return memory;
+}
+
+inline void releaseFastMemory(void*& memory, size_t writableSize, size_t mappedCapacity, Memory::Mode mode)
+{
+    if (mode != Memory::Signaling || !memory)
+        return;
+
+    RELEASE_ASSERT(memory && mappedCapacity == fastMemoryMappedBytes);
+    ASSERT(fastMemoryEnabled());
+
+    memset(memory, 0, writableSize);
+    if (mprotect(memory, writableSize, PROT_NONE))
+        CRASH();
+
+    LockHolder locker(memoryLock);
+    ASSERT(availableFastMemories(locker).size() < allocatedFastMemories);
+    availableFastMemories(locker).append(memory);
+    memory = nullptr;
+}
+
+Memory::Memory(PageCount initial, PageCount maximum)
+    : m_initial(initial)
+    , m_maximum(maximum)
+{
+    ASSERT(!initial.bytes());
+}
+
+Memory::Memory(void* memory, PageCount initial, PageCount maximum, size_t mappedCapacity, Mode mode)
+    : m_memory(memory)
+    , m_size(initial.bytes())
     , m_initial(initial)
     , m_maximum(maximum)
-    , m_mode(Mode::BoundsChecking)
-    // FIXME: If we add signal based bounds checking then we need extra space for overflow on load.
-    // see: https://bugs.webkit.org/show_bug.cgi?id=162693
+    , m_mappedCapacity(mappedCapacity)
+    , m_mode(mode)
+{
+    dataLogLnIf(verbose, "Memory::Memory allocating ", *this);
+}
+
+RefPtr<Memory> Memory::createImpl(VM& vm, PageCount initial, PageCount maximum, std::optional<Mode> requiredMode)
 {
     RELEASE_ASSERT(!maximum || maximum >= initial); // This should be guaranteed by our caller.
 
-    m_mappedCapacity = maximum ? maximum.bytes() : PageCount::max().bytes();
-    if (!m_mappedCapacity) {
+    Mode mode = requiredMode ? *requiredMode : BoundsChecking;
+    const size_t size = initial.bytes();
+    size_t mappedCapacity = maximum ? maximum.bytes() : PageCount::max().bytes();
+    void* memory = nullptr;
+
+    auto makeEmptyMemory = [&] () -> RefPtr<Memory> {
+        if (mode == Signaling)
+            return nullptr;
+
+        lastAllocatedMemoryMode = BoundsChecking;
+        return adoptRef(new Memory(initial, maximum));
+    };
+
+    if (!mappedCapacity) {
         // This means we specified a zero as maximum (which means we also have zero as initial size).
-        RELEASE_ASSERT(m_size == 0);
-        m_memory = nullptr;
-        m_mappedCapacity = 0;
-        failed = false;
-        if (verbose)
-            dataLogLn("Memory::Memory allocating nothing ", *this);
-        return;
-    }
-
-    // FIXME: It would be nice if we had a VM tag for wasm memory. https://bugs.webkit.org/show_bug.cgi?id=163600
-    void* result = Options::simulateWebAssemblyLowMemory() ? MAP_FAILED : mmap(nullptr, m_mappedCapacity, PROT_NONE, MAP_PRIVATE | MAP_ANON, -1, 0);
-    if (result == MAP_FAILED) {
-        // Try again with a different number.
-        if (verbose)
-            dataLogLn("Memory::Memory mmap failed once for capacity, trying again", *this);
-        m_mappedCapacity = m_size;
-        if (!m_mappedCapacity) {
-            m_memory = nullptr;
-            failed = false;
-            if (verbose)
-                dataLogLn("Memory::Memory mmap not trying again because size is zero ", *this);
-            return;
+        RELEASE_ASSERT(!size);
+        dataLogLnIf(verbose, "Memory::create allocating nothing");
+        return makeEmptyMemory();
+    }
+
+    bool canUseFastMemory = !requiredMode || requiredMode == Signaling;
+    if (!canUseFastMemory || !tryGetFastMemory(vm, memory, mappedCapacity, mode)) {
+        if (mode == Signaling)
+            return nullptr;
+
+        if (Options::simulateWebAssemblyLowMemory() ? true : !mmapBytes(mappedCapacity, memory)) {
+            // Try again with a different number.
+            dataLogLnIf(verbose, "Memory::create mmap failed once for capacity, trying again");
+            mappedCapacity = size;
+            if (!mappedCapacity) {
+                dataLogLnIf(verbose, "Memory::create mmap not trying again because size is zero");
+                return makeEmptyMemory();
+            }
+
+            if (!mmapBytes(mappedCapacity, memory)) {
+                dataLogLnIf(verbose, "Memory::create mmap failed twice");
+                return nullptr;
+            }
         }
-
-        result = mmap(nullptr, m_mappedCapacity, PROT_NONE, MAP_PRIVATE | MAP_ANON, -1, 0);
-        if (result == MAP_FAILED) {
-            if (verbose)
-                dataLogLn("Memory::Memory mmap failed twice ", *this);
-            failed = true;
-            return;
+    }
+
+    ASSERT(memory && size <= mappedCapacity);
+    if (mprotect(memory, size, PROT_READ | PROT_WRITE)) {
+        dataLogLnIf(verbose, "Memory::create mprotect failed");
+        releaseFastMemory(memory, 0, mappedCapacity, mode);
+        if (memory) {
+            if (munmap(memory, mappedCapacity))
+                CRASH();
         }
-    }
-
-    ASSERT(m_size <= m_mappedCapacity);
-    {
-        bool success = !mprotect(result, static_cast<size_t>(m_size), PROT_READ | PROT_WRITE);
-        RELEASE_ASSERT(success);
-    }
-
-    m_memory = result;
-    failed = false;
-    if (verbose)
-        dataLogLn("Memory::Memory mmap succeeded ", *this);
+        return nullptr;
+    }
+    
+    lastAllocatedMemoryMode = mode;
+    dataLogLnIf(verbose, "Memory::create mmap succeeded");
+    return adoptRef(new Memory(memory, initial, maximum, mappedCapacity, mode));
+}
+
+RefPtr<Memory> Memory::create(VM& vm, PageCount initial, PageCount maximum, std::optional<Mode> mode)
+{
+    RELEASE_ASSERT(!maximum || maximum >= initial); // This should be guaranteed by our caller.
+    RefPtr<Memory> result = createImpl(vm, initial, maximum, mode);
+    if (result) {
+        if (result->mode() == Signaling)
+            RELEASE_ASSERT(result->m_mappedCapacity == fastMemoryMappedBytes);
+        if (mode)
+            ASSERT(*mode == result->mode());
+        ASSERT(lastAllocatedMemoryMode == result->mode());
+    }
+    return result;
 }
 
 Memory::~Memory()
 {
-    if (verbose)
-        dataLogLn("Memory::~Memory ", *this);
+    dataLogLnIf(verbose, "Memory::~Memory ", *this);
+    releaseFastMemory(m_memory, m_size, m_mappedCapacity, m_mode);
     if (m_memory) {
         if (munmap(m_memory, m_mappedCapacity))
@@ -129 +247 @@
     RELEASE_ASSERT(newSize > PageCount::fromBytes(m_size));
 
-    if (verbose)
-        dataLogLn("Memory::grow to ", newSize, " from ", *this);
+    dataLogLnIf(verbose, "Memory::grow to ", newSize, " from ", *this);
 
     if (maximum() && newSize > maximum())
         return false;
 
-    uint64_t desiredSize = newSize.bytes();
+    size_t desiredSize = newSize.bytes();
 
     if (m_memory && desiredSize <= m_mappedCapacity) {
-        bool success = !mprotect(static_cast<uint8_t*>(m_memory) + m_size, static_cast<size_t>(desiredSize - m_size), PROT_READ | PROT_WRITE);
-        RELEASE_ASSERT(success);
+        if (mprotect(static_cast<uint8_t*>(m_memory) + m_size, static_cast<size_t>(desiredSize - m_size), PROT_READ | PROT_WRITE)) {
+            dataLogLnIf(verbose, "Memory::grow in-place failed ", *this);
+            return false;
+        }
+
         m_size = desiredSize;
-        if (verbose)
-            dataLogLn("Memory::grow in-place ", *this);
+        dataLogLnIf(verbose, "Memory::grow in-place ", *this);
         return true;
     }
 
+    ASSERT(mode() != Signaling);
     // Otherwise, let's try to make some new memory.
     void* newMemory = mmap(nullptr, desiredSize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0);
@@ -160 +280 @@
     m_size = desiredSize;
 
-    if (verbose)
-        dataLogLn("Memory::grow ", *this);
+    dataLogLnIf(verbose, "Memory::grow ", *this);
     return true;
 }
 
+void Memory::dump(PrintStream& out) const
+{
+    out.print("Memory at ", RawPointer(m_memory), ", size ", m_size, "B capacity ", m_mappedCapacity, "B, initial ", m_initial, " maximum ", m_maximum, " mode ", makeString(m_mode));
+}
+
+const char* Memory::makeString(Mode mode) const
+{
+    switch (mode) {
+    case Mode::BoundsChecking: return "BoundsChecking";
+    case Mode::Signaling: return "Signaling";
+    case Mode::NumberOfModes: break;
+    }
+    RELEASE_ASSERT_NOT_REACHED();
+    return "";
+}
+
 } // namespace JSC
 

trunk/Source/JavaScriptCore/wasm/WasmMemory.h

@@ -28 +28 @@
 #if ENABLE(WEBASSEMBLY)
 
-#include "WasmCallingConvention.h"
 #include "WasmPageCount.h"
+
+#include <wtf/Optional.h>
+#include <wtf/RefCounted.h>
+#include <wtf/RefPtr.h>
 
 namespace WTF {
@@ -35 +38 @@
 }
 
-namespace JSC { namespace Wasm {
+namespace JSC {
 
-class Memory {
+class VM;
+
+namespace Wasm {
+
+class Memory : public RefCounted<Memory> {
     WTF_MAKE_NONCOPYABLE(Memory);
     WTF_MAKE_FAST_ALLOCATED;
@@ -44 +51 @@
 
     // FIXME: We should support other modes. see: https://bugs.webkit.org/show_bug.cgi?id=162693
-    enum class Mode {
-        BoundsChecking
+    enum Mode {
+        BoundsChecking,
+        Signaling,
+        NumberOfModes
     };
     const char* makeString(Mode) const;
 
+    explicit operator bool() const { return !!m_memory; }
+
+    static RefPtr<Memory> create(VM&, PageCount initial, PageCount maximum, std::optional<Mode> requiredMode = std::nullopt);
+
     Memory() = default;
-    JS_EXPORT_PRIVATE Memory(PageCount initial, PageCount maximum, bool& failed);
-    Memory(Memory&& other)
-        : m_memory(other.m_memory)
-        , m_size(other.m_size)
-        , m_initial(other.m_initial)
-        , m_maximum(other.m_maximum)
-        , m_mappedCapacity(other.m_mappedCapacity)
-        , m_mode(other.m_mode)
-    {
-        // Moving transfers ownership of the allocated memory.
-        other.m_memory = nullptr;
-    }
     ~Memory();
 
     void* memory() const { return m_memory; }
-    uint64_t size() const { return m_size; }
+    size_t size() const { return m_size; }
     PageCount sizeInPages() const { return PageCount::fromBytes(m_size); }
 
@@ -71 +72 @@
     PageCount maximum() const { return m_maximum; }
 
+    static Mode lastAllocatedMode();
     Mode mode() const { return m_mode; }
 
+    // grow() should only be called from the JSWebAssemblyMemory object since that object needs to update internal
+    // pointers with the current base and size.
     bool grow(PageCount);
 
-    static ptrdiff_t offsetOfMemory() { return OBJECT_OFFSETOF(Memory, m_memory); }
-    static ptrdiff_t offsetOfSize() { return OBJECT_OFFSETOF(Memory, m_size); }
-    
+    void check() {  ASSERT(!deletionHasBegun()); }
 private:
+    static RefPtr<Memory> createImpl(VM&, PageCount initial, PageCount maximum, std::optional<Mode> requiredMode = std::nullopt);
+    Memory(void* memory, PageCount initial, PageCount maximum, size_t mappedCapacity, Mode);
+    Memory(PageCount initial, PageCount maximum);
+
+    // FIXME: we should move these to the instance to avoid a load on instance->instance calls.
     void* m_memory { nullptr };
-    uint64_t m_size { 0 };
+    size_t m_size { 0 };
     PageCount m_initial;
     PageCount m_maximum;
-    uint64_t m_mappedCapacity { 0 };
+    size_t m_mappedCapacity { 0 };
     Mode m_mode { Mode::BoundsChecking };
 };

trunk/Source/JavaScriptCore/wasm/WasmMemoryInformation.cpp

@@ -30 +30 @@
 
 #include "WasmCallingConvention.h"
+#include "WasmMemory.h"
 #include <wtf/NeverDestroyed.h>
 
@@ -49 +50 @@
         unsigned remainingPinnedRegisters = pinnedSizes.size() + 1;
         jscCallingConvention().m_calleeSaveRegisters.forEach([&] (Reg reg) {
+            if (!reg.isGPR())
+                return;
             GPRReg gpr = reg.gpr();
             if (!remainingPinnedRegisters || RegisterSet::stackRegisters().get(reg))
@@ -72 +75 @@
 }
 
-MemoryInformation::MemoryInformation(PageCount initial, PageCount maximum,  bool isImport)
+MemoryInformation::MemoryInformation(VM& vm, PageCount initial, PageCount maximum, std::optional<Memory::Mode> recompileMode, bool isImport)
     : m_initial(initial)
     , m_maximum(maximum)
@@ -80 +83 @@
     RELEASE_ASSERT(!m_maximum || m_maximum >= m_initial);
     ASSERT(!!*this);
+
+    if (!recompileMode) {
+        if (!isImport) {
+            m_reservedMemory = Memory::create(vm, initial, maximum, Memory::Signaling);
+            if (m_reservedMemory) {
+                ASSERT(!!*m_reservedMemory);
+                m_mode = m_reservedMemory->mode();
+                return;
+            }
+        }
+        m_mode = Memory::lastAllocatedMode();
+    } else
+        m_mode = *recompileMode;
 }
 

trunk/Source/JavaScriptCore/wasm/WasmMemoryInformation.h

@@ -29 +29 @@
 
 #include "GPRInfo.h"
+#include "WasmMemory.h"
 #include "WasmPageCount.h"
+#include <wtf/Ref.h>
 #include <wtf/Vector.h>
 
@@ -53 +55 @@
     }
 
-    MemoryInformation(PageCount initial, PageCount maximum, bool isImport);
+    MemoryInformation(VM&, PageCount initial, PageCount maximum, std::optional<Memory::Mode>, bool isImport);
 
     PageCount initial() const { return m_initial; }
     PageCount maximum() const { return m_maximum; }
+    bool hasReservedMemory() const { return m_reservedMemory; }
+    RefPtr<Memory> takeReservedMemory() { ASSERT(hasReservedMemory()); return m_reservedMemory.release(); }
+    Memory::Mode mode() const { return m_mode; }
     bool isImport() const { return m_isImport; }
 
@@ -62 +67 @@
 
 private:
+    RefPtr<Memory> m_reservedMemory;
     PageCount m_initial { };
     PageCount m_maximum { };
+    Memory::Mode m_mode { Memory::Mode::BoundsChecking };
     bool m_isImport { false };
 };

trunk/Source/JavaScriptCore/wasm/WasmModuleParser.cpp

@@ -317 +317 @@
     ASSERT(!maximumPageCount || maximumPageCount >= initialPageCount);
 
-    m_result.module->memory = MemoryInformation(initialPageCount, maximumPageCount, isImport);
+    m_result.module->memory = MemoryInformation(*m_vm, initialPageCount, maximumPageCount, m_mode, isImport);
     return { };
 }

trunk/Source/JavaScriptCore/wasm/WasmModuleParser.h

@@ -31 +31 @@
 #include "WasmOps.h"
 #include "WasmParser.h"
+#include <wtf/Optional.h>
 #include <wtf/Vector.h>
 
@@ -44 +45 @@
 public:
 
-    ModuleParser(VM* vm, const uint8_t* sourceBuffer, size_t sourceLength)
+    ModuleParser(VM* vm, const uint8_t* sourceBuffer, size_t sourceLength, std::optional<Memory::Mode> mode)
         : Parser(vm, sourceBuffer, sourceLength)
-    {
-    }
-    ModuleParser(VM* vm, const Vector<uint8_t>& sourceBuffer)
-        : ModuleParser(vm, sourceBuffer.data(), sourceBuffer.size())
+        , m_mode(mode)
     {
     }
@@ -69 +67 @@
 
     ModuleParserResult m_result;
+    std::optional<Memory::Mode> m_mode { std::nullopt };
     bool m_hasTable { false };
 };

trunk/Source/JavaScriptCore/wasm/WasmPlan.cpp

@@ -36 +36 @@
 #include "WasmBinding.h"
 #include "WasmCallingConvention.h"
+#include "WasmFaultSignalHandler.h"
 #include "WasmMemory.h"
 #include "WasmModuleParser.h"
@@ -62 +63 @@
 }
 
-bool Plan::parseAndValidateModule()
+bool Plan::parseAndValidateModule(std::optional<Memory::Mode> recompileMode)
 {
     MonotonicTime startTime;
@@ -69 +70 @@
 
     {
-        ModuleParser moduleParser(m_vm, m_source, m_sourceLength);
+        ModuleParser moduleParser(m_vm, m_source, m_sourceLength, recompileMode);
         auto parseResult = moduleParser.parse();
         if (!parseResult) {
@@ -110 +111 @@
 // that could touch its stack are done executing.
 SUPPRESS_ASAN 
-void Plan::run()
-{
-    if (!parseAndValidateModule())
+void Plan::run(std::optional<Memory::Mode> recompileMode)
+{
+    if (!parseAndValidateModule(recompileMode))
         return;
+    if (recompileMode)
+        ASSERT(m_moduleInformation->memory.mode() == recompileMode);
 
     auto tryReserveCapacity = [this] (auto& vector, size_t size, const char* what) {

trunk/Source/JavaScriptCore/wasm/WasmPlan.h

@@ -50 +50 @@
     JS_EXPORT_PRIVATE ~Plan();
 
-    bool parseAndValidateModule();
+    bool parseAndValidateModule(std::optional<Memory::Mode> = std::nullopt);
 
-    JS_EXPORT_PRIVATE void run();
+    JS_EXPORT_PRIVATE void run(std::optional<Memory::Mode> = std::nullopt);
 
     JS_EXPORT_PRIVATE void initializeCallees(JSGlobalObject*, std::function<void(unsigned, JSWebAssemblyCallee*, JSWebAssemblyCallee*)>);
@@ -93 +93 @@
     }
 
+    Memory::Mode mode() const { return m_moduleInformation->memory.mode(); }
+
 private:
     std::unique_ptr<ModuleInformation> m_moduleInformation;

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyCallee.cpp

@@ -30 +30 @@
 
 #include "JSCInlines.h"
+#include "WasmFaultSignalHandler.h"
 
 namespace JSC {
@@ -44 +45 @@
 
     m_entrypoint = WTFMove(entrypoint);
+    Wasm::registerCode(vm, m_entrypoint.compilation->codeRef().executableMemory()->start(), m_entrypoint.compilation->codeRef().executableMemory()->end());
 }
 
@@ -49 +51 @@
 {
     JSWebAssemblyCallee* thisObject = static_cast<JSWebAssemblyCallee*>(cell);
+    Wasm::unregisterCode(*cell->vm(), thisObject->m_entrypoint.compilation->codeRef().executableMemory()->start(), thisObject->m_entrypoint.compilation->codeRef().executableMemory()->end());
     thisObject->JSWebAssemblyCallee::~JSWebAssemblyCallee();
 }

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyInstance.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -39 +39 @@
 namespace JSC {
 
+void JSWebAssemblyInstance::setMemory(VM& vm, ExecState* exec, JSWebAssemblyMemory* memory)
+{
+    auto scope = DECLARE_THROW_SCOPE(vm);
+    // We create stub memories even for modules that should eventually get a memory so we want to avoid recompling there.
+    if (memory->memory()) {
+        auto codeBlock = m_codeBlock->module()->codeBlock(vm, exec, memory);
+        RETURN_IF_EXCEPTION(scope,);
+        m_codeBlock.set(vm, this, codeBlock);
+    }
+    m_memory.set(vm, this, memory);
+}
+
 JSWebAssemblyInstance* JSWebAssemblyInstance::create(VM& vm, Structure* structure, JSWebAssemblyModule* module, JSModuleNamespaceObject* moduleNamespaceObject)
 {
@@ -68 +80 @@
     heap()->reportExtraMemoryAllocated(extraMemorySize);
 
-    m_module.set(vm, this, module);
+    m_codeBlock.set(vm, this, module->codeBlock());
     m_moduleNamespaceObject.set(vm, this, moduleNamespaceObject);
     putDirect(vm, Identifier::fromString(&vm, "exports"), moduleNamespaceObject, None);
@@ -84 +96 @@
 
     Base::visitChildren(thisObject, visitor);
-    visitor.append(thisObject->m_module);
+    visitor.append(thisObject->m_codeBlock);
     visitor.append(thisObject->m_moduleNamespaceObject);
     visitor.append(thisObject->m_memory);

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyInstance.h

@@ -30 +30 @@
 #include "JSDestructibleObject.h"
 #include "JSObject.h"
+#include "JSWebAssemblyCodeBlock.h"
 #include "JSWebAssemblyMemory.h"
 #include "JSWebAssemblyTable.h"
@@ -48 +49 @@
     DECLARE_INFO;
 
-    JSWebAssemblyModule* module()
+    JSWebAssemblyModule* module() const
     {
-        ASSERT(m_module);
-        return m_module.get();
+        ASSERT(m_codeBlock);
+        return m_codeBlock->module();
+    }
+
+    JSWebAssemblyCodeBlock* codeBlock() const
+    {
+        ASSERT(m_codeBlock);
+        return m_codeBlock.get();
     }
 
@@ -71 +78 @@
 
     JSWebAssemblyMemory* memory() { return m_memory.get(); }
-    void setMemory(VM& vm, JSWebAssemblyMemory* memory) { m_memory.set(vm, this, memory); }
+    // Calling this might trigger a recompile.
+    void setMemory(VM&, ExecState*, JSWebAssemblyMemory*);
+    Wasm::Memory::Mode memoryMode() { return memory()->memory().mode(); }
 
     JSWebAssemblyTable* table() { return m_table.get(); }
@@ -105 +114 @@
 
 private:
-    WriteBarrier<JSWebAssemblyModule> m_module;
+    WriteBarrier<JSWebAssemblyCodeBlock> m_codeBlock;
     WriteBarrier<JSModuleNamespaceObject> m_moduleNamespaceObject;
     WriteBarrier<JSWebAssemblyMemory> m_memory;

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyMemory.cpp

@@ -38 +38 @@
 const ClassInfo JSWebAssemblyMemory::s_info = { "WebAssembly.Memory", &Base::s_info, 0, CREATE_METHOD_TABLE(JSWebAssemblyMemory) };
 
-JSWebAssemblyMemory* JSWebAssemblyMemory::create(VM& vm, Structure* structure, Wasm::Memory&& memory)
+JSWebAssemblyMemory* JSWebAssemblyMemory::create(VM& vm, Structure* structure, Ref<Wasm::Memory>&& memory)
 {
-    auto* instance = new (NotNull, allocateCell<JSWebAssemblyMemory>(vm.heap)) JSWebAssemblyMemory(vm, structure, std::forward<Wasm::Memory>(memory));
+    auto* instance = new (NotNull, allocateCell<JSWebAssemblyMemory>(vm.heap)) JSWebAssemblyMemory(vm, structure, WTFMove(memory));
+    instance->m_memory->check();
     instance->finishCreation(vm);
     return instance;
@@ -50 +51 @@
 }
 
-JSWebAssemblyMemory::JSWebAssemblyMemory(VM& vm, Structure* structure, Wasm::Memory&& memory)
+JSWebAssemblyMemory::JSWebAssemblyMemory(VM& vm, Structure* structure, Ref<Wasm::Memory>&& memory)
     : Base(vm, structure)
     , m_memory(WTFMove(memory))
 {
+    ASSERT(m_memory->refCount() == 1);
+    m_memoryBase = m_memory->memory();
+    m_memorySize = m_memory->size();
 }
 
@@ -61 +65 @@
         return m_bufferWrapper.get();
 
-    auto destructor = [] (void*) {
-        // We don't need to do anything here to destroy the memory.
-        // The ArrayBuffer backing the JSArrayBuffer is only owned by us,
-        // so we guarantee its lifecycle.
-    };
-    m_buffer = ArrayBuffer::createFromBytes(memory()->memory(), memory()->size(), WTFMove(destructor));
+    // We can't use a ref here since it doesn't have a copy constructor...
+    Ref<Wasm::Memory> protectedMemory = m_memory.get();
+    auto destructor = [protectedMemory = WTFMove(protectedMemory)] (void*) { };
+    m_buffer = ArrayBuffer::createFromBytes(memory().memory(), memory().size(), WTFMove(destructor));
     m_bufferWrapper.set(vm, this, JSArrayBuffer::create(vm, globalObject->m_arrayBufferStructure.get(), m_buffer.get()));
     RELEASE_ASSERT(m_bufferWrapper);
@@ -77 +79 @@
     auto throwScope = DECLARE_THROW_SCOPE(vm);
 
-    Wasm::PageCount oldPageCount = memory()->sizeInPages();
+    Wasm::PageCount oldPageCount = memory().sizeInPages();
 
     if (!Wasm::PageCount::isValid(delta)) {
@@ -93 +95 @@
 
     if (delta) {
-        bool success = memory()->grow(newSize);
+        bool success = memory().grow(newSize);
         if (!success) {
+            ASSERT(m_memoryBase == memory().memory());
+            ASSERT(m_memorySize == memory().size());
             if (shouldThrowExceptionsOnFailure)
                 throwException(exec, throwScope, createOutOfMemoryError(exec));
             return Wasm::PageCount();
         }
+        m_memoryBase = memory().memory();
+        m_memorySize = memory().size();
     }
 
@@ -111 +117 @@
     }
 
+    memory().check();
     return oldPageCount;
 }
@@ -124 +131 @@
     auto memory = static_cast<JSWebAssemblyMemory*>(cell);
     ASSERT(memory->classInfo() == info());
-    VM& vm = *memory->vm();
-
-    if (memory->m_buffer) {
-        ArrayBufferContents dummyContents;
-        memory->m_buffer->transferTo(vm, dummyContents);
-        memory->m_buffer = nullptr;
-    }
 
     memory->JSWebAssemblyMemory::~JSWebAssemblyMemory();

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyMemory.h

@@ -42 +42 @@
     typedef JSDestructibleObject Base;
 
-    static JSWebAssemblyMemory* create(VM&, Structure*, Wasm::Memory&&);
+    static JSWebAssemblyMemory* create(VM&, Structure*, Ref<Wasm::Memory>&&);
     static Structure* createStructure(VM&, JSGlobalObject*, JSValue);
 
     DECLARE_INFO;
 
-    Wasm::Memory* memory() { return &m_memory; }
+    Wasm::Memory& memory() { return m_memory.get(); }
     JSArrayBuffer* buffer(VM& vm, JSGlobalObject*);
     Wasm::PageCount grow(ExecState*, uint32_t delta, bool shouldThrowExceptionsOnFailure);
 
-    static ptrdiff_t offsetOfMemory() { return OBJECT_OFFSETOF(JSWebAssemblyMemory, m_memory) + Wasm::Memory::offsetOfMemory(); }
-    static ptrdiff_t offsetOfSize() { return OBJECT_OFFSETOF(JSWebAssemblyMemory, m_memory) + Wasm::Memory::offsetOfSize(); }
+    static ptrdiff_t offsetOfMemory() { return OBJECT_OFFSETOF(JSWebAssemblyMemory, m_memoryBase); }
+    static ptrdiff_t offsetOfSize() { return OBJECT_OFFSETOF(JSWebAssemblyMemory, m_memorySize); }
 
-protected:
-    JSWebAssemblyMemory(VM&, Structure*, Wasm::Memory&&);
+private:
+    JSWebAssemblyMemory(VM&, Structure*, Ref<Wasm::Memory>&&);
     void finishCreation(VM&);
     static void destroy(JSCell*);
     static void visitChildren(JSCell*, SlotVisitor&);
 
-    Wasm::Memory m_memory;
+    void* m_memoryBase;
+    size_t m_memorySize;
+    Ref<Wasm::Memory> m_memory;
     WriteBarrier<JSArrayBuffer> m_bufferWrapper;
     RefPtr<ArrayBuffer> m_buffer;

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyModule.cpp

@@ -31 +31 @@
 #include "JSCInlines.h"
 #include "JSWebAssemblyCallee.h"
+#include "JSWebAssemblyCodeBlock.h"
+#include "JSWebAssemblyCompileError.h"
+#include "JSWebAssemblyMemory.h"
 #include "WasmFormat.h"
 #include "WasmMemory.h"
+#include "WasmPlan.h"
 #include <wtf/StdLibExtras.h>
 
@@ -39 +43 @@
 const ClassInfo JSWebAssemblyModule::s_info = { "WebAssembly.Module", &Base::s_info, nullptr, CREATE_METHOD_TABLE(JSWebAssemblyModule) };
 
-JSWebAssemblyModule* JSWebAssemblyModule::create(VM& vm, Structure* structure, std::unique_ptr<Wasm::ModuleInformation>&& moduleInformation, Bag<CallLinkInfo>&& callLinkInfos, Vector<Wasm::WasmExitStubs>&& wasmExitStubs, SymbolTable* exportSymbolTable, unsigned calleeCount)
+JSWebAssemblyCodeBlock* JSWebAssemblyModule::buildCodeBlock(VM& vm, ExecState* exec, Wasm::Plan& plan, std::optional<Wasm::Memory::Mode> mode)
 {
-    auto* instance = new (NotNull, allocateCell<JSWebAssemblyModule>(vm.heap, allocationSize(calleeCount))) JSWebAssemblyModule(vm, structure, std::forward<std::unique_ptr<Wasm::ModuleInformation>>(moduleInformation), std::forward<Bag<CallLinkInfo>>(callLinkInfos), std::forward<Vector<Wasm::WasmExitStubs>>(wasmExitStubs), calleeCount);
-    instance->finishCreation(vm, exportSymbolTable);
+    auto scope = DECLARE_THROW_SCOPE(vm);
+    // On failure, a new WebAssembly.CompileError is thrown.
+    plan.run(mode);
+    if (plan.failed()) {
+        throwException(exec, scope, createJSWebAssemblyCompileError(exec, vm, plan.errorMessage()));
+        return nullptr;
+    }
+    if (mode)
+        ASSERT(*mode == plan.mode());
+
+    unsigned calleeCount = plan.internalFunctionCount();
+    auto* codeBlock = JSWebAssemblyCodeBlock::create(vm, this, plan.takeCallLinkInfos(), plan.takeWasmExitStubs(), plan.mode(), calleeCount);
+
+    plan.initializeCallees(exec->jsCallee()->globalObject(),
+        [&] (unsigned calleeIndex, JSWebAssemblyCallee* jsEntrypointCallee, JSWebAssemblyCallee* wasmEntrypointCallee) {
+            codeBlock->setJSEntrypointCallee(vm, calleeIndex, jsEntrypointCallee);
+            codeBlock->setWasmEntrypointCallee(vm, calleeIndex, wasmEntrypointCallee);
+        });
+    return codeBlock;
+}
+
+JSWebAssemblyModule* JSWebAssemblyModule::create(VM& vm, ExecState* exec, Structure* structure, uint8_t* source, size_t byteSize)
+{
+    auto* instance = new (NotNull, allocateCell<JSWebAssemblyModule>(vm.heap)) JSWebAssemblyModule(vm, structure);
+
+    instance->finishCreation(vm, exec, source, byteSize);
     return instance;
 }
@@ -51 +79 @@
 }
 
-JSWebAssemblyModule::JSWebAssemblyModule(VM& vm, Structure* structure, std::unique_ptr<Wasm::ModuleInformation>&& moduleInformation, Bag<CallLinkInfo>&& callLinkInfos, Vector<Wasm::WasmExitStubs>&& wasmExitStubs, unsigned calleeCount)
+JSWebAssemblyModule::JSWebAssemblyModule(VM& vm, Structure* structure)
     : Base(vm, structure)
-    , m_moduleInformation(WTFMove(moduleInformation))
-    , m_callLinkInfos(WTFMove(callLinkInfos))
-    , m_wasmExitStubs(WTFMove(wasmExitStubs))
-    , m_calleeCount(calleeCount)
 {
-    memset(callees(), 0, m_calleeCount * sizeof(WriteBarrier<JSWebAssemblyCallee>) * 2);
 }
 
-void JSWebAssemblyModule::finishCreation(VM& vm, SymbolTable* exportSymbolTable)
+JSWebAssemblyCodeBlock* JSWebAssemblyModule::codeBlock(VM& vm, ExecState* exec, JSWebAssemblyMemory* memory)
+{
+    Wasm::Memory::Mode mode = memory->memory().mode();
+
+    for (unsigned i = 0; i < Wasm::Memory::NumberOfModes; ++i) {
+        if (m_codeBlocks[i] && m_codeBlocks[i]->isSafeToRun(memory))
+            return m_codeBlocks[i].get();
+    }
+
+    ASSERT(!m_codeBlocks[mode]);
+    auto scope = DECLARE_THROW_SCOPE(vm);
+    // We don't have a code block for this mode, we need to recompile...
+    Wasm::Plan plan(&vm, static_cast<uint8_t*>(m_sourceBuffer->data()), m_sourceBuffer->byteLength());
+
+    auto* codeBlock = buildCodeBlock(vm, exec, plan, mode);
+    RETURN_IF_EXCEPTION(scope, nullptr);
+
+    ASSERT(plan.exports().size() == m_exportSymbolTable->size());
+    if (!ASSERT_DISABLED) {
+        for (auto& exp : plan.exports())
+            ASSERT_UNUSED(exp, m_exportSymbolTable->contains(exp.field.impl()));
+    }
+
+    ASSERT(mode == codeBlock->mode());
+    m_codeBlocks[mode].set(vm, this, codeBlock);
+    return codeBlock;
+}
+
+void JSWebAssemblyModule::finishCreation(VM& vm, ExecState* exec, uint8_t* source, size_t byteSize)
 {
     Base::finishCreation(vm);
     ASSERT(inherits(vm, info()));
+
+    auto scope = DECLARE_THROW_SCOPE(vm);
+    Wasm::Plan plan(&vm, source, byteSize);
+
+    auto* codeBlock = buildCodeBlock(vm, exec, plan);
+    RETURN_IF_EXCEPTION(scope,);
+
+    // On success, a new WebAssembly.Module object is returned with [[Module]] set to the validated Ast.module.
+    SymbolTable* exportSymbolTable = SymbolTable::create(vm);
+    for (auto& exp : plan.exports()) {
+        auto offset = exportSymbolTable->takeNextScopeOffset(NoLockingNecessary);
+        exportSymbolTable->set(NoLockingNecessary, exp.field.impl(), SymbolTableEntry(VarOffset(offset)));
+    }
+
+    m_sourceBuffer = ArrayBuffer::create(source, byteSize);
+    m_moduleInformation = plan.takeModuleInformation();
     m_exportSymbolTable.set(vm, this, exportSymbolTable);
+    m_codeBlocks[codeBlock->mode()].set(vm, this, codeBlock);
 }
 
@@ -80 +148 @@
     Base::visitChildren(thisObject, visitor);
     visitor.append(thisObject->m_exportSymbolTable);
-    for (unsigned i = 0; i < thisObject->m_calleeCount * 2; i++)
-        visitor.append(thisObject->callees()[i]);
-
-    visitor.addUnconditionalFinalizer(&thisObject->m_unconditionalFinalizer);
-}
-
-void JSWebAssemblyModule::UnconditionalFinalizer::finalizeUnconditionally()
-{
-    JSWebAssemblyModule* thisObject = bitwise_cast<JSWebAssemblyModule*>(
-        bitwise_cast<char*>(this) - OBJECT_OFFSETOF(JSWebAssemblyModule, m_unconditionalFinalizer));
-    for (auto iter = thisObject->m_callLinkInfos.begin(); !!iter; ++iter)
-        (*iter)->visitWeak(*thisObject->vm());
+    for (unsigned i = 0; i < Wasm::Memory::NumberOfModes; ++i)
+        visitor.append(thisObject->m_codeBlocks[i]);
 }
 

trunk/Source/JavaScriptCore/wasm/js/JSWebAssemblyModule.h

@@ -1 +1 @@
 /*
- * Copyright (C) 2016 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -30 +30 @@
 #include "JSDestructibleObject.h"
 #include "JSObject.h"
-#include "JSWebAssemblyCallee.h"
+#include "JSWebAssemblyCodeBlock.h"
 #include "UnconditionalFinalizer.h"
 #include "WasmFormat.h"
@@ -38 +38 @@
 namespace JSC {
 
+namespace Wasm {
+class Plan;
+}
+
 class SymbolTable;
+class JSWebAssemblyMemory;
 
 class JSWebAssemblyModule : public JSDestructibleObject {
@@ -44 +49 @@
     typedef JSDestructibleObject Base;
 
-    static JSWebAssemblyModule* create(VM&, Structure*, std::unique_ptr<Wasm::ModuleInformation>&&, Bag<CallLinkInfo>&&, Vector<Wasm::WasmExitStubs>&&, SymbolTable*, unsigned);
+    static JSWebAssemblyModule* create(VM&, ExecState*, Structure*, uint8_t* source, size_t byteSize);
     static Structure* createStructure(VM&, JSGlobalObject*, JSValue);
 
@@ -50 +55 @@
 
     const Wasm::ModuleInformation& moduleInformation() const { return *m_moduleInformation.get(); }
+    RefPtr<Wasm::Memory> takeReservedMemory() { return m_moduleInformation->memory.takeReservedMemory(); }
     SymbolTable* exportSymbolTable() const { return m_exportSymbolTable.get(); }
     Wasm::SignatureIndex signatureIndexFromFunctionIndexSpace(unsigned functionIndexSpace) const
@@ -55 +61 @@
         return m_moduleInformation->signatureIndexFromFunctionIndexSpace(functionIndexSpace);
     }
-    unsigned functionImportCount() const { return m_wasmExitStubs.size(); }
 
-    JSWebAssemblyCallee* jsEntrypointCalleeFromFunctionIndexSpace(unsigned functionIndexSpace)
-    {
-        RELEASE_ASSERT(functionIndexSpace >= functionImportCount());
-        unsigned calleeIndex = functionIndexSpace - functionImportCount();
-        RELEASE_ASSERT(calleeIndex < m_calleeCount);
-        return callees()[calleeIndex].get();
-    }
+    // Returns the code block that this module was originally compiled expecting to use. This won't need to recompile.
+    JSWebAssemblyCodeBlock* codeBlock() { return m_codeBlocks[m_moduleInformation->memory.mode()].get(); }
+    // Returns the appropriate code block for the given memory, possibly triggering a recompile.
+    JSWebAssemblyCodeBlock* codeBlock(VM&, ExecState*, JSWebAssemblyMemory*);
 
-    JSWebAssemblyCallee* wasmEntrypointCalleeFromFunctionIndexSpace(unsigned functionIndexSpace)
-    {
-        RELEASE_ASSERT(functionIndexSpace >= functionImportCount());
-        unsigned calleeIndex = functionIndexSpace - functionImportCount();
-        RELEASE_ASSERT(calleeIndex < m_calleeCount);
-        return callees()[calleeIndex + m_calleeCount].get();
-    }
+private:
+    JSWebAssemblyCodeBlock* buildCodeBlock(VM&, ExecState*, Wasm::Plan&, std::optional<Wasm::Memory::Mode> mode = std::nullopt);
 
-    void setJSEntrypointCallee(VM& vm, unsigned calleeIndex, JSWebAssemblyCallee* callee)
-    {
-        RELEASE_ASSERT(calleeIndex < m_calleeCount);
-        callees()[calleeIndex].set(vm, this, callee);
-    }
-
-    void setWasmEntrypointCallee(VM& vm, unsigned calleeIndex, JSWebAssemblyCallee* callee)
-    {
-        RELEASE_ASSERT(calleeIndex < m_calleeCount);
-        callees()[calleeIndex + m_calleeCount].set(vm, this, callee);
-    }
-
-    WriteBarrier<JSWebAssemblyCallee>* callees()
-    {
-        return bitwise_cast<WriteBarrier<JSWebAssemblyCallee>*>(bitwise_cast<char*>(this) + offsetOfCallees());
-    }
-
-protected:
-    JSWebAssemblyModule(VM&, Structure*, std::unique_ptr<Wasm::ModuleInformation>&&, Bag<CallLinkInfo>&&, Vector<Wasm::WasmExitStubs>&&, unsigned calleeCount);
-    void finishCreation(VM&, SymbolTable*);
+    JSWebAssemblyModule(VM&, Structure*);
+    void finishCreation(VM&, ExecState*, uint8_t* source, size_t byteSize);
     static void destroy(JSCell*);
     static void visitChildren(JSCell*, SlotVisitor&);
 
-private:
-    static size_t offsetOfCallees()
-    {
-        return WTF::roundUpToMultipleOf<sizeof(WriteBarrier<JSWebAssemblyCallee>)>(sizeof(JSWebAssemblyModule));
-    }
-
-    static size_t allocationSize(unsigned numCallees)
-    {
-        return offsetOfCallees() + sizeof(WriteBarrier<JSWebAssemblyCallee>) * numCallees * 2;
-    }
-
-    class UnconditionalFinalizer : public JSC::UnconditionalFinalizer { 
-        void finalizeUnconditionally() override;
-    };
-
-    UnconditionalFinalizer m_unconditionalFinalizer;
+    RefPtr<ArrayBuffer> m_sourceBuffer;
     std::unique_ptr<Wasm::ModuleInformation> m_moduleInformation;
-    Bag<CallLinkInfo> m_callLinkInfos;
     WriteBarrier<SymbolTable> m_exportSymbolTable;
-    Vector<Wasm::WasmExitStubs> m_wasmExitStubs;
-    unsigned m_calleeCount;
+    WriteBarrier<JSWebAssemblyCodeBlock> m_codeBlocks[Wasm::Memory::NumberOfModes];
 };
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp

@@ -57 +57 @@
     const Wasm::Signature* signature = Wasm::SignatureInformation::get(&vm, signatureIndex);
 
+    // Make sure that the memory we think we are going to run with matches the one we expect.
+    ASSERT(wasmFunction->instance()->codeBlock()->isSafeToRun(wasmFunction->instance()->memory()));
     {
         // Check if we have a disallowed I64 use.

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyInstanceConstructor.cpp

@@ -85 +85 @@
     JSWebAssemblyInstance* instance = JSWebAssemblyInstance::create(vm, instanceStructure, jsModule, moduleRecord->getModuleNamespace(exec));
     RETURN_IF_EXCEPTION(throwScope, encodedJSValue());
-    {
-        // Always start with a dummy Memory, so that wasm -> wasm thunks avoid checking for a nullptr Memory when trying to set pinned registers.
-        Wasm::Memory memory;
-        instance->setMemory(vm, JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), WTFMove(memory)));
-    }
+
 
     // Let funcs, memories and tables be initially-empty lists of callable JavaScript objects, WebAssembly.Memory objects and WebAssembly.Table objects, respectively.
@@ -179 +175 @@
 
             Wasm::PageCount expectedInitial = moduleInformation.memory.initial();
-            Wasm::PageCount actualInitial = memory->memory()->initial();
+            Wasm::PageCount actualInitial = memory->memory().initial();
             if (actualInitial < expectedInitial)
                 return JSValue::encode(throwException(exec, throwScope, createJSWebAssemblyLinkError(exec, vm, ASCIILiteral("Memory import provided an 'initial' that is too small"))));
 
             if (Wasm::PageCount expectedMaximum = moduleInformation.memory.maximum()) {
-                Wasm::PageCount actualMaximum = memory->memory()->maximum();
+                Wasm::PageCount actualMaximum = memory->memory().maximum();
                 if (!actualMaximum) {
                     return JSValue::encode(
@@ -195 +191 @@
                 }
             }
+
             // ii. Append v to memories.
             // iii. Append v.[[Memory]] to imports.
-            instance->setMemory(vm, memory);
+            instance->setMemory(vm, exec, memory);
+            RETURN_IF_EXCEPTION(throwScope, encodedJSValue());
             break;
         }
@@ -238 +236 @@
             RELEASE_ASSERT(!moduleInformation.memory.isImport());
             // We create a memory when it's a memory definition.
-            bool failed;
-            Wasm::Memory memory(moduleInformation.memory.initial(), moduleInformation.memory.maximum(), failed);
-            if (failed)
-                return JSValue::encode(throwException(exec, throwScope, createOutOfMemoryError(exec)));
-            instance->setMemory(vm,
-               JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), WTFMove(memory)));
+            RefPtr<Wasm::Memory> memory;
+            if (moduleInformation.memory.hasReservedMemory())
+                memory = jsModule->takeReservedMemory();
+            else {
+                memory = Wasm::Memory::create(vm, moduleInformation.memory.initial(), moduleInformation.memory.maximum());
+                if (!memory)
+                    return JSValue::encode(throwException(exec, throwScope, createOutOfMemoryError(exec)));
+            }
+            instance->setMemory(vm, exec,
+                JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), memory.releaseNonNull()));
+            RETURN_IF_EXCEPTION(throwScope, encodedJSValue());
         }
     }
@@ -264 +267 @@
             instance->setTable(vm, table);
         }
+    }
+
+    if (!instance->memory()) {
+        // Make sure we have a dummy memory, so that wasm -> wasm thunks avoid checking for a nullptr Memory when trying to set pinned registers.
+        instance->setMemory(vm, exec, JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), adoptRef(*(new Wasm::Memory()))));
     }
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyMemoryConstructor.cpp

@@ -97 +97 @@
     }
 
-    bool failed;
-    Wasm::Memory memory(initialPageCount, maximumPageCount, failed);
-    if (failed)
+    RefPtr<Wasm::Memory> memory = Wasm::Memory::create(vm, initialPageCount, maximumPageCount);
+    if (!memory)
         return JSValue::encode(throwException(exec, throwScope, createOutOfMemoryError(exec)));
 
-    return JSValue::encode(JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), WTFMove(memory)));
+    return JSValue::encode(JSWebAssemblyMemory::create(vm, exec->lexicalGlobalObject()->WebAssemblyMemoryStructure(), adoptRef(*memory.leakRef())));
 }
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyModuleConstructor.cpp

@@ -81 +81 @@
     RETURN_IF_EXCEPTION(scope, { });
 
-    Wasm::Plan plan(&vm, base + byteOffset, byteSize);
-    // On failure, a new WebAssembly.CompileError is thrown.
-    plan.run();
-    if (plan.failed())
-        return throwException(state, scope, createJSWebAssemblyCompileError(state, vm, plan.errorMessage()));
-
-    // On success, a new WebAssembly.Module object is returned with [[Module]] set to the validated Ast.module.
-
-    // The export symbol table is the same for all Instances of a Module.
-    SymbolTable* exportSymbolTable = SymbolTable::create(vm);
-    for (auto& exp : plan.exports()) {
-        auto offset = exportSymbolTable->takeNextScopeOffset(NoLockingNecessary);
-        exportSymbolTable->set(NoLockingNecessary, exp.field.impl(), SymbolTableEntry(VarOffset(offset)));
-    }
-
-    // Only wasm-internal functions have a callee, stubs to JS do not.
-    unsigned calleeCount = plan.internalFunctionCount();
-    JSWebAssemblyModule* result = JSWebAssemblyModule::create(vm, structure, plan.takeModuleInformation(), plan.takeCallLinkInfos(), plan.takeWasmExitStubs(), exportSymbolTable, calleeCount);
-    plan.initializeCallees(state->jsCallee()->globalObject(), 
-        [&] (unsigned calleeIndex, JSWebAssemblyCallee* jsEntrypointCallee, JSWebAssemblyCallee* wasmEntrypointCallee) {
-            result->setJSEntrypointCallee(vm, calleeIndex, jsEntrypointCallee);
-            result->setWasmEntrypointCallee(vm, calleeIndex, wasmEntrypointCallee);
-        });
-
-    return result;
+    scope.release();
+    return JSWebAssemblyModule::create(vm, state, structure, base + byteOffset, byteSize);
 }
 

trunk/Source/JavaScriptCore/wasm/js/WebAssemblyModuleRecord.cpp

@@ -93 +93 @@
 
     JSWebAssemblyModule* module = instance->module();
+    JSWebAssemblyCodeBlock* codeBlock = instance->codeBlock();
     const Wasm::ModuleInformation& moduleInformation = module->moduleInformation();
 
     SymbolTable* exportSymbolTable = module->exportSymbolTable();
-    unsigned functionImportCount = module->functionImportCount();
+    unsigned functionImportCount = codeBlock->functionImportCount();
 
     // FIXME wire up the imports. https://bugs.webkit.org/show_bug.cgi?id=165118
@@ -117 +118 @@
             //     b. Append func to funcs.
             //     c. Return func.
-            JSWebAssemblyCallee* jsEntrypointCallee = module->jsEntrypointCalleeFromFunctionIndexSpace(exp.kindIndex);
-            JSWebAssemblyCallee* wasmEntrypointCallee = module->wasmEntrypointCalleeFromFunctionIndexSpace(exp.kindIndex);
+            JSWebAssemblyCallee* jsEntrypointCallee = codeBlock->jsEntrypointCalleeFromFunctionIndexSpace(exp.kindIndex);
+            JSWebAssemblyCallee* wasmEntrypointCallee = codeBlock->wasmEntrypointCalleeFromFunctionIndexSpace(exp.kindIndex);
             Wasm::SignatureIndex signatureIndex = module->signatureIndexFromFunctionIndexSpace(exp.kindIndex);
             const Wasm::Signature* signature = Wasm::SignatureInformation::get(&vm, signatureIndex);
@@ -183 +184 @@
         ASSERT(!signature->argumentCount());
         ASSERT(signature->returnType() == Wasm::Void);
-        if (startFunctionIndexSpace < module->functionImportCount()) {
+        if (startFunctionIndexSpace < codeBlock->functionImportCount()) {
             JSCell* startFunction = instance->importFunction(startFunctionIndexSpace)->get();
             m_startFunction.set(vm, this, startFunction);
         } else {
-            JSWebAssemblyCallee* jsEntrypointCallee = module->jsEntrypointCalleeFromFunctionIndexSpace(startFunctionIndexSpace);
-            JSWebAssemblyCallee* wasmEntrypointCallee = module->wasmEntrypointCalleeFromFunctionIndexSpace(startFunctionIndexSpace);
+            JSWebAssemblyCallee* jsEntrypointCallee = codeBlock->jsEntrypointCalleeFromFunctionIndexSpace(startFunctionIndexSpace);
+            JSWebAssemblyCallee* wasmEntrypointCallee = codeBlock->wasmEntrypointCalleeFromFunctionIndexSpace(startFunctionIndexSpace);
             WebAssemblyFunction* function = WebAssemblyFunction::create(vm, globalObject, signature->argumentCount(), "start", instance, jsEntrypointCallee, wasmEntrypointCallee, signatureIndex);
             m_startFunction.set(vm, this, function);
@@ -212 +213 @@
     {
         JSWebAssemblyModule* module = m_instance->module();
+        JSWebAssemblyCodeBlock* codeBlock = m_instance->codeBlock();
         const Wasm::ModuleInformation& moduleInformation = module->moduleInformation();
         JSWebAssemblyTable* table = m_instance->table();
@@ -234 +236 @@
                 // https://bugs.webkit.org/show_bug.cgi?id=165510
                 uint32_t functionIndex = element.functionIndices[i];
-                if (functionIndex < module->functionImportCount()) {
+                if (functionIndex < codeBlock->functionImportCount()) {
                     return JSValue::decode(
                         throwVMRangeError(state, scope, ASCIILiteral("Element is setting the table value with an import. This is not yet implemented. FIXME.")));
                 }
 
-                JSWebAssemblyCallee* jsEntrypointCallee = module->jsEntrypointCalleeFromFunctionIndexSpace(functionIndex);
-                JSWebAssemblyCallee* wasmEntrypointCallee = module->wasmEntrypointCalleeFromFunctionIndexSpace(functionIndex);
+                JSWebAssemblyCallee* jsEntrypointCallee = codeBlock->jsEntrypointCalleeFromFunctionIndexSpace(functionIndex);
+                JSWebAssemblyCallee* wasmEntrypointCallee = codeBlock->wasmEntrypointCalleeFromFunctionIndexSpace(functionIndex);
                 Wasm::SignatureIndex signatureIndex = module->signatureIndexFromFunctionIndexSpace(functionIndex);
                 const Wasm::Signature* signature = Wasm::SignatureInformation::get(&vm, signatureIndex);
@@ -260 +262 @@
         JSWebAssemblyMemory* jsMemory = m_instance->memory();
         if (!data.isEmpty()) {
-            uint8_t* memory = reinterpret_cast<uint8_t*>(jsMemory->memory()->memory());
-            uint64_t sizeInBytes = jsMemory->memory()->size();
+            uint8_t* memory = reinterpret_cast<uint8_t*>(jsMemory->memory().memory());
+            uint64_t sizeInBytes = jsMemory->memory().size();
             for (auto& segment : data) {
                 if (segment->sizeInBytes) {

trunk/Source/WTF/ChangeLog

@@ +1 @@
+2017-03-03  Keith Miller  <keith_miller@apple.com>
+
+        WASM should support faster loads.
+        https://bugs.webkit.org/show_bug.cgi?id=162693
+
+        Reviewed by Saam Barati.
+
+        Add new forms of dataLog that take a boolean which describes if the log should happen. This makes cases where we have a static const bool for printing nicer since you can do:
+
+        dataLogIf(verbose, things, to, print);
+
+        instead of:
+
+        if (verbose)
+            dataLog(things, to, print);
+
+        Also, add a operator! to Ref that has the same semantics as C++ refs.
+
+        * wtf/DataLog.h:
+        (WTF::dataLogLn):
+        (WTF::dataLogIf):
+        (WTF::dataLogLnIf):
+        * wtf/Ref.h:
+        (WTF::Ref::operator!):
+
 2017-03-02  Jer Noble  <jer.noble@apple.com>
 

trunk/Source/WTF/wtf/DataLog.h

@@ -50 +50 @@
 void dataLogLn(const Types&... values)
 {
-    dataFile().print(values..., "\n");
+    dataLog(values..., "\n");
+}
+
+template<typename... Types>
+void dataLogIf(bool shouldLog, const Types&... values)
+{
+    if (shouldLog)
+        dataLog(values...);
+}
+
+template<typename... Types>
+void dataLogLnIf(bool shouldLog, const Types&... values)
+{
+    if (shouldLog)
+        dataLogLn(values...);
 }
 
@@ -57 +71 @@
 using WTF::dataLog;
 using WTF::dataLogLn;
+using WTF::dataLogIf;
+using WTF::dataLogLnIf;
 using WTF::dataLogF;
 using WTF::dataLogFString;

trunk/Source/WTF/wtf/Ref.h

@@ -138 +138 @@
     T& get() const { ASSERT(m_ptr); return *m_ptr; }
     operator T&() const { ASSERT(m_ptr); return *m_ptr; }
+    bool operator!() const { ASSERT(m_ptr); return !*m_ptr; }
 
     template<typename U> Ref<T> replace(Ref<U>&&) WARN_UNUSED_RETURN;