Changeset 287801 in webkit

Timestamp:

Jan 7, 2022, 5:57:04 PM (5 years ago)

Author:

sbarati@apple.com

Message:

Add support for Wasm exceptions in the Air generator
​https://bugs.webkit.org/show_bug.cgi?id=231211
<rdar://problem/84132861>

Reviewed by Filip Pizlo.

This patch adds support to Air for Wasm exceptions. The implementation
is very similar to how we implement it in the B3 Wasm tier. This patch
shares code with the B3 tier where it can.

This patch also fixes a bug where you the early clobbered registers
of a patchpoint could prevent the prior instruction from register allocating.
For example, you can have the instructions I1, I2. Where I2 clobbers the
entire register file. It doesn't mean I1 shouldn't be able to allocate
registers. Instead, the clobber should occur after I1 executes. This patch
fixes the issue.

• JavaScriptCore.xcodeproj/project.pbxproj:
• b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp:

(JSC::B3::Air::GenerateAndAllocateRegisters::generate):

• wasm/WasmAirIRGenerator.cpp:

(JSC::Wasm::AirIRGenerator::ControlData::ControlData):
(JSC::Wasm::AirIRGenerator::ControlData::isTry):
(JSC::Wasm::AirIRGenerator::ControlData::isCatch):
(JSC::Wasm::AirIRGenerator::ControlData::convertTryToCatch):
(JSC::Wasm::AirIRGenerator::ControlData::convertTryToCatchAll):
(JSC::Wasm::AirIRGenerator::ControlData::tryStart const):
(JSC::Wasm::AirIRGenerator::ControlData::tryEnd const):
(JSC::Wasm::AirIRGenerator::ControlData::tryDepth const):
(JSC::Wasm::AirIRGenerator::ControlData::catchKind const):
(JSC::Wasm::AirIRGenerator::ControlData::exception const):
(JSC::Wasm::AirIRGenerator::emitCallPatchpoint):
(JSC::Wasm::AirIRGenerator::addStackMap):
(JSC::Wasm::AirIRGenerator::takeStackmaps):
(JSC::Wasm::AirIRGenerator::takeExceptionHandlers):
(JSC::Wasm::AirIRGenerator::newTmp):
(JSC::Wasm::AirIRGenerator::emitPatchpoint):
(JSC::Wasm::AirIRGenerator::emitLoad):
(JSC::Wasm::AirIRGenerator::AirIRGenerator):
(JSC::Wasm::AirIRGenerator::finalizeEntrypoints):
(JSC::Wasm::AirIRGenerator::forEachLiveValue):
(JSC::Wasm::AirIRGenerator::emitLoopTierUpCheck):
(JSC::Wasm::AirIRGenerator::addTry):
(JSC::Wasm::AirIRGenerator::addCatch):
(JSC::Wasm::AirIRGenerator::addCatchAll):
(JSC::Wasm::AirIRGenerator::addCatchToUnreachable):
(JSC::Wasm::AirIRGenerator::addCatchAllToUnreachable):
(JSC::Wasm::AirIRGenerator::emitCatchImpl):
(JSC::Wasm::AirIRGenerator::addDelegate):
(JSC::Wasm::AirIRGenerator::addDelegateToUnreachable):
(JSC::Wasm::AirIRGenerator::addThrow):
(JSC::Wasm::AirIRGenerator::addRethrow):
(JSC::Wasm::AirIRGenerator::addEndToUnreachable):
(JSC::Wasm::AirIRGenerator::addCall):
(JSC::Wasm::AirIRGenerator::emitIndirectCall):
(JSC::Wasm::parseAndCompileAir):
(JSC::Wasm::AirIRGenerator::preparePatchpointForExceptions):

• wasm/WasmB3IRGenerator.cpp:

(JSC::Wasm::B3IRGenerator::insertEntrySwitch):
(JSC::Wasm::B3IRGenerator::emitCatchImpl):
(JSC::Wasm::B3IRGenerator::addThrow):
(JSC::Wasm::B3IRGenerator::addRethrow):
(JSC::Wasm::PatchpointExceptionHandle::generate const): Deleted.
(JSC::Wasm::buildEntryBufferForCatch): Deleted.
(JSC::Wasm::computeExceptionHandlerLocations): Deleted.

• wasm/WasmB3IRGenerator.h:
• wasm/WasmBBQPlan.cpp:

(JSC::Wasm::BBQPlan::compileFunction):

• wasm/WasmIRGeneratorHelpers.h: Added.

(JSC::Wasm::PatchpointExceptionHandle::generate const):
(JSC::Wasm::computeExceptionHandlerLocations):
(JSC::Wasm::emitRethrowImpl):
(JSC::Wasm::emitThrowImpl):
(JSC::Wasm::buildEntryBufferForCatch):
(JSC::Wasm::emitCatchPrologueShared):

• wasm/WasmLLIntGenerator.cpp:

(JSC::Wasm::LLIntGenerator::finalize):

• wasm/WasmModuleInformation.h:
• wasm/WasmOMGPlan.cpp:
• wasm/WasmOSREntryPlan.cpp:
• wasm/WasmStreamingParser.cpp:

(JSC::Wasm::StreamingParser::parseCodeSectionSize):

Location:

trunk/Source/JavaScriptCore

Files:

• ChangeLog (modified) (1 diff)
• JavaScriptCore.xcodeproj/project.pbxproj (modified) (4 diffs)
• b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp (modified) (5 diffs)
• wasm/WasmAirIRGenerator.cpp (modified) (40 diffs)
• wasm/WasmB3IRGenerator.cpp (modified) (10 diffs)
• wasm/WasmB3IRGenerator.h (modified) (1 diff)
• wasm/WasmBBQPlan.cpp (modified) (2 diffs)
• wasm/WasmIRGeneratorHelpers.h (added)
• wasm/WasmLLIntGenerator.cpp (modified) (1 diff)
• wasm/WasmModuleInformation.h (modified) (1 diff)
• wasm/WasmOMGPlan.cpp (modified) (1 diff)
• wasm/WasmOSREntryPlan.cpp (modified) (1 diff)
• wasm/WasmStreamingParser.cpp (modified) (1 diff)

trunk/Source/JavaScriptCore/ChangeLog

@@ +1 @@
+2022-01-07  Saam Barati  <sbarati@apple.com>
+
+        Add support for Wasm exceptions in the Air generator
+        https://bugs.webkit.org/show_bug.cgi?id=231211
+        <rdar://problem/84132861>
+
+        Reviewed by Filip Pizlo.
+
+        This patch adds support to Air for Wasm exceptions. The implementation
+        is very similar to how we implement it in the B3 Wasm tier. This patch
+        shares code with the B3 tier where it can.
+        
+        This patch also fixes a bug where you the early clobbered registers
+        of a patchpoint could prevent the prior instruction from register allocating.
+        For example, you can have the instructions I1, I2. Where I2 clobbers the
+        entire register file. It doesn't mean I1 shouldn't be able to allocate
+        registers. Instead, the clobber should occur after I1 executes. This patch
+        fixes the issue.
+
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp:
+        (JSC::B3::Air::GenerateAndAllocateRegisters::generate):
+        * wasm/WasmAirIRGenerator.cpp:
+        (JSC::Wasm::AirIRGenerator::ControlData::ControlData):
+        (JSC::Wasm::AirIRGenerator::ControlData::isTry):
+        (JSC::Wasm::AirIRGenerator::ControlData::isCatch):
+        (JSC::Wasm::AirIRGenerator::ControlData::convertTryToCatch):
+        (JSC::Wasm::AirIRGenerator::ControlData::convertTryToCatchAll):
+        (JSC::Wasm::AirIRGenerator::ControlData::tryStart const):
+        (JSC::Wasm::AirIRGenerator::ControlData::tryEnd const):
+        (JSC::Wasm::AirIRGenerator::ControlData::tryDepth const):
+        (JSC::Wasm::AirIRGenerator::ControlData::catchKind const):
+        (JSC::Wasm::AirIRGenerator::ControlData::exception const):
+        (JSC::Wasm::AirIRGenerator::emitCallPatchpoint):
+        (JSC::Wasm::AirIRGenerator::addStackMap):
+        (JSC::Wasm::AirIRGenerator::takeStackmaps):
+        (JSC::Wasm::AirIRGenerator::takeExceptionHandlers):
+        (JSC::Wasm::AirIRGenerator::newTmp):
+        (JSC::Wasm::AirIRGenerator::emitPatchpoint):
+        (JSC::Wasm::AirIRGenerator::emitLoad):
+        (JSC::Wasm::AirIRGenerator::AirIRGenerator):
+        (JSC::Wasm::AirIRGenerator::finalizeEntrypoints):
+        (JSC::Wasm::AirIRGenerator::forEachLiveValue):
+        (JSC::Wasm::AirIRGenerator::emitLoopTierUpCheck):
+        (JSC::Wasm::AirIRGenerator::addTry):
+        (JSC::Wasm::AirIRGenerator::addCatch):
+        (JSC::Wasm::AirIRGenerator::addCatchAll):
+        (JSC::Wasm::AirIRGenerator::addCatchToUnreachable):
+        (JSC::Wasm::AirIRGenerator::addCatchAllToUnreachable):
+        (JSC::Wasm::AirIRGenerator::emitCatchImpl):
+        (JSC::Wasm::AirIRGenerator::addDelegate):
+        (JSC::Wasm::AirIRGenerator::addDelegateToUnreachable):
+        (JSC::Wasm::AirIRGenerator::addThrow):
+        (JSC::Wasm::AirIRGenerator::addRethrow):
+        (JSC::Wasm::AirIRGenerator::addEndToUnreachable):
+        (JSC::Wasm::AirIRGenerator::addCall):
+        (JSC::Wasm::AirIRGenerator::emitIndirectCall):
+        (JSC::Wasm::parseAndCompileAir):
+        (JSC::Wasm::AirIRGenerator::preparePatchpointForExceptions):
+        * wasm/WasmB3IRGenerator.cpp:
+        (JSC::Wasm::B3IRGenerator::insertEntrySwitch):
+        (JSC::Wasm::B3IRGenerator::emitCatchImpl):
+        (JSC::Wasm::B3IRGenerator::addThrow):
+        (JSC::Wasm::B3IRGenerator::addRethrow):
+        (JSC::Wasm::PatchpointExceptionHandle::generate const): Deleted.
+        (JSC::Wasm::buildEntryBufferForCatch): Deleted.
+        (JSC::Wasm::computeExceptionHandlerLocations): Deleted.
+        * wasm/WasmB3IRGenerator.h:
+        * wasm/WasmBBQPlan.cpp:
+        (JSC::Wasm::BBQPlan::compileFunction):
+        * wasm/WasmIRGeneratorHelpers.h: Added.
+        (JSC::Wasm::PatchpointExceptionHandle::generate const):
+        (JSC::Wasm::computeExceptionHandlerLocations):
+        (JSC::Wasm::emitRethrowImpl):
+        (JSC::Wasm::emitThrowImpl):
+        (JSC::Wasm::buildEntryBufferForCatch):
+        (JSC::Wasm::emitCatchPrologueShared):
+        * wasm/WasmLLIntGenerator.cpp:
+        (JSC::Wasm::LLIntGenerator::finalize):
+        * wasm/WasmModuleInformation.h:
+        * wasm/WasmOMGPlan.cpp:
+        * wasm/WasmOSREntryPlan.cpp:
+        * wasm/WasmStreamingParser.cpp:
+        (JSC::Wasm::StreamingParser::parseCodeSectionSize):
+
 2022-01-07  Alexey Shvayka  <ashvayka@apple.com>
 

trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -901 +901 @@
                 52EED7942492B870008F4C93 /* FunctionAllowlist.h in Headers */ = {isa = PBXBuildFile; fileRef = 52EED7932492B868008F4C93 /* FunctionAllowlist.h */; };
                 52F6C35E1E71EB080081F4CC /* WebAssemblyWrapperFunction.h in Headers */ = {isa = PBXBuildFile; fileRef = 52F6C35C1E71EB080081F4CC /* WebAssemblyWrapperFunction.h */; };
+                52FDABC32788076B00C15B59 /* WasmIRGeneratorHelpers.h in Headers */ = {isa = PBXBuildFile; fileRef = 52FDABC22788076900C15B59 /* WasmIRGeneratorHelpers.h */; };
                 530A66B91FA3E78B0026A545 /* UnifiedSource3-mm.mm in Sources */ = {isa = PBXBuildFile; fileRef = 530A66B11FA3E77A0026A545 /* UnifiedSource3-mm.mm */; };
                 530A66BA1FA3E78B0026A545 /* UnifiedSource4-mm.mm in Sources */ = {isa = PBXBuildFile; fileRef = 530A66B81FA3E77E0026A545 /* UnifiedSource4-mm.mm */; };
@@ -3761 +3762 @@
                 52F6C35B1E71EB080081F4CC /* WebAssemblyWrapperFunction.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = WebAssemblyWrapperFunction.cpp; path = js/WebAssemblyWrapperFunction.cpp; sourceTree = "<group>"; };
                 52F6C35C1E71EB080081F4CC /* WebAssemblyWrapperFunction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WebAssemblyWrapperFunction.h; path = js/WebAssemblyWrapperFunction.h; sourceTree = "<group>"; };
+                52FDABC22788076900C15B59 /* WasmIRGeneratorHelpers.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmIRGeneratorHelpers.h; sourceTree = "<group>"; };
                 5300740C22DD6F6600B9ACB3 /* JSFinalizationRegistry.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = JSFinalizationRegistry.cpp; sourceTree = "<group>"; };
                 530A63401FA3E31C0026A545 /* SourcesCocoa.txt */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = SourcesCocoa.txt; sourceTree = "<group>"; };
@@ -7306 +7308 @@
                                 148521D526EAEBDF00CC1D1A /* WasmHandlerInfo.cpp */,
                                 148521D726EAEBFE00CC1D1A /* WasmHandlerInfo.h */,
+                                52FDABC22788076900C15B59 /* WasmIRGeneratorHelpers.h */,
                                 AD8FF3961EB5BD850087FF82 /* WasmIndexOrName.cpp */,
                                 AD8FF3951EB5BD850087FF82 /* WasmIndexOrName.h */,
@@ -11161 +11164 @@
                                 E393ADD81FE702D00022D681 /* WeakMapImplInlines.h in Headers */,
                                 A7CA3AE617DA41AE006538AF /* WeakMapPrototype.h in Headers */,
+                                52FDABC32788076B00C15B59 /* WasmIRGeneratorHelpers.h in Headers */,
                                 539930C822AD3B9A0051CDE2 /* WeakObjectRefConstructor.h in Headers */,
                                 539BFBAE22AD3C3A0023F4C0 /* WeakObjectRefPrototype.h in Headers */,

trunk/Source/JavaScriptCore/b3/air/AirAllocateRegistersAndStackAndGenerateCode.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2019 Apple Inc. All rights reserved.
+ * Copyright (C) 2019-2022 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -613 +613 @@
 
             RegisterSet clobberedRegisters;
+            RegisterSet earlyNextClobberedRegisters;
             {
                 Inst* nextInst = block->get(instIndex + 1);
@@ -619 +620 @@
                         clobberedRegisters.merge(inst.extraClobberedRegs());
                     if (nextInst && nextInst->kind.opcode == Patch)
-                        clobberedRegisters.merge(nextInst->extraEarlyClobberedRegs());
+                        earlyNextClobberedRegisters.merge(nextInst->extraEarlyClobberedRegs());
 
                     clobberedRegisters.filter(m_allowedRegisters);
                     clobberedRegisters.exclude(m_namedDefdRegs);
+                    earlyNextClobberedRegisters.filter(m_allowedRegisters);
 
                     m_namedDefdRegs.merge(clobberedRegisters);
@@ -740 +742 @@
             }
 
-            if (inst.isTerminal() && block->numSuccessors()) {
-                // By default, we spill everything between block boundaries. However, we have a small
-                // heuristic to pass along register state. We should eventually make this better.
-                // What we do now is if we have a successor with a single predecessor (us), and we
-                // haven't yet generated code for it, we give it our register state. If all our successors
-                // can take on our register state, we don't flush at the end of this block.
-
-                bool everySuccessorGetsOurRegisterState = true;
-                for (unsigned i = 0; i < block->numSuccessors(); ++i) {
-                    BasicBlock* successor = block->successorBlock(i);
-                    if (successor->numPredecessors() == 1 && !context.blockLabels[successor]->isSet())
-                        currentAllocationMap[successor] = currentAllocation;
-                    else
-                        everySuccessorGetsOurRegisterState = false;
-                }
-                if (!everySuccessorGetsOurRegisterState) {
-                    for (Tmp tmp : m_liveness->liveAtTail(block)) {
-                        if (tmp.isReg() && isDisallowedRegister(tmp.reg()))
-                            continue;
-                        if (Reg reg = m_map[tmp].reg)
-                            flush(tmp, reg);
-                    }
-                }
-            }
+            auto clobber = [&] (const RegisterSet& set) {
+                for (Reg reg : set) {
+                    Tmp tmp(reg);
+                    ASSERT(currentAllocation[reg] == tmp);
+                    m_availableRegs[tmp.bank()].set(reg);
+                    m_currentAllocation->at(reg) = Tmp();
+                    m_map[tmp].reg = Reg();
+                }
+            };
 
             if (!inst.isTerminal()) {
@@ -771 +758 @@
                 ASSERT_UNUSED(jump, !jump.isSet());
 
-                for (Reg reg : clobberedRegisters) {
-                    Tmp tmp(reg);
-                    ASSERT(currentAllocation[reg] == tmp);
-                    m_availableRegs[tmp.bank()].set(reg);
-                    m_currentAllocation->at(reg) = Tmp();
-                    m_map[tmp].reg = Reg();
-                }
+                allocNamed(earlyNextClobberedRegisters, true);
+                clobber(clobberedRegisters);
+                clobber(earlyNextClobberedRegisters);
             } else {
                 ASSERT(needsToGenerate);
+
+                clobber(clobberedRegisters);
+                ASSERT(earlyNextClobberedRegisters.isEmpty());
+
+                if (block->numSuccessors()) {
+                    // By default, we spill everything between block boundaries. However, we have a small
+                    // heuristic to pass along register state. We should eventually make this better.
+                    // What we do now is if we have a successor with a single predecessor (us), and we
+                    // haven't yet generated code for it, we give it our register state. If all our successors
+                    // can take on our register state, we don't flush at the end of this block.
+
+
+                    bool everySuccessorGetsOurRegisterState = true;
+                    for (unsigned i = 0; i < block->numSuccessors(); ++i) {
+                        BasicBlock* successor = block->successorBlock(i);
+                        if (successor->numPredecessors() == 1 && !context.blockLabels[successor]->isSet())
+                            currentAllocationMap[successor] = currentAllocation;
+                        else
+                            everySuccessorGetsOurRegisterState = false;
+                    }
+                    if (!everySuccessorGetsOurRegisterState) {
+                        for (Tmp tmp : m_liveness->liveAtTail(block)) {
+                            if (tmp.isReg() && isDisallowedRegister(tmp.reg()))
+                                continue;
+                            if (Reg reg = m_map[tmp].reg)
+                                flush(tmp, reg);
+                        }
+                    }
+                }
+
                 if (inst.kind.opcode == Jump && block->successorBlock(0) == m_code.findNextBlock(block))
                     needsToGenerate = false;

trunk/Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2019-2021 Apple Inc. All rights reserved.
+ * Copyright (C) 2019-2022 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -49 +49 @@
 #include "WasmExceptionType.h"
 #include "WasmFunctionParser.h"
+#include "WasmIRGeneratorHelpers.h"
 #include "WasmInstance.h"
 #include "WasmMemory.h"
@@ -133 +134 @@
 
     struct ControlData {
-        ControlData(B3::Origin origin, BlockSignature result, ResultList resultTmps, BlockType type, BasicBlock* continuation, BasicBlock* special = nullptr)
+        ControlData(B3::Origin, BlockSignature result, ResultList resultTmps, BlockType type, BasicBlock* continuation, BasicBlock* special = nullptr)
             : controlBlockType(type)
             , continuation(continuation)
@@ -140 +141 @@
             , returnType(result)
         {
-            UNUSED_PARAM(origin);
+        }
+
+        ControlData(B3::Origin, BlockSignature result, ResultList resultTmps, BlockType type, BasicBlock* continuation, unsigned tryStart, unsigned tryDepth)
+            : controlBlockType(type)
+            , continuation(continuation)
+            , special(nullptr)
+            , results(resultTmps)
+            , returnType(result)
+            , m_tryStart(tryStart)
+            , m_tryCatchDepth(tryDepth)
+        {
         }
 
@@ -149 +160 @@
         static bool isIf(const ControlData& control) { return control.blockType() == BlockType::If; }
         static bool isTry(const ControlData& control) { return control.blockType() == BlockType::Try; }
-        static bool isCatch(const ControlData& control) { return control.blockType() == BlockType::Catch; }
         static bool isAnyCatch(const ControlData& control) { return control.blockType() == BlockType::Catch; }
+        static bool isCatch(const ControlData& control) { return isAnyCatch(control) && control.catchKind() == CatchKind::Catch; }
         static bool isTopLevel(const ControlData& control) { return control.blockType() == BlockType::TopLevel; }
         static bool isLoop(const ControlData& control) { return control.blockType() == BlockType::Loop; }
@@ -222 +233 @@
         }
 
+        void convertTryToCatch(unsigned tryEndCallSiteIndex, TypedTmp exception)
+        {
+            ASSERT(blockType() == BlockType::Try);
+            controlBlockType = BlockType::Catch;
+            m_catchKind = CatchKind::Catch;
+            m_tryEnd = tryEndCallSiteIndex;
+            m_exception = exception;
+        }
+
+        void convertTryToCatchAll(unsigned tryEndCallSiteIndex, TypedTmp exception)
+        {
+            ASSERT(blockType() == BlockType::Try);
+            controlBlockType = BlockType::Catch;
+            m_catchKind = CatchKind::CatchAll;
+            m_tryEnd = tryEndCallSiteIndex;
+            m_exception = exception;
+        }
+
+        unsigned tryStart() const
+        {
+            ASSERT(controlBlockType == BlockType::Try || controlBlockType == BlockType::Catch);
+            return m_tryStart;
+        }
+
+        unsigned tryEnd() const
+        {
+            ASSERT(controlBlockType == BlockType::Catch);
+            return m_tryEnd;
+        }
+
+        unsigned tryDepth() const
+        {
+            ASSERT(controlBlockType == BlockType::Try || controlBlockType == BlockType::Catch);
+            return m_tryCatchDepth;
+        }
+
+        CatchKind catchKind() const
+        {
+            ASSERT(controlBlockType == BlockType::Catch);
+            return m_catchKind;
+        }
+
+        TypedTmp exception() const
+        {
+            ASSERT(controlBlockType == BlockType::Catch);
+            return m_exception;
+        }
+
     private:
         friend class AirIRGenerator;
@@ -229 +288 @@
         ResultList results;
         BlockSignature returnType;
+        unsigned m_tryStart;
+        unsigned m_tryEnd;
+        unsigned m_tryCatchDepth;
+        CatchKind m_catchKind;
+        TypedTmp m_exception;
     };
 
@@ -260 +324 @@
 
     AirIRGenerator(const ModuleInformation&, B3::Procedure&, InternalFunction*, Vector<UnlinkedWasmToWasmCall>&, MemoryMode, unsigned functionIndex, TierUpCount*, const Signature&);
+
+    void finalizeEntrypoints();
 
     PartialResult WARN_UNUSED_RETURN addArguments(const Signature&);
@@ -350 +416 @@
     PartialResult WARN_UNUSED_RETURN addUnreachable();
     PartialResult WARN_UNUSED_RETURN emitIndirectCall(TypedTmp calleeInstance, ExpressionType calleeCode, const Signature&, const Vector<ExpressionType>& args, ResultList&);
-    B3::PatchpointValue* WARN_UNUSED_RETURN emitCallPatchpoint(BasicBlock*, const Signature&, const ResultList& results, const Vector<TypedTmp>& args, Vector<ConstrainedTmp>&& extraArgs = { });
+    std::pair<B3::PatchpointValue*, PatchpointExceptionHandle> WARN_UNUSED_RETURN emitCallPatchpoint(BasicBlock*, const Signature&, const ResultList& results, const Vector<TypedTmp>& args, Vector<ConstrainedTmp> extraArgs = { });
 
     PartialResult addShift(Type, B3::Air::Opcode, ExpressionType value, ExpressionType shift, ExpressionType& result);
@@ -362 +428 @@
     void didPopValueFromStack() { }
 
+    Tmp emitCatchImpl(CatchKind, ControlType&, unsigned exceptionIndex = 0);
+    template <size_t inlineCapacity>
+    PatchpointExceptionHandle preparePatchpointForExceptions(B3::PatchpointValue*, Vector<ConstrainedTmp, inlineCapacity>& args);
+
     const Bag<B3::PatchpointValue*>& patchpoints() const
     {
         return m_patchpoints;
+    }
+
+    void addStackMap(unsigned callSiteIndex, StackMap&& stackmap)
+    {
+        m_stackmaps.add(CallSiteIndex(callSiteIndex), WTFMove(stackmap));
+    }
+
+    StackMaps&& takeStackmaps()
+    {
+        return WTFMove(m_stackmaps);
+    }
+
+    Vector<UnlinkedHandlerInfo>&& takeExceptionHandlers()
+    {
+        return WTFMove(m_exceptionHandlers);
     }
 
@@ -418 +503 @@
     Tmp newTmp(B3::Bank bank)
     {
-        switch (bank) {
-        case B3::GP:
-            if (m_freeGPs.size())
-                return m_freeGPs.takeLast();
-            break;
-        case B3::FP:
-            if (m_freeFPs.size())
-                return m_freeFPs.takeLast();
-            break;
-        }
         return m_code.newTmp(bank);
     }
@@ -497 +572 @@
     }
 
+    template <size_t inlineSize>
+    void emitPatchpoint(BasicBlock* basicBlock, B3::PatchpointValue* patch, Tmp result, Vector<ConstrainedTmp, inlineSize>&& args)
+    {
+        emitPatchpoint(basicBlock, patch, Vector<Tmp, 8> { result }, WTFMove(args));
+    }
+
     template <typename ResultTmpType, size_t inlineSize>
-    void emitPatchpoint(BasicBlock* basicBlock, B3::PatchpointValue* patch, const Vector<ResultTmpType, 8>&  results, Vector<ConstrainedTmp, inlineSize>&& args)
+    void emitPatchpoint(BasicBlock* basicBlock, B3::PatchpointValue* patch, const Vector<ResultTmpType, 8>& results, Vector<ConstrainedTmp, inlineSize>&& args)
     {
         if (!m_patchpointSpecial)
@@ -550 +631 @@
             patch->append(dummyValue, tmp.rep);
             switch (tmp.rep.kind()) {
-            case B3::ValueRep::ColdAny: // B3::Value propagates ColdAny information and later Air will allocate appropriate stack.
+            // B3::Value propagates (Late)ColdAny information and later Air will allocate appropriate stack.
+            case B3::ValueRep::ColdAny: 
+            case B3::ValueRep::LateColdAny:
             case B3::ValueRep::SomeRegister:
                 inst.args.append(tmp.tmp);
@@ -560 +643 @@
                 break;
             case B3::ValueRep::StackArgument: {
-                ASSERT(!patch->effects.terminal);
                 Arg arg = Arg::callArg(tmp.rep.offsetFromSP());
                 append(basicBlock, tmp.tmp.isGP() ? Move : MoveDouble, tmp.tmp, arg);
@@ -690 +772 @@
     }
 
+    void emitLoad(B3::Air::Opcode op, B3::Type type, Tmp base, size_t offset, Tmp result)
+    {
+        if (Arg::isValidAddrForm(offset, B3::widthForType(type)))
+            append(op, Arg::addr(base, offset), result);
+        else {
+            auto temp2 = g64();
+            append(Move, Arg::bigImm(offset), temp2);
+            append(Add64, temp2, base, temp2);
+            append(op, Arg::addr(temp2), result);
+        }
+    }
+
+    void emitLoad(Tmp base, size_t offset, TypedTmp result)
+    {
+        emitLoad(moveOpForValueType(result.type()), toB3Type(result.type()), base, offset, result.tmp());
+    }
+
     void emitThrowException(CCallHelpers&, ExceptionType);
 
@@ -738 +837 @@
     }
 
+    template <typename Function>
+    void forEachLiveValue(Function);
+
     FunctionParser<AirIRGenerator>* m_parser { nullptr };
     const ModuleInformation& m_info;
@@ -749 +851 @@
     BasicBlock* m_currentBlock { nullptr };
     BasicBlock* m_rootBlock { nullptr };
+    BasicBlock* m_mainEntrypointStart { nullptr };
     Vector<TypedTmp> m_locals;
     Vector<UnlinkedWasmToWasmCall>& m_unlinkedWasmToWasmCalls; // List each call site and the function index whose address it should be patched with.
@@ -754 +857 @@
     GPRReg m_boundsCheckingSizeGPR { InvalidGPRReg };
     GPRReg m_wasmContextInstanceGPR { InvalidGPRReg };
+    GPRReg m_prologueWasmContextGPR { InvalidGPRReg };
     bool m_makesCalls { false };
-
-    Vector<Tmp, 8> m_freeGPs;
-    Vector<Tmp, 8> m_freeFPs;
 
     HashMap<BlockSignature, B3::Type> m_tupleMap;
@@ -775 +876 @@
 
     B3::PatchpointSpecial* m_patchpointSpecial { nullptr };
+
+    RefPtr<B3::Air::PrologueGenerator> m_prologueGenerator;
+
+    Vector<BasicBlock*> m_catchEntrypoints;
+
+    Checked<unsigned> m_tryCatchDepth { 0 };
+    Checked<unsigned> m_callSiteIndex { 0 };
+    StackMaps m_stackmaps;
+    Vector<UnlinkedHandlerInfo> m_exceptionHandlers;
 };
 
@@ -848 +958 @@
     }
 
-    m_code.setNumEntrypoints(1);
-
-    GPRReg contextInstance = Context::useFastTLS() ? wasmCallingConvention().prologueScratchGPRs[1] : m_wasmContextInstanceGPR;
-
-    Ref<B3::Air::PrologueGenerator> prologueGenerator = createSharedTask<B3::Air::PrologueGeneratorFunction>([=] (CCallHelpers& jit, B3::Air::Code& code) {
+    m_prologueWasmContextGPR = Context::useFastTLS() ? wasmCallingConvention().prologueScratchGPRs[1] : m_wasmContextInstanceGPR;
+
+    m_prologueGenerator = createSharedTask<B3::Air::PrologueGeneratorFunction>([=] (CCallHelpers& jit, B3::Air::Code& code) {
         AllowMacroScratchRegisterUsage allowScratch(jit);
         code.emitDefaultPrologue(jit);
@@ -892 +1000 @@
 
                 if (Context::useFastTLS())
-                    jit.loadWasmContextInstance(contextInstance);
+                    jit.loadWasmContextInstance(m_prologueWasmContextGPR);
 
                 jit.addPtr(CCallHelpers::TrustedImm32(-checkSize), GPRInfo::callFrameRegister, scratch);
@@ -898 +1006 @@
                 if (UNLIKELY(needUnderflowCheck))
                     overflow.append(jit.branchPtr(CCallHelpers::Above, scratch, GPRInfo::callFrameRegister));
-                overflow.append(jit.branchPtr(CCallHelpers::Below, scratch, CCallHelpers::Address(contextInstance, Instance::offsetOfCachedStackLimit())));
+                overflow.append(jit.branchPtr(CCallHelpers::Below, scratch, CCallHelpers::Address(m_prologueWasmContextGPR, Instance::offsetOfCachedStackLimit())));
                 jit.addLinkTask([overflow] (LinkBuffer& linkBuffer) {
                     linkBuffer.link(overflow, CodeLocationLabel<JITThunkPtrTag>(Thunks::singleton().stub(throwStackOverflowFromWasmThunkGenerator).code()));
@@ -904 +1012 @@
             } else if (m_usesInstanceValue && Context::useFastTLS()) {
                 // No overflow check is needed, but the instance values still needs to be correct.
-                jit.loadWasmContextInstance(contextInstance);
+                jit.loadWasmContextInstance(m_prologueWasmContextGPR);
             }
-        }
-    });
-
-    m_code.setPrologueForEntrypoint(0, WTFMove(prologueGenerator));
+
+            if (m_catchEntrypoints.size()) {
+                GPRReg scratch = wasmCallingConvention().prologueScratchGPRs[0];
+                jit.loadPtr(CCallHelpers::Address(m_prologueWasmContextGPR, Instance::offsetOfOwner()), scratch);
+                jit.store64(scratch, CCallHelpers::Address(GPRInfo::callFrameRegister, CallFrameSlot::thisArgument * sizeof(Register)));
+            }
+        }
+    });
 
     if (Context::useFastTLS()) {
         m_instanceValue = g64();
         // FIXME: Would be nice to only do this if we use instance value.
-        append(Move, Tmp(contextInstance), m_instanceValue);
+        append(Move, Tmp(m_prologueWasmContextGPR), m_instanceValue);
     } else
-        m_instanceValue = { Tmp(contextInstance), Types::I64 };
+        m_instanceValue = { Tmp(m_prologueWasmContextGPR), Types::I64 };
+
+    append(EntrySwitch);
+    m_mainEntrypointStart = m_code.addBlock();
+    m_currentBlock = m_mainEntrypointStart;
 
     ASSERT(!m_locals.size());
@@ -955 +1071 @@
 }
 
+void AirIRGenerator::finalizeEntrypoints()
+{
+    unsigned numEntrypoints = 1 + m_catchEntrypoints.size();
+    m_proc.setNumEntrypoints(numEntrypoints);
+    m_code.setPrologueForEntrypoint(0, Ref<B3::Air::PrologueGenerator>(*m_prologueGenerator));
+
+    if (m_catchEntrypoints.size()) {
+        Ref<B3::Air::PrologueGenerator> catchPrologueGenerator = createSharedTask<B3::Air::PrologueGeneratorFunction>([this] (CCallHelpers& jit, B3::Air::Code& code) {
+            AllowMacroScratchRegisterUsage allowScratch(jit);
+            emitCatchPrologueShared(code, jit);
+
+            if (Context::useFastTLS()) {
+                // Shared prologue expects this in this register when entering the function using fast TLS.
+                jit.loadWasmContextInstance(m_prologueWasmContextGPR);
+            }
+        });
+
+        for (unsigned i = 0; i < m_catchEntrypoints.size(); ++i)
+            m_code.setPrologueForEntrypoint(1 + i, catchPrologueGenerator.copyRef());
+    }
+
+    BasicBlock::SuccessorList successors;
+    successors.append(m_mainEntrypointStart);
+    successors.appendVector(m_catchEntrypoints);
+
+    RELEASE_ASSERT(numEntrypoints == successors.size());
+    m_rootBlock->successors() = successors;
+}
+
 B3::Type AirIRGenerator::toB3ResultType(BlockSignature returnType)
 {
@@ -1026 +1171 @@
         linkBuffer.link(jumpToExceptionStub, CodeLocationLabel<JITThunkPtrTag>(Thunks::singleton().stub(throwExceptionFromWasmThunkGenerator).code()));
     });
+}
+
+template <typename Function>
+void AirIRGenerator::forEachLiveValue(Function function)
+{
+    for (const auto& local : m_locals)
+        function(local);
+    for (unsigned controlIndex = 0; controlIndex < m_parser->controlStack().size(); ++controlIndex) {
+        ControlData& data = m_parser->controlStack()[controlIndex].controlData;
+        Stack& expressionStack = m_parser->controlStack()[controlIndex].enclosedExpressionStack;
+        for (const auto& tmp : expressionStack)
+            function(tmp.value());
+        if (ControlType::isAnyCatch(data))
+            function(data.exception());
+    }
 }
 
@@ -2904 +3064 @@
     patchArgs.append(countdownPtr);
 
-    for (auto& local : m_locals)
-        patchArgs.append(ConstrainedTmp(local, B3::ValueRep::ColdAny));
-    for (unsigned controlIndex = 0; controlIndex < m_parser->controlStack().size(); ++controlIndex) {
-        Stack& expressionStack = m_parser->controlStack()[controlIndex].enclosedExpressionStack;
-        for (TypedExpression value : expressionStack)
-            patchArgs.append(ConstrainedTmp(value.value(), B3::ValueRep::ColdAny));
-    }
+    forEachLiveValue([&] (Tmp tmp) {
+        patchArgs.append(ConstrainedTmp(tmp, B3::ValueRep::ColdAny));
+    });
     for (TypedExpression value : enclosingStack)
         patchArgs.append(ConstrainedTmp(value.value(), B3::ValueRep::ColdAny));
@@ -3013 +3169 @@
 }
 
-// FIXME: Add support for Wasm exceptions in the Air generator
-// https://bugs.webkit.org/show_bug.cgi?id=231211
-auto AirIRGenerator::addTry(BlockSignature, Stack&, ControlType&, Stack&) -> PartialResult
-{
-    return { };
-}
-
-auto AirIRGenerator::addCatch(unsigned, const Signature&, Stack&, ControlType&, ResultList&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addCatchToUnreachable(unsigned, const Signature&, ControlType&, ResultList&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addCatchAll(Stack&, ControlType&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addCatchAllToUnreachable(ControlType&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addDelegate(ControlType&, ControlType&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addDelegateToUnreachable(ControlType&, ControlType&) -> PartialResult
-{
-    RELEASE_ASSERT_NOT_REACHED();
-    return { };
-}
-
-auto AirIRGenerator::addThrow(unsigned exceptionIndex, Vector<ExpressionType>&, Stack&) -> PartialResult
-{
-    UNUSED_PARAM(exceptionIndex);
-    return { };
-}
-
-auto AirIRGenerator::addRethrow(unsigned, ControlType&) -> PartialResult
-{
+auto AirIRGenerator::addTry(BlockSignature signature, Stack& enclosingStack, ControlType& result, Stack& newStack) -> PartialResult
+{
+    ++m_tryCatchDepth;
+
+    BasicBlock* continuation = m_code.addBlock();
+    splitStack(signature, enclosingStack, newStack);
+    result = ControlData(origin(), signature, tmpsForSignature(signature), BlockType::Try, continuation, ++m_callSiteIndex, m_tryCatchDepth);
+    return { };
+}
+
+auto AirIRGenerator::addCatch(unsigned exceptionIndex, const Signature& signature, Stack& currentStack, ControlType& data, ResultList& results) -> PartialResult
+{
+    unifyValuesWithBlock(currentStack, data.results);
+    append(Jump);
+    m_currentBlock->setSuccessors(data.continuation);
+    return addCatchToUnreachable(exceptionIndex, signature, data, results);
+}
+
+auto AirIRGenerator::addCatchAll(Stack& currentStack, ControlType& data) -> PartialResult
+{
+    unifyValuesWithBlock(currentStack, data.results);
+    append(Jump);
+    m_currentBlock->setSuccessors(data.continuation);
+    return addCatchAllToUnreachable(data);
+}
+
+auto AirIRGenerator::addCatchToUnreachable(unsigned exceptionIndex, const Signature& signature, ControlType& data, ResultList& results) -> PartialResult
+{
+    Tmp buffer = emitCatchImpl(CatchKind::Catch, data, exceptionIndex);
+    for (unsigned i = 0; i < signature.argumentCount(); ++i) {
+        Type type = signature.argument(i);
+        TypedTmp tmp = tmpForType(type);
+        emitLoad(buffer, i * sizeof(uint64_t), tmp);
+        results.append(tmp);
+    }
+    return { };
+}
+
+auto AirIRGenerator::addCatchAllToUnreachable(ControlType& data) -> PartialResult
+{
+    emitCatchImpl(CatchKind::CatchAll, data);
+    return { };
+}
+
+Tmp AirIRGenerator::emitCatchImpl(CatchKind kind, ControlType& data, unsigned exceptionIndex)
+{
+    m_currentBlock = m_code.addBlock();
+    m_catchEntrypoints.append(m_currentBlock);
+
+    if (ControlType::isTry(data)) {
+        if (kind == CatchKind::Catch)
+            data.convertTryToCatch(++m_callSiteIndex, g64());
+        else
+            data.convertTryToCatchAll(++m_callSiteIndex, g64());
+    }
+    // We convert from "try" to "catch" ControlType above. This doesn't
+    // happen if ControlType is already a "catch". This can happen when
+    // we have multiple catches like "try {} catch(A){} catch(B){}...CatchAll(E){}".
+    // We just convert the first ControlType to a catch, then the others will
+    // use its fields.
+    ASSERT(ControlType::isAnyCatch(data));
+
+    HandlerType handlerType = kind == CatchKind::Catch ? HandlerType::Catch : HandlerType::CatchAll;
+    m_exceptionHandlers.append({ handlerType, data.tryStart(), data.tryEnd(), 0, m_tryCatchDepth, exceptionIndex });
+
+    restoreWebAssemblyGlobalState(RestoreCachedStackLimit::Yes, m_info.memory, instanceValue(), m_currentBlock);
+
+    unsigned indexInBuffer = 0;
+    auto loadFromScratchBuffer = [&] (TypedTmp result) {
+        size_t offset = sizeof(uint64_t) * indexInBuffer;
+        ++indexInBuffer;
+        Tmp bufferPtr = Tmp(GPRInfo::argumentGPR0);
+        emitLoad(bufferPtr, offset, result);
+    };
+    forEachLiveValue([&] (TypedTmp tmp) {
+        // We set our current ControlEntry's exception below after the patchpoint, it's
+        // not in the incoming buffer of live values.
+        auto toIgnore = data.exception();
+        if (tmp.tmp() != toIgnore.tmp())
+            loadFromScratchBuffer(tmp);
+    });
+
+    B3::PatchpointValue* patch = addPatchpoint(m_proc.addTuple({ B3::pointerType(), B3::pointerType() }));
+    patch->effects.exitsSideways = true;
+    patch->clobber(RegisterSet::macroScratchRegisters());
+    RegisterSet clobberLate = RegisterSet::volatileRegistersForJSCall();
+    clobberLate.add(GPRInfo::argumentGPR0);
+    patch->clobberLate(clobberLate);
+    patch->resultConstraints.append(B3::ValueRep::reg(GPRInfo::returnValueGPR));
+    patch->resultConstraints.append(B3::ValueRep::reg(GPRInfo::returnValueGPR2));
+    patch->setGenerator([=] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
+        AllowMacroScratchRegisterUsage allowScratch(jit);
+        jit.move(params[2].gpr(), GPRInfo::argumentGPR0);
+        CCallHelpers::Call call = jit.call(OperationPtrTag);
+        jit.addLinkTask([call] (LinkBuffer& linkBuffer) {
+            linkBuffer.link(call, FunctionPtr<OperationPtrTag>(operationWasmRetrieveAndClearExceptionIfCatchable));
+        });
+    });
+
+    Tmp exception = Tmp(GPRInfo::returnValueGPR);
+    Tmp buffer = Tmp(GPRInfo::returnValueGPR2);
+    emitPatchpoint(m_currentBlock, patch, Vector<Tmp, 8>::from(exception, buffer), Vector<ConstrainedTmp, 1>::from(instanceValue()));
+    append(Move, exception, data.exception());
+
+    return buffer;
+}
+
+auto AirIRGenerator::addDelegate(ControlType& target, ControlType& data) -> PartialResult
+{
+    return addDelegateToUnreachable(target, data);
+}
+
+auto AirIRGenerator::addDelegateToUnreachable(ControlType& target, ControlType& data) -> PartialResult
+{
+    unsigned targetDepth = 0;
+    if (ControlType::isTry(target))
+        targetDepth = target.tryDepth();
+
+    m_exceptionHandlers.append({ HandlerType::Delegate, data.tryStart(), ++m_callSiteIndex, 0, m_tryCatchDepth, targetDepth });
+    return { };
+}
+
+auto AirIRGenerator::addThrow(unsigned exceptionIndex, Vector<ExpressionType>& args, Stack&) -> PartialResult
+{
+    B3::PatchpointValue* patch = addPatchpoint(B3::Void);
+    patch->effects.terminal = true;
+    patch->clobber(RegisterSet::volatileRegistersForJSCall());
+
+    Vector<ConstrainedTmp, 8> patchArgs;
+    patchArgs.append(ConstrainedTmp(instanceValue(), B3::ValueRep::reg(GPRInfo::argumentGPR0)));
+    patchArgs.append(ConstrainedTmp(Tmp(GPRInfo::callFrameRegister), B3::ValueRep::reg(GPRInfo::argumentGPR1)));
+    for (unsigned i = 0; i < args.size(); ++i)
+        patchArgs.append(ConstrainedTmp(args[i], B3::ValueRep::stackArgument(i * sizeof(EncodedJSValue))));
+
+    PatchpointExceptionHandle handle = preparePatchpointForExceptions(patch, patchArgs);
+
+    patch->setGenerator([this, exceptionIndex, handle] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
+        AllowMacroScratchRegisterUsage allowScratch(jit);
+        handle.generate(jit, params, this);
+        emitThrowImpl(jit, exceptionIndex); 
+    });
+
+    emitPatchpoint(m_currentBlock, patch, Tmp(), WTFMove(patchArgs));
+
+    return { };
+}
+
+auto AirIRGenerator::addRethrow(unsigned, ControlType& data) -> PartialResult
+{
+    B3::PatchpointValue* patch = addPatchpoint(B3::Void);
+    patch->clobber(RegisterSet::volatileRegistersForJSCall());
+    patch->effects.terminal = true;
+
+    Vector<ConstrainedTmp, 3> patchArgs;
+    patchArgs.append(ConstrainedTmp(instanceValue(), B3::ValueRep::reg(GPRInfo::argumentGPR0)));
+    patchArgs.append(ConstrainedTmp(Tmp(GPRInfo::callFrameRegister), B3::ValueRep::reg(GPRInfo::argumentGPR1)));
+    patchArgs.append(ConstrainedTmp(data.exception(), B3::ValueRep::reg(GPRInfo::argumentGPR2)));
+
+    PatchpointExceptionHandle handle = preparePatchpointForExceptions(patch, patchArgs);
+    patch->setGenerator([this, handle] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
+        AllowMacroScratchRegisterUsage allowScratch(jit);
+        handle.generate(jit, params, this);
+        emitRethrowImpl(jit);
+    });
+
+    emitPatchpoint(m_currentBlock, patch, Tmp(), WTFMove(patchArgs));
+
     return { };
 }
@@ -3210 +3482 @@
         append(data.special, Jump);
         data.special->setSuccessors(m_currentBlock);
-    }
+    } else if (data.blockType() == BlockType::Try || data.blockType() == BlockType::Catch)
+        --m_tryCatchDepth;
 
     if (data.blockType() == BlockType::Loop) {
@@ -3234 +3507 @@
 }
 
-B3::PatchpointValue* AirIRGenerator::emitCallPatchpoint(BasicBlock* block, const Signature& signature, const ResultList& results, const Vector<TypedTmp>& args, Vector<ConstrainedTmp>&& patchArgs)
+std::pair<B3::PatchpointValue*, PatchpointExceptionHandle> AirIRGenerator::emitCallPatchpoint(BasicBlock* block, const Signature& signature, const ResultList& results, const Vector<TypedTmp>& args, Vector<ConstrainedTmp> patchArgs)
 {
     auto* patchpoint = addPatchpoint(toB3ResultType(&signature));
@@ -3259 +3532 @@
         patchpoint->resultConstraints = WTFMove(resultConstraints);
     }
+    PatchpointExceptionHandle exceptionHandle = preparePatchpointForExceptions(patchpoint, patchArgs);
     emitPatchpoint(block, patchpoint, results, WTFMove(patchArgs));
-    return patchpoint;
+    return { patchpoint, exceptionHandle };
 }
 
@@ -3296 +3570 @@
 
         {
-            auto* patchpoint = emitCallPatchpoint(isWasmBlock, signature, results, args);
+            auto pair = emitCallPatchpoint(isWasmBlock, signature, results, args);
+            auto* patchpoint = pair.first;
+            auto exceptionHandle = pair.second;
             // We need to clobber all potential pinned registers since we might be leaving the instance.
             // We pessimistically assume we could be calling to something that is bounds checking.
@@ -3302 +3578 @@
             patchpoint->clobberLate(PinnedRegisterInfo::get().toSave(MemoryMode::BoundsChecking));
 
-            patchpoint->setGenerator([unlinkedWasmToWasmCalls, functionIndex] (CCallHelpers& jit, const B3::StackmapGenerationParams&) {
+            patchpoint->setGenerator([=] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
                 AllowMacroScratchRegisterUsage allowScratch(jit);
+                exceptionHandle.generate(jit, params, this);
                 CCallHelpers::Call call = jit.threadSafePatchableNearCall();
                 jit.addLinkTask([unlinkedWasmToWasmCalls, call, functionIndex] (LinkBuffer& linkBuffer) {
@@ -3322 +3599 @@
             Vector<ConstrainedTmp> jumpArgs;
             jumpArgs.append({ jumpDestination, B3::ValueRep::SomeRegister });
-            auto* patchpoint = emitCallPatchpoint(isEmbedderBlock, signature, results, args, WTFMove(jumpArgs));
+            auto pair = emitCallPatchpoint(isEmbedderBlock, signature, results, args, WTFMove(jumpArgs));
+            auto* patchpoint = pair.first;
+            auto exceptionHandle = pair.second;
+
             // We need to clobber all potential pinned registers since we might be leaving the instance.
             // We pessimistically assume we could be calling to something that is bounds checking.
             // FIXME: We shouldn't have to do this: https://bugs.webkit.org/show_bug.cgi?id=172181
             patchpoint->clobberLate(PinnedRegisterInfo::get().toSave(MemoryMode::BoundsChecking));
-            patchpoint->setGenerator([] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
+            patchpoint->setGenerator([=] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
                 AllowMacroScratchRegisterUsage allowScratch(jit);
+                exceptionHandle.generate(jit, params, this);
                 jit.call(params[params.proc().resultCount(params.value()->type())].gpr(), WasmEntryPtrTag);
             });
@@ -3340 +3621 @@
         restoreWebAssemblyGlobalState(RestoreCachedStackLimit::Yes, m_info.memory, currentInstance, continuation);
     } else {
-        auto* patchpoint = emitCallPatchpoint(m_currentBlock, signature, results, args);
+        auto pair = emitCallPatchpoint(m_currentBlock, signature, results, args);
+        auto* patchpoint = pair.first;
+        auto exceptionHandle = pair.second;
         // We need to clobber the size register since the LLInt always bounds checks
         if (m_mode == MemoryMode::Signaling || m_info.memory.isShared())
             patchpoint->clobberLate(RegisterSet { PinnedRegisterInfo::get().boundsCheckingSizeRegister });
-        patchpoint->setGenerator([unlinkedWasmToWasmCalls, functionIndex] (CCallHelpers& jit, const B3::StackmapGenerationParams&) {
+        patchpoint->setGenerator([=] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
             AllowMacroScratchRegisterUsage allowScratch(jit);
+            exceptionHandle.generate(jit, params, this);
             CCallHelpers::Call call = jit.threadSafePatchableNearCall();
             jit.addLinkTask([unlinkedWasmToWasmCalls, call, functionIndex] (LinkBuffer& linkBuffer) {
@@ -3524 +3808 @@
         results.append(tmpForType(signature.returnType(i)));
 
-    auto* patchpoint = emitCallPatchpoint(m_currentBlock, signature, results, args, WTFMove(extraArgs));
+    auto pair = emitCallPatchpoint(m_currentBlock, signature, results, args, WTFMove(extraArgs));
+    auto* patchpoint = pair.first;
+    auto exceptionHandle = pair.second;
 
     // We need to clobber all potential pinned registers since we might be leaving the instance.
@@ -3536 +3822 @@
     patchpoint->setGenerator([=] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
         AllowMacroScratchRegisterUsage allowScratch(jit);
+        exceptionHandle.generate(jit, params, this);
         jit.call(params[params.proc().resultCount(params.value()->type())].gpr(), WasmEntryPtrTag);
     });
@@ -3596 +3883 @@
     compilationContext.wasmEntrypointJIT = makeUnique<CCallHelpers>();
 
-    B3::Procedure procedure;
+    compilationContext.procedure = makeUnique<B3::Procedure>();
+    auto& procedure = *compilationContext.procedure;
     Code& code = procedure.code();
 
@@ -3616 +3904 @@
     WASM_FAIL_IF_HELPER_FAILS(parser.parse());
 
+    irGenerator.finalizeEntrypoints();
 
     for (BasicBlock* block : code) {
@@ -3622 +3911 @@
     }
 
-    {
-        if (UNLIKELY(shouldDumpIRAtEachPhase(B3::AirMode))) {
-            dataLogLn("Generated patchpoints");
-            for (B3::PatchpointValue** patch : irGenerator.patchpoints())
-                dataLogLn(deepDump(procedure, *patch));
-        }
-
-        B3::Air::prepareForGeneration(code);
-        B3::Air::generate(code, *compilationContext.wasmEntrypointJIT);
-        compilationContext.wasmEntrypointByproducts = procedure.releaseByproducts();
-        result->entrypoint.calleeSaveRegisters = code.calleeSaveRegisterAtOffsetList();
-    }
+    if (UNLIKELY(shouldDumpIRAtEachPhase(B3::AirMode))) {
+        dataLogLn("Generated patchpoints");
+        for (B3::PatchpointValue** patch : irGenerator.patchpoints())
+            dataLogLn(deepDump(procedure, *patch));
+    }
+
+    B3::Air::prepareForGeneration(code);
+    B3::Air::generate(code, *compilationContext.wasmEntrypointJIT);
+
+    compilationContext.wasmEntrypointByproducts = procedure.releaseByproducts();
+    result->entrypoint.calleeSaveRegisters = code.calleeSaveRegisterAtOffsetList();
+    result->stackmaps = irGenerator.takeStackmaps();
+    result->exceptionHandlers = irGenerator.takeExceptionHandlers();
 
     return result;
@@ -5179 +5469 @@
 }
 
+template <size_t inlineCapacity>
+PatchpointExceptionHandle AirIRGenerator::preparePatchpointForExceptions(B3::PatchpointValue* patch, Vector<ConstrainedTmp, inlineCapacity>& args)
+{
+    ++m_callSiteIndex;
+    if (!m_tryCatchDepth)
+        return { };
+
+    unsigned numLiveValues = 0;
+    forEachLiveValue([&] (Tmp tmp) {
+        ++numLiveValues;
+        args.append(ConstrainedTmp(tmp, B3::ValueRep::LateColdAny));
+    });
+
+    patch->effects.exitsSideways = true;
+
+    return PatchpointExceptionHandle { m_callSiteIndex, numLiveValues };
+}
+
 } } // namespace JSC::Wasm
 

trunk/Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp

@@ -1 +1 @@
 /*
- * Copyright (C) 2016-2021 Apple Inc. All rights reserved.
+ * Copyright (C) 2016-2022 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -58 +58 @@
 #include "WasmExceptionType.h"
 #include "WasmFunctionParser.h"
+#include "WasmIRGeneratorHelpers.h"
 #include "WasmInstance.h"
 #include "WasmMemory.h"
@@ -87 +88 @@
 }
 }
-
-class B3IRGenerator;
-struct PatchpointExceptionHandle {
-    void generate(CCallHelpers&, const B3::StackmapGenerationParams&, B3IRGenerator*) const;
-
-    static constexpr unsigned s_invalidCallSiteIndex =  std::numeric_limits<unsigned>::max();
-
-    unsigned m_callSiteIndex { s_invalidCallSiteIndex };
-    unsigned m_numLiveValues;
-};
 
 class B3IRGenerator {
@@ -579 +570 @@
 };
 
-void PatchpointExceptionHandle::generate(CCallHelpers& jit, const B3::StackmapGenerationParams& params, B3IRGenerator* generator) const
-{
-    if (m_callSiteIndex == s_invalidCallSiteIndex)
-        return;
-
-    StackMap values(m_numLiveValues);
-    unsigned paramsOffset = params.size() - m_numLiveValues;
-    unsigned childrenOffset = params.value()->numChildren() - m_numLiveValues;
-    for (unsigned i = 0; i < m_numLiveValues; ++i)
-        values[i] = OSREntryValue(params[i + paramsOffset], params.value()->child(i + childrenOffset)->type());
-
-    generator->addStackMap(m_callSiteIndex, WTFMove(values));
-    jit.store32(CCallHelpers::TrustedImm32(m_callSiteIndex), CCallHelpers::tagFor(CallFrameSlot::argumentCountIncludingThis));
-}
-
 // Memory accesses in WebAssembly have unsigned 32-bit offsets, whereas they have signed 32-bit offsets in B3.
 int32_t B3IRGenerator::fixupPointerPlusOffset(Value*& ptr, uint32_t offset)
@@ -858 +834 @@
 }
 
-static void buildEntryBufferForCatch(Probe::Context& context)
-{
-    CallFrame* callFrame = context.fp<CallFrame*>();
-    CallSiteIndex callSiteIndex = callFrame->callSiteIndex();
-    OptimizingJITCallee* callee = bitwise_cast<OptimizingJITCallee*>(callFrame->callee().asWasmCallee());
-    const StackMap& stackmap = callee->stackmap(callSiteIndex);
-    VM* vm = context.gpr<VM*>(GPRInfo::regT0);
-    uint64_t* buffer = vm->wasmContext.scratchBufferForSize(stackmap.size() * 8);
-    loadValuesIntoBuffer(context, stackmap, buffer);
-
-    context.gpr(GPRInfo::argumentGPR0) = bitwise_cast<uintptr_t>(buffer);
-}
-
 void B3IRGenerator::insertEntrySwitch()
 {
@@ -877 +840 @@
     Ref<B3::Air::PrologueGenerator> catchPrologueGenerator = createSharedTask<B3::Air::PrologueGeneratorFunction>([] (CCallHelpers& jit, B3::Air::Code& code) {
         AllowMacroScratchRegisterUsage allowScratch(jit);
-
-        jit.emitGetFromCallFrameHeaderPtr(CallFrameSlot::callee, GPRInfo::regT0);
-        {
-            // FIXME: Handling precise allocations in WasmB3IRGenerator catch entrypoints might be unnecessary
-            // https://bugs.webkit.org/show_bug.cgi?id=231213
-            auto preciseAllocationCase = jit.branchTestPtr(CCallHelpers::NonZero, GPRInfo::regT0, CCallHelpers::TrustedImm32(PreciseAllocation::halfAlignment));
-            jit.andPtr(CCallHelpers::TrustedImmPtr(MarkedBlock::blockMask), GPRInfo::regT0);
-            jit.loadPtr(CCallHelpers::Address(GPRInfo::regT0, MarkedBlock::offsetOfFooter + MarkedBlock::Footer::offsetOfVM()), GPRInfo::regT0);
-            auto loadedCase = jit.jump();
-
-            preciseAllocationCase.link(&jit);
-            jit.loadPtr(CCallHelpers::Address(GPRInfo::regT0, PreciseAllocation::offsetOfWeakSet() + WeakSet::offsetOfVM() - PreciseAllocation::headerSize()), GPRInfo::regT0);
-
-            loadedCase.link(&jit);
-        }
-        jit.restoreCalleeSavesFromVMEntryFrameCalleeSavesBuffer(GPRInfo::regT0, GPRInfo::regT3);
-
-        jit.loadPtr(CCallHelpers::Address(GPRInfo::regT0, VM::calleeForWasmCatchOffset()), GPRInfo::regT3);
-        jit.storePtr(CCallHelpers::TrustedImmPtr(nullptr), CCallHelpers::Address(GPRInfo::regT0, VM::calleeForWasmCatchOffset()));
-        jit.emitPutToCallFrameHeader(GPRInfo::regT3, CallFrameSlot::callee);
-
-        jit.load64(CCallHelpers::Address(GPRInfo::regT0, VM::callFrameForCatchOffset()), GPRInfo::callFrameRegister);
-        jit.storePtr(CCallHelpers::TrustedImmPtr(nullptr), CCallHelpers::Address(GPRInfo::regT0, VM::callFrameForCatchOffset()));
-
-        jit.loadPtr(CCallHelpers::Address(GPRInfo::callFrameRegister, CallFrameSlot::thisArgument * sizeof(Register)), GPRInfo::regT3);
-        jit.loadPtr(CCallHelpers::Address(GPRInfo::regT3, JSWebAssemblyInstance::offsetOfInstance()), GPRInfo::regT3);
-        jit.storeWasmContextInstance(GPRInfo::regT3);
-
-        jit.probe(tagCFunction<JITProbePtrTag>(buildEntryBufferForCatch), nullptr);
-
-        jit.addPtr(CCallHelpers::TrustedImm32(-code.frameSize()), GPRInfo::callFrameRegister, CCallHelpers::stackPointerRegister);
+        emitCatchPrologueShared(code, jit);
     });
 
@@ -2653 +2586 @@
             data.convertTryToCatchAll(++m_callSiteIndex, m_proc.addVariable(pointerType()));
     }
+    // We convert from "try" to "catch" ControlType above. This doesn't
+    // happen if ControlType is already a "catch". This can happen when
+    // we have multiple catches like "try {} catch(A){} catch(B){}...CatchAll(E){}"
+    ASSERT(ControlType::isAnyCatch(data));
 
     HandlerType handlerType = kind == CatchKind::Catch ? HandlerType::Catch : HandlerType::CatchAll;
@@ -2723 +2660 @@
     patch->setGenerator([this, exceptionIndex, handle] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
         AllowMacroScratchRegisterUsage allowScratch(jit);
-        GPRReg scratch = GPRInfo::nonPreservedNonArgumentGPR0;
         handle.generate(jit, params, this);
-
-        jit.loadPtr(CCallHelpers::Address(GPRInfo::argumentGPR0, Instance::offsetOfOwner()), scratch);
-        {
-            auto preciseAllocationCase = jit.branchTestPtr(CCallHelpers::NonZero, scratch, CCallHelpers::TrustedImm32(PreciseAllocation::halfAlignment));
-            jit.andPtr(CCallHelpers::TrustedImmPtr(MarkedBlock::blockMask), scratch);
-            jit.loadPtr(CCallHelpers::Address(scratch, MarkedBlock::offsetOfFooter + MarkedBlock::Footer::offsetOfVM()), scratch);
-            auto loadedCase = jit.jump();
-
-            preciseAllocationCase.link(&jit);
-            jit.loadPtr(CCallHelpers::Address(scratch, PreciseAllocation::offsetOfWeakSet() + WeakSet::offsetOfVM() - PreciseAllocation::headerSize()), scratch);
-
-            loadedCase.link(&jit);
-        }
-        jit.copyCalleeSavesToVMEntryFrameCalleeSavesBuffer(scratch);
-
-        jit.move(MacroAssembler::TrustedImm32(exceptionIndex), GPRInfo::argumentGPR2);
-        jit.move(MacroAssembler::stackPointerRegister, GPRInfo::argumentGPR3);
-        CCallHelpers::Call call = jit.call(OperationPtrTag);
-        jit.farJump(GPRInfo::returnValueGPR, ExceptionHandlerPtrTag);
-        jit.addLinkTask([call] (LinkBuffer& linkBuffer) {
-            linkBuffer.link(call, FunctionPtr<OperationPtrTag>(operationWasmThrow));
-        });
+        emitThrowImpl(jit, exceptionIndex); 
     });
     m_currentBlock->append(patch);
@@ -2764 +2679 @@
     patch->setGenerator([this, handle] (CCallHelpers& jit, const B3::StackmapGenerationParams& params) {
         AllowMacroScratchRegisterUsage allowScratch(jit);
-
-        GPRReg scratch = GPRInfo::nonPreservedNonArgumentGPR0;
-        jit.loadPtr(CCallHelpers::Address(GPRInfo::argumentGPR0, Instance::offsetOfOwner()), scratch);
-        {
-            auto preciseAllocationCase = jit.branchTestPtr(CCallHelpers::NonZero, scratch, CCallHelpers::TrustedImm32(PreciseAllocation::halfAlignment));
-            jit.andPtr(CCallHelpers::TrustedImmPtr(MarkedBlock::blockMask), scratch);
-            jit.loadPtr(CCallHelpers::Address(scratch, MarkedBlock::offsetOfFooter + MarkedBlock::Footer::offsetOfVM()), scratch);
-            auto loadedCase = jit.jump();
-
-            preciseAllocationCase.link(&jit);
-            jit.loadPtr(CCallHelpers::Address(scratch, PreciseAllocation::offsetOfWeakSet() + WeakSet::offsetOfVM() - PreciseAllocation::headerSize()), scratch);
-
-            loadedCase.link(&jit);
-        }
-        jit.copyCalleeSavesToVMEntryFrameCalleeSavesBuffer(scratch);
-
         handle.generate(jit, params, this);
-        CCallHelpers::Call call = jit.call(OperationPtrTag);
-        jit.farJump(GPRInfo::returnValueGPR, ExceptionHandlerPtrTag);
-        jit.addLinkTask([call] (LinkBuffer& linkBuffer) {
-            linkBuffer.link(call, FunctionPtr<OperationPtrTag>(operationWasmRethrow));
-        });
+        emitRethrowImpl(jit);
     });
     m_currentBlock->append(patch);
@@ -3321 +3216 @@
 }
 
-void computeExceptionHandlerLocations(Vector<CodeLocationLabel<ExceptionHandlerPtrTag>>& handlers, const InternalFunction* function, const CompilationContext& context, LinkBuffer& linkBuffer)
-{
-    if (!context.procedure)
-        return;
-
-    unsigned entrypointIndex = 0;
-    unsigned numEntrypoints = context.procedure->numEntrypoints();
-    for (const UnlinkedHandlerInfo& handlerInfo : function->exceptionHandlers) {
-        RELEASE_ASSERT(entrypointIndex < numEntrypoints);
-        if (handlerInfo.m_type == HandlerType::Delegate) {
-            handlers.append({ });
-            continue;
-        }
-
-        ++entrypointIndex;
-        handlers.append(linkBuffer.locationOf<ExceptionHandlerPtrTag>(context.procedure->code().entrypointLabel(entrypointIndex)));
-    }
-    RELEASE_ASSERT(entrypointIndex == numEntrypoints - 1);
-}
-
 // Custom wasm ops. These are the ones too messy to do in wasm.json.
 

trunk/Source/JavaScriptCore/wasm/WasmB3IRGenerator.h

@@ -60 +60 @@
 Expected<std::unique_ptr<InternalFunction>, String> parseAndCompile(CompilationContext&, const FunctionData&, const Signature&, Vector<UnlinkedWasmToWasmCall>&, unsigned& osrEntryScratchBufferSize, const ModuleInformation&, MemoryMode, CompilationMode, uint32_t functionIndex, uint32_t loopIndexForOSREntry, TierUpCount* = nullptr);
 
-void computeExceptionHandlerLocations(Vector<CodeLocationLabel<ExceptionHandlerPtrTag>>& handlers, const InternalFunction*, const CompilationContext&, LinkBuffer&);
 void computePCToCodeOriginMap(CompilationContext&, LinkBuffer&);
 

trunk/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp

@@ -37 +37 @@
 #include "WasmCalleeGroup.h"
 #include "WasmCalleeRegistry.h"
+#include "WasmIRGeneratorHelpers.h"
 #include "WasmSignatureInlines.h"
 #include "WasmTierUpCount.h"
@@ -204 +205 @@
     if (Options::webAssemblyBBQAirModeThreshold() && m_moduleInformation->codeSectionSize >= Options::webAssemblyBBQAirModeThreshold())
         forceUsingB3 = true;
-    else if (!m_moduleInformation->m_functionDoesNotUseExceptions.quickGet(functionIndex))
-        forceUsingB3 = true;
     else if (!Options::wasmBBQUsesAir())
         forceUsingB3 = true;

trunk/Source/JavaScriptCore/wasm/WasmLLIntGenerator.cpp

@@ -559 +559 @@
     RELEASE_ASSERT(usedBuffer.capacity() == oldCapacity);
     *threadSpecific = WTFMove(usedBuffer);
-
-    if (!m_usesExceptions)
-        m_info.m_functionDoesNotUseExceptions.quickSet(m_functionIndex);
 
     return WTFMove(m_codeBlock);

trunk/Source/JavaScriptCore/wasm/WasmModuleInformation.h

@@ -120 +120 @@
     BitVector m_declaredFunctions;
     BitVector m_declaredExceptions;
-    BitVector m_functionDoesNotUseExceptions;
     mutable BitVector m_referencedFunctions;
 };

trunk/Source/JavaScriptCore/wasm/WasmOMGPlan.cpp

@@ -34 +34 @@
 #include "WasmCallee.h"
 #include "WasmCalleeRegistry.h"
+#include "WasmIRGeneratorHelpers.h"
 #include "WasmNameSection.h"
 #include "WasmSignatureInlines.h"

trunk/Source/JavaScriptCore/wasm/WasmOSREntryPlan.cpp

@@ -33 +33 @@
 #include "WasmB3IRGenerator.h"
 #include "WasmCallee.h"
+#include "WasmIRGeneratorHelpers.h"
 #include "WasmMachineThreads.h"
 #include "WasmNameSection.h"

trunk/Source/JavaScriptCore/wasm/WasmStreamingParser.cpp

@@ -118 +118 @@
     m_functionIndex = 0;
     m_codeOffset = m_offset;
-    m_info->m_functionDoesNotUseExceptions.ensureSize(functionCount);
 
     WASM_PARSER_FAIL_IF(functionCount == std::numeric_limits<uint32_t>::max(), "Code section's count is too big ", functionCount);