Changeset 276145 in webkit
- Timestamp:
- Apr 16, 2021, 11:02:48 AM (5 years ago)
- Location:
- trunk/Source/WebKit
- Files:
-
- 7 edited
-
ChangeLog (modified) (1 diff)
-
GPUProcess/GPUConnectionToWebProcess.cpp (modified) (2 diffs)
-
GPUProcess/GPUConnectionToWebProcess.h (modified) (1 diff)
-
GPUProcess/GPUConnectionToWebProcess.messages.in (modified) (1 diff)
-
Platform/IPC/Connection.cpp (modified) (8 diffs)
-
Platform/IPC/Connection.h (modified) (1 diff)
-
WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp (modified) (1 diff)
Legend:
- Unmodified
- Added
- Removed
-
trunk/Source/WebKit/ChangeLog
r276136 r276145 1 2021-04-16 Chris Dumez <cdumez@apple.com> 2 3 Early IPC messages to a WorkQueueMessageReceiver may get processed out of order 4 https://bugs.webkit.org/show_bug.cgi?id=224623 5 6 Reviewed by Geoffrey Garen. 7 8 Bug 224566 exposed an issue where early IPC being sent to WorkQueueMessageReceiver might get received 9 out of order. The reason behind it is that the WorkQueueMessageReceiver registers itself on the main 10 thread while we receive the IPC on the IPC thread. When we receive the IPC on the IPC thread, we check 11 if there is a WorkQueueMessageReceiver for it and if there is, we dispatch the message straight to its 12 WorkQueue. However, if the WorkQueueMessageReceiver has not registered itself yet on the main thread, 13 we hop to the main thread first, before dispatching the IPC back to the receiver's WorkQueue. The 14 extra hop to the main thread means that 2 IPC messages to the WorkQueueMessageReceiver sent one after 15 the other may get dispatched on the WorkQueue in an inconsistent order, if the WorkQueueMessageReceiver 16 registers itself as a receiver in between the 2 IPC messages. 17 18 We actually were trying to deal with this issue in Connection::addWorkQueueMessageReceiver(). When 19 the WorkQueueMessageReceiver would register itself on the main thread, we would grab the incomingMessages 20 lock and check m_incomingMessages for messages that should be dispatched to the WorkQueue. Those are 21 async messages that should have been dispatched straight to the WorkQueue on the IPC thread but didn't 22 because the WorkQueueMessageReceiver has not registered itself yet. 23 24 However, this logic in Connection::addWorkQueueMessageReceiver() was insufficient because it only checked 25 m_incomingMessages. m_incomingMessages only contains async messages. Sync messages (and special async 26 messages with the IPC::SendOption::DispatchMessageEvenWhenWaitingForSyncReply flag) are stored in 27 Connection::SyncMessageState::m_messagesToDispatchWhileWaitingForSyncReply. This is what was causing 28 Bug 224566 since RemoteRenderingBackendProxy's CreateImageBuffer IPC was async with the 29 DispatchMessageEvenWhenWaitingForSyncReply flag and its GetDataURLForImageBuffer was synchronous. The 30 ordering of these 2 IPC messages could get reversed and it would cause correctness issues and flaky 31 crashes. 32 33 To address the issue, I updated Connection::addWorkQueueMessageReceiver() to ask the 34 Connection::SyncMessageState to enqueue its matching messages to the WorkQueue. There was one issue 35 though because Connection::SyncMessageState::dispatchMessagesAndResetDidScheduleDispatchMessagesForConnection() 36 was taking messages out of m_messagesToDispatchWhileWaitingForSyncReply and storing them in a local 37 container and then iterating over this container to dispatch the messages. The dispatching of one 38 of these messages could cause a WorkQueueMessageReceiver to register itself (call addWorkQueueMessageReceiver()). 39 When this would happen, addWorkQueueMessageReceiver() would try and enqueue matching messages in 40 m_messagesToDispatchWhileWaitingForSyncReply and would miss the messages in the local container 41 that dispatchMessagesAndResetDidScheduleDispatchMessagesForConnection() is currently iterating on. 42 To address this issue, I introduced a new m_messagesBeingDispatched data member and used that to 43 store the messages being dispatched instead of the local container. As a result, 44 addWorkQueueMessageReceiver() can now enqueue the messages in m_messagesBeingDispatched first and 45 then enqueue the ones in m_messagesToDispatchWhileWaitingForSyncReply. 46 47 * GPUProcess/GPUConnectionToWebProcess.cpp: 48 (WebKit::GPUConnectionToWebProcess::createRenderingBackend): 49 * GPUProcess/GPUConnectionToWebProcess.h: 50 * GPUProcess/GPUConnectionToWebProcess.messages.in: 51 * WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp: 52 (WebKit::RemoteRenderingBackendProxy::ensureGPUProcessConnection): 53 Revert r276007 that was committed as a temporary workaround for this bug. 54 55 * Platform/IPC/Connection.cpp: 56 (IPC::Connection::SyncMessageState::enqueueMatchingMessages): 57 Add utility function to SyncMessageState to enqueue its matching messages in m_messagesBeingDispatched 58 and m_messagesToDispatchWhileWaitingForSyncReply to the provided MessageReceiveQueue. This is called 59 by Connection::addMessageReceiveQueue(). The logic is similar to the one in the 60 enqueueMatchingMessagesToMessageReceiveQueue() function but works on a Deque<ConnectionAndIncomingMessage> 61 instead of a Deque<std::unique_ptr<Decoder>>. 62 63 (IPC::Connection::SyncMessageState::dispatchMessages): 64 (IPC::Connection::SyncMessageState::dispatchMessagesAndResetDidScheduleDispatchMessagesForConnection): 65 Use m_messagesBeingDispatched instead of a local container to store the messages we are about to 66 dispatch. This allows enqueueMatchingMessages() to check those messages to see if they should be 67 dispatched to a MessageReceiveQueue. We also need to make sure we don't iterate over 68 m_messagesBeingDispatched to call dispatch() on the messages. This is important because any message 69 dispatch may cause a WorkQueueMessageReceiver to register itself, which would call 70 enqueueMatchingMessages() and potentially extract matching messages from m_messagesBeingDispatched. 71 For this reason, we take messages from m_messagesBeingDispatched one by one, until the container 72 becomes empty. 73 74 (IPC::Connection::addMessageReceiveQueue): 75 (IPC::Connection::addWorkQueueMessageReceiver): 76 (IPC::Connection::addThreadMessageReceiver): 77 - This used to only check m_incomingMessages for matching messages that should be enqueued on the 78 MessageReceiveQueue in order to preserve IPC ordering. This was insufficient because it would fail 79 to consider sync IPC messages (or async IPC messages with the 80 IPC::SendOption::DispatchMessageEvenWhenWaitingForSyncReply flag). Since those are stored in separate 81 containers in Connection::SyncMessageState, we now also call 82 Connection::SyncMessageState::enqueueMatchingMessages() to enqueue those messages and preserve their 83 ordering too. This fixes IPC ordering bug identified via Bug 224566. 84 - Avoid some code duplication by moving more logic to a shared enqueueMatchingMessagesToMessageReceiveQueue() 85 function. The function is no longer templated because I don't think it is worth increasing binary 86 size just to avoid the virtual enqueueMessage() function call on the MessageReceiverQueue. We do not 87 register message receivers very often and then only have a few early messages at most to enqueue. 88 1 89 2021-04-16 Carlos Garcia Campos <cgarcia@igalia.com> 2 90 -
trunk/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
r276007 r276145 363 363 #endif 364 364 365 void GPUConnectionToWebProcess::createRenderingBackend(RemoteRenderingBackendCreationParameters&& creationParameters , CompletionHandler<void()>&& completionHandler)365 void GPUConnectionToWebProcess::createRenderingBackend(RemoteRenderingBackendCreationParameters&& creationParameters) 366 366 { 367 367 auto addResult = m_remoteRenderingBackendMap.ensure(creationParameters.identifier, [&]() { … … 369 369 }); 370 370 ASSERT_UNUSED(addResult, addResult.isNewEntry); 371 completionHandler();372 371 } 373 372 -
trunk/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
r276007 r276145 162 162 #endif 163 163 164 void createRenderingBackend(RemoteRenderingBackendCreationParameters&& , CompletionHandler<void()>&&);164 void createRenderingBackend(RemoteRenderingBackendCreationParameters&&); 165 165 void releaseRenderingBackend(RenderingBackendIdentifier); 166 166 -
trunk/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.messages.in
r276007 r276145 24 24 25 25 messages -> GPUConnectionToWebProcess WantsDispatchMessage { 26 void CreateRenderingBackend(struct WebKit::RemoteRenderingBackendCreationParameters creationParameters) -> () Synchronous26 void CreateRenderingBackend(struct WebKit::RemoteRenderingBackendCreationParameters creationParameters) 27 27 void ReleaseRenderingBackend(WebKit::RenderingBackendIdentifier renderingBackendIdentifier) 28 28 #if ENABLE(WEBGL) -
trunk/Source/WebKit/Platform/IPC/Connection.cpp
r274565 r276145 94 94 void dispatchMessages(); 95 95 96 // Add matching pending messages to the provided MessageReceiveQueue. 97 void enqueueMatchingMessages(Connection&, MessageReceiveQueue&, ReceiverName, uint64_t destinationID); 98 96 99 private: 97 100 friend class LazyNeverDestroyed<Connection::SyncMessageState>; … … 118 121 } 119 122 }; 120 Vector<ConnectionAndIncomingMessage> m_messagesToDispatchWhileWaitingForSyncReply; 123 Deque<ConnectionAndIncomingMessage> m_messagesBeingDispatched; // Only used on the main thread. 124 Deque<ConnectionAndIncomingMessage> m_messagesToDispatchWhileWaitingForSyncReply; 121 125 }; 122 126 … … 131 135 132 136 return syncMessageState; 137 } 138 139 void Connection::SyncMessageState::enqueueMatchingMessages(Connection& connection, MessageReceiveQueue& receiveQueue, ReceiverName receiverName, uint64_t destinationID) 140 { 141 ASSERT(isMainRunLoop()); 142 auto enqueueMatchingMessagesInContainer = [&](Deque<ConnectionAndIncomingMessage>& connectionAndMessages) { 143 Deque<ConnectionAndIncomingMessage> rest; 144 for (auto& connectionAndMessage : connectionAndMessages) { 145 if (connectionAndMessage.connection.ptr() == &connection && connectionAndMessage.message->messageReceiverName() == receiverName && (connectionAndMessage.message->destinationID() == destinationID || !destinationID)) 146 receiveQueue.enqueueMessage(connection, WTFMove(connectionAndMessage.message)); 147 else 148 rest.append(WTFMove(connectionAndMessage)); 149 } 150 connectionAndMessages = WTFMove(rest); 151 }; 152 auto locker = holdLock(m_mutex); 153 enqueueMatchingMessagesInContainer(m_messagesBeingDispatched); 154 enqueueMatchingMessagesInContainer(m_messagesToDispatchWhileWaitingForSyncReply); 133 155 } 134 156 … … 174 196 ASSERT(RunLoop::isMain()); 175 197 176 Vector<ConnectionAndIncomingMessage> messagesToDispatchWhileWaitingForSyncReply;177 198 { 178 199 auto locker = holdLock(m_mutex); 179 m_messagesToDispatchWhileWaitingForSyncReply.swap(messagesToDispatchWhileWaitingForSyncReply); 180 } 181 182 for (auto& connectionAndIncomingMessage : messagesToDispatchWhileWaitingForSyncReply) 183 connectionAndIncomingMessage.dispatch(); 200 if (m_messagesBeingDispatched.isEmpty()) 201 m_messagesBeingDispatched = std::exchange(m_messagesToDispatchWhileWaitingForSyncReply, { }); 202 else { 203 while (!m_messagesToDispatchWhileWaitingForSyncReply.isEmpty()) 204 m_messagesBeingDispatched.append(m_messagesToDispatchWhileWaitingForSyncReply.takeLast()); 205 } 206 } 207 208 while (!m_messagesBeingDispatched.isEmpty()) 209 m_messagesBeingDispatched.takeFirst().dispatch(); 184 210 } 185 211 … … 188 214 ASSERT(RunLoop::isMain()); 189 215 190 Vector<ConnectionAndIncomingMessage> messagesToDispatchWhileWaitingForSyncReply;191 216 { 192 217 auto locker = holdLock(m_mutex); 193 218 ASSERT(m_didScheduleDispatchMessagesWorkSet.contains(&connection)); 194 219 m_didScheduleDispatchMessagesWorkSet.remove(&connection); 195 m_messagesToDispatchWhileWaitingForSyncReply.swap(messagesToDispatchWhileWaitingForSyncReply); 196 } 197 198 Vector<ConnectionAndIncomingMessage> messagesToPutBack; 199 for (auto& connectionAndIncomingMessage : messagesToDispatchWhileWaitingForSyncReply) { 200 if (&connection == connectionAndIncomingMessage.connection.ptr()) 201 connectionAndIncomingMessage.dispatch(); 202 else 203 messagesToPutBack.append(WTFMove(connectionAndIncomingMessage)); 204 } 205 206 if (!messagesToPutBack.isEmpty()) { 207 auto locker = holdLock(m_mutex); 208 messagesToPutBack.appendVector(WTFMove(m_messagesToDispatchWhileWaitingForSyncReply)); 220 ASSERT(m_messagesBeingDispatched.isEmpty()); 221 Deque<ConnectionAndIncomingMessage> messagesToPutBack; 222 for (auto& connectionAndIncomingMessage : m_messagesToDispatchWhileWaitingForSyncReply) { 223 if (&connection == connectionAndIncomingMessage.connection.ptr()) 224 m_messagesBeingDispatched.append(WTFMove(connectionAndIncomingMessage)); 225 else 226 messagesToPutBack.append(WTFMove(connectionAndIncomingMessage)); 227 } 209 228 m_messagesToDispatchWhileWaitingForSyncReply = WTFMove(messagesToPutBack); 210 229 } 230 231 while (!m_messagesBeingDispatched.isEmpty()) 232 m_messagesBeingDispatched.takeFirst().dispatch(); 211 233 } 212 234 … … 320 342 } 321 343 322 namespace { 323 template <typename T> 324 Deque<std::unique_ptr<Decoder>> filterWithMessageReceiveQueue(Connection& connection, T& receiveQueue, ReceiverName receiverName, uint64_t destinationID, Deque<std::unique_ptr<Decoder>>&& incomingMessages) 325 { 326 Deque<std::unique_ptr<Decoder>> rest; 327 for (auto& message : incomingMessages) { 344 // Enqueue any pending message to the MessageReceiveQueue that is meant to go on that queue. This is important to maintain the ordering of 345 // IPC messages as some messages may get received on the IPC thread before the message receiver registered itself on the main thread. 346 void Connection::enqueueMatchingMessagesToMessageReceiveQueue(Locker<Lock>&, MessageReceiveQueue& receiveQueue, ReceiverName receiverName, uint64_t destinationID) 347 { 348 ASSERT(isMainRunLoop()); 349 350 SyncMessageState::singleton().enqueueMatchingMessages(*this, receiveQueue, receiverName, destinationID); 351 352 Deque<std::unique_ptr<Decoder>> remainingIncomingMessages; 353 for (auto& message : m_incomingMessages) { 328 354 if (message->messageReceiverName() == receiverName && (message->destinationID() == destinationID || !destinationID)) 329 receiveQueue.enqueueMessage( connection, WTFMove(message));355 receiveQueue.enqueueMessage(*this, WTFMove(message)); 330 356 else 331 rest.append(WTFMove(message)); 332 } 333 return rest; 334 } 357 remainingIncomingMessages.append(WTFMove(message)); 358 } 359 m_incomingMessages = WTFMove(remainingIncomingMessages); 335 360 } 336 361 337 362 void Connection::addMessageReceiveQueue(MessageReceiveQueue& receiveQueue, ReceiverName receiverName, uint64_t destinationID) 338 363 { 339 auto locker = holdLock(m_incomingMessagesMutex);340 m_incomingMessages = filterWithMessageReceiveQueue(*this, receiveQueue, receiverName, destinationID, WTFMove(m_incomingMessages));364 auto incomingMessagesLocker = holdLock(m_incomingMessagesMutex); 365 enqueueMatchingMessagesToMessageReceiveQueue(incomingMessagesLocker, receiveQueue, receiverName, destinationID); 341 366 m_receiveQueues.add(receiveQueue, receiverName, destinationID); 342 367 } … … 345 370 { 346 371 auto receiveQueue = makeUnique<WorkQueueMessageReceiverQueue>(workQueue, *receiver); 347 auto locker = holdLock(m_incomingMessagesMutex);348 m_incomingMessages = filterWithMessageReceiveQueue(*this, *receiveQueue, receiverName, destinationID, WTFMove(m_incomingMessages));372 auto incomingMessagesLocker = holdLock(m_incomingMessagesMutex); 373 enqueueMatchingMessagesToMessageReceiveQueue(incomingMessagesLocker, *receiveQueue, receiverName, destinationID); 349 374 m_receiveQueues.add(WTFMove(receiveQueue), receiverName, destinationID); 350 375 } … … 353 378 { 354 379 auto receiveQueue = makeUnique<ThreadMessageReceiverQueue>(*receiver); 355 auto locker = holdLock(m_incomingMessagesMutex);356 m_incomingMessages = filterWithMessageReceiveQueue(*this, *receiveQueue, receiverName, destinationID, WTFMove(m_incomingMessages));380 auto incomingMessagesLocker = holdLock(m_incomingMessagesMutex); 381 enqueueMatchingMessagesToMessageReceiveQueue(incomingMessagesLocker, *receiveQueue, receiverName, destinationID); 357 382 m_receiveQueues.add(WTFMove(receiveQueue), receiverName, destinationID); 358 383 } -
trunk/Source/WebKit/Platform/IPC/Connection.h
r274565 r276145 330 330 std::unique_ptr<Decoder> waitForSyncReply(uint64_t syncRequestID, MessageName, Timeout, OptionSet<SendSyncOption>); 331 331 332 void enqueueMatchingMessagesToMessageReceiveQueue(Locker<Lock>& incomingMessagesLocker, MessageReceiveQueue&, ReceiverName, uint64_t destinationID); 333 332 334 // Called on the connection work queue. 333 335 void processIncomingMessage(std::unique_ptr<Decoder>); -
trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp
r276007 r276145 76 76 gpuProcessConnection.addClient(*this); 77 77 gpuProcessConnection.messageReceiverMap().addMessageReceiver(Messages::RemoteRenderingBackendProxy::messageReceiverName(), renderingBackendIdentifier().toUInt64(), *this); 78 // This message is synchronous to ensure that the RemoteRenderingBackend has been created and has registered itself as a WorkQueueMessageReceiver before we send it IPC. 79 // Without this synchronization, some IPC messages may get received by the GPUProcess before the RemoteRenderingBackend has registered itself as a WorkQueueMessageReceiver 80 // and IPC may get processed out of order. 81 gpuProcessConnection.connection().sendSync(Messages::GPUConnectionToWebProcess::CreateRenderingBackend(m_parameters), Messages::GPUConnectionToWebProcess::CreateRenderingBackend::Reply(), 0); 78 gpuProcessConnection.connection().send(Messages::GPUConnectionToWebProcess::CreateRenderingBackend(m_parameters), 0, IPC::SendOption::DispatchMessageEvenWhenWaitingForSyncReply); 82 79 m_gpuProcessConnection = makeWeakPtr(gpuProcessConnection); 83 80 }
Note:
See TracChangeset
for help on using the changeset viewer.