Changeset 235343 in webkit


Ignore:
Timestamp:
Aug 26, 2018 7:37:22 PM (6 years ago)
Author:
Wenson Hsieh
Message:

[Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
https://bugs.webkit.org/show_bug.cgi?id=188933
<rdar://problem/43699724>

Reviewed by Darin Adler.

Source/WebCore:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

Tests: WKAttachmentTests.CutAndPastePastedImage

WKAttachmentTests.MovePastedImageByDragging
WKAttachmentTests.RemoveNewlinesBeforePastedImage

  • editing/Editor.h:
  • editing/cocoa/EditorCocoa.mm:

(WebCore::Editor::getPasteboardTypesAndDataForAttachment):

Adjust this helper to take an Element& rather than an HTMLAttachmentElement&, and address a FIXME by writing the
document origin identifier to the pasteboard via custom pasteboard data when dragging an attachment. This allows
us to avoid creating extra image and attachment elements when dragging an image backed by an attachment within
the same document.

  • editing/cocoa/WebContentReaderCocoa.mm:

(WebCore::contentTypeIsSuitableForInlineImageRepresentation):

Add a helper to determine whether a content type (UTI or MIME type) should be read as an inline image.

(WebCore::createFragmentForImageAttachment):
(WebCore::replaceRichContentWithAttachments):
(WebCore::WebContentReader::readFilePaths):

Teach codepaths where we currently create attachment elements to instead create image elements if the MIME type,
is something suitable for display via an inline image element; add the attachment element under the shadow root
of the image element.

  • editing/markup.cpp:

(WebCore::StyledMarkupAccumulator::appendCustomAttributes):
(WebCore::restoreAttachmentElementsInFragment):

When dragging or copying an image element, we need to make sure that any attachment element backing the image
is preserved in the pasted or dropped fragment. To do this, we use a technique similar to what was done for
r180785 and r224593 and write a temporary "webkitattachmentid" attribute to the serialized markup on copy. Upon
deserializing the markup back to a fragment, we then create an attachment element with the same identifier under
the image.

(WebCore::createFragmentFromMarkup):

  • html/HTMLAttachmentElement.h:
  • html/HTMLImageElement.cpp:

(WebCore::HTMLImageElement::setAttachmentElement):
(WebCore::HTMLImageElement::attachmentElement const):

Helper methods to get and set an attachment element under an image element. Setting an image's attachment
element puts that attachment element under the shadow root of the image, and also hides the attachment element.

(WebCore::HTMLImageElement::attachmentIdentifier const):

Returns the identifier of an attachment element associated with the image element, or null.

  • html/HTMLImageElement.h:
  • html/HTMLImageElement.idl:

Add HTMLImageElement.webkitAttachmentIdentifier, a readonly attribute guarded by runtime-enabled attachment
element feature.

  • page/DragController.cpp:

(WebCore::DragController::startDrag):

In the case of dragging an image, if that image element is backed by an attachment element, don't bother writing
the image data to the clipboard; instead, write the attachment data as a promise.

(WebCore::DragController::doImageDrag):

Plumb promised attachment information to DragController::doSystemDrag.

(WebCore::DragController::promisedAttachmentInfo):

Teach this to handle attachment elements as well as image elements that are backed by attachment elements.

  • page/DragController.h:
  • platform/PromisedAttachmentInfo.h:

(WebCore::PromisedAttachmentInfo::operator bool const):

A valid PromisedAttachmentInfo no longer requires a contentType to be set; instead, an attachment identifier
alone is sufficient, since an up-to-date content type can be requested in the UI process from the API attachment
object.

Source/WebKit:

Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
elements are enabled. See changes below for more detail.

  • Shared/WebCoreArgumentCoders.cpp:

(IPC::ArgumentCoder<PromisedAttachmentInfo>::encode):
(IPC::ArgumentCoder<PromisedAttachmentInfo>::decode):

Rename "filename" to "fileName", for consistency with WKContentView and WebViewImpl.

  • UIProcess/API/APIAttachment.cpp:

(API::Attachment::mimeType const):
(API::Attachment::fileName const):

  • UIProcess/API/APIAttachment.h:

Push getters for MIME type, UTI, and the file name down from _WKAttachment to API::Attachment. This allows
WKContentView and WebViewImpl to ask an API::Attachment questions about its UTI and file name without
additionally creating a wrapper object.

  • UIProcess/API/Cocoa/APIAttachmentCocoa.mm: Added.

(API::mimeTypeInferredFromFileExtension):
(API::isDeclaredOrDynamicTypeIdentifier):
(API::Attachment::mimeType const):
(API::Attachment::utiType const):
(API::Attachment::fileName const):

  • UIProcess/API/Cocoa/WKUIDelegatePrivate.h:

Add a private delegate hook to notify the UI delegate when a drop has been performed. This is used by tests to
know when a drop with file promises has been processed by the page.

  • UIProcess/API/Cocoa/WKWebView.mm:

(-[WKWebView _web_didPerformDragOperation:]):

  • UIProcess/API/Cocoa/_WKAttachment.mm:

(-[_WKAttachmentInfo initWithFileWrapper:filePath:mimeType:utiType:]):
(-[_WKAttachmentInfo fileWrapper]):
(-[_WKAttachmentInfo contentType]):
(-[_WKAttachment info]):
(-[_WKAttachmentInfo initWithFileWrapper:filePath:contentType:]): Deleted.
(isDeclaredOrDynamicTypeIdentifier): Deleted.
(-[_WKAttachmentInfo _typeIdentifierFromPathExtension]): Deleted.
(-[_WKAttachmentInfo mimeType]): Deleted.
(-[_WKAttachmentInfo utiType]): Deleted.

Moved to APIAttachmentCocoa.mm.

  • UIProcess/API/mac/WKView.mm:

(-[WKView _web_didPerformDragOperation:]):

  • UIProcess/Cocoa/WebViewImpl.h:
  • UIProcess/Cocoa/WebViewImpl.mm:

(-[WKPromisedAttachmentContext initWithIdentifier:blobURL:fileName:]):

Adjust this constructor to take each piece of data separately. This is because, in the case where our
PromisedAttachmentInfo contains an attachment identifier, we determine the UTI and file name from the associated
file wrapper.

(-[WKPromisedAttachmentContext fileName]):
(WebKit::WebViewImpl::fileNameForFilePromiseProvider):
(WebKit::WebViewImpl::didPerformDragOperation):
(WebKit::WebViewImpl::startDrag):

Determine UTI and file name from the attachment element corresponding to the attachment identifier when
dragging. This is because the attachment element in the web process shouldn't need to have type and title
attributes set when starting a drag if it already has an identifier that maps to attachment data in the UI
process. An example of this is in inline images backed by attachments, for which we don't need to bother keeping
specifying display attributes, since they are never visible.

(-[WKPromisedAttachmentContext initWithAttachmentInfo:]): Deleted.
(-[WKPromisedAttachmentContext filename]): Deleted.

  • UIProcess/PageClient.h:

(WebKit::PageClient::didPerformDragOperation):

  • UIProcess/WebPageProxy.cpp:

(WebKit::WebPageProxy::didPerformDragOperation):

  • UIProcess/WebPageProxy.h:
  • UIProcess/WebPageProxy.messages.in:

Rename DidPerformDataInteractionControllerOperation to DidPerformDragOperation, and make it cross-platform (this
was previously only sent on iOS). Add plumbing through PageClient and friends on macOS to notify the UI
delegate when a drop is handled by the web process.

  • UIProcess/ios/PageClientImplIOS.h:
  • UIProcess/ios/PageClientImplIOS.mm:

(WebKit::PageClientImpl::didPerformDragOperation):
(WebKit::PageClientImpl::didPerformDataInteractionControllerOperation): Deleted.

  • UIProcess/ios/WKContentViewInteraction.h:
  • UIProcess/ios/WKContentViewInteraction.mm:

(-[WKContentView _didPerformDragOperation:]):
(-[WKContentView _prepareToDragPromisedAttachment:]):

Just like in WebViewImpl::startDrag, infer content type and file name from the API attachment object.

(-[WKContentView _didPerformDataInteractionControllerOperation:]): Deleted.

  • UIProcess/ios/WebPageProxyIOS.mm:

(WebKit::WebPageProxy::didPerformDataInteractionControllerOperation): Deleted.

  • UIProcess/mac/PageClientImplMac.h:
  • UIProcess/mac/PageClientImplMac.mm:

(WebKit::PageClientImpl::didPerformDragOperation):

  • WebKit.xcodeproj/project.pbxproj:
  • WebProcess/WebPage/WebPage.cpp:

(WebKit::WebPage::performDragControllerAction):

Tools:

Rebaseline existing API tests that involve dropping or pasting image files, and additionally write some new
tests. These new tests exercise the following cases:

  • Inserting and removing newlines before an inline image with an attachment element does not cause new

_WKAttachments to be created and destroyed.

  • Pasting an image, cutting it, and then pasting it again propagates an attachment update to the UI

process with the original _WKAttachment.

  • A pasted attachment in the document can be moved around by dragging, and doing so does not cause us to

lose a _WKAttachment.

  • TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:

(-[TestWKWebView expectElementCount:tagName:]):
(TestWebKitAPI::TEST):

Add the new tests described above, and also adjust existing tests to check that images are dropped or pasted
as image elements, but still have associated attachment elements whose attachment identifiers (observed via
script) match that of the corresponding _WKAttachment's uniqueIdentifier in the UI process.

  • TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:

(-[DragAndDropSimulator runFrom:to:]):
(-[DragAndDropSimulator continueDragSession]):
(-[DragAndDropSimulator performDragInWebView:atLocation:withImage:pasteboard:source:]):

Teach DragAndDropSimulator on macOS to wait until the drop has been handled by the web process before returning
execution to the caller. This ensures that tests which involve dropping promised files as attachments aren't
flaky, due to how the promised data is retrieved asynchronously when performing the drop.

(-[DragAndDropSimulator _webView:didPerformDragOperation:]):

Location:
trunk
Files:
1 added
38 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r235342 r235343  
     12018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
     4        https://bugs.webkit.org/show_bug.cgi?id=188933
     5        <rdar://problem/43699724>
     6
     7        Reviewed by Darin Adler.
     8
     9        Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
     10        elements are enabled. See changes below for more detail.
     11
     12        Tests:  WKAttachmentTests.CutAndPastePastedImage
     13                WKAttachmentTests.MovePastedImageByDragging
     14                WKAttachmentTests.RemoveNewlinesBeforePastedImage
     15
     16        * editing/Editor.h:
     17        * editing/cocoa/EditorCocoa.mm:
     18        (WebCore::Editor::getPasteboardTypesAndDataForAttachment):
     19
     20        Adjust this helper to take an Element& rather than an HTMLAttachmentElement&, and address a FIXME by writing the
     21        document origin identifier to the pasteboard via custom pasteboard data when dragging an attachment. This allows
     22        us to avoid creating extra image and attachment elements when dragging an image backed by an attachment within
     23        the same document.
     24
     25        * editing/cocoa/WebContentReaderCocoa.mm:
     26        (WebCore::contentTypeIsSuitableForInlineImageRepresentation):
     27
     28        Add a helper to determine whether a content type (UTI or MIME type) should be read as an inline image.
     29
     30        (WebCore::createFragmentForImageAttachment):
     31        (WebCore::replaceRichContentWithAttachments):
     32        (WebCore::WebContentReader::readFilePaths):
     33
     34        Teach codepaths where we currently create attachment elements to instead create image elements if the MIME type,
     35        is something suitable for display via an inline image element; add the attachment element under the shadow root
     36        of the image element.
     37
     38        * editing/markup.cpp:
     39        (WebCore::StyledMarkupAccumulator::appendCustomAttributes):
     40        (WebCore::restoreAttachmentElementsInFragment):
     41
     42        When dragging or copying an image element, we need to make sure that any attachment element backing the image
     43        is preserved in the pasted or dropped fragment. To do this, we use a technique similar to what was done for
     44        r180785 and r224593 and write a temporary "webkitattachmentid" attribute to the serialized markup on copy. Upon
     45        deserializing the markup back to a fragment, we then create an attachment element with the same identifier under
     46        the image.
     47
     48        (WebCore::createFragmentFromMarkup):
     49        * html/HTMLAttachmentElement.h:
     50        * html/HTMLImageElement.cpp:
     51        (WebCore::HTMLImageElement::setAttachmentElement):
     52        (WebCore::HTMLImageElement::attachmentElement const):
     53
     54        Helper methods to get and set an attachment element under an image element. Setting an image's attachment
     55        element puts that attachment element under the shadow root of the image, and also hides the attachment element.
     56
     57        (WebCore::HTMLImageElement::attachmentIdentifier const):
     58
     59        Returns the identifier of an attachment element associated with the image element, or null.
     60
     61        * html/HTMLImageElement.h:
     62        * html/HTMLImageElement.idl:
     63
     64        Add HTMLImageElement.webkitAttachmentIdentifier, a readonly attribute guarded by runtime-enabled attachment
     65        element feature.
     66
     67        * page/DragController.cpp:
     68        (WebCore::DragController::startDrag):
     69
     70        In the case of dragging an image, if that image element is backed by an attachment element, don't bother writing
     71        the image data to the clipboard; instead, write the attachment data as a promise.
     72
     73        (WebCore::DragController::doImageDrag):
     74
     75        Plumb promised attachment information to DragController::doSystemDrag.
     76
     77        (WebCore::DragController::promisedAttachmentInfo):
     78
     79        Teach this to handle attachment elements as well as image elements that are backed by attachment elements.
     80
     81        * page/DragController.h:
     82        * platform/PromisedAttachmentInfo.h:
     83        (WebCore::PromisedAttachmentInfo::operator bool const):
     84
     85        A valid PromisedAttachmentInfo no longer requires a contentType to be set; instead, an attachment identifier
     86        alone is sufficient, since an up-to-date content type can be requested in the UI process from the API attachment
     87        object.
     88
    1892018-08-26  Andy Estes  <aestes@apple.com>
    290
  • trunk/Source/WebCore/editing/Editor.h

    r235137 r235343  
    512512
    513513#if PLATFORM(COCOA)
    514     void getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement&, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData);
     514    void getPasteboardTypesAndDataForAttachment(Element&, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData);
    515515#endif
    516516#endif
  • trunk/Source/WebCore/editing/cocoa/EditorCocoa.mm

    r235202 r235343  
    161161#if ENABLE(ATTACHMENT_ELEMENT)
    162162
    163 void Editor::getPasteboardTypesAndDataForAttachment(HTMLAttachmentElement& attachment, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData)
    164 {
    165     auto attachmentRange = Range::create(attachment.document(), { &attachment, Position::PositionIsBeforeAnchor }, { &attachment, Position::PositionIsAfterAnchor });
    166     client()->getClientPasteboardDataForRange(attachmentRange.ptr(), outTypes, outData);
    167     // FIXME: We should additionally write the attachment as a web archive here, such that drag and drop within the
    168     // same page doesn't destroy and recreate attachments unnecessarily. This is also needed to preserve the attachment
    169     // display mode when dragging and dropping or cutting and pasting. For the time being, this is disabled because
    170     // inserting attachment elements from web archive data sometimes causes attachment data to be lost; this requires
    171     // further investigation.
    172 #if PLATFORM(MAC)
    173     // On macOS, we currently write the attachment as a web archive; we can't do the same for iOS and remove the platform guard above
    174     // quite yet without breaking drag moves. This investigation is tracked in <https://bugs.webkit.org/show_bug.cgi?id=181514>.
    175     // See the above FIXME for more details.
    176     if (auto archive = LegacyWebArchive::create(attachmentRange.ptr())) {
     163void Editor::getPasteboardTypesAndDataForAttachment(Element& element, Vector<String>& outTypes, Vector<RefPtr<SharedBuffer>>& outData)
     164{
     165    auto& document = element.document();
     166    auto elementRange = Range::create(document, { &element, Position::PositionIsBeforeAnchor }, { &element, Position::PositionIsAfterAnchor });
     167    client()->getClientPasteboardDataForRange(elementRange.ptr(), outTypes, outData);
     168
     169    outTypes.append(PasteboardCustomData::cocoaType());
     170    outData.append(PasteboardCustomData { document.originIdentifierForPasteboard(), { }, { }, { } }.createSharedBuffer());
     171
     172    if (auto archive = LegacyWebArchive::create(elementRange.ptr())) {
    177173        if (auto webArchiveData = archive->rawDataRepresentation()) {
    178174            outTypes.append(WebArchivePboardType);
     
    180176        }
    181177    }
    182 #endif
    183178}
    184179
  • trunk/Source/WebCore/editing/cocoa/WebContentReaderCocoa.mm

    r235137 r235343  
    4848#import "HTMLObjectElement.h"
    4949#import "LegacyWebArchive.h"
     50#import "MIMETypeRegistry.h"
    5051#import "Page.h"
    5152#import "PublicURLManager.h"
     
    208209#if ENABLE(ATTACHMENT_ELEMENT)
    209210
     211static bool contentTypeIsSuitableForInlineImageRepresentation(const String& contentType)
     212{
     213    return MIMETypeRegistry::isSupportedImageMIMEType(isDeclaredUTI(contentType) ? MIMETypeFromUTI(contentType) : contentType);
     214}
     215
    210216static bool supportsClientSideAttachmentData(const Frame& frame)
    211217{
     
    225231    String defaultImageAttachmentName { "image"_s };
    226232
     233    auto fragment = document.createDocumentFragment();
    227234    if (supportsClientSideAttachmentData(frame)) {
    228         attachment->updateAttributes(buffer->size(), contentType, defaultImageAttachmentName);
    229235        frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), contentType, defaultImageAttachmentName, WTFMove(buffer));
    230     } else
     236        if (contentTypeIsSuitableForInlineImageRepresentation(contentType)) {
     237            auto image = HTMLImageElement::create(document);
     238            image->setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(document, Blob::create(buffer.get(), contentType)));
     239            image->setAttachmentElement(WTFMove(attachment));
     240            fragment->appendChild(WTFMove(image));
     241        } else {
     242            attachment->updateAttributes(buffer->size(), contentType, defaultImageAttachmentName);
     243            fragment->appendChild(WTFMove(attachment));
     244        }
     245    } else {
    231246        attachment->setFile(File::create(Blob::create(buffer.get(), contentType), defaultImageAttachmentName), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
    232 
    233     auto fragment = document.createDocumentFragment();
    234     fragment->appendChild(attachment);
    235 
     247        fragment->appendChild(WTFMove(attachment));
     248    }
    236249    return fragment;
    237250#else
     
    244257{
    245258#if ENABLE(ATTACHMENT_ELEMENT)
    246     struct AttachmentReplacementInfo {
     259    struct AttachmentInsertionInfo {
    247260        String fileName;
    248261        String contentType;
    249262        Ref<SharedBuffer> data;
    250         Ref<Element> elementToReplace;
     263        Ref<Element> originalElement;
    251264    };
    252265
     
    264277
    265278    Vector<Ref<Element>> elementsToRemove;
    266     Vector<AttachmentReplacementInfo> attachmentReplacementInfo;
     279    Vector<AttachmentInsertionInfo> attachmentInsertionInfo;
    267280    for (auto& image : descendantsOfType<HTMLImageElement>(fragment)) {
    268281        auto resourceURLString = image.attributeWithoutSynchronization(HTMLNames::srcAttr);
     
    278291            name = AtomicString("media");
    279292
    280         attachmentReplacementInfo.append({ name, resource->value->mimeType(), resource->value->data(), image });
     293        attachmentInsertionInfo.append({ name, resource->value->mimeType(), resource->value->data(), image });
    281294    }
    282295
     
    296309            name = AtomicString("file");
    297310
    298         attachmentReplacementInfo.append({ name, resource->value->mimeType(), resource->value->data(), object });
    299     }
    300 
    301     for (auto& info : attachmentReplacementInfo) {
    302         auto elementToReplace = WTFMove(info.elementToReplace);
    303         auto parent = makeRefPtr(elementToReplace->parentNode());
     311        attachmentInsertionInfo.append({ name, resource->value->mimeType(), resource->value->data(), object });
     312    }
     313
     314    for (auto& info : attachmentInsertionInfo) {
     315        auto originalElement = WTFMove(info.originalElement);
     316        auto parent = makeRefPtr(originalElement->parentNode());
    304317        if (!parent)
    305318            continue;
     
    307320        auto attachment = HTMLAttachmentElement::create(HTMLNames::attachmentTag, fragment.document());
    308321        if (supportsClientSideAttachmentData(frame)) {
    309             attachment->updateAttributes(info.data->size(), info.contentType, info.fileName);
     322            if (is<HTMLImageElement>(originalElement.get()) && contentTypeIsSuitableForInlineImageRepresentation(info.contentType)) {
     323                auto& image = downcast<HTMLImageElement>(originalElement.get());
     324                image.setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(*frame.document(), Blob::create(info.data, info.contentType)));
     325                image.setAttachmentElement(WTFMove(attachment));
     326            } else {
     327                attachment->updateAttributes(info.data->size(), info.contentType, info.fileName);
     328                parent->replaceChild(attachment, WTFMove(originalElement));
     329            }
    310330            frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), WTFMove(info.contentType), WTFMove(info.fileName), WTFMove(info.data));
    311         } else
    312             attachment->setFile(File::create(Blob::create(info.data, info.contentType), info.fileName), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
    313 
    314         parent->replaceChild(attachment, elementToReplace);
     331        } else {
     332            attachment->setFile(File::create(Blob::create(WTFMove(info.data), WTFMove(info.contentType)), WTFMove(info.fileName)), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
     333            parent->replaceChild(WTFMove(attachment), WTFMove(originalElement));
     334        }
    315335    }
    316336
     
    685705                FileSystem::getFileSize(path, fileSize);
    686706                auto contentType = File::contentTypeForFile(path);
    687                 attachment->updateAttributes(fileSize, contentType, FileSystem::pathGetFileName(path));
    688707                frame.editor().registerAttachmentIdentifier(attachment->ensureUniqueIdentifier(), contentType, path);
    689             } else
     708                if (contentTypeIsSuitableForInlineImageRepresentation(contentType)) {
     709                    auto image = HTMLImageElement::create(document);
     710                    image->setAttributeWithoutSynchronization(HTMLNames::srcAttr, DOMURL::createObjectURL(document, File::create(path)));
     711                    image->setAttachmentElement(WTFMove(attachment));
     712                    fragment->appendChild(image);
     713                } else {
     714                    attachment->updateAttributes(fileSize, contentType, FileSystem::pathGetFileName(path));
     715                    fragment->appendChild(attachment);
     716                }
     717            } else {
    690718                attachment->setFile(File::create(path), HTMLAttachmentElement::UpdateDisplayAttributes::Yes);
    691 
    692             fragment->appendChild(attachment);
     719                fragment->appendChild(attachment);
     720            }
    693721        }
    694722    }
  • trunk/Source/WebCore/editing/markup.cpp

    r234278 r235343  
    6868#include "Range.h"
    6969#include "RenderBlock.h"
     70#include "RuntimeEnabledFeatures.h"
    7071#include "Settings.h"
    7172#include "SocketProvider.h"
     
    407408{
    408409#if ENABLE(ATTACHMENT_ELEMENT)
    409     if (!is<HTMLAttachmentElement>(element))
     410    if (!RuntimeEnabledFeatures::sharedFeatures().attachmentElementEnabled())
    410411        return;
    411412   
    412     const HTMLAttachmentElement& attachment = downcast<HTMLAttachmentElement>(element);
    413     appendAttribute(out, element, { webkitattachmentidAttr, attachment.uniqueIdentifier() }, namespaces);
    414     if (auto* file = attachment.file()) {
    415         // These attributes are only intended for File deserialization, and are removed from the generated attachment
    416         // element after we've deserialized and set its backing File.
    417         appendAttribute(out, element, { webkitattachmentpathAttr, file->path() }, namespaces);
    418         appendAttribute(out, element, { webkitattachmentbloburlAttr, file->url().string() }, namespaces);
     413    if (is<HTMLAttachmentElement>(element)) {
     414        auto& attachment = downcast<HTMLAttachmentElement>(element);
     415        appendAttribute(out, element, { webkitattachmentidAttr, attachment.uniqueIdentifier() }, namespaces);
     416        if (auto* file = attachment.file()) {
     417            // These attributes are only intended for File deserialization, and are removed from the generated attachment
     418            // element after we've deserialized and set its backing File, in restoreAttachmentElementsInFragment.
     419            appendAttribute(out, element, { webkitattachmentpathAttr, file->path() }, namespaces);
     420            appendAttribute(out, element, { webkitattachmentbloburlAttr, file->url().string() }, namespaces);
     421        }
     422    } else if (is<HTMLImageElement>(element)) {
     423        if (auto attachment = downcast<HTMLImageElement>(element).attachmentElement())
     424            appendAttribute(out, element, { webkitattachmentidAttr, attachment->uniqueIdentifier() }, namespaces);
    419425    }
    420426#else
     
    882888}
    883889
    884 Ref<DocumentFragment> createFragmentFromMarkup(Document& document, const String& markup, const String& baseURL, ParserContentPolicy parserContentPolicy)
    885 {
    886     // We use a fake body element here to trick the HTML parser to using the InBody insertion mode.
    887     auto fakeBody = HTMLBodyElement::create(document);
    888     auto fragment = DocumentFragment::create(document);
    889 
    890     fragment->parseHTML(markup, fakeBody.ptr(), parserContentPolicy);
    891 
     890static void restoreAttachmentElementsInFragment(DocumentFragment& fragment)
     891{
    892892#if ENABLE(ATTACHMENT_ELEMENT)
     893    if (!RuntimeEnabledFeatures::sharedFeatures().attachmentElementEnabled())
     894        return;
     895
    893896    // When creating a fragment we must strip the webkit-attachment-path attribute after restoring the File object.
    894897    Vector<Ref<HTMLAttachmentElement>> attachments;
     
    906909            attachment->setFile(File::deserialize({ }, blobURL, attachment->attachmentType(), attachment->attachmentTitle()));
    907910
     911        // Remove temporary attributes that were previously added in StyledMarkupAccumulator::appendCustomAttributes.
    908912        attachment->removeAttribute(webkitattachmentidAttr);
    909913        attachment->removeAttribute(webkitattachmentpathAttr);
    910914        attachment->removeAttribute(webkitattachmentbloburlAttr);
    911915    }
     916
     917    Vector<Ref<HTMLImageElement>> images;
     918    for (auto& image : descendantsOfType<HTMLImageElement>(fragment))
     919        images.append(image);
     920
     921    for (auto& image : images) {
     922        auto attachmentIdentifier = image->attributeWithoutSynchronization(webkitattachmentidAttr);
     923        if (attachmentIdentifier.isEmpty())
     924            continue;
     925
     926        auto attachment = HTMLAttachmentElement::create(HTMLNames::attachmentTag, *fragment.ownerDocument());
     927        attachment->setUniqueIdentifier(attachmentIdentifier);
     928        image->setAttachmentElement(WTFMove(attachment));
     929        image->removeAttribute(webkitattachmentidAttr);
     930    }
     931#else
     932    UNUSED_PARAM(fragment);
    912933#endif
     934}
     935
     936Ref<DocumentFragment> createFragmentFromMarkup(Document& document, const String& markup, const String& baseURL, ParserContentPolicy parserContentPolicy)
     937{
     938    // We use a fake body element here to trick the HTML parser into using the InBody insertion mode.
     939    auto fakeBody = HTMLBodyElement::create(document);
     940    auto fragment = DocumentFragment::create(document);
     941
     942    fragment->parseHTML(markup, fakeBody.ptr(), parserContentPolicy);
     943    restoreAttachmentElementsInFragment(fragment);
    913944    if (!baseURL.isEmpty() && baseURL != blankURL() && baseURL != document.baseURL())
    914945        completeURLs(fragment.ptr(), baseURL);
  • trunk/Source/WebCore/html/HTMLAttachmentElement.h

    r235137 r235343  
    4747    void setFile(RefPtr<File>&&, UpdateDisplayAttributes = UpdateDisplayAttributes::No);
    4848
    49     String uniqueIdentifier() const { return m_uniqueIdentifier; }
     49    const String& uniqueIdentifier() const { return m_uniqueIdentifier; }
    5050    void setUniqueIdentifier(const String& uniqueIdentifier) { m_uniqueIdentifier = uniqueIdentifier; }
    5151
  • trunk/Source/WebCore/html/HTMLImageElement.cpp

    r234610 r235343  
    2727#include "CSSValueKeywords.h"
    2828#include "CachedImage.h"
     29#include "ElementIterator.h"
    2930#include "FrameView.h"
    3031#include "HTMLAnchorElement.h"
     32#include "HTMLAttachmentElement.h"
    3133#include "HTMLDocument.h"
    3234#include "HTMLFormElement.h"
     
    604606}
    605607
     608#if ENABLE(ATTACHMENT_ELEMENT)
     609
     610void HTMLImageElement::setAttachmentElement(Ref<HTMLAttachmentElement>&& attachment)
     611{
     612    if (auto existingAttachment = attachmentElement())
     613        existingAttachment->remove();
     614
     615    attachment->setInlineStyleProperty(CSSPropertyDisplay, CSSValueNone, true);
     616    ensureUserAgentShadowRoot().appendChild(WTFMove(attachment));
     617}
     618
     619RefPtr<HTMLAttachmentElement> HTMLImageElement::attachmentElement() const
     620{
     621    if (auto shadowRoot = userAgentShadowRoot())
     622        return childrenOfType<HTMLAttachmentElement>(*shadowRoot).first();
     623
     624    return nullptr;
     625}
     626
     627const String& HTMLImageElement::attachmentIdentifier() const
     628{
     629    if (auto attachment = attachmentElement())
     630        return attachment->uniqueIdentifier();
     631
     632    return nullAtom();
     633}
     634
     635#endif // ENABLE(ATTACHMENT_ELEMENT)
     636
    606637#if ENABLE(SERVICE_CONTROLS)
    607638void HTMLImageElement::updateImageControls()
  • trunk/Source/WebCore/html/HTMLImageElement.h

    r231329 r235343  
    3232namespace WebCore {
    3333
     34class HTMLAttachmentElement;
    3435class HTMLFormElement;
    3536class HTMLMapElement;
     
    9091#if PLATFORM(IOS)
    9192    bool willRespondToMouseClickEvents() override;
     93#endif
     94
     95#if ENABLE(ATTACHMENT_ELEMENT)
     96    void setAttachmentElement(Ref<HTMLAttachmentElement>&&);
     97    RefPtr<HTMLAttachmentElement> attachmentElement() const;
     98    const String& attachmentIdentifier() const;
    9299#endif
    93100
  • trunk/Source/WebCore/html/HTMLImageElement.idl

    r225616 r235343  
    4343    [Reflect] attribute DOMString decoding;
    4444
     45    [Conditional=ATTACHMENT_ELEMENT, EnabledAtRuntime=AttachmentElement, ImplementedAs=attachmentIdentifier] readonly attribute DOMString webkitAttachmentIdentifier;
     46
    4547    // Extensions
    4648    readonly attribute boolean complete;
  • trunk/Source/WebCore/page/DragController.cpp

    r235137 r235343  
    911911    includeShadowDOM = state.source->isMediaElement();
    912912#endif
    913 #if ENABLE(ATTACHMENT_ELEMENT)
    914     includeShadowDOM = includeShadowDOM || is<HTMLAttachmentElement>(state.source.get());
    915 #endif
    916913#if ENABLE(INPUT_TYPE_COLOR)
    917914    bool isColorControl = is<HTMLInputElement>(state.source) && downcast<HTMLInputElement>(*state.source).isColorControl();
     
    10491046        // This is an early detection for problems encountered later upon drop.
    10501047        ASSERT(!image->filenameExtension().isEmpty());
     1048
     1049#if ENABLE(ATTACHMENT_ELEMENT)
     1050        auto attachmentInfo = promisedAttachmentInfo(src, element);
     1051#else
     1052        PromisedAttachmentInfo attachmentInfo;
     1053#endif
     1054
    10511055        if (hasData == HasNonDefaultPasteboardData::No) {
    10521056            m_draggingImageURL = imageURL;
    10531057            if (element.isContentRichlyEditable())
    10541058                selectElement(element);
    1055             declareAndWriteDragImage(dataTransfer, element, !linkURL.isEmpty() ? linkURL : imageURL, hitTestResult.altDisplayString());
     1059            if (!attachmentInfo)
     1060                declareAndWriteDragImage(dataTransfer, element, !linkURL.isEmpty() ? linkURL : imageURL, hitTestResult.altDisplayString());
    10561061        }
    10571062
     
    10591064
    10601065        if (!dragImage)
    1061             doImageDrag(element, dragOrigin, hitTestResult.imageRect(), src, m_dragOffset, state);
     1066            doImageDrag(element, dragOrigin, hitTestResult.imageRect(), src, m_dragOffset, state, WTFMove(attachmentInfo));
    10621067        else {
    10631068            // DHTML defined drag image
    1064             doSystemDrag(WTFMove(dragImage), dragLoc, dragOrigin, src, state, { });
     1069            doSystemDrag(WTFMove(dragImage), dragLoc, dragOrigin, src, state, WTFMove(attachmentInfo));
    10651070        }
    10661071
     
    12011206}
    12021207
    1203 void DragController::doImageDrag(Element& element, const IntPoint& dragOrigin, const IntRect& layoutRect, Frame& frame, IntPoint& dragImageOffset, const DragState& state)
     1208void DragController::doImageDrag(Element& element, const IntPoint& dragOrigin, const IntRect& layoutRect, Frame& frame, IntPoint& dragImageOffset, const DragState& state, PromisedAttachmentInfo&& attachmentInfo)
    12041209{
    12051210    IntPoint mouseDownPoint = dragOrigin;
     
    12441249
    12451250    dragImageOffset = mouseDownPoint + scaledOrigin;
    1246     doSystemDrag(WTFMove(dragImage), dragImageOffset, dragOrigin, frame, state, { });
     1251    doSystemDrag(WTFMove(dragImage), dragImageOffset, dragOrigin, frame, state, WTFMove(attachmentInfo));
    12471252}
    12481253
     
    13681373#if ENABLE(ATTACHMENT_ELEMENT)
    13691374
    1370 PromisedAttachmentInfo DragController::promisedAttachmentInfo(Frame& frame, HTMLAttachmentElement& attachment)
    1371 {
     1375PromisedAttachmentInfo DragController::promisedAttachmentInfo(Frame& frame, Element& element)
     1376{
     1377    auto* client = frame.editor().client();
     1378    if (!client || !client->supportsClientSideAttachmentData())
     1379        return { };
     1380
     1381    RefPtr<HTMLAttachmentElement> attachment;
     1382    if (is<HTMLAttachmentElement>(element))
     1383        attachment = &downcast<HTMLAttachmentElement>(element);
     1384    else if (is<HTMLImageElement>(element))
     1385        attachment = downcast<HTMLImageElement>(element).attachmentElement();
     1386
     1387    if (!attachment)
     1388        return { };
     1389
    13721390    Vector<String> additionalTypes;
    13731391    Vector<RefPtr<SharedBuffer>> additionalData;
    13741392#if PLATFORM(COCOA)
    1375     if (frame.editor().client())
    1376         frame.editor().getPasteboardTypesAndDataForAttachment(attachment, additionalTypes, additionalData);
    1377 #endif
    1378 
    1379     if (auto* file = attachment.file())
    1380         return { file->url(), platformContentTypeForBlobType(file->type()), file->name(), attachment.uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
    1381 
    1382     return { { }, platformContentTypeForBlobType(attachment.attachmentType()), attachment.attachmentTitle(), attachment.uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
     1393    frame.editor().getPasteboardTypesAndDataForAttachment(element, additionalTypes, additionalData);
     1394#endif
     1395
     1396    if (auto* file = attachment->file())
     1397        return { file->url(), platformContentTypeForBlobType(file->type()), file->name(), { }, WTFMove(additionalTypes), WTFMove(additionalData) };
     1398
     1399    return { { }, { }, { }, attachment->uniqueIdentifier(), WTFMove(additionalTypes), WTFMove(additionalData) };
    13831400}
    13841401
  • trunk/Source/WebCore/page/DragController.h

    r235137 r235343  
    4141class Frame;
    4242class FrameSelection;
    43 class HTMLAttachmentElement;
    4443class HTMLInputElement;
    4544class IntRect;
     
    116115        bool shouldUseCachedImageForDragImage(const Image&) const;
    117116
    118         void doImageDrag(Element&, const IntPoint&, const IntRect&, Frame&, IntPoint&, const DragState&);
     117        void doImageDrag(Element&, const IntPoint&, const IntRect&, Frame&, IntPoint&, const DragState&, PromisedAttachmentInfo&&);
    119118        void doSystemDrag(DragImage, const IntPoint&, const IntPoint&, Frame&, const DragState&, PromisedAttachmentInfo&&);
    120119
     
    136135
    137136#if ENABLE(ATTACHMENT_ELEMENT)
    138         PromisedAttachmentInfo promisedAttachmentInfo(Frame&, HTMLAttachmentElement&);
     137        PromisedAttachmentInfo promisedAttachmentInfo(Frame&, Element&);
    139138#endif
    140139        Page& m_page;
  • trunk/Source/WebCore/platform/PromisedAttachmentInfo.h

    r235137 r235343  
    3737    URL blobURL;
    3838    String contentType;
    39     String filename;
     39    String fileName;
    4040
    4141#if ENABLE(ATTACHMENT_ELEMENT)
     
    4848    operator bool() const
    4949    {
    50         if (contentType.isEmpty())
    51             return false;
    52 
    5350#if ENABLE(ATTACHMENT_ELEMENT)
    5451        if (!attachmentIdentifier.isEmpty())
     
    5653#endif
    5754
    58         return !blobURL.isEmpty();
     55        return !contentType.isEmpty() && !blobURL.isEmpty();
    5956    }
    6057};
  • trunk/Source/WebKit/ChangeLog

    r235332 r235343  
     12018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
     4        https://bugs.webkit.org/show_bug.cgi?id=188933
     5        <rdar://problem/43699724>
     6
     7        Reviewed by Darin Adler.
     8
     9        Support the ability to drop and paste images as image elements, with attachment elements, only if attachment
     10        elements are enabled. See changes below for more detail.
     11
     12        * Shared/WebCoreArgumentCoders.cpp:
     13        (IPC::ArgumentCoder<PromisedAttachmentInfo>::encode):
     14        (IPC::ArgumentCoder<PromisedAttachmentInfo>::decode):
     15
     16        Rename "filename" to "fileName", for consistency with WKContentView and WebViewImpl.
     17
     18        * UIProcess/API/APIAttachment.cpp:
     19        (API::Attachment::mimeType const):
     20        (API::Attachment::fileName const):
     21        * UIProcess/API/APIAttachment.h:
     22
     23        Push getters for MIME type, UTI, and the file name down from _WKAttachment to API::Attachment. This allows
     24        WKContentView and WebViewImpl to ask an API::Attachment questions about its UTI and file name without
     25        additionally creating a wrapper object.
     26
     27        * UIProcess/API/Cocoa/APIAttachmentCocoa.mm: Added.
     28        (API::mimeTypeInferredFromFileExtension):
     29        (API::isDeclaredOrDynamicTypeIdentifier):
     30        (API::Attachment::mimeType const):
     31        (API::Attachment::utiType const):
     32        (API::Attachment::fileName const):
     33        * UIProcess/API/Cocoa/WKUIDelegatePrivate.h:
     34
     35        Add a private delegate hook to notify the UI delegate when a drop has been performed. This is used by tests to
     36        know when a drop with file promises has been processed by the page.
     37
     38        * UIProcess/API/Cocoa/WKWebView.mm:
     39        (-[WKWebView _web_didPerformDragOperation:]):
     40        * UIProcess/API/Cocoa/_WKAttachment.mm:
     41        (-[_WKAttachmentInfo initWithFileWrapper:filePath:mimeType:utiType:]):
     42        (-[_WKAttachmentInfo fileWrapper]):
     43        (-[_WKAttachmentInfo contentType]):
     44        (-[_WKAttachment info]):
     45        (-[_WKAttachmentInfo initWithFileWrapper:filePath:contentType:]): Deleted.
     46        (isDeclaredOrDynamicTypeIdentifier): Deleted.
     47        (-[_WKAttachmentInfo _typeIdentifierFromPathExtension]): Deleted.
     48        (-[_WKAttachmentInfo mimeType]): Deleted.
     49        (-[_WKAttachmentInfo utiType]): Deleted.
     50
     51        Moved to APIAttachmentCocoa.mm.
     52
     53        * UIProcess/API/mac/WKView.mm:
     54        (-[WKView _web_didPerformDragOperation:]):
     55        * UIProcess/Cocoa/WebViewImpl.h:
     56        * UIProcess/Cocoa/WebViewImpl.mm:
     57        (-[WKPromisedAttachmentContext initWithIdentifier:blobURL:fileName:]):
     58
     59        Adjust this constructor to take each piece of data separately. This is because, in the case where our
     60        PromisedAttachmentInfo contains an attachment identifier, we determine the UTI and file name from the associated
     61        file wrapper.
     62
     63        (-[WKPromisedAttachmentContext fileName]):
     64        (WebKit::WebViewImpl::fileNameForFilePromiseProvider):
     65        (WebKit::WebViewImpl::didPerformDragOperation):
     66        (WebKit::WebViewImpl::startDrag):
     67
     68        Determine UTI and file name from the attachment element corresponding to the attachment identifier when
     69        dragging. This is because the attachment element in the web process shouldn't need to have type and title
     70        attributes set when starting a drag if it already has an identifier that maps to attachment data in the UI
     71        process. An example of this is in inline images backed by attachments, for which we don't need to bother keeping
     72        specifying display attributes, since they are never visible.
     73
     74        (-[WKPromisedAttachmentContext initWithAttachmentInfo:]): Deleted.
     75        (-[WKPromisedAttachmentContext filename]): Deleted.
     76        * UIProcess/PageClient.h:
     77        (WebKit::PageClient::didPerformDragOperation):
     78        * UIProcess/WebPageProxy.cpp:
     79        (WebKit::WebPageProxy::didPerformDragOperation):
     80        * UIProcess/WebPageProxy.h:
     81        * UIProcess/WebPageProxy.messages.in:
     82
     83        Rename DidPerformDataInteractionControllerOperation to DidPerformDragOperation, and make it cross-platform (this
     84        was previously only sent on iOS). Add plumbing through PageClient and friends on macOS to notify the UI
     85        delegate when a drop is handled by the web process.
     86
     87        * UIProcess/ios/PageClientImplIOS.h:
     88        * UIProcess/ios/PageClientImplIOS.mm:
     89        (WebKit::PageClientImpl::didPerformDragOperation):
     90        (WebKit::PageClientImpl::didPerformDataInteractionControllerOperation): Deleted.
     91        * UIProcess/ios/WKContentViewInteraction.h:
     92        * UIProcess/ios/WKContentViewInteraction.mm:
     93        (-[WKContentView _didPerformDragOperation:]):
     94        (-[WKContentView _prepareToDragPromisedAttachment:]):
     95
     96        Just like in WebViewImpl::startDrag, infer content type and file name from the API attachment object.
     97
     98        (-[WKContentView _didPerformDataInteractionControllerOperation:]): Deleted.
     99        * UIProcess/ios/WebPageProxyIOS.mm:
     100        (WebKit::WebPageProxy::didPerformDataInteractionControllerOperation): Deleted.
     101        * UIProcess/mac/PageClientImplMac.h:
     102        * UIProcess/mac/PageClientImplMac.mm:
     103        (WebKit::PageClientImpl::didPerformDragOperation):
     104        * WebKit.xcodeproj/project.pbxproj:
     105        * WebProcess/WebPage/WebPage.cpp:
     106        (WebKit::WebPage::performDragControllerAction):
     107
    11082018-08-23  Jeff Miller  <jeffm@apple.com>
    2109
  • trunk/Source/WebKit/Shared/WebCoreArgumentCoders.cpp

    r235137 r235343  
    28682868    encoder << info.blobURL;
    28692869    encoder << info.contentType;
    2870     encoder << info.filename;
     2870    encoder << info.fileName;
    28712871#if ENABLE(ATTACHMENT_ELEMENT)
    28722872    encoder << info.attachmentIdentifier;
     
    28832883        return false;
    28842884
    2885     if (!decoder.decode(info.filename))
     2885    if (!decoder.decode(info.fileName))
    28862886        return false;
    28872887
  • trunk/Source/WebKit/UIProcess/API/APIAttachment.cpp

    r235156 r235343  
    8282}
    8383
     84#if !PLATFORM(COCOA)
     85
     86WTF::String Attachment::mimeType() const
     87{
     88    return m_contentType;
     89}
     90
     91WTF::String Attachment::fileName() const
     92{
     93    return { };
     94}
     95
     96#endif // !PLATFORM(COCOA)
     97
    8498}
    8599
  • trunk/Source/WebKit/UIProcess/API/APIAttachment.h

    r235156 r235343  
    6565    NSFileWrapper *fileWrapper() const { return m_fileWrapper.get(); }
    6666    void setFileWrapper(NSFileWrapper *fileWrapper) { m_fileWrapper = fileWrapper; }
     67    WTF::String utiType() const;
    6768#endif
     69    WTF::String mimeType() const;
    6870
    6971    const WTF::String& filePath() const { return m_filePath; }
    7072    void setFilePath(const WTF::String& filePath) { m_filePath = filePath; }
     73    WTF::String fileName() const;
    7174
    7275    const WTF::String& contentType() const { return m_contentType; }
  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKUIDelegatePrivate.h

    r235259 r235343  
    192192- (NSMenu *)_webView:(WKWebView *)webView contextMenu:(NSMenu *)menu forElement:(_WKContextMenuElementInfo *)element userInfo:(id <NSSecureCoding>)userInfo WK_API_DEPRECATED_WITH_REPLACEMENT("_webView:getContextMenuFromProposedMenu:forElement:userInfo:completionHandler:", macosx(10.12, WK_MAC_TBA));
    193193- (void)_webView:(WKWebView *)webView getContextMenuFromProposedMenu:(NSMenu *)menu forElement:(_WKContextMenuElementInfo *)element userInfo:(id <NSSecureCoding>)userInfo completionHandler:(void (^)(NSMenu *))completionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA));
     194- (void)_webView:(WKWebView *)webView didPerformDragOperation:(BOOL)handled WK_API_AVAILABLE(macosx(WK_MAC_TBA));
    194195#endif // TARGET_OS_IPHONE
    195196
  • trunk/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm

    r235328 r235343  
    40694069}
    40704070
     4071- (void)_web_didPerformDragOperation:(BOOL)handled
     4072{
     4073    id <WKUIDelegatePrivate> uiDelegate = (id <WKUIDelegatePrivate>)self.UIDelegate;
     4074    if ([uiDelegate respondsToSelector:@selector(_webView:didPerformDragOperation:)])
     4075        [uiDelegate _webView:self didPerformDragOperation:handled];
     4076}
     4077
    40714078#endif
    40724079
  • trunk/Source/WebKit/UIProcess/API/Cocoa/_WKAttachment.mm

    r235156 r235343  
    5757@implementation _WKAttachmentInfo {
    5858    RetainPtr<NSFileWrapper> _fileWrapper;
    59     RetainPtr<NSString> _contentType;
     59    RetainPtr<NSString> _mimeType;
     60    RetainPtr<NSString> _utiType;
    6061    RetainPtr<NSString> _filePath;
    6162}
    6263
    63 - (instancetype)initWithFileWrapper:(NSFileWrapper *)fileWrapper filePath:(NSString *)filePath contentType:(NSString *)contentType
     64- (instancetype)initWithFileWrapper:(NSFileWrapper *)fileWrapper filePath:(NSString *)filePath mimeType:(NSString *)mimeType utiType:(NSString *)utiType
    6465{
    6566    if (!(self = [super init]))
     
    6869    _fileWrapper = fileWrapper;
    6970    _filePath = filePath;
    70     _contentType = contentType;
     71    _mimeType = mimeType;
     72    _utiType = utiType;
    7173    return self;
    7274}
     
    9597}
    9698
    97 static BOOL isDeclaredOrDynamicTypeIdentifier(NSString *type)
    98 {
    99     return UTTypeIsDeclared((CFStringRef)type) || UTTypeIsDynamic((CFStringRef)type);
    100 }
    101 
    102 - (NSString *)_typeIdentifierFromPathExtension
    103 {
    104     if (NSString *extension = self.name.pathExtension)
    105         return WebCore::MIMETypeRegistry::getMIMETypeForExtension(extension);
    106 
    107     return nil;
     99- (NSFileWrapper *)fileWrapper
     100{
     101    return _fileWrapper.get();
    108102}
    109103
    110104- (NSString *)contentType
    111105{
    112     // A "content type" can refer to either a UTI or a MIME type. We prefer MIME type here to preserve existing behavior.
    113     return self.mimeType ?: self.utiType;
    114 }
    115 
    116 - (NSString *)mimeType
    117 {
    118     NSString *contentType = [_contentType length] ? _contentType.get() : [self _typeIdentifierFromPathExtension];
    119     if (!isDeclaredOrDynamicTypeIdentifier(contentType))
    120         return contentType;
    121 
    122     auto mimeType = adoptCF(UTTypeCopyPreferredTagWithClass((CFStringRef)contentType, kUTTagClassMIMEType));
    123     return (NSString *)mimeType.autorelease();
    124 }
    125 
    126 - (NSString *)utiType
    127 {
    128     NSString *contentType = [_contentType length] ? _contentType.get() : [self _typeIdentifierFromPathExtension];
    129     if (isDeclaredOrDynamicTypeIdentifier(contentType))
    130         return contentType;
    131 
    132     auto utiType = adoptCF(UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (CFStringRef)contentType, nil));
    133     return (NSString *)utiType.autorelease();
    134 }
    135 
    136 - (NSFileWrapper *)fileWrapper
    137 {
    138     return _fileWrapper.get();
     106    if ([_mimeType length])
     107        return _mimeType.get();
     108
     109    return _utiType.get();
    139110}
    140111
     
    153124        return nil;
    154125
    155     return [[[_WKAttachmentInfo alloc] initWithFileWrapper:_attachment->fileWrapper() filePath:_attachment->filePath() contentType:_attachment->contentType()] autorelease];
     126    return [[[_WKAttachmentInfo alloc] initWithFileWrapper:_attachment->fileWrapper() filePath:_attachment->filePath() mimeType:_attachment->mimeType() utiType:_attachment->utiType()] autorelease];
    156127}
    157128
  • trunk/Source/WebKit/UIProcess/API/mac/WKView.mm

    r235202 r235343  
    10321032#if ENABLE(DRAG_SUPPORT) && WK_API_ENABLED
    10331033
     1034- (void)_web_didPerformDragOperation:(BOOL)handled
     1035{
     1036    UNUSED_PARAM(handled);
     1037}
     1038
    10341039- (WKDragDestinationAction)_web_dragDestinationActionForDraggingInfo:(id <NSDraggingInfo>)draggingInfo
    10351040{
  • trunk/Source/WebKit/UIProcess/Cocoa/WebViewImpl.h

    r235202 r235343  
    105105#if ENABLE(DRAG_SUPPORT) && WK_API_ENABLED
    106106- (WKDragDestinationAction)_web_dragDestinationActionForDraggingInfo:(id <NSDraggingInfo>)draggingInfo;
     107- (void)_web_didPerformDragOperation:(BOOL)handled;
    107108#endif
    108109
     
    431432    NSString *fileNameForFilePromiseProvider(NSFilePromiseProvider *, NSString *fileType);
    432433    void writeToURLForFilePromiseProvider(NSFilePromiseProvider *, NSURL *, void(^)(NSError *));
     434
     435    void didPerformDragOperation(bool handled);
    433436#endif
    434437
  • trunk/Source/WebKit/UIProcess/Cocoa/WebViewImpl.mm

    r235265 r235343  
    881881@private
    882882    RetainPtr<NSURL> _blobURL;
    883     RetainPtr<NSString> _filename;
     883    RetainPtr<NSString> _fileName;
    884884    RetainPtr<NSString> _attachmentIdentifier;
    885885}
    886886
    887 - (instancetype)initWithAttachmentInfo:(const WebCore::PromisedAttachmentInfo&)info;
     887- (instancetype)initWithIdentifier:(NSString *)identifier blobURL:(NSURL *)url fileName:(NSString *)fileName;
    888888
    889889@property (nonatomic, readonly) NSURL *blobURL;
    890 @property (nonatomic, readonly) NSString *filename;
     890@property (nonatomic, readonly) NSString *fileName;
    891891@property (nonatomic, readonly) NSString *attachmentIdentifier;
    892892
     
    895895@implementation WKPromisedAttachmentContext
    896896
    897 - (instancetype)initWithAttachmentInfo:(const WebCore::PromisedAttachmentInfo&)info
     897- (instancetype)initWithIdentifier:(NSString *)identifier blobURL:(NSURL *)blobURL fileName:(NSString *)fileName
    898898{
    899899    if (!(self = [super init]))
    900900        return nil;
    901901
    902     _blobURL = info.blobURL;
    903     _filename = info.filename;
    904     _attachmentIdentifier = info.attachmentIdentifier;
     902    _blobURL = blobURL;
     903    _fileName = fileName;
     904    _attachmentIdentifier = identifier;
    905905    return self;
    906906}
     
    911911}
    912912
    913 - (NSString *)filename
    914 {
    915     return _filename.get();
     913- (NSString *)fileName
     914{
     915    return _fileName.get();
    916916}
    917917
     
    39093909        return nil;
    39103910
    3911     return [(WKPromisedAttachmentContext *)userInfo filename];
     3911    return [(WKPromisedAttachmentContext *)userInfo fileName];
    39123912}
    39133913
     
    39183918#else
    39193919    return [NSError errorWithDomain:@"WKErrorDomain" code:1 userInfo:nil];
     3920#endif
     3921}
     3922
     3923void WebViewImpl::didPerformDragOperation(bool handled)
     3924{
     3925#if WK_API_ENABLED
     3926    [m_view _web_didPerformDragOperation:handled];
     3927#else
     3928    UNUSED_PARAM(handled);
    39203929#endif
    39213930}
     
    39934002#pragma clang diagnostic pop
    39944003
    3995     if (auto& attachmentInfo = item.promisedAttachmentInfo) {
    3996         auto provider = adoptNS([[NSFilePromiseProvider alloc] initWithFileType:attachmentInfo.contentType delegate:(id <NSFilePromiseProviderDelegate>)m_view.getAutoreleased()]);
    3997         [provider setUserInfo:[[[WKPromisedAttachmentContext alloc] initWithAttachmentInfo:attachmentInfo] autorelease]];
     4004    if (auto& info = item.promisedAttachmentInfo) {
     4005        NSString *utiType = info.contentType;
     4006        NSString *fileName = info.fileName;
     4007        if (auto attachment = m_page->attachmentForIdentifier(info.attachmentIdentifier)) {
     4008            utiType = attachment->utiType();
     4009            fileName = attachment->fileName();
     4010        }
     4011
     4012        auto provider = adoptNS([[NSFilePromiseProvider alloc] initWithFileType:utiType delegate:(id <NSFilePromiseProviderDelegate>)m_view.getAutoreleased()]);
     4013        auto context = adoptNS([[WKPromisedAttachmentContext alloc] initWithIdentifier:info.attachmentIdentifier blobURL:info.blobURL fileName:fileName]);
     4014        [provider setUserInfo:context.get()];
    39984015        auto draggingItem = adoptNS([[NSDraggingItem alloc] initWithPasteboardWriter:provider.get()]);
    39994016        [draggingItem setDraggingFrame:NSMakeRect(clientDragLocation.x(), clientDragLocation.y() - size.height(), size.width(), size.height()) contents:dragNSImage.get()];
    40004017        [m_view beginDraggingSessionWithItems:@[draggingItem.get()] event:m_lastMouseDownEvent.get() source:(id <NSDraggingSource>)m_view.getAutoreleased()];
    40014018
    4002         ASSERT(attachmentInfo.additionalTypes.size() == attachmentInfo.additionalData.size());
    4003         if (attachmentInfo.additionalTypes.size() == attachmentInfo.additionalData.size()) {
    4004             for (size_t index = 0; index < attachmentInfo.additionalTypes.size(); ++index) {
    4005                 auto nsData = attachmentInfo.additionalData[index]->createNSData();
    4006                 [pasteboard setData:nsData.get() forType:attachmentInfo.additionalTypes[index]];
     4019        ASSERT(info.additionalTypes.size() == info.additionalData.size());
     4020        if (info.additionalTypes.size() == info.additionalData.size()) {
     4021            for (size_t index = 0; index < info.additionalTypes.size(); ++index) {
     4022                auto nsData = info.additionalData[index]->createNSData();
     4023                [pasteboard setData:nsData.get() forType:info.additionalTypes[index]];
    40074024            }
    40084025        }
  • trunk/Source/WebKit/UIProcess/PageClient.h

    r235156 r235343  
    211211    virtual void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle&) { }
    212212#endif
     213    virtual void didPerformDragOperation(bool) { }
    213214#endif // ENABLE(DRAG_SUPPORT)
    214215
     
    431432
    432433#if ENABLE(DATA_INTERACTION)
    433     virtual void didPerformDataInteractionControllerOperation(bool handled) = 0;
    434434    virtual void didHandleStartDataInteractionRequest(bool started) = 0;
    435435    virtual void didHandleAdditionalDragItemsRequest(bool added) = 0;
  • trunk/Source/WebKit/UIProcess/WebPageProxy.cpp

    r235265 r235343  
    18711871}
    18721872
     1873void WebPageProxy::didPerformDragOperation(bool handled)
     1874{
     1875    m_pageClient.didPerformDragOperation(handled);
     1876}
     1877
    18731878void WebPageProxy::didStartDrag()
    18741879{
  • trunk/Source/WebKit/UIProcess/WebPageProxy.h

    r235205 r235343  
    629629    void cancelAutoscroll();
    630630#if ENABLE(DATA_INTERACTION)
    631     void didPerformDataInteractionControllerOperation(bool handled);
    632631    void didHandleStartDataInteractionRequest(bool started);
    633632    void didHandleAdditionalDragItemsRequest(bool added);
     
    924923    void dragExited(WebCore::DragData&, const String& dragStorageName = String());
    925924    void performDragOperation(WebCore::DragData&, const String& dragStorageName, SandboxExtension::Handle&&, SandboxExtension::HandleArray&&);
     925    void didPerformDragOperation(bool handled);
    926926
    927927    void didPerformDragControllerAction(uint64_t dragOperation, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const WebCore::IntRect& insertionRect);
  • trunk/Source/WebKit/UIProcess/WebPageProxy.messages.in

    r235200 r235343  
    322322#endif
    323323
     324#if ENABLE(DRAG_SUPPORT)
     325    DidPerformDragOperation(bool handled)
     326#endif
     327
    324328#if ENABLE(DATA_INTERACTION)
    325     DidPerformDataInteractionControllerOperation(bool handled)
    326329    DidHandleStartDataInteractionRequest(bool started)
    327330    DidHandleAdditionalDragItemsRequest(bool added)
  • trunk/Source/WebKit/UIProcess/ios/PageClientImplIOS.h

    r235137 r235343  
    211211
    212212#if ENABLE(DATA_INTERACTION)
    213     void didPerformDataInteractionControllerOperation(bool handled) override;
     213    void didPerformDragOperation(bool handled) override;
    214214    void didHandleStartDataInteractionRequest(bool started) override;
    215215    void didHandleAdditionalDragItemsRequest(bool added) override;
  • trunk/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm

    r235265 r235343  
    779779
    780780#if ENABLE(DATA_INTERACTION)
    781 void PageClientImpl::didPerformDataInteractionControllerOperation(bool handled)
    782 {
    783     [m_contentView _didPerformDataInteractionControllerOperation:handled];
     781void PageClientImpl::didPerformDragOperation(bool handled)
     782{
     783    [m_contentView _didPerformDragOperation:handled];
    784784}
    785785
  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h

    r235137 r235343  
    342342#if ENABLE(DATA_INTERACTION)
    343343- (void)_didChangeDragInteractionPolicy;
    344 - (void)_didPerformDataInteractionControllerOperation:(BOOL)handled;
     344- (void)_didPerformDragOperation:(BOOL)handled;
    345345- (void)_didHandleStartDataInteractionRequest:(BOOL)started;
    346346- (void)_didHandleAdditionalDragItemsRequest:(BOOL)added;
  • trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm

    r235245 r235343  
    50775077}
    50785078
    5079 - (void)_didPerformDataInteractionControllerOperation:(BOOL)handled
     5079- (void)_didPerformDragOperation:(BOOL)handled
    50805080{
    50815081    RELEASE_LOG(DragAndDrop, "Finished performing drag controller operation (handled: %d)", handled);
     
    51305130    RELEASE_LOG(DragAndDrop, "Drag session: %p preparing to drag blob: %s with attachment identifier: %s", session.get(), info.blobURL.string().utf8().data(), info.attachmentIdentifier.utf8().data());
    51315131
     5132    NSString *utiType = info.contentType;
     5133    NSString *fileName = info.fileName;
     5134    if (auto attachment = _page->attachmentForIdentifier(info.attachmentIdentifier)) {
     5135        utiType = attachment->utiType();
     5136        fileName = attachment->fileName();
     5137    }
     5138
    51325139    auto registrationList = adoptNS([[WebItemProviderRegistrationInfoList alloc] init]);
    51335140    [registrationList setPreferredPresentationStyle:WebPreferredPresentationStyleAttachment];
    5134     if (!info.filename.isEmpty())
    5135         [registrationList setSuggestedName:info.filename];
     5141    if ([fileName length])
     5142        [registrationList setSuggestedName:fileName];
    51365143    if (numberOfAdditionalTypes == info.additionalData.size() && numberOfAdditionalTypes) {
    51375144        for (size_t index = 0; index < numberOfAdditionalTypes; ++index) {
     
    51415148    }
    51425149
    5143     [registrationList addPromisedType:info.contentType fileCallback:[session = WTFMove(session), weakSelf = WeakObjCPtr<WKContentView>(self), info] (WebItemProviderFileCallback callback) {
     5150    [registrationList addPromisedType:utiType fileCallback:[session = WTFMove(session), weakSelf = WeakObjCPtr<WKContentView>(self), info] (WebItemProviderFileCallback callback) {
    51445151        auto strongSelf = weakSelf.get();
    51455152        if (!strongSelf) {
  • trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm

    r235265 r235343  
    10891089#if ENABLE(DATA_INTERACTION)
    10901090
    1091 void WebPageProxy::didPerformDataInteractionControllerOperation(bool handled)
    1092 {
    1093     m_pageClient.didPerformDataInteractionControllerOperation(handled);
    1094 }
    1095 
    10961091void WebPageProxy::didHandleStartDataInteractionRequest(bool started)
    10971092{
  • trunk/Source/WebKit/UIProcess/mac/PageClientImplMac.h

    r235137 r235343  
    234234    bool effectiveAppearanceIsDark() const override;
    235235
     236#if ENABLE(DRAG_SUPPORT)
     237    void didPerformDragOperation(bool handled) final;
     238#endif
     239
    236240#if WK_API_ENABLED
    237241    NSView *inspectorAttachmentView() override;
  • trunk/Source/WebKit/UIProcess/mac/PageClientImplMac.mm

    r235265 r235343  
    875875}
    876876
     877#if ENABLE(DRAG_SUPPORT)
     878
     879void PageClientImpl::didPerformDragOperation(bool handled)
     880{
     881    m_impl->didPerformDragOperation(handled);
     882}
     883
     884#endif
     885
    877886#if WK_API_ENABLED
    878887NSView *PageClientImpl::inspectorAttachmentView()
  • trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj

    r235323 r235343  
    18121812                EDCA71B7128DDA8C00201B26 /* WKBundlePageOverlay.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 1A22F1001289FCD90085E74F /* WKBundlePageOverlay.cpp */; };
    18131813                F409BA181E6E64BC009DA28E /* WKDragDestinationAction.h in Headers */ = {isa = PBXBuildFile; fileRef = F409BA171E6E64B3009DA28E /* WKDragDestinationAction.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1814                F41056622130699A0092281D /* APIAttachmentCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = F41056602130699A0092281D /* APIAttachmentCocoa.h */; };
     1815                F41056632130699A0092281D /* APIAttachmentCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = F41056612130699A0092281D /* APIAttachmentCocoa.mm */; };
    18141816                F44291921FA591C9002CC93E /* _WKAttachment.h in Headers */ = {isa = PBXBuildFile; fileRef = F44291911FA59107002CC93E /* _WKAttachment.h */; settings = {ATTRIBUTES = (Private, ); }; };
    18151817                F44291941FA59311002CC93E /* _WKAttachment.mm in Sources */ = {isa = PBXBuildFile; fileRef = F44291931FA59311002CC93E /* _WKAttachment.mm */; };
     
    46554657                F036978715F4BF0500C3A80E /* WebColorPicker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebColorPicker.cpp; sourceTree = "<group>"; };
    46564658                F409BA171E6E64B3009DA28E /* WKDragDestinationAction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKDragDestinationAction.h; sourceTree = "<group>"; };
     4659                F41056602130699A0092281D /* APIAttachmentCocoa.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = APIAttachmentCocoa.h; sourceTree = "<group>"; };
     4660                F41056612130699A0092281D /* APIAttachmentCocoa.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = APIAttachmentCocoa.mm; sourceTree = "<group>"; };
    46574661                F44291911FA59107002CC93E /* _WKAttachment.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = _WKAttachment.h; sourceTree = "<group>"; };
    46584662                F44291931FA59311002CC93E /* _WKAttachment.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; path = _WKAttachment.mm; sourceTree = "<group>"; };
     
    62106214                                5CB2378A1DF0DD4300117AA3 /* _WKWebsitePolicies.mm */,
    62116215                                5CB2378D1DF0E0C200117AA3 /* _WKWebsitePoliciesInternal.h */,
     6216                                F41056602130699A0092281D /* APIAttachmentCocoa.h */,
     6217                                F41056612130699A0092281D /* APIAttachmentCocoa.mm */,
    62126218                                7CEFA9601AC0999300B910FD /* APIContentRuleListStoreCocoa.mm */,
    62136219                                FED3C1DA1B447AE800E0EB7F /* APISerializedScriptValueCocoa.mm */,
     
    89999005                                BC64697011DBE603006455B0 /* APIArray.h in Headers */,
    90009006                                2E5C770E1FA7D429005932C3 /* APIAttachment.h in Headers */,
     9007                                F41056622130699A0092281D /* APIAttachmentCocoa.h in Headers */,
    90019008                                99C81D5D1C21F38B005C4C82 /* APIAutomationClient.h in Headers */,
    90029009                                990D28C01C6553F100986977 /* APIAutomationSessionClient.h in Headers */,
     
    1088210889                                BC64696F11DBE603006455B0 /* APIArray.cpp in Sources */,
    1088310890                                2E5C770F1FA7D429005932C3 /* APIAttachment.cpp in Sources */,
     10891                                F41056632130699A0092281D /* APIAttachmentCocoa.mm in Sources */,
    1088410892                                7C89D2B31A6B068C003A5FDE /* APIContentRuleList.cpp in Sources */,
    1088510893                                7C3A06A71AAB903E009D74BA /* APIContentRuleListStore.cpp in Sources */,
  • trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp

    r235331 r235343  
    34603460
    34613461        m_pendingDropExtensionsForFileUpload.clear();
    3462 #if ENABLE(DATA_INTERACTION)
    3463         send(Messages::WebPageProxy::DidPerformDataInteractionControllerOperation(handled));
    3464 #else
    3465         UNUSED_PARAM(handled);
    3466 #endif
     3462        send(Messages::WebPageProxy::DidPerformDragOperation(handled));
    34673463        return;
    34683464    }
  • trunk/Tools/ChangeLog

    r235339 r235343  
     12018-08-26  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        [Attachment Support] Dropping and pasting images should insert inline image elements with _WKAttachments
     4        https://bugs.webkit.org/show_bug.cgi?id=188933
     5        <rdar://problem/43699724>
     6
     7        Reviewed by Darin Adler.
     8
     9        Rebaseline existing API tests that involve dropping or pasting image files, and additionally write some new
     10        tests. These new tests exercise the following cases:
     11        •       Inserting and removing newlines before an inline image with an attachment element does not cause new
     12                _WKAttachments to be created and destroyed.
     13        •       Pasting an image, cutting it, and then pasting it again propagates an attachment update to the UI
     14                process with the original _WKAttachment.
     15        •       A pasted attachment in the document can be moved around by dragging, and doing so does not cause us to
     16                lose a _WKAttachment.
     17
     18        * TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
     19        (-[TestWKWebView expectElementCount:tagName:]):
     20        (TestWebKitAPI::TEST):
     21
     22        Add the new tests described above, and also adjust existing tests to check that images are dropped or pasted
     23        as image elements, but still have associated attachment elements whose attachment identifiers (observed via
     24        script) match that of the corresponding _WKAttachment's uniqueIdentifier in the UI process.
     25
     26        * TestWebKitAPI/mac/DragAndDropSimulatorMac.mm:
     27        (-[DragAndDropSimulator runFrom:to:]):
     28        (-[DragAndDropSimulator continueDragSession]):
     29        (-[DragAndDropSimulator performDragInWebView:atLocation:withImage:pasteboard:source:]):
     30
     31        Teach DragAndDropSimulator on macOS to wait until the drop has been handled by the web process before returning
     32        execution to the caller. This ensures that tests which involve dropping promised files as attachments aren't
     33        flaky, due to how the promised data is retrieved asynchronously when performing the drop.
     34
     35        (-[DragAndDropSimulator _webView:didPerformDragOperation:]):
     36
    1372018-08-26  Lucas Forschler  <lforschler@apple.com>
    238
  • trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm

    r235229 r235343  
    226226}
    227227
     228- (void)expectElementCount:(NSInteger)count tagName:(NSString *)tagName
     229{
     230    NSString *script = [NSString stringWithFormat:@"document.querySelectorAll('%@').length", tagName];
     231    EXPECT_EQ(count, [self stringByEvaluatingJavaScript:script].integerValue);
     232}
     233
    228234- (void)expectElementTag:(NSString *)tagName toComeBefore:(NSString *)otherTagName
    229235{
     
    780786}
    781787
    782 TEST(WKAttachmentTests, InsertPastedImageAsAttachment)
     788TEST(WKAttachmentTests, RemoveNewlinesBeforePastedImage)
    783789{
    784790    platformCopyPNG();
     
    796802    EXPECT_EQ(215., size.width);
    797803    EXPECT_EQ(174., size.height);
    798     EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').uniqueIdentifier"]);
    799 
     804    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
     805
     806    [webView stringByEvaluatingJavaScript:@"getSelection().collapse(document.body, 0)"];
     807    {
     808        ObserveAttachmentUpdatesForScope observer(webView.get());
     809        [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
     810        [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
     811        observer.expectAttachmentUpdates(@[ ], @[ ]);
     812        [webView expectElementTagsInOrder:@[ @"BR", @"BR", @"IMG" ]];
     813    }
     814    {
     815        ObserveAttachmentUpdatesForScope observer(webView.get());
     816        [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
     817        [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
     818        observer.expectAttachmentUpdates(@[ ], @[ ]);
     819        [webView expectElementCount:0 tagName:@"BR"];
     820    }
     821}
     822
     823TEST(WKAttachmentTests, CutAndPastePastedImage)
     824{
     825    platformCopyPNG();
     826
     827    RetainPtr<_WKAttachment> attachment;
     828    auto webView = webViewForTestingAttachments();
     829    {
     830        ObserveAttachmentUpdatesForScope observer(webView.get());
     831        [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
     832        EXPECT_EQ(1U, observer.observer().inserted.count);
     833        attachment = observer.observer().inserted[0];
     834    }
    800835    {
    801836        ObserveAttachmentUpdatesForScope observer(webView.get());
    802837        [webView _synchronouslyExecuteEditCommand:@"SelectAll" argument:nil];
    803         [webView _synchronouslyExecuteEditCommand:@"DeleteBackward" argument:nil];
    804         observer.expectAttachmentUpdates(@[attachment.get()], @[]);
    805     }
     838        [webView _synchronouslyExecuteEditCommand:@"Cut" argument:nil];
     839        observer.expectAttachmentUpdates(@[ attachment.get() ], @[ ]);
     840    }
     841    {
     842        ObserveAttachmentUpdatesForScope observer(webView.get());
     843        [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
     844        observer.expectAttachmentUpdates(@[ ], @[ attachment.get() ]);
     845    }
     846}
     847
     848TEST(WKAttachmentTests, MovePastedImageByDragging)
     849{
     850    auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
     851    [configuration _setAttachmentElementEnabled:YES];
     852    auto simulator = adoptNS([[DragAndDropSimulator alloc] initWithWebViewFrame:NSMakeRect(0, 0, 400, 400) configuration:configuration.get()]);
     853    TestWKWebView *webView = [simulator webView];
     854    [webView synchronouslyLoadHTMLString:attachmentEditingTestMarkup];
     855
     856    platformCopyPNG();
     857    [webView _synchronouslyExecuteEditCommand:@"Paste" argument:nil];
     858    [webView _executeEditCommand:@"InsertParagraph" argument:nil completion:nil];
     859    [webView _executeEditCommand:@"InsertHTML" argument:@"<strong>text</strong>" completion:nil];
     860    [webView _synchronouslyExecuteEditCommand:@"InsertParagraph" argument:nil];
     861    [webView expectElementTag:@"IMG" toComeBefore:@"STRONG"];
     862    [webView expectElementCount:1 tagName:@"IMG"];
     863
     864    // Drag the attachment element to somewhere below the strong text.
     865    [simulator runFrom:CGPointMake(50, 50) to:CGPointMake(50, 350)];
     866
     867    [webView expectElementTag:@"STRONG" toComeBefore:@"IMG"];
     868    [webView expectElementCount:1 tagName:@"IMG"];
     869    EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
     870
     871    [simulator endDataTransfer];
    806872}
    807873
     
    822888    [attachment expectRequestedDataToBe:testImageData()];
    823889    EXPECT_WK_STREQ("Lorem ipsum  dolor sit amet.", [webView stringByEvaluatingJavaScript:@"document.body.textContent"]);
    824     EXPECT_WK_STREQ("image/png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
    825     EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').uniqueIdentifier"]);
     890    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
    826891
    827892    {
     
    858923
    859924    EXPECT_TRUE(zipAttachment && imageAttachment && pdfAttachment);
    860     EXPECT_EQ(3, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].integerValue);
    861     EXPECT_WK_STREQ("image/png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
    862     EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"]);
    863 
    864     NSString *zipAttachmentType = [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[2].getAttribute('type')"];
     925    [webView expectElementCount:2 tagName:@"ATTACHMENT"];
     926    [webView expectElementCount:1 tagName:@"IMG"];
     927    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
     928
     929    NSString *zipAttachmentType = [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"];
    865930#if USES_MODERN_ATTRIBUTED_STRING_CONVERSION
    866931    EXPECT_WK_STREQ("application/zip", zipAttachmentType);
     
    869934#endif
    870935
    871     EXPECT_WK_STREQ([imageAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].uniqueIdentifier"]);
    872     EXPECT_WK_STREQ([pdfAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].uniqueIdentifier"]);
    873     EXPECT_WK_STREQ([zipAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[2].uniqueIdentifier"]);
     936    EXPECT_WK_STREQ([imageAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
     937    EXPECT_WK_STREQ([pdfAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].uniqueIdentifier"]);
     938    EXPECT_WK_STREQ([zipAttachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].uniqueIdentifier"]);
    874939
    875940    {
     
    9931058    [webView expectElementTagsInOrder:@[ @"IMG", @"ATTACHMENT", @"ATTACHMENT" ]];
    9941059    EXPECT_WK_STREQ("cid:foo-bar", [webView valueOfAttribute:@"src" forQuerySelector:@"img"]);
     1060    EXPECT_WK_STREQ(@"", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('img')[0].webkitAttachmentIdentifier"]);
    9951061}
    9961062
     
    11251191    [simulator runFrom:[webView attachmentElementMidPoint] to:CGPointMake(50, 300)];
    11261192
    1127     EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
    1128     [attachment expectRequestedDataToBe:data.get()];
    11291193    EXPECT_WK_STREQ("document.pdf", [webView valueOfAttribute:@"title" forQuerySelector:@"attachment"]);
    11301194    EXPECT_WK_STREQ("application/pdf", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
     1195    [attachment expectRequestedDataToBe:data.get()];
     1196    EXPECT_EQ([simulator insertedAttachments].count, [simulator removedAttachments].count);
    11311197#if PLATFORM(MAC)
    11321198    EXPECT_FALSE(isCompletelyTransparent([simulator draggingInfo].draggedImage));
     
    11571223    }
    11581224
    1159     NSArray<NSData *> *expectedAttachmentData = @[ testPDFData(), testImageData() ];
    1160     EXPECT_TRUE([expectedAttachmentData containsObject:[insertedAttachments firstObject].info.data]);
    1161     EXPECT_TRUE([expectedAttachmentData containsObject:[insertedAttachments lastObject].info.data]);
    1162     EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
    1163     EXPECT_WK_STREQ("test.pdf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('title')"]);
    1164     EXPECT_WK_STREQ("image/png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('type')"]);
    1165     EXPECT_WK_STREQ("icon.png", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[1].getAttribute('title')"]);
     1225    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
     1226    [webView expectElementCount:1 tagName:@"IMG"];
     1227    EXPECT_WK_STREQ("application/pdf", [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').getAttribute('type')"]);
     1228    EXPECT_WK_STREQ("test.pdf", [webView stringByEvaluatingJavaScript:@"document.querySelector('attachment').getAttribute('title')"]);
     1229
     1230    NSString *imageAttachmentIdentifier = [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"];
     1231    if ([testImageData() isEqualToData:[insertedAttachments firstObject].info.data])
     1232        EXPECT_WK_STREQ([insertedAttachments firstObject].uniqueIdentifier, imageAttachmentIdentifier);
     1233    else
     1234        EXPECT_WK_STREQ([insertedAttachments lastObject].uniqueIdentifier, imageAttachmentIdentifier);
    11661235
    11671236    for (_WKAttachment *attachment in insertedAttachments.get())
     
    11891258
    11901259    [simulator runFrom:CGPointMake(0, 0) to:CGPointMake(50, 50)];
    1191     while ([[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantPast]]) {
    1192         if ([simulator insertedAttachments].count == 2)
    1193             break;
    1194     }
    1195     EXPECT_EQ(2, [[webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"] intValue]);
     1260
     1261    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
     1262    [webView expectElementCount:1 tagName:@"IMG"];
     1263    EXPECT_EQ(2U, [simulator insertedAttachments].count);
    11961264
    11971265    auto insertedAttachments = retainPtr([simulator insertedAttachments]);
     
    12021270        if ([testPDFData() isEqualToData:attachment.info.data])
    12031271            EXPECT_WK_STREQ("application/pdf", attachment.info.contentType);
    1204         else if ([testImageData() isEqualToData:attachment.info.data])
     1272        else if ([testImageData() isEqualToData:attachment.info.data]) {
    12051273            EXPECT_WK_STREQ("image/png", attachment.info.contentType);
     1274            EXPECT_WK_STREQ(attachment.uniqueIdentifier, [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
     1275        }
    12061276    }
    12071277
     
    12101280    auto removedAttachments = retainPtr([simulator removedAttachments]);
    12111281    EXPECT_EQ(2U, [removedAttachments count]);
    1212     EXPECT_EQ(0, [[webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"] intValue]);
     1282    [webView expectElementCount:0 tagName:@"ATTACHMENT"];
     1283    [webView expectElementCount:0 tagName:@"IMG"];
    12131284    EXPECT_TRUE([removedAttachments containsObject:[insertedAttachments firstObject]]);
    12141285    EXPECT_TRUE([removedAttachments containsObject:[insertedAttachments lastObject]]);
     
    12291300    NSArray<NSURL *> *urls = [simulator receivePromisedFiles];
    12301301    EXPECT_EQ(1U, urls.count);
     1302    EXPECT_WK_STREQ("test.pdf", urls.lastObject.lastPathComponent);
    12311303    EXPECT_TRUE([[NSData dataWithContentsOfURL:urls.firstObject] isEqualToData:testPDFData()]);
    12321304    EXPECT_FALSE(isCompletelyTransparent([simulator draggingInfo].draggedImage));
     
    12501322    auto attachment = retainPtr([dragAndDropSimulator insertedAttachments].firstObject);
    12511323    [attachment expectRequestedDataToBe:testImageData()];
    1252     EXPECT_WK_STREQ("public.png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
     1324    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
    12531325
    12541326    {
     
    12781350    EXPECT_EQ(215., size.width);
    12791351    EXPECT_EQ(174., size.height);
    1280     EXPECT_WK_STREQ("image/png", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
     1352    EXPECT_WK_STREQ([attachment uniqueIdentifier], [webView stringByEvaluatingJavaScript:@"document.querySelector('img').webkitAttachmentIdentifier"]);
    12811353
    12821354    {
     
    13161388        EXPECT_GT([attachment info].data.length, 0U);
    13171389
    1318     EXPECT_EQ(2, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].intValue);
     1390    [webView expectElementCount:2 tagName:@"ATTACHMENT"];
    13191391    EXPECT_WK_STREQ("hello.rtf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('title')"]);
    13201392    EXPECT_WK_STREQ("text/rtf", [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment')[0].getAttribute('type')"]);
     
    13411413    EXPECT_EQ(0U, [dragAndDropSimulator removedAttachments].count);
    13421414    [[dragAndDropSimulator insertedAttachments].firstObject expectRequestedDataToBe:data];
    1343     EXPECT_EQ(1, [webView stringByEvaluatingJavaScript:@"document.querySelectorAll('attachment').length"].intValue);
     1415    [webView expectElementCount:1 tagName:@"ATTACHMENT"];
    13441416    EXPECT_WK_STREQ("archive.zip", [webView valueOfAttribute:@"title" forQuerySelector:@"attachment"]);
    13451417    EXPECT_WK_STREQ("application/zip", [webView valueOfAttribute:@"type" forQuerySelector:@"attachment"]);
  • trunk/Tools/TestWebKitAPI/mac/DragAndDropSimulatorMac.mm

    r235202 r235343  
    104104    double _progress;
    105105    bool _doneWaitingForDraggingSession;
     106    bool _doneWaitingForDrop;
    106107}
    107108
     
    156157    _removedAttachments = adoptNS([NSMutableArray new]);
    157158    _doneWaitingForDraggingSession = true;
     159    _doneWaitingForDrop = true;
    158160    _startLocationInWindow = [self flipAboutXAxisInHostWindow:flippedStartLocation];
    159161    _endLocationInWindow = [self flipAboutXAxisInHostWindow:flippedEndLocation];
     
    168170        NSImage *dragImage = self.externalDragImage ?: defaultExternalDragImage();
    169171        [self performDragInWebView:_webView.get() atLocation:startLocationInView withImage:dragImage pasteboard:pasteboard source:nil];
     172        TestWebKitAPI::Util::run(&_doneWaitingForDrop);
    170173        return;
    171174    }
     
    187190    [_webView mouseUpAtPoint:_endLocationInWindow];
    188191    [_webView waitForPendingMouseEvents];
     192
     193    TestWebKitAPI::Util::run(&_doneWaitingForDrop);
    189194}
    190195
     
    239244        _willEndDraggingHandler();
    240245
    241     if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()])
     246    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()]) {
     247        _doneWaitingForDrop = false;
    242248        [_webView performDragOperation:_draggingInfo.get()];
    243     else if (_currentDragOperation == NSDragOperationNone)
     249    } else if (_currentDragOperation == NSDragOperationNone)
    244250        [_webView draggingExited:_draggingInfo.get()];
    245251    [_webView waitForNextPresentationUpdate];
     
    269275        _willEndDraggingHandler();
    270276
    271     if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()])
     277    if (_currentDragOperation != NSDragOperationNone && [_webView prepareForDragOperation:_draggingInfo.get()]) {
     278        _doneWaitingForDrop = false;
    272279        [_webView performDragOperation:_draggingInfo.get()];
    273     else if (_currentDragOperation == NSDragOperationNone)
     280    } else if (_currentDragOperation == NSDragOperationNone)
    274281        [_webView draggingExited:_draggingInfo.get()];
    275282    [_webView waitForNextPresentationUpdate];
     
    438445}
    439446
     447#pragma mark - WKUIDelegatePrivate
     448
    440449- (void)_webView:(WKWebView *)webView didInsertAttachment:(_WKAttachment *)attachment withSource:(NSString *)source
    441450{
     
    448457}
    449458
     459- (void)_webView:(WKWebView *)webView didPerformDragOperation:(BOOL)handled
     460{
     461    _doneWaitingForDrop = true;
     462}
     463
    450464@end
    451465
Note: See TracChangeset for help on using the changeset viewer.