Changeset 283592 in webkit


Ignore:
Timestamp:
Oct 5, 2021, 5:42:35 PM (4 years ago)
Author:
Aditya Keerthi
Message:

[iOS] Transcode videos selected from UIImagePickerController
​https://bugs.webkit.org/show_bug.cgi?id=230639
rdar://79665678

Reviewed by Tim Horton.

Source/WebCore:

  • en.lproj/Localizable.strings:

Add a localizable string for the message displayed while transcoding
video.

Source/WebCore/PAL:

Add AVFoundation API needed to transcode video.

  • pal/cocoa/AVFoundationSoftLink.h:
  • pal/cocoa/AVFoundationSoftLink.mm:

Source/WebKit:

File inputs on iOS allow users to choose images/videos from the system
photo picker, using UIImagePickerController. In single selection mode,
UIImagePickerController transcodes the selected video to H.264. However,
in multiple selection mode, video is not transcoded and is left in its
original format.

Today, videos on most iOS devices are encoded with HEVC by default.
However, some sites, such as Twitter, only accept H.264 encoded video.
Thus, the current video upload behavior is problematic, as users may be
unable to upload video.

Unfortunately, the photo picking functionality of UIImagePickerController
is deprecated. The best solution would be to adopt PHPickerViewController,
the replacement API, which performs transcoding when retrieving selected
items (regardless of single/multiple selection). However,
PHPickerViewController currently lacks other functionality that WebKit
requires, preventing adoption.

Consequently, the short term solution is to transcode the videos in
WebKit, ensuring H.264 encoded video is always provided to sites. See
below for implementation details.

  • Platform/spi/ios/PhotosUISPI.h: Added.
  • UIProcess/ios/forms/WKFileUploadPanel.mm:

(-[_WKFileUploadItem setFileURL:]):

Add a setter to update the file URL. Called after transcoding a _WKFileUploadItem.

(-[WKFileUploadMediaTranscoder initWithItems:videoCount:completionHandler:]):

Introduce WKFileUploadMediaTranscoder to manage transcoding of videos
and the display of progress UI. Transcoding is performed serially
(one video at a time), but occurs off the main thread.

(-[WKFileUploadMediaTranscoder start]):

Begin transcoding. Run a timer to update the progress UI, as
AVAssetExportSession does not provide progress updates on its own.

The progress UI is implemented using PUActivityProgressController, to
match system Photos UI.

(-[WKFileUploadMediaTranscoder _processItemAtIndex:]):

Transcode a single video, using AVAssetExportSession. If transcoding
fails for any reason, the original video is used as a fallback.
Transcoding can also be cancelled using the progress UI, in which case
no more videos are processed.

(-[WKFileUploadMediaTranscoder _finishedProcessing]):
(-[WKFileUploadMediaTranscoder _dismissProgress]):
(-[WKFileUploadMediaTranscoder _updateProgress:]):
(-[WKFileUploadMediaTranscoder _temporaryDirectoryCreateIfNecessary]):
(-[WKFileUploadPanel _chooseMediaItems:]):

Refactor the common aspects of single/multiple media selection into a
single method.

(-[WKFileUploadPanel imagePickerController:didFinishPickingMediaWithInfo:]):
(-[WKFileUploadPanel imagePickerController:didFinishPickingMultipleMediaWithInfo:]):
(-[WKFileUploadPanel _processMediaInfoDictionaries:successBlock:failureBlock:]):
(-[WKFileUploadPanel _processMediaInfoDictionaries:atIndex:processedResults:successBlock:failureBlock:]):
(-[WKFileUploadPanel _uploadItemFromMediaInfo:successBlock:failureBlock:]):

Remove redundant platform conditional.

(-[WKFileUploadPanel _uploadMediaItemsTranscodingVideo:]):

If any videos were selected, transcode them prior to uploading.

  • WebKit.xcodeproj/project.pbxproj:
Location:
trunk/Source
Files:
1 added
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r283590 r283592  
     12021-10-05  Aditya Keerthi  <akeerthi@apple.com>
     2
     3        [iOS] Transcode videos selected from UIImagePickerController
     4        https://bugs.webkit.org/show_bug.cgi?id=230639
     5        rdar://79665678
     6
     7        Reviewed by Tim Horton.
     8
     9        * en.lproj/Localizable.strings:
     10
     11        Add a localizable string for the message displayed while transcoding
     12        video.
     13
    1142021-10-05  Chris Dumez  <cdumez@apple.com>
    215

  • trunk/Source/WebCore/PAL/ChangeLog

    r283577 r283592  
     12021-10-05  Aditya Keerthi  <akeerthi@apple.com>
     2
     3        [iOS] Transcode videos selected from UIImagePickerController
     4        https://bugs.webkit.org/show_bug.cgi?id=230639
     5        rdar://79665678
     6
     7        Reviewed by Tim Horton.
     8
     9        Add AVFoundation API needed to transcode video.
     10
     11        * pal/cocoa/AVFoundationSoftLink.h:
     12        * pal/cocoa/AVFoundationSoftLink.mm:
     13
    1142021-10-05  Ayumi Kojima  <ayumi_kojima@apple.com>
    215

  • trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h

    r283217 r283592  
    3939SOFT_LINK_CLASS_FOR_HEADER(PAL, AVAssetCache)
    4040SOFT_LINK_CLASS_FOR_HEADER(PAL, AVAssetCollection)
     41SOFT_LINK_CLASS_FOR_HEADER(PAL, AVAssetExportSession)
    4142SOFT_LINK_CLASS_FOR_HEADER(PAL, AVAssetImageGenerator)
    4243SOFT_LINK_CLASS_FOR_HEADER(PAL, AVAssetReader)
    … …  
    191192SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVFileTypeMPEG4, NSString *)
    192193#define AVFileTypeMPEG4 PAL::get_AVFoundation_AVFileTypeMPEG4()
     194SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVFileTypeQuickTimeMovie, NSString *)
     195#define AVFileTypeQuickTimeMovie PAL::get_AVFoundation_AVFileTypeQuickTimeMovie()
    193196SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVVideoCodecKey, NSString *)
    194197#define AVVideoCodecKey PAL::get_AVFoundation_AVVideoCodecKey()
    … …  
    337340#endif // PLATFORM(COCOA)
    338341
     342SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVAssetExportPresetHighestQuality, NSString *)
     343#define AVAssetExportPresetHighestQuality PAL::get_AVFoundation_AVAssetExportPresetHighestQuality()
     344
    339345#endif // USE(AVFOUNDATION)

  • trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm

    r283431 r283592  
    6464
    6565SOFT_LINK_CLASS_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetCache, PAL_EXPORT)
     66SOFT_LINK_CLASS_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetExportSession, PAL_EXPORT)
    6667SOFT_LINK_CLASS_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetImageGenerator, PAL_EXPORT)
    6768SOFT_LINK_CLASS_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetReader, PAL_EXPORT)
    … …  
    134135SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVCaptureDeviceWasDisconnectedNotification, NSString *, PAL_EXPORT)
    135136SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVFileTypeMPEG4, NSString *, PAL_EXPORT)
     137SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVFileTypeQuickTimeMovie, NSString *, PAL_EXPORT)
    136138SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVLayerVideoGravityResize, NSString *, PAL_EXPORT)
    137139SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVLayerVideoGravityResizeAspect, NSString *, PAL_EXPORT)
    … …  
    254256#endif
    255257
     258SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetExportPresetHighestQuality, NSString *, PAL_EXPORT)
     259
    256260#endif // USE(AVFOUNDATION)

  • trunk/Source/WebCore/en.lproj/Localizable.strings

    r279912 r283592  
    755755"PostScript" = "PostScript";
    756756
     757/* Title for file upload progress view */
     758"Preparing (file upload)" = "Preparing…";
     759
    757760/* Title for Quick Look action button */
    758761"Quick Look" = "Quick Look";

  • trunk/Source/WebKit/ChangeLog

    r283586 r283592  
     12021-10-05  Aditya Keerthi  <akeerthi@apple.com>
     2
     3        [iOS] Transcode videos selected from UIImagePickerController
     4        https://bugs.webkit.org/show_bug.cgi?id=230639
     5        rdar://79665678
     6
     7        Reviewed by Tim Horton.
     8
     9        File inputs on iOS allow users to choose images/videos from the system
     10        photo picker, using UIImagePickerController. In single selection mode,
     11        UIImagePickerController transcodes the selected video to H.264. However,
     12        in multiple selection mode, video is not transcoded and is left in its
     13        original format.
     14
     15        Today, videos on most iOS devices are encoded with HEVC by default.
     16        However, some sites, such as Twitter, only accept H.264 encoded video.
     17        Thus, the current video upload behavior is problematic, as users may be
     18        unable to upload video.
     19
     20        Unfortunately, the photo picking functionality of UIImagePickerController
     21        is deprecated. The best solution would be to adopt PHPickerViewController,
     22        the replacement API, which performs transcoding when retrieving selected
     23        items (regardless of single/multiple selection). However,
     24        PHPickerViewController currently lacks other functionality that WebKit
     25        requires, preventing adoption.
     26
     27        Consequently, the short term solution is to transcode the videos in
     28        WebKit, ensuring H.264 encoded video is always provided to sites. See
     29        below for implementation details.
     30
     31        * Platform/spi/ios/PhotosUISPI.h: Added.
     32        * UIProcess/ios/forms/WKFileUploadPanel.mm:
     33        (-[_WKFileUploadItem setFileURL:]):
     34
     35        Add a setter to update the file URL. Called after transcoding a _WKFileUploadItem.
     36
     37        (-[WKFileUploadMediaTranscoder initWithItems:videoCount:completionHandler:]):
     38
     39        Introduce WKFileUploadMediaTranscoder to manage transcoding of videos
     40        and the display of progress UI. Transcoding is performed serially
     41        (one video at a time), but occurs off the main thread.
     42
     43        (-[WKFileUploadMediaTranscoder start]):
     44
     45        Begin transcoding. Run a timer to update the progress UI, as
     46        AVAssetExportSession does not provide progress updates on its own.
     47
     48        The progress UI is implemented using PUActivityProgressController, to
     49        match system Photos UI.
     50
     51        (-[WKFileUploadMediaTranscoder _processItemAtIndex:]):
     52
     53        Transcode a single video, using AVAssetExportSession. If transcoding
     54        fails for any reason, the original video is used as a fallback.
     55        Transcoding can also be cancelled using the progress UI, in which case
     56        no more videos are processed.
     57
     58        (-[WKFileUploadMediaTranscoder _finishedProcessing]):
     59        (-[WKFileUploadMediaTranscoder _dismissProgress]):
     60        (-[WKFileUploadMediaTranscoder _updateProgress:]):
     61        (-[WKFileUploadMediaTranscoder _temporaryDirectoryCreateIfNecessary]):
     62        (-[WKFileUploadPanel _chooseMediaItems:]):
     63
     64        Refactor the common aspects of single/multiple media selection into a
     65        single method.
     66
     67        (-[WKFileUploadPanel imagePickerController:didFinishPickingMediaWithInfo:]):
     68        (-[WKFileUploadPanel imagePickerController:didFinishPickingMultipleMediaWithInfo:]):
     69        (-[WKFileUploadPanel _processMediaInfoDictionaries:successBlock:failureBlock:]):
     70        (-[WKFileUploadPanel _processMediaInfoDictionaries:atIndex:processedResults:successBlock:failureBlock:]):
     71        (-[WKFileUploadPanel _uploadItemFromMediaInfo:successBlock:failureBlock:]):
     72
     73        Remove redundant platform conditional.
     74
     75        (-[WKFileUploadPanel _uploadMediaItemsTranscodingVideo:]):
     76
     77        If any videos were selected, transcode them prior to uploading.
     78
     79        * WebKit.xcodeproj/project.pbxproj:
     80
    1812021-10-05  Alex Christensen  <achristensen@webkit.org>
    282

  • trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm

    r281612 r283592  
    3333#import "APIOpenPanelParameters.h"
    3434#import "APIString.h"
     35#import "PhotosUISPI.h"
    3536#import "UIKitSPI.h"
    3637#import "UserInterfaceIdiom.h"
    … …  
    4647#import <WebCore/LocalizedStrings.h>
    4748#import <WebCore/MIMETypeRegistry.h>
     49#import <wtf/MainThread.h>
    4850#import <wtf/OptionSet.h>
    4951#import <wtf/RetainPtr.h>
    … …  
    5153#import <wtf/text/StringView.h>
    5254
     55#import <pal/cocoa/AVFoundationSoftLink.h>
     56
     57SOFT_LINK_FRAMEWORK(PhotosUI)
     58SOFT_LINK_CLASS(PhotosUI, PUActivityProgressController)
     59
    5360using namespace WebKit;
    5461
    … …  
    8087- (instancetype)initWithFileURL:(NSURL *)fileURL;
    8188@property (nonatomic, readonly, getter=isVideo) BOOL video;
    82 @property (nonatomic, readonly) NSURL *fileURL;
    8389@property (nonatomic, readonly) RetainPtr<UIImage> displayImage;
    8490@end
    … …  
    110116}
    111117
     118- (void)setFileURL:(NSURL *)fileURL
     119{
     120    _fileURL = fileURL;
     121}
     122
    112123- (RetainPtr<UIImage>)displayImage
    113124{
    … …  
    154165@end
    155166
     167#pragma mark - WKFileUploadMediaTranscoder
     168
     169@interface WKFileUploadMediaTranscoder : NSObject
     170
     171- (instancetype)initWithItems:(NSArray *)items videoCount:(NSUInteger)videoCount completionHandler:(WTF::Function<void(NSArray<_WKFileUploadItem *> *)>&&)completionHandler;
     172
     173- (void)start;
     174
     175@end
     176
     177@implementation WKFileUploadMediaTranscoder {
     178    RetainPtr<NSTimer> _progressTimer;
     179    RetainPtr<PUActivityProgressController> _progressController;
     180    RetainPtr<AVAssetExportSession> _exportSession;
     181    RetainPtr<NSArray<_WKFileUploadItem *>> _items;
     182    RetainPtr<NSString> _temporaryDirectoryPath;
     183
     184    // Only called if the transcoding is not cancelled.
     185    WTF::Function<void(NSArray<_WKFileUploadItem *> *)> _completionHandler;
     186
     187    NSUInteger _videoCount;
     188    NSUInteger _processedVideoCount;
     189}
     190
     191- (instancetype)initWithItems:(NSArray<_WKFileUploadItem *> *)items videoCount:(NSUInteger)videoCount completionHandler:(WTF::Function<void(NSArray<_WKFileUploadItem *> *)>&&)completionHandler
     192{
     193    if (!(self = [super init]))
     194        return nil;
     195
     196    _items = items;
     197    _processedVideoCount = 0;
     198    _videoCount = videoCount;
     199
     200    _completionHandler = WTFMove(completionHandler);
     201
     202    return self;
     203}
     204
     205- (void)start
     206{
     207    _progressController = adoptNS([allocPUActivityProgressControllerInstance() init]);
     208    [_progressController setTitle:WEB_UI_STRING_KEY("Preparing…", "Preparing (file upload)", "Title for file upload progress view")];
     209    [_progressController showAnimated:YES allowDelay:YES];
     210
     211    [_progressController setCancellationHandler:makeBlockPtr([weakSelf = WeakObjCPtr<WKFileUploadMediaTranscoder>(self)] {
     212        auto strongSelf = weakSelf.get();
     213        if (!strongSelf)
     214            return;
     215
     216        [strongSelf->_exportSession cancelExport];
     217        [strongSelf _dismissProgress];
     218    }).get()];
     219
     220    _progressTimer = [NSTimer scheduledTimerWithTimeInterval:0.1f target:self selector:@selector(_updateProgress:) userInfo:nil repeats:YES];
     221
     222    [self _processItemAtIndex:0];
     223}
     224
     225- (void)_processItemAtIndex:(NSUInteger)index
     226{
     227    if ([_progressController isCancelled])
     228        return;
     229
     230    if (index >= [_items count]) {
     231        [self _finishedProcessing];
     232        return;
     233    }
     234
     235    _WKFileUploadItem *item = [_items objectAtIndex:index];
     236
     237    while (!item.isVideo) {
     238        index++;
     239
     240        if (index == [_items count]) {
     241            [self _finishedProcessing];
     242            return;
     243        }
     244
     245        item = [_items objectAtIndex:index];
     246    }
     247
     248    NSString *temporaryDirectory = [self _temporaryDirectoryCreateIfNecessary];
     249    if (!temporaryDirectory) {
     250        LOG_ERROR("WKFileUploadMediaTranscoder: Failed to make temporary directory");
     251        [self _finishedProcessing];
     252        return;
     253    }
     254
     255    NSString *fileName = [item.fileURL.lastPathComponent.stringByDeletingPathExtension stringByAppendingPathExtension:UTTypeQuickTimeMovie.preferredFilenameExtension.uppercaseString];
     256    NSString *filePath = [temporaryDirectory stringByAppendingPathComponent:fileName];
     257    NSURL *outputURL = [NSURL fileURLWithPath:filePath isDirectory:NO];
     258
     259    RetainPtr<AVURLAsset> asset = adoptNS([PAL::allocAVURLAssetInstance() initWithURL:item.fileURL options:nil]);
     260    _exportSession = adoptNS([PAL::allocAVAssetExportSessionInstance() initWithAsset:asset.get() presetName:AVAssetExportPresetHighestQuality]);
     261    [_exportSession setOutputURL:outputURL];
     262    [_exportSession setOutputFileType:AVFileTypeQuickTimeMovie];
     263
     264    [_exportSession exportAsynchronouslyWithCompletionHandler:makeBlockPtr([weakSelf = WeakObjCPtr<WKFileUploadMediaTranscoder>(self), index] () mutable {
     265        ensureOnMainRunLoop([weakSelf = WTFMove(weakSelf), index] {
     266            auto strongSelf = weakSelf.get();
     267            if (!strongSelf)
     268                return;
     269
     270            AVAssetExportSessionStatus status = [strongSelf->_exportSession status];
     271
     272            if (status == AVAssetExportSessionStatusCancelled)
     273                return;
     274
     275            if (status == AVAssetExportSessionStatusCompleted) {
     276                _WKFileUploadItem *item = [strongSelf->_items objectAtIndex:index];
     277                [item setFileURL:[strongSelf->_exportSession outputURL]];
     278            }
     279
     280            strongSelf->_exportSession = nil;
     281
     282            strongSelf->_processedVideoCount++;
     283            [strongSelf _processItemAtIndex:index + 1];
     284        });
     285    }).get()];
     286}
     287
     288- (void)_finishedProcessing
     289{
     290    [self _dismissProgress];
     291
     292    if (auto completionHandler = std::exchange(_completionHandler, nullptr))
     293        completionHandler(_items.get());
     294}
     295
     296- (void)_dismissProgress
     297{
     298    [_progressTimer invalidate];
     299    [_progressController hideAnimated:NO allowDelay:NO];
     300}
     301
     302- (void)_updateProgress:(NSTimer *)timer
     303{
     304    auto currentSessionProgress = [_exportSession progress];
     305    [_progressController setFractionCompleted:(currentSessionProgress + _processedVideoCount) / _videoCount];
     306}
     307
     308- (NSString *)_temporaryDirectoryCreateIfNecessary
     309{
     310    if (_temporaryDirectoryPath) {
     311        BOOL isDirectory = NO;
     312        BOOL exists = [[NSFileManager defaultManager] fileExistsAtPath:_temporaryDirectoryPath.get() isDirectory:&isDirectory];
     313
     314        if (exists && isDirectory)
     315            return _temporaryDirectoryPath.get();
     316    }
     317
     318    _temporaryDirectoryPath = FileSystem::createTemporaryDirectory(@"WKVideoUpload");
     319    return _temporaryDirectoryPath.get();
     320}
     321
     322@end
    156323
    157324#pragma mark - WKFileUploadPanel
    158 
    159325
    160326@interface WKFileUploadPanel () <UIPopoverControllerDelegate, UINavigationControllerDelegate, UIImagePickerControllerDelegate, UIDocumentPickerDelegate, UIAdaptivePresentationControllerDelegate
    … …  
    173339    BOOL _allowMultipleFiles;
    174340    BOOL _usingCamera;
     341    RetainPtr<WKFileUploadMediaTranscoder> _mediaTranscoder;
    175342    RetainPtr<UIImagePickerController> _imagePicker;
    176343    RetainPtr<UIViewController> _presentationViewController; // iPhone always. iPad for Fullscreen Camera.
    … …  
    219386   
    220387    [self _dispatchDidDismiss];
     388}
     389
     390- (void)_chooseMediaItems:(NSArray<_WKFileUploadItem *> *)mediaItems
     391{
     392    RetainPtr<UIImage> iconImage = nil;
     393    NSMutableArray *fileURLs = [NSMutableArray array];
     394    NSUInteger videoCount = 0;
     395
     396    for (_WKFileUploadItem *item in mediaItems) {
     397        [fileURLs addObject:item.fileURL];
     398
     399        if (!iconImage)
     400            iconImage = item.displayImage;
     401
     402        if (item.isVideo)
     403            videoCount++;
     404    }
     405
     406    NSUInteger imageCount = mediaItems.count - videoCount;
     407
     408    NSString *displayString = (imageCount || videoCount) ? [NSString localizedStringWithFormat:WEB_UI_NSSTRING(@"%lu photo(s) and %lu video(s)", "label next to file upload control; parameters are the number of photos and the number of videos"), (unsigned long)imageCount, (unsigned long)videoCount] : nil;
     409
     410    [self _dismissDisplayAnimated:YES];
     411    [self _chooseFiles:fileURLs displayString:displayString iconImage:iconImage.get()];
    221412}
    222413
    … …  
    730921        return;
    731922
    732     [self _dismissDisplayAnimated:YES];
    733 
    734923    [self _processMediaInfoDictionaries:@[info]
    735         successBlock:^(NSArray *processedResults, NSString *displayString) {
    736             ASSERT([processedResults count] == 1);
    737             _WKFileUploadItem *result = [processedResults objectAtIndex:0];
    738             RunLoop::main().dispatch([self, strongSelf = retainPtr(self), result = retainPtr(result), displayString = retainPtr(displayString)] {
    739                 [self _chooseFiles:@[result.get().fileURL] displayString:displayString.get() iconImage:result.get().displayImage.get()];
     924        successBlock:^(NSArray<_WKFileUploadItem *> *items) {
     925            ASSERT([items count] == 1);
     926            ensureOnMainRunLoop([self, strongSelf = retainPtr(self), items = retainPtr(items)] {
     927                [self _chooseMediaItems:items.get()];
    740928            });
    741929        }
    742930        failureBlock:^{
    743             RunLoop::main().dispatch([self, strongSelf = retainPtr(self)] {
     931            ensureOnMainRunLoop([self, strongSelf = retainPtr(self)] {
     932                [self _dismissDisplayAnimated:YES];
    744933                [self _cancel];
    745934            });
    … …  
    750939- (void)imagePickerController:(UIImagePickerController *)imagePicker didFinishPickingMultipleMediaWithInfo:(NSArray *)infos
    751940{
    752     [self _dismissDisplayAnimated:YES];
    753 
    754941    [self _processMediaInfoDictionaries:infos
    755         successBlock:^(NSArray *processedResults, NSString *displayString) {
    756             RetainPtr<UIImage> iconImage = nil;
    757             NSMutableArray *fileURLs = [NSMutableArray array];
    758             for (_WKFileUploadItem *result in processedResults) {
    759                 NSURL *fileURL = result.fileURL;
    760                 if (!fileURL)
    761                     continue;
    762                 [fileURLs addObject:result.fileURL];
    763                 if (!iconImage)
    764                     iconImage = result.displayImage;
    765             }
    766 
    767             RunLoop::main().dispatch([self, strongSelf = retainPtr(self), fileURLs = retainPtr(fileURLs), displayString = retainPtr(displayString), iconImage] {
    768                 [self _chooseFiles:fileURLs.get() displayString:displayString.get() iconImage:iconImage.get()];
    769             });
     942        successBlock:^(NSArray<_WKFileUploadItem *> *items) {
     943            [self _uploadMediaItemsTranscodingVideo:items];
    770944        }
    771945        failureBlock:^{
    772             RunLoop::main().dispatch([self, strongSelf = retainPtr(self)] {
     946            ensureOnMainRunLoop([self, strongSelf = retainPtr(self)] {
     947                [self _dismissDisplayAnimated:YES];
    773948                [self _cancel];
    774949            });
    … …  
    785960#pragma mark - Process UIImagePicker results
    786961
    787 - (void)_processMediaInfoDictionaries:(NSArray *)infos successBlock:(void (^)(NSArray *processedResults, NSString *displayString))successBlock failureBlock:(void (^)(void))failureBlock
    788 {
    789     [self _processMediaInfoDictionaries:infos atIndex:0 processedResults:[NSMutableArray array] processedImageCount:0 processedVideoCount:0 successBlock:successBlock failureBlock:failureBlock];
    790 }
    791 
    792 - (void)_processMediaInfoDictionaries:(NSArray *)infos atIndex:(NSUInteger)index processedResults:(NSMutableArray *)processedResults processedImageCount:(NSUInteger)processedImageCount processedVideoCount:(NSUInteger)processedVideoCount successBlock:(void (^)(NSArray *processedResults, NSString *displayString))successBlock failureBlock:(void (^)(void))failureBlock
     962- (void)_processMediaInfoDictionaries:(NSArray *)infos successBlock:(void (^)(NSArray<_WKFileUploadItem *> *processedResults))successBlock failureBlock:(void (^)(void))failureBlock
     963{
     964    [self _processMediaInfoDictionaries:infos atIndex:0 processedResults:[NSMutableArray array] successBlock:successBlock failureBlock:failureBlock];
     965}
     966
     967- (void)_processMediaInfoDictionaries:(NSArray *)infos atIndex:(NSUInteger)index processedResults:(NSMutableArray<_WKFileUploadItem *> *)processedResults successBlock:(void (^)(NSArray<_WKFileUploadItem *> *processedResults))successBlock failureBlock:(void (^)(void))failureBlock
    793968{
    794969    NSUInteger count = [infos count];
    795970    if (index == count) {
    796         NSString *displayString = (processedImageCount || processedVideoCount) ? [NSString localizedStringWithFormat:WEB_UI_NSSTRING(@"%lu photo(s) and %lu video(s)", "label next to file upload control; parameters are the number of photos and the number of videos"), (unsigned long)processedImageCount, (unsigned long)processedVideoCount] : nil;
    797         successBlock(processedResults, displayString);
     971        successBlock(processedResults);
    798972        return;
    799973    }
    … …  
    804978
    805979    auto uploadItemSuccessBlock = ^(_WKFileUploadItem *uploadItem) {
    806         NSUInteger newProcessedVideoCount = processedVideoCount + (uploadItem.isVideo ? 1 : 0);
    807         NSUInteger newProcessedImageCount = processedImageCount + (uploadItem.isVideo ? 0 : 1);
    808980        [processedResults addObject:uploadItem];
    809         [self _processMediaInfoDictionaries:infos atIndex:index processedResults:processedResults processedImageCount:newProcessedImageCount processedVideoCount:newProcessedVideoCount successBlock:successBlock failureBlock:failureBlock];
     981        [self _processMediaInfoDictionaries:infos atIndex:index processedResults:processedResults successBlock:successBlock failureBlock:failureBlock];
    810982    };
    811983
    … …  
    8911063    }
    8921064
    893 #if PLATFORM(IOS_FAMILY)
    8941065    if (NSURL *imageURL = info[UIImagePickerControllerImageURL]) {
    8951066        if (!imageURL.isFileURL) {
    … …  
    9031074        return;
    9041075    }
    905 #endif
    9061076
    9071077    UIImage *originalImage = [info objectForKey:UIImagePickerControllerOriginalImage];
    … …  
    9171087}
    9181088
     1089- (void)_uploadMediaItemsTranscodingVideo:(NSArray<_WKFileUploadItem *> *)items
     1090{
     1091    auto videoCount = [[items indexesOfObjectsPassingTest:^(_WKFileUploadItem *item, NSUInteger, BOOL*) {
     1092        return item.isVideo;
     1093    }] count];
     1094
     1095    ensureOnMainRunLoop([self, strongSelf = retainPtr(self), items = retainPtr(items), videoCount] {
     1096        if (!videoCount) {
     1097            [self _chooseMediaItems:items.get()];
     1098            return;
     1099        }
     1100
     1101        _mediaTranscoder = adoptNS([[WKFileUploadMediaTranscoder alloc] initWithItems:items.get() videoCount:videoCount completionHandler:[weakSelf = WeakObjCPtr<WKFileUploadPanel>(self)] (NSArray<_WKFileUploadItem *> *items) {
     1102            auto strongSelf = weakSelf.get();
     1103            if (!strongSelf)
     1104                return;
     1105
     1106            [strongSelf _chooseMediaItems:items];
     1107        }]);
     1108
     1109        [_mediaTranscoder start];
     1110    });
     1111}
     1112
    9191113- (BOOL)platformSupportsPickerViewController
    9201114{

  • trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj

    r283559 r283592  
    20342034                E5BEF6822130C48000F31111 /* WebDataListSuggestionsDropdownIOS.h in Headers */ = {isa = PBXBuildFile; fileRef = E5BEF6802130C47F00F31111 /* WebDataListSuggestionsDropdownIOS.h */; };
    20352035                E5CB07DC20E1678F0022C183 /* WKFormColorControl.h in Headers */ = {isa = PBXBuildFile; fileRef = E5CB07DA20E1678F0022C183 /* WKFormColorControl.h */; };
     2036                E5DEFA6826F8F42600AB68DB /* PhotosUISPI.h in Headers */ = {isa = PBXBuildFile; fileRef = E5DEFA6726F8F42600AB68DB /* PhotosUISPI.h */; };
    20362037                ED82A7F2128C6FAF004477B3 /* WKBundlePageOverlay.h in Headers */ = {isa = PBXBuildFile; fileRef = 1A22F0FF1289FCD90085E74F /* WKBundlePageOverlay.h */; settings = {ATTRIBUTES = (Private, ); }; };
    20372038                F4094CBD2553053D003D73E3 /* DisplayListReaderHandle.h in Headers */ = {isa = PBXBuildFile; fileRef = F4094CBB255304AF003D73E3 /* DisplayListReaderHandle.h */; };
    … …  
    60936094                E5CB07DA20E1678F0022C183 /* WKFormColorControl.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = WKFormColorControl.h; path = ios/forms/WKFormColorControl.h; sourceTree = "<group>"; };
    60946095                E5CB07DB20E1678F0022C183 /* WKFormColorControl.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; name = WKFormColorControl.mm; path = ios/forms/WKFormColorControl.mm; sourceTree = "<group>"; };
     6096                E5DEFA6726F8F42600AB68DB /* PhotosUISPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PhotosUISPI.h; sourceTree = "<group>"; };
    60956097                ECA680D31E6904B500731D20 /* ExtraPrivateSymbolsForTAPI.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ExtraPrivateSymbolsForTAPI.h; sourceTree = "<group>"; };
    60966098                ECBFC1DB1E6A4D66000300C7 /* ExtraPublicSymbolsForTAPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = ExtraPublicSymbolsForTAPI.h; sourceTree = "<group>"; };
    … …  
    1180611808                                A13B3DA1207F39DE0090C58D /* MobileWiFiSPI.h */,
    1180711809                                3178AF9720E2A7F80074DE94 /* PDFKitSPI.h */,
     11810                                E5DEFA6726F8F42600AB68DB /* PhotosUISPI.h */,
    1180811811                                2D279E1826955768004B3EEB /* PrototypeToolsSPI.h */,
    1180911812                                46F38E8B2416E66D0059375A /* RunningBoardServicesSPI.h */,
    … …  
    1248212485                                832ED18C1E2FE157006BA64A /* PerActivityStateCPUUsageSampler.h in Headers */,
    1248312486                                7AFBD36F21E546F8005DBACB /* PersistencyUtils.h in Headers */,
     12487                            E5DEFA6826F8F42600AB68DB /* PhotosUISPI.h in Headers */,
    1248412488                                5CE85B201C88E64B0070BFCE /* PingLoad.h in Headers */,
    1248512489                                0F5E200418E77051003EC3E5 /* PlatformCAAnimationRemote.h in Headers */,
Note: See TracChangeset for help on using the changeset viewer.