Changeset 255424 in webkit


Ignore:
Timestamp:
Jan 30, 2020 6:52:29 AM (4 years ago)
Author:
youenn@apple.com
Message:

[Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
​https://bugs.webkit.org/show_bug.cgi?id=206582

Reviewed by Eric Carlson.

Source/WebCore:

AVAssetWriterDelegate allows to grab recorded data whenever wanted.
This delegate requires passing compressed samples to AVAssetWriter.
Implement video encoding and audio encoding in dedicated classes and use these classes before adding buffers to AVAssetWriter.
Since AVAssetWriterDelegate is Apple SDK only, keep the existing file based implementation as a fallback.

Covered by existing tests.

  • platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
  • platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:

(WebCore::AudioSampleBufferCompressor::create):
(WebCore::AudioSampleBufferCompressor::AudioSampleBufferCompressor):
(WebCore::AudioSampleBufferCompressor::~AudioSampleBufferCompressor):
(WebCore::AudioSampleBufferCompressor::initialize):
(WebCore::AudioSampleBufferCompressor::finish):
(WebCore::AudioSampleBufferCompressor::audioConverterComplexInputDataProc):
(WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
(WebCore::AudioSampleBufferCompressor::computeBufferSizeForAudioFormat):
(WebCore::AudioSampleBufferCompressor::attachPrimingTrimsIfNeeded):
(WebCore::AudioSampleBufferCompressor::gradualDecoderRefreshCount):
(WebCore::AudioSampleBufferCompressor::sampleBufferWithNumPackets):
(WebCore::AudioSampleBufferCompressor::processSampleBuffersUntilLowWaterTime):
(WebCore::AudioSampleBufferCompressor::provideSourceDataNumOutputPackets):
(WebCore::AudioSampleBufferCompressor::processSampleBuffer):
(WebCore::AudioSampleBufferCompressor::addSampleBuffer):
(WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer):
(WebCore::AudioSampleBufferCompressor::takeOutputSampleBuffer):

  • platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
  • platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:

(WebCore::MediaRecorderPrivateWriter::create):
(WebCore::MediaRecorderPrivateWriter::MediaRecorderPrivateWriter):
(WebCore::MediaRecorderPrivateWriter::initialize):
(WebCore::MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers):
(WebCore::MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendVideoSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::stopRecording):

  • platform/mediarecorder/cocoa/VideoSampleBufferCompressor.h:
  • platform/mediarecorder/cocoa/VideoSampleBufferCompressor.mm:

(WebCore::VideoSampleBufferCompressor::create):
(WebCore::VideoSampleBufferCompressor::VideoSampleBufferCompressor):
(WebCore::VideoSampleBufferCompressor::~VideoSampleBufferCompressor):
(WebCore::VideoSampleBufferCompressor::initialize):
(WebCore::VideoSampleBufferCompressor::finish):
(WebCore::VideoSampleBufferCompressor::videoCompressionCallback):
(WebCore::VideoSampleBufferCompressor::initCompressionSession):
(WebCore::VideoSampleBufferCompressor::processSampleBuffer):
(WebCore::VideoSampleBufferCompressor::addSampleBuffer):
(WebCore::VideoSampleBufferCompressor::getOutputSampleBuffer):
(WebCore::VideoSampleBufferCompressor::takeOutputSampleBuffer):

Source/WebCore/PAL:

Add soft link macros for VideoToolbox and AudioToolbox.

  • PAL.xcodeproj/project.pbxproj:
  • pal/cf/AudioToolboxSoftLink.cpp: Added.
  • pal/cf/AudioToolboxSoftLink.h: Added.
  • pal/cf/CoreMediaSoftLink.cpp:
  • pal/cf/CoreMediaSoftLink.h:
  • pal/cf/VideoToolboxSoftLink.cpp: Added.
  • pal/cf/VideoToolboxSoftLink.h: Added.

Source/WebKit:

  • GPUProcess/webrtc/RemoteMediaRecorder.cpp:

(WebKit::RemoteMediaRecorder::create):
Use new constructor.

LayoutTests:

  • http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html:

Remove web audio generation since there seems to be some unstability in web audio -> stream -> media recorder.
which should be fixed as follow-up specific patches.

Location:
trunk
Files:
8 added
13 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r255422 r255424  
     12020-01-30  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        * http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html:
     9        Remove web audio generation since there seems to be some unstability in web audio -> stream -> media recorder.
     10        which should be fixed as follow-up specific patches.
     11
    1122020-01-30  Antoine Quint  <graouts@apple.com>
    213

  • trunk/LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html

    r255356 r255424  
    5757
    5858    async_test(t => {
    59         const ac = new AudioContext();
    60         const osc = ac.createOscillator();
    61         const dest = ac.createMediaStreamDestination();
    62         const audio = dest.stream;
    63         osc.connect(dest);
    64 
    6559        const video = createVideoStream();
    66         assert_equals(video.getAudioTracks().length, 0, "video mediastream starts with no audio track");
    67         assert_equals(audio.getAudioTracks().length, 1, "audio mediastream starts with one audio track");
    68         video.addTrack(audio.getAudioTracks()[0]);
    69         assert_equals(video.getAudioTracks().length, 1, "video mediastream starts with one audio track");
    7060        const recorder = new MediaRecorder(video);
    7161        let mode = 0;

  • trunk/Source/WebCore/ChangeLog

    r255423 r255424  
     12020-01-30  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        AVAssetWriterDelegate allows to grab recorded data whenever wanted.
     9        This delegate requires passing compressed samples to AVAssetWriter.
     10        Implement video encoding and audio encoding in dedicated classes and use these classes before adding buffers to AVAssetWriter.
     11        Since AVAssetWriterDelegate is Apple SDK only, keep the existing file based implementation as a fallback.
     12
     13        Covered by existing tests.
     14
     15        * platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
     16        * platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:
     17        (WebCore::AudioSampleBufferCompressor::create):
     18        (WebCore::AudioSampleBufferCompressor::AudioSampleBufferCompressor):
     19        (WebCore::AudioSampleBufferCompressor::~AudioSampleBufferCompressor):
     20        (WebCore::AudioSampleBufferCompressor::initialize):
     21        (WebCore::AudioSampleBufferCompressor::finish):
     22        (WebCore::AudioSampleBufferCompressor::audioConverterComplexInputDataProc):
     23        (WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
     24        (WebCore::AudioSampleBufferCompressor::computeBufferSizeForAudioFormat):
     25        (WebCore::AudioSampleBufferCompressor::attachPrimingTrimsIfNeeded):
     26        (WebCore::AudioSampleBufferCompressor::gradualDecoderRefreshCount):
     27        (WebCore::AudioSampleBufferCompressor::sampleBufferWithNumPackets):
     28        (WebCore::AudioSampleBufferCompressor::processSampleBuffersUntilLowWaterTime):
     29        (WebCore::AudioSampleBufferCompressor::provideSourceDataNumOutputPackets):
     30        (WebCore::AudioSampleBufferCompressor::processSampleBuffer):
     31        (WebCore::AudioSampleBufferCompressor::addSampleBuffer):
     32        (WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer):
     33        (WebCore::AudioSampleBufferCompressor::takeOutputSampleBuffer):
     34        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
     35        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
     36        (WebCore::MediaRecorderPrivateWriter::create):
     37        (WebCore::MediaRecorderPrivateWriter::MediaRecorderPrivateWriter):
     38        (WebCore::MediaRecorderPrivateWriter::initialize):
     39        (WebCore::MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers):
     40        (WebCore::MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers):
     41        (WebCore::MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer):
     42        (WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer):
     43        (WebCore::MediaRecorderPrivateWriter::appendVideoSampleBuffer):
     44        (WebCore::MediaRecorderPrivateWriter::appendAudioSampleBuffer):
     45        (WebCore::MediaRecorderPrivateWriter::stopRecording):
     46        * platform/mediarecorder/cocoa/VideoSampleBufferCompressor.h:
     47        * platform/mediarecorder/cocoa/VideoSampleBufferCompressor.mm:
     48        (WebCore::VideoSampleBufferCompressor::create):
     49        (WebCore::VideoSampleBufferCompressor::VideoSampleBufferCompressor):
     50        (WebCore::VideoSampleBufferCompressor::~VideoSampleBufferCompressor):
     51        (WebCore::VideoSampleBufferCompressor::initialize):
     52        (WebCore::VideoSampleBufferCompressor::finish):
     53        (WebCore::VideoSampleBufferCompressor::videoCompressionCallback):
     54        (WebCore::VideoSampleBufferCompressor::initCompressionSession):
     55        (WebCore::VideoSampleBufferCompressor::processSampleBuffer):
     56        (WebCore::VideoSampleBufferCompressor::addSampleBuffer):
     57        (WebCore::VideoSampleBufferCompressor::getOutputSampleBuffer):
     58        (WebCore::VideoSampleBufferCompressor::takeOutputSampleBuffer):
     59
    1602020-01-29  Sergio Villar Senin  <svillar@igalia.com>
    261

  • trunk/Source/WebCore/PAL/ChangeLog

    r255356 r255424  
     12020-01-30  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        Add soft link macros for VideoToolbox and AudioToolbox.
     9
     10        * PAL.xcodeproj/project.pbxproj:
     11        * pal/cf/AudioToolboxSoftLink.cpp: Added.
     12        * pal/cf/AudioToolboxSoftLink.h: Added.
     13        * pal/cf/CoreMediaSoftLink.cpp:
     14        * pal/cf/CoreMediaSoftLink.h:
     15        * pal/cf/VideoToolboxSoftLink.cpp: Added.
     16        * pal/cf/VideoToolboxSoftLink.h: Added.
     17
    1182020-01-29  Commit Queue  <commit-queue@webkit.org>
    219

  • trunk/Source/WebCore/PAL/PAL.xcodeproj/project.pbxproj

    r255356 r255424  
    116116                2E1342CD215AA10A007199D2 /* UIKitSoftLink.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E1342CB215AA10A007199D2 /* UIKitSoftLink.mm */; };
    117117                31308B1420A21705003FB929 /* SystemPreviewSPI.h in Headers */ = {isa = PBXBuildFile; fileRef = 31308B1320A21705003FB929 /* SystemPreviewSPI.h */; };
     118                416E995323DAE6BE00E871CB /* AudioToolboxSoftLink.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */; };
     119                416E995423DAE6BE00E871CB /* AudioToolboxSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = 416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */; };
     120                416E995723DAEFF800E871CB /* VideoToolboxSoftLink.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */; };
     121                416E995823DAEFF800E871CB /* VideoToolboxSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = 416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */; };
    118122                442956CD218A72DF0080DB54 /* RevealSPI.h in Headers */ = {isa = PBXBuildFile; fileRef = 442956CC218A72DE0080DB54 /* RevealSPI.h */; };
    119123                4450FC9F21F5F602004DFA56 /* QuickLookSoftLink.mm in Sources */ = {isa = PBXBuildFile; fileRef = 4450FC9D21F5F602004DFA56 /* QuickLookSoftLink.mm */; };
    … …  
    292296                31308B1320A21705003FB929 /* SystemPreviewSPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SystemPreviewSPI.h; sourceTree = "<group>"; };
    293297                37119A7820CCB5FF002C6DC9 /* WebKitTargetConditionals.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = WebKitTargetConditionals.xcconfig; sourceTree = "<group>"; };
     298                416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AudioToolboxSoftLink.cpp; sourceTree = "<group>"; };
     299                416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioToolboxSoftLink.h; sourceTree = "<group>"; };
     300                416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = VideoToolboxSoftLink.cpp; sourceTree = "<group>"; };
     301                416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoToolboxSoftLink.h; sourceTree = "<group>"; };
    294302                442956CC218A72DE0080DB54 /* RevealSPI.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RevealSPI.h; sourceTree = "<group>"; };
    295303                4450FC9D21F5F602004DFA56 /* QuickLookSoftLink.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = QuickLookSoftLink.mm; sourceTree = "<group>"; };
    … …  
    525533                        isa = PBXGroup;
    526534                        children = (
     535                                416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */,
     536                                416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */,
    527537                                0CF99CA61F738436007EE793 /* CoreMediaSoftLink.cpp */,
    528538                                0CF99CA71F738437007EE793 /* CoreMediaSoftLink.h */,
     539                                416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */,
     540                                416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */,
    529541                        );
    530542                        path = cf;
    … …  
    716728                                57FD318B22B35989008D0E8B /* AppSSOSoftLink.h in Headers */,
    717729                                576CA9D622B854AB0030143C /* AppSSOSPI.h in Headers */,
     730                                416E995423DAE6BE00E871CB /* AudioToolboxSoftLink.h in Headers */,
    718731                                2D02E93C2056FAA700A13797 /* AudioToolboxSPI.h in Headers */,
    719732                                572A107822B456F500F410C8 /* AuthKitSPI.h in Headers */,
    … …  
    839852                                0C5AF9221F43A4C7002EAC02 /* UIKitSPI.h in Headers */,
    840853                                0C2DA1471F3BEB4900DBC317 /* URLFormattingSPI.h in Headers */,
     854                                416E995823DAEFF800E871CB /* VideoToolboxSoftLink.h in Headers */,
    841855                                0C2DA1591F3BEB4900DBC317 /* WebFilterEvaluatorSPI.h in Headers */,
    842856                                A10826F91F576292004772AC /* WebPanel.h in Headers */,
    … …  
    931945                        files = (
    932946                                57FD318A22B3593E008D0E8B /* AppSSOSoftLink.mm in Sources */,
     947                                416E995323DAE6BE00E871CB /* AudioToolboxSoftLink.cpp in Sources */,
    933948                                077E87B1226A460200A2AFF0 /* AVFoundationSoftLink.mm in Sources */,
    934949                                0C5FFF0F1F78D9DA009EFF1A /* ClockCM.mm in Sources */,
    … …  
    957972                                A3AB6E651F3D217F009C14B1 /* SystemSleepListenerMac.mm in Sources */,
    958973                                2E1342CD215AA10A007199D2 /* UIKitSoftLink.mm in Sources */,
     974                                416E995723DAEFF800E871CB /* VideoToolboxSoftLink.cpp in Sources */,
    959975                                A10826FA1F576292004772AC /* WebPanel.mm in Sources */,
    960976                        );

  • trunk/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp

    r255356 r255424  
    4747SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferCopyDataBytes, OSStatus, (CMBlockBufferRef theSourceBuffer, size_t offsetToData, size_t dataLength, void* destination), (theSourceBuffer, offsetToData, dataLength, destination))
    4848SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferGetDataLength, size_t, (CMBlockBufferRef theBuffer), (theBuffer))
     49SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferReplaceDataBytes, OSStatus, (const void* sourceBytes, CMBlockBufferRef destinationBuffer, size_t offsetIntoDestination, size_t dataLength), (sourceBytes, destinationBuffer, offsetIntoDestination, dataLength))
    4950SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMFormatDescriptionGetExtensions, CFDictionaryRef, (CMFormatDescriptionRef desc), (desc))
    5051SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetTypeID, CFTypeID, (void), ())
    … …  
    137138SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueGetEndPresentationTimeStamp, CMTime, (CMBufferQueueRef queue), (queue))
    138139SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
     140SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueMarkEndOfData, OSStatus, (CMBufferQueueRef queue), (queue))
    139141
    140142SOFT_LINK_CONSTANT_FOR_SOURCE(PAL, CoreMedia, kCMSampleAttachmentKey_DoNotDisplay, CFStringRef)
    … …  
    164166SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferSetDataReady, OSStatus, (CMSampleBufferRef sbuf), (sbuf))
    165167SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionCreate, OSStatus, (CFAllocatorRef allocator, const AudioStreamBasicDescription* asbd, size_t layoutSize, const AudioChannelLayout* layout, size_t magicCookieSize, const void* magicCookie, CFDictionaryRef extensions, CMAudioFormatDescriptionRef* outDesc), (allocator, asbd, layoutSize, layout, magicCookieSize, magicCookie, extensions, outDesc))
     168SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionGetMagicCookie, const void*, (CMAudioFormatDescriptionRef desc, size_t* sizeOut), (desc, sizeOut))
     169SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionGetRichestDecodableFormat, const AudioFormatListItem *, (CMAudioFormatDescriptionRef desc), (desc))
     170
    166171SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMClockGetHostTimeClock, CMClockRef, (void), ())
    167172SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMClockGetTime, CMTime, (CMClockRef clock), (clock))

  • trunk/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h

    r255356 r255424  
    5050SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBlockBufferGetDataLength, size_t, (CMBlockBufferRef theBuffer), (theBuffer))
    5151#define CMBlockBufferGetDataLength softLink_CoreMedia_CMBlockBufferGetDataLength
     52SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBlockBufferReplaceDataBytes, OSStatus, (const void* sourceBytes, CMBlockBufferRef destinationBuffer, size_t offsetIntoDestination, size_t dataLength), (sourceBytes, destinationBuffer, offsetIntoDestination, dataLength))
     53#define CMBlockBufferReplaceDataBytes softLink_CoreMedia_CMBlockBufferReplaceDataBytes
    5254SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMFormatDescriptionGetExtensions, CFDictionaryRef, (CMFormatDescriptionRef desc), (desc))
    5355#define CMFormatDescriptionGetExtensions softLink_CoreMedia_CMFormatDescriptionGetExtensions
    … …  
    228230SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
    229231#define CMBufferQueueInstallTriggerWithIntegerThreshold softLink_CoreMedia_CMBufferQueueInstallTriggerWithIntegerThreshold
     232SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBufferQueueMarkEndOfData, OSStatus, (CMBufferQueueRef queue), (queue))
     233#define CMBufferQueueMarkEndOfData softLink_CoreMedia_CMBufferQueueMarkEndOfData
    230234
    231235SOFT_LINK_CONSTANT_FOR_HEADER(PAL, CoreMedia, kCMSampleAttachmentKey_DoNotDisplay, CFStringRef)
    … …  
    257261SOFT_LINK_CONSTANT_FOR_HEADER(PAL, CoreMedia, kCMSampleBufferConsumerNotification_BufferConsumed, CFStringRef)
    258262#define kCMSampleBufferConsumerNotification_BufferConsumed get_CoreMedia_kCMSampleBufferConsumerNotification_BufferConsumed()
     263
     264SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetMagicCookie, const void*, (CMAudioFormatDescriptionRef desc, size_t* sizeOut), (desc, sizeOut))
     265#define CMAudioFormatDescriptionGetMagicCookie softLink_CoreMedia_CMAudioFormatDescriptionGetMagicCookie
    259266SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetStreamBasicDescription, const AudioStreamBasicDescription *, (CMAudioFormatDescriptionRef desc), (desc))
    260267#define CMAudioFormatDescriptionGetStreamBasicDescription softLink_CoreMedia_CMAudioFormatDescriptionGetStreamBasicDescription
    … …  
    265272SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetNumSamples, CMItemCount, (CMSampleBufferRef sbuf), (sbuf))
    266273#define CMSampleBufferGetNumSamples softLink_CoreMedia_CMSampleBufferGetNumSamples
     274SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetRichestDecodableFormat, const AudioFormatListItem *, (CMAudioFormatDescriptionRef desc), (desc))
     275#define CMAudioFormatDescriptionGetRichestDecodableFormat softLink_CoreMedia_CMAudioFormatDescriptionGetRichestDecodableFormat
    267276SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCopySampleBufferForRange, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CFRange sampleRange, CMSampleBufferRef* sBufOut), (allocator, sbuf, sampleRange, sBufOut))
    268277#define CMSampleBufferCopySampleBufferForRange softLink_CoreMedia_CMSampleBufferCopySampleBufferForRange

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r255406 r255424  
    10981098                416E6FE91BBD12E5000A6043 /* ReadableStreamBuiltins.h in Headers */ = {isa = PBXBuildFile; fileRef = 9B03D8061BB3110D00B764D8 /* ReadableStreamBuiltins.h */; settings = {ATTRIBUTES = (Private, ); }; };
    10991099                416E6FE91BBD12E5000A6053 /* WritableStreamBuiltins.h in Headers */ = {isa = PBXBuildFile; fileRef = 9B03D8061BB3110D00B764E8 /* WritableStreamBuiltins.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1100                416F799023D750CF00829FC1 /* AudioSampleBufferCompressor.mm in Sources */ = {isa = PBXBuildFile; fileRef = 416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */; };
    11001101                417253AB1354BBBC00360F2A /* MediaControlElements.h in Headers */ = {isa = PBXBuildFile; fileRef = 417253A91354BBBC00360F2A /* MediaControlElements.h */; };
    11011102                417612AF1E3A994000C3D81D /* LibWebRTCMediaEndpoint.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 417612AB1E3A993B00C3D81D /* LibWebRTCMediaEndpoint.cpp */; };
    … …  
    11381139                41BF204922BA7BE80004F812 /* RealtimeVideoSource.h in Headers */ = {isa = PBXBuildFile; fileRef = 41BF204022B947160004F812 /* RealtimeVideoSource.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11391140                41C760B10EDE03D300C1655F /* ScriptState.h in Headers */ = {isa = PBXBuildFile; fileRef = 41C760B00EDE03D300C1655F /* ScriptState.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1141                41CD6F8C23D6E82100B16421 /* VideoSampleBufferCompressor.mm in Sources */ = {isa = PBXBuildFile; fileRef = 41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */; };
    11401142                41D015CA0F4B5C71004A662F /* ContentType.h in Headers */ = {isa = PBXBuildFile; fileRef = 41D015C80F4B5C71004A662F /* ContentType.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11411143                41D129CE1F3D0EF600D15E47 /* WorkerGlobalScopeCaches.h in Headers */ = {isa = PBXBuildFile; fileRef = 41FB278D1F34C28200795487 /* WorkerGlobalScopeCaches.h */; };
    … …  
    73617363                416E0B37209BC3C2004A95D9 /* FetchIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FetchIdentifier.h; sourceTree = "<group>"; };
    73627364                416E29A5102FA962007FC14E /* WorkerReportingProxy.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WorkerReportingProxy.h; sourceTree = "<group>"; };
     7365                416F798D23D750CA00829FC1 /* AudioSampleBufferCompressor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioSampleBufferCompressor.h; sourceTree = "<group>"; };
     7366                416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = AudioSampleBufferCompressor.mm; sourceTree = "<group>"; };
     7367                416F799323D8B69C00829FC1 /* MediaRecorderPrivateWriterCocoaFileBased.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaRecorderPrivateWriterCocoaFileBased.mm; sourceTree = "<group>"; };
    73637368                4170A2E91D8C0CC000318452 /* JSDOMWrapper.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSDOMWrapper.cpp; sourceTree = "<group>"; };
    73647369                417253A81354BBBC00360F2A /* MediaControlElements.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaControlElements.cpp; sourceTree = "<group>"; };
    … …  
    74687473                41C7E1061E6A54360027B4DE /* CanvasCaptureMediaStreamTrack.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CanvasCaptureMediaStreamTrack.h; sourceTree = "<group>"; };
    74697474                41C7E1081E6AA37C0027B4DE /* CanvasCaptureMediaStreamTrack.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = CanvasCaptureMediaStreamTrack.idl; sourceTree = "<group>"; };
     7475                41CD6F8923D6E81C00B16421 /* VideoSampleBufferCompressor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoSampleBufferCompressor.h; sourceTree = "<group>"; };
     7476                41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = VideoSampleBufferCompressor.mm; sourceTree = "<group>"; };
    74707477                41CF8BE41D46222000707DC9 /* FetchBodyConsumer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FetchBodyConsumer.cpp; sourceTree = "<group>"; };
    74717478                41CF8BE51D46222000707DC9 /* FetchBodyConsumer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FetchBodyConsumer.h; sourceTree = "<group>"; };
    … …  
    1889118898                        isa = PBXGroup;
    1889218899                        children = (
     18900                                416F798D23D750CA00829FC1 /* AudioSampleBufferCompressor.h */,
     18901                                416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */,
    1889318902                                4D73F94C218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h */,
    1889418903                                4D73F94D218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.mm */,
     18904                                416F799323D8B69C00829FC1 /* MediaRecorderPrivateWriterCocoaFileBased.mm */,
     18905                                41CD6F8923D6E81C00B16421 /* VideoSampleBufferCompressor.h */,
     18906                                41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */,
    1889518907                        );
    1889618908                        path = cocoa;
    … …  
    3353933551                                CD0EEE0E14743F39003EAFA2 /* AudioDestinationIOS.cpp in Sources */,
    3354033552                                CD5596911475B678001D0BD0 /* AudioFileReaderIOS.cpp in Sources */,
     33553                                416F799023D750CF00829FC1 /* AudioSampleBufferCompressor.mm in Sources */,
    3354133554                                CDA79827170A279100D45C55 /* AudioSessionIOS.mm in Sources */,
    3354233555                                CD8A7BBB197735FE00CBD643 /* AudioSourceProviderAVFObjC.mm in Sources */,
    … …  
    3427134284                                3FBC4AF3189881560046EE38 /* VideoFullscreenInterfaceAVKit.mm in Sources */,
    3427234285                                52D5A18F1C54592300DE34A3 /* VideoFullscreenLayerManagerObjC.mm in Sources */,
     34286                                41CD6F8C23D6E82100B16421 /* VideoSampleBufferCompressor.mm in Sources */,
    3427334287                                BE88E0DE1715D2A200658D98 /* VideoTrack.cpp in Sources */,
    3427434288                                BE88E0E11715D2A200658D98 /* VideoTrackList.cpp in Sources */,

  • trunk/Source/WebCore/platform/graphics/cv/ImageTransferSessionVT.mm

    r255356 r255424  
    9696    };
    9797
     98
    9899    NSDictionary* pixelBufferPoolOptions = @{
    99100        (__bridge NSString *)kCVPixelBufferPoolMinimumBufferCountKey: @(6)

  • trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp

    r255356 r255424  
    3030
    3131#include "AudioStreamDescription.h"
     32#include "MediaRecorderPrivateWriterCocoa.h"
    3233#include "MediaSample.h"
    3334#include "MediaStreamPrivate.h"
    … …  
    6970        }
    7071    }
    71     auto writer = MediaRecorderPrivateWriter::create(audioTrack, videoTrack);
     72
     73    int width = 0, height = 0;
     74    if (videoTrack) {
     75        auto& settings = videoTrack->settings();
     76        width = settings.width();
     77        height = settings.height();
     78    }
     79    auto writer = MediaRecorderPrivateWriter::create(!!audioTrack, width, height);
    7280    if (!writer)
    7381        return nullptr;

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h

    r255356 r255424  
    2626
    2727#if ENABLE(MEDIA_STREAM)
     28#include "AudioStreamDescription.h"
    2829
    2930#include "SharedBuffer.h"
    … …  
    3536#include <wtf/WeakPtr.h>
    3637#include <wtf/threads/BinarySemaphore.h>
     38#include <CoreAudio/CoreAudioTypes.h>
     39#include <CoreMedia/CMTime.h>
    3740
    3841typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
     42typedef const struct opaqueCMFormatDescription* CMFormatDescriptionRef;
     43typedef struct opaqueCMBufferQueueTriggerToken *CMBufferQueueTriggerToken;
    3944
    4045OBJC_CLASS AVAssetWriter;
    4146OBJC_CLASS AVAssetWriterInput;
     47OBJC_CLASS WebAVAssetWriterDelegate;
    4248
    4349namespace WTF {
    … …  
    4753namespace WebCore {
    4854
     55class AudioSampleBufferCompressor;
    4956class AudioStreamDescription;
    5057class MediaStreamTrackPrivate;
    5158class PlatformAudioData;
     59class VideoSampleBufferCompressor;
    5260
    53 class WEBCORE_EXPORT MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter> {
     61class WEBCORE_EXPORT MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter, WeakPtrFactoryInitialization::Eager> {
    5462public:
    55     static RefPtr<MediaRecorderPrivateWriter> create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack);
    5663    static RefPtr<MediaRecorderPrivateWriter> create(bool hasAudio, int width, int height);
    5764    ~MediaRecorderPrivateWriter();
    58    
    59     bool setupWriter();
    60     bool setVideoInput(int width, int height);
    61     bool setAudioInput();
     65
    6266    void appendVideoSampleBuffer(CMSampleBufferRef);
    6367    void appendAudioSampleBuffer(const PlatformAudioData&, const AudioStreamDescription&, const WTF::MediaTime&, size_t);
    … …  
    6569    void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&)>&&);
    6670
     71#if USE(APPLE_INTERNAL_SDK)
     72    void appendData(const char*, size_t);
     73    void appendData(Ref<SharedBuffer>&&);
     74#endif
     75
    6776private:
     77#if USE(APPLE_INTERNAL_SDK)
     78    MediaRecorderPrivateWriter(bool hasAudio, bool hasVideo);
     79    void clear();
     80
     81    bool initialize();
     82
     83    static void compressedVideoOutputBufferCallback(void*, CMBufferQueueTriggerToken);
     84    static void compressedAudioOutputBufferCallback(void*, CMBufferQueueTriggerToken);
     85
     86    void startAssetWriter();
     87    void appendCompressedSampleBuffers();
     88
     89    bool appendCompressedAudioSampleBuffer();
     90    bool appendCompressedVideoSampleBuffer();
     91
     92    void processNewCompressedAudioSampleBuffers();
     93    void processNewCompressedVideoSampleBuffers();
     94
     95    void flushCompressedSampleBuffers(CompletionHandler<void()>&&);
     96    void appendEndOfVideoSampleDurationIfNeeded(CompletionHandler<void()>&&);
     97#else
    6898    MediaRecorderPrivateWriter(RetainPtr<AVAssetWriter>&&, String&& path);
    6999    void clear();
    70100
     101    bool setupWriter();
     102    bool setVideoInput(int width, int height);
     103    bool setAudioInput();
     104#endif
     105
     106    bool m_hasStartedWriting { false };
     107    bool m_isStopped { false };
     108
    71109    RetainPtr<AVAssetWriter> m_writer;
     110
     111    bool m_isStopping { false };
     112    RefPtr<SharedBuffer> m_data;
     113    CompletionHandler<void(RefPtr<SharedBuffer>&&)> m_fetchDataCompletionHandler;
     114
     115#if USE(APPLE_INTERNAL_SDK)
     116    bool m_hasAudio;
     117    bool m_hasVideo;
     118
     119    RetainPtr<CMFormatDescriptionRef> m_audioFormatDescription;
     120    std::unique_ptr<AudioSampleBufferCompressor> m_audioCompressor;
     121    RetainPtr<AVAssetWriterInput> m_audioAssetWriterInput;
     122
     123    RetainPtr<CMFormatDescriptionRef> m_videoFormatDescription;
     124    std::unique_ptr<VideoSampleBufferCompressor> m_videoCompressor;
     125    RetainPtr<AVAssetWriterInput> m_videoAssetWriterInput;
     126    CMTime m_lastVideoPresentationTime;
     127    CMTime m_lastVideoDecodingTime;
     128    bool m_hasEncodedVideoSamples { false };
     129
     130    RetainPtr<WebAVAssetWriterDelegate> m_writerDelegate;
     131#else
    72132    RetainPtr<AVAssetWriterInput> m_videoInput;
    73133    RetainPtr<AVAssetWriterInput> m_audioInput;
    … …  
    79139    BinarySemaphore m_finishWritingAudioSemaphore;
    80140    BinarySemaphore m_finishWritingVideoSemaphore;
    81     bool m_hasStartedWriting { false };
    82     bool m_isStopped { false };
    83141    bool m_isFirstAudioSample { true };
    84142    dispatch_queue_t m_audioPullQueue;
    … …  
    87145    Deque<RetainPtr<CMSampleBufferRef>> m_audioBufferPool;
    88146
    89     bool m_isStopping { false };
    90     RefPtr<SharedBuffer> m_data;
    91     CompletionHandler<void(RefPtr<SharedBuffer>&&)> m_fetchDataCompletionHandler;
     147#endif
    92148};
    93149

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm

    r255356 r255424  
    2727#include "MediaRecorderPrivateWriterCocoa.h"
    2828
     29#if !USE(APPLE_INTERNAL_SDK)
     30#include "MediaRecorderPrivateWriterCocoaFileBased.mm"
     31#else
     32
    2933#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
    3034
     35#include "AudioSampleBufferCompressor.h"
    3136#include "AudioStreamDescription.h"
    3237#include "Logging.h"
    3338#include "MediaStreamTrackPrivate.h"
     39#include "VideoSampleBufferCompressor.h"
    3440#include "WebAudioBufferList.h"
    3541#include <AVFoundation/AVAssetWriter.h>
    3642#include <AVFoundation/AVAssetWriterInput.h>
     43#include <AVFoundation/AVAssetWriter_Private.h>
     44#include <pal/avfoundation/MediaTimeAVFoundation.h>
    3745#include <pal/cf/CoreMediaSoftLink.h>
     46#include <wtf/BlockPtr.h>
    3847#include <wtf/CompletionHandler.h>
    3948#include <wtf/FileSystem.h>
     49#include <wtf/cf/TypeCastsCF.h>
    4050
    4151#import <pal/cocoa/AVFoundationSoftLink.h>
    4252
    43 #undef AVEncoderBitRateKey
    44 #define AVEncoderBitRateKey getAVEncoderBitRateKeyWithFallback()
    45 #undef AVFormatIDKey
    46 #define AVFormatIDKey getAVFormatIDKeyWithFallback()
    47 #undef AVNumberOfChannelsKey
    48 #define AVNumberOfChannelsKey getAVNumberOfChannelsKeyWithFallback()
    49 #undef AVSampleRateKey
    50 #define AVSampleRateKey getAVSampleRateKeyWithFallback()
     53@interface WebAVAssetWriterDelegate : NSObject <AVAssetWriterDelegate> {
     54    WeakPtr<WebCore::MediaRecorderPrivateWriter> m_writer;
     55}
     56
     57- (instancetype)initWithWriter:(WebCore::MediaRecorderPrivateWriter*)writer;
     58- (void)close;
     59- (void)finish;
     60
     61@end
     62
     63@implementation WebAVAssetWriterDelegate {
     64};
     65
     66- (instancetype)initWithWriter:(WebCore::MediaRecorderPrivateWriter*)writer
     67{
     68    ASSERT(isMainThread());
     69    self = [super init];
     70    if (self)
     71        self->m_writer = makeWeakPtr(writer);
     72
     73    return self;
     74}
     75
     76- (void)dealloc
     77{
     78    [super dealloc];
     79}
     80
     81- (void)assetWriter:(AVAssetWriter *)assetWriter didProduceFragmentedHeaderData:(NSData *)fragmentedHeaderData
     82{
     83    UNUSED_PARAM(assetWriter);
     84    if (!isMainThread()) {
     85        if (auto size = [fragmentedHeaderData length]) {
     86            callOnMainThread([protectedSelf = RetainPtr<WebAVAssetWriterDelegate>(self), buffer = WebCore::SharedBuffer::create(static_cast<const char*>([fragmentedHeaderData bytes]), size)]() mutable {
     87                if (protectedSelf->m_writer)
     88                    protectedSelf->m_writer->appendData(WTFMove(buffer));
     89            });
     90        }
     91        return;
     92    }
     93
     94    if (m_writer)
     95        m_writer->appendData(static_cast<const char*>([fragmentedHeaderData bytes]), [fragmentedHeaderData length]);
     96}
     97
     98- (void)assetWriter:(AVAssetWriter *)assetWriter didProduceFragmentedMediaData:(NSData *)fragmentedMediaData fragmentedMediaDataReport:(AVFragmentedMediaDataReport *)fragmentedMediaDataReport
     99{
     100    UNUSED_PARAM(assetWriter);
     101    UNUSED_PARAM(fragmentedMediaDataReport);
     102    if (!isMainThread()) {
     103        if (auto size = [fragmentedMediaData length]) {
     104            callOnMainThread([protectedSelf = RetainPtr<WebAVAssetWriterDelegate>(self), buffer = WebCore::SharedBuffer::create(static_cast<const char*>([fragmentedMediaData bytes]), size)]() mutable {
     105                if (protectedSelf->m_writer)
     106                    protectedSelf->m_writer->appendData(WTFMove(buffer));
     107            });
     108        }
     109        return;
     110    }
     111
     112    if (m_writer)
     113        m_writer->appendData(static_cast<const char*>([fragmentedMediaData bytes]), [fragmentedMediaData length]);
     114}
     115
     116- (void)close
     117{
     118    m_writer = nullptr;
     119}
     120
     121- (void)finish
     122{
     123    m_writer = nullptr;
     124}
     125
     126@end
    51127
    52128namespace WebCore {
    … …  
    54130using namespace PAL;
    55131
    56 static NSString *getAVFormatIDKeyWithFallback()
    57 {
    58     if (PAL::canLoad_AVFoundation_AVFormatIDKey())
    59         return PAL::get_AVFoundation_AVFormatIDKey();
    60 
    61     RELEASE_LOG_ERROR(Media, "Failed to load AVFormatIDKey");
    62     return @"AVFormatIDKey";
    63 }
    64 
    65 static NSString *getAVNumberOfChannelsKeyWithFallback()
    66 {
    67     if (PAL::canLoad_AVFoundation_AVNumberOfChannelsKey())
    68         return PAL::get_AVFoundation_AVNumberOfChannelsKey();
    69 
    70     RELEASE_LOG_ERROR(Media, "Failed to load AVNumberOfChannelsKey");
    71     return @"AVNumberOfChannelsKey";
    72 }
    73 
    74 static NSString *getAVSampleRateKeyWithFallback()
    75 {
    76     if (PAL::canLoad_AVFoundation_AVSampleRateKey())
    77         return PAL::get_AVFoundation_AVSampleRateKey();
    78 
    79     RELEASE_LOG_ERROR(Media, "Failed to load AVSampleRateKey");
    80     return @"AVSampleRateKey";
    81 }
    82 
    83 static NSString *getAVEncoderBitRateKeyWithFallback()
    84 {
    85     if (PAL::canLoad_AVFoundation_AVEncoderBitRateKey())
    86         return PAL::get_AVFoundation_AVEncoderBitRateKey();
    87 
    88     RELEASE_LOG_ERROR(Media, "Failed to load AVEncoderBitRateKey");
    89     return @"AVEncoderBitRateKey";
    90 }
    91 
    92 RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack)
    93 {
    94     int width = 0, height = 0;
    95     if (videoTrack) {
    96         auto& settings = videoTrack->settings();
    97         width = settings.width();
    98         height = settings.height();
    99     }
    100     return create(!!audioTrack, width, height);
    101 }
    102 
    103132RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(bool hasAudio, int width, int height)
    104133{
    105     NSString *directory = FileSystem::createTemporaryDirectory(@"videos");
    106     NSString *filename = [NSString stringWithFormat:@"/%lld.mp4", CMClockGetTime(CMClockGetHostTimeClock()).value];
    107     NSString *path = [directory stringByAppendingString:filename];
    108 
    109     NSURL *outputURL = [NSURL fileURLWithPath:path];
    110     String filePath = [path UTF8String];
     134    auto writer = adoptRef(*new MediaRecorderPrivateWriter(hasAudio, width && height));
     135    if (!writer->initialize())
     136        return nullptr;
     137    return writer;
     138}
     139
     140void MediaRecorderPrivateWriter::compressedVideoOutputBufferCallback(void *mediaRecorderPrivateWriter, CMBufferQueueTriggerToken)
     141{
     142    auto *writer = static_cast<MediaRecorderPrivateWriter*>(mediaRecorderPrivateWriter);
     143    writer->processNewCompressedVideoSampleBuffers();
     144}
     145
     146void MediaRecorderPrivateWriter::compressedAudioOutputBufferCallback(void *mediaRecorderPrivateWriter, CMBufferQueueTriggerToken)
     147{
     148    auto *writer = static_cast<MediaRecorderPrivateWriter*>(mediaRecorderPrivateWriter);
     149    writer->processNewCompressedAudioSampleBuffers();
     150}
     151
     152MediaRecorderPrivateWriter::MediaRecorderPrivateWriter(bool hasAudio, bool hasVideo)
     153    : m_hasAudio(hasAudio)
     154    , m_hasVideo(hasVideo)
     155{
     156}
     157
     158MediaRecorderPrivateWriter::~MediaRecorderPrivateWriter()
     159{
     160    clear();
     161}
     162
     163bool MediaRecorderPrivateWriter::initialize()
     164{
    111165    NSError *error = nil;
    112     auto avAssetWriter = adoptNS([PAL::allocAVAssetWriterInstance() initWithURL:outputURL fileType:AVFileTypeMPEG4 error:&error]);
     166    m_writer = adoptNS([PAL::allocAVAssetWriterInstance() initWithFileType:AVFileTypeMPEG4 error:&error]);
    113167    if (error) {
    114168        RELEASE_LOG_ERROR(MediaStream, "create AVAssetWriter instance failed with error code %ld", (long)error.code);
    115         return nullptr;
    116     }
    117 
    118     auto writer = adoptRef(*new MediaRecorderPrivateWriter(WTFMove(avAssetWriter), WTFMove(filePath)));
    119 
    120     if (hasAudio && !writer->setAudioInput())
    121         return nullptr;
    122 
    123     if (width && height) {
    124         if (!writer->setVideoInput(width, height))
    125             return nullptr;
    126     }
    127 
    128     return WTFMove(writer);
    129 }
    130 
    131 MediaRecorderPrivateWriter::MediaRecorderPrivateWriter(RetainPtr<AVAssetWriter>&& avAssetWriter, String&& filePath)
    132     : m_writer(WTFMove(avAssetWriter))
    133     , m_path(WTFMove(filePath))
    134 {
    135 }
    136 
    137 MediaRecorderPrivateWriter::~MediaRecorderPrivateWriter()
    138 {
    139     clear();
     169        return false;
     170    }
     171
     172    m_writerDelegate = adoptNS([[WebAVAssetWriterDelegate alloc] initWithWriter: this]);
     173    [m_writer.get() setDelegate:m_writerDelegate.get()];
     174
     175    if (m_hasAudio) {
     176        m_audioCompressor = AudioSampleBufferCompressor::create(compressedAudioOutputBufferCallback, this);
     177        if (!m_audioCompressor)
     178            return false;
     179    }
     180    if (m_hasVideo) {
     181        m_videoCompressor = VideoSampleBufferCompressor::create(kCMVideoCodecType_H264, compressedVideoOutputBufferCallback, this);
     182        if (!m_videoCompressor)
     183            return false;
     184    }
     185    return true;
     186}
     187
     188void MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers()
     189{
     190    ASSERT(m_hasVideo);
     191    if (!m_videoFormatDescription) {
     192        m_videoFormatDescription = CMSampleBufferGetFormatDescription(m_videoCompressor->getOutputSampleBuffer());
     193        callOnMainThread([weakThis = makeWeakPtr(this), this] {
     194            if (!weakThis)
     195                return;
     196           
     197            if (m_hasAudio && !m_audioFormatDescription)
     198                return;
     199
     200            startAssetWriter();
     201        });
     202    }
     203    if (!m_hasStartedWriting)
     204        return;
     205    appendCompressedSampleBuffers();
     206}
     207
     208void MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers()
     209{
     210    ASSERT(m_hasAudio);
     211    if (!m_audioFormatDescription) {
     212        m_audioFormatDescription = CMSampleBufferGetFormatDescription(m_audioCompressor->getOutputSampleBuffer());
     213        callOnMainThread([weakThis = makeWeakPtr(this), this] {
     214            if (!weakThis)
     215                return;
     216           
     217            if (m_hasVideo && !m_videoFormatDescription)
     218                return;
     219
     220            startAssetWriter();
     221        });
     222    }
     223    if (!m_hasStartedWriting)
     224        return;
     225    appendCompressedSampleBuffers();
     226}
     227
     228void MediaRecorderPrivateWriter::startAssetWriter()
     229{
     230    if (m_hasVideo) {
     231        m_videoAssetWriterInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeVideo outputSettings:nil sourceFormatHint:m_videoFormatDescription.get()]);
     232        [m_videoAssetWriterInput setExpectsMediaDataInRealTime:true];
     233        if (![m_writer.get() canAddInput:m_videoAssetWriterInput.get()]) {
     234            RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed canAddInput for video");
     235            return;
     236        }
     237        [m_writer.get() addInput:m_videoAssetWriterInput.get()];
     238    }
     239   
     240    if (m_hasAudio) {
     241        m_audioAssetWriterInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeAudio outputSettings:nil sourceFormatHint:m_audioFormatDescription.get()]);
     242        [m_audioAssetWriterInput setExpectsMediaDataInRealTime:true];
     243        if (![m_writer.get() canAddInput:m_audioAssetWriterInput.get()]) {
     244            RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed canAddInput for audio");
     245            return;
     246        }
     247        [m_writer.get() addInput:m_audioAssetWriterInput.get()];
     248    }
     249   
     250    if (![m_writer.get() startWriting]) {
     251        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed startWriting");
     252        return;
     253    }
     254
     255    [m_writer.get() startSessionAtSourceTime:kCMTimeZero];
     256
     257    appendCompressedSampleBuffers();
     258
     259    m_hasStartedWriting = true;
     260}
     261
     262bool MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer()
     263{
     264    if (!m_audioCompressor)
     265        return false;
     266
     267    if (![m_audioAssetWriterInput isReadyForMoreMediaData])
     268        return false;
     269
     270    auto buffer = m_audioCompressor->takeOutputSampleBuffer();
     271    if (!buffer)
     272        return false;
     273
     274    [m_audioAssetWriterInput.get() appendSampleBuffer:buffer.get()];
     275    return true;
     276}
     277
     278bool MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer()
     279{
     280    if (!m_videoCompressor)
     281        return false;
     282
     283    if (![m_videoAssetWriterInput isReadyForMoreMediaData])
     284        return false;
     285
     286    auto buffer = m_videoCompressor->takeOutputSampleBuffer();
     287    if (!buffer)
     288        return false;
     289
     290    m_lastVideoPresentationTime = CMSampleBufferGetPresentationTimeStamp(buffer.get());
     291    m_lastVideoDecodingTime = CMSampleBufferGetDecodeTimeStamp(buffer.get());
     292    m_hasEncodedVideoSamples = true;
     293
     294    [m_videoAssetWriterInput.get() appendSampleBuffer:buffer.get()];
     295    return true;
     296}
     297
     298void MediaRecorderPrivateWriter::appendCompressedSampleBuffers()
     299{
     300    while (appendCompressedVideoSampleBuffer() || appendCompressedAudioSampleBuffer()) { };
     301}
     302
     303static inline void appendEndsPreviousSampleDurationMarker(AVAssetWriterInput *assetWriterInput, CMTime presentationTimeStamp, CMTime decodingTimeStamp)
     304{
     305    CMSampleTimingInfo timingInfo = { kCMTimeInvalid, presentationTimeStamp, decodingTimeStamp};
     306
     307    CMSampleBufferRef buffer = NULL;
     308    auto error = CMSampleBufferCreate(kCFAllocatorDefault, NULL, true, NULL, NULL, NULL, 0, 1, &timingInfo, 0, NULL, &buffer);
     309    if (error) {
     310        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter appendEndsPreviousSampleDurationMarker failed CMSampleBufferCreate with %d", error);
     311        return;
     312    }
     313    auto sampleBuffer = adoptCF(buffer);
     314
     315    CMSetAttachment(sampleBuffer.get(), kCMSampleBufferAttachmentKey_EndsPreviousSampleDuration, kCFBooleanTrue, kCMAttachmentMode_ShouldPropagate);
     316    if (![assetWriterInput appendSampleBuffer:sampleBuffer.get()])
     317        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter appendSampleBuffer to writer input failed");
     318}
     319
     320void MediaRecorderPrivateWriter::appendEndOfVideoSampleDurationIfNeeded(CompletionHandler<void()>&& completionHandler)
     321{
     322    if (!m_hasEncodedVideoSamples) {
     323        completionHandler();
     324        return;
     325    }
     326    if ([m_videoAssetWriterInput isReadyForMoreMediaData]) {
     327        appendEndsPreviousSampleDurationMarker(m_videoAssetWriterInput.get(), m_lastVideoPresentationTime, m_lastVideoDecodingTime);
     328        completionHandler();
     329        return;
     330    }
     331
     332    auto block = makeBlockPtr([this, weakThis = makeWeakPtr(this), completionHandler = WTFMove(completionHandler)]() mutable {
     333        if (weakThis) {
     334            appendEndsPreviousSampleDurationMarker(m_videoAssetWriterInput.get(), m_lastVideoPresentationTime, m_lastVideoDecodingTime);
     335            [m_videoAssetWriterInput markAsFinished];
     336        }
     337        completionHandler();
     338    });
     339    [m_videoAssetWriterInput requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:block.get()];
     340}
     341
     342void MediaRecorderPrivateWriter::flushCompressedSampleBuffers(CompletionHandler<void()>&& completionHandler)
     343{
     344    appendCompressedSampleBuffers();
     345    appendEndOfVideoSampleDurationIfNeeded(WTFMove(completionHandler));
    140346}
    141347
    142348void MediaRecorderPrivateWriter::clear()
    143349{
    144     if (m_videoInput) {
    145         m_videoInput.clear();
    146         dispatch_release(m_videoPullQueue);
    147     }
    148     if (m_audioInput) {
    149         m_audioInput.clear();
    150         dispatch_release(m_audioPullQueue);
    151     }
    152350    if (m_writer)
    153351        m_writer.clear();
    … …  
    158356}
    159357
    160 bool MediaRecorderPrivateWriter::setVideoInput(int width, int height)
    161 {
    162     ASSERT(!m_videoInput);
    163    
    164     NSDictionary *compressionProperties = @{
    165         AVVideoAverageBitRateKey : [NSNumber numberWithInt:width * height * 12],
    166         AVVideoExpectedSourceFrameRateKey : @(30),
    167         AVVideoMaxKeyFrameIntervalKey : @(120),
    168         AVVideoProfileLevelKey : AVVideoProfileLevelH264MainAutoLevel
    169     };
    170 
    171     NSDictionary *videoSettings = @{
    172         AVVideoCodecKey: AVVideoCodecH264,
    173         AVVideoWidthKey: [NSNumber numberWithInt:width],
    174         AVVideoHeightKey: [NSNumber numberWithInt:height],
    175         AVVideoCompressionPropertiesKey: compressionProperties
    176     };
    177    
    178     m_videoInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeVideo outputSettings:videoSettings sourceFormatHint:nil]);
    179     [m_videoInput setExpectsMediaDataInRealTime:true];
    180    
    181     if (![m_writer canAddInput:m_videoInput.get()]) {
    182         m_videoInput = nullptr;
    183         RELEASE_LOG_ERROR(MediaStream, "the video input is not allowed to add to the AVAssetWriter");
    184         return false;
    185     }
    186     [m_writer addInput:m_videoInput.get()];
    187     m_videoPullQueue = dispatch_queue_create("WebCoreVideoRecordingPullBufferQueue", DISPATCH_QUEUE_SERIAL);
    188     return true;
    189 }
    190 
    191 bool MediaRecorderPrivateWriter::setAudioInput()
    192 {
    193     ASSERT(!m_audioInput);
    194 
    195     NSDictionary *audioSettings = @{
    196         AVEncoderBitRateKey : @(28000),
    197         AVFormatIDKey : @(kAudioFormatMPEG4AAC),
    198         AVNumberOfChannelsKey : @(1),
    199         AVSampleRateKey : @(22050)
    200     };
    201 
    202     m_audioInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeAudio outputSettings:audioSettings sourceFormatHint:nil]);
    203     [m_audioInput setExpectsMediaDataInRealTime:true];
    204    
    205     if (![m_writer canAddInput:m_audioInput.get()]) {
    206         m_audioInput = nullptr;
    207         RELEASE_LOG_ERROR(MediaStream, "the audio input is not allowed to add to the AVAssetWriter");
    208         return false;
    209     }
    210     [m_writer addInput:m_audioInput.get()];
    211     m_audioPullQueue = dispatch_queue_create("WebCoreAudioRecordingPullBufferQueue", DISPATCH_QUEUE_SERIAL);
    212     return true;
    213 }
    214358
    215359static inline RetainPtr<CMSampleBufferRef> copySampleBufferWithCurrentTimeStamp(CMSampleBufferRef originalBuffer)
    … …  
    218362    CMItemCount count = 0;
    219363    CMSampleBufferGetSampleTimingInfoArray(originalBuffer, 0, nil, &count);
    220    
     364
    221365    Vector<CMSampleTimingInfo> timeInfo(count);
    222366    CMSampleBufferGetSampleTimingInfoArray(originalBuffer, count, timeInfo.data(), &count);
    223    
    224     for (CMItemCount i = 0; i < count; i++) {
     367
     368    for (auto i = 0; i < count; i++) {
    225369        timeInfo[i].decodeTimeStamp = kCMTimeInvalid;
    226370        timeInfo[i].presentationTimeStamp = startTime;
    227371    }
     372
     373    CMSampleBufferRef newBuffer = nullptr;
     374    if (auto error = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, originalBuffer, count, timeInfo.data(), &newBuffer)) {
     375        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMSampleBufferCreateCopyWithNewTiming failed with %d", error);
     376        return nullptr;
     377    }
     378    return adoptCF(newBuffer);
     379}
    228380   
    229     CMSampleBufferRef newBuffer = nullptr;
    230     auto error = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, originalBuffer, count, timeInfo.data(), &newBuffer);
    231     if (error)
    232         return nullptr;
    233     return adoptCF(newBuffer);
    234 }
    235 
    236381void MediaRecorderPrivateWriter::appendVideoSampleBuffer(CMSampleBufferRef sampleBuffer)
    237382{
    238     ASSERT(m_videoInput);
    239     if (m_isStopped)
    240         return;
    241 
    242     if (!m_hasStartedWriting) {
    243         if (![m_writer startWriting]) {
    244             m_isStopped = true;
    245             RELEASE_LOG_ERROR(MediaStream, "create AVAssetWriter instance failed with error code %ld", (long)[m_writer error]);
    246             return;
    247         }
    248         [m_writer startSessionAtSourceTime:CMClockGetTime(CMClockGetHostTimeClock())];
    249         m_hasStartedWriting = true;
    250         RefPtr<MediaRecorderPrivateWriter> protectedThis = this;
    251         [m_videoInput requestMediaDataWhenReadyOnQueue:m_videoPullQueue usingBlock:[this, protectedThis = WTFMove(protectedThis)] {
    252             do {
    253                 if (![m_videoInput isReadyForMoreMediaData])
    254                     break;
    255                 auto locker = holdLock(m_videoLock);
    256                 if (m_videoBufferPool.isEmpty())
    257                     break;
    258                 auto buffer = m_videoBufferPool.takeFirst();
    259                 locker.unlockEarly();
    260                 if (![m_videoInput appendSampleBuffer:buffer.get()])
    261                     break;
    262             } while (true);
    263             if (m_isStopped && m_videoBufferPool.isEmpty()) {
    264                 [m_videoInput markAsFinished];
    265                 m_finishWritingVideoSemaphore.signal();
    266             }
    267         }];
    268         return;
    269     }
    270     auto bufferWithCurrentTime = copySampleBufferWithCurrentTimeStamp(sampleBuffer);
    271     if (!bufferWithCurrentTime)
    272         return;
    273 
    274     auto locker = holdLock(m_videoLock);
    275     m_videoBufferPool.append(WTFMove(bufferWithCurrentTime));
     383    // FIXME: We should not set the timestamps if they are already set.
     384    if (auto bufferWithCurrentTime = copySampleBufferWithCurrentTimeStamp(sampleBuffer))
     385        m_videoCompressor->addSampleBuffer(bufferWithCurrentTime.get());
    276386}
    277387
    … …  
    281391    CMFormatDescriptionRef format = nullptr;
    282392    auto error = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, basicDescription, 0, NULL, 0, NULL, NULL, &format);
    283     if (error)
     393    if (error) {
     394        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMAudioFormatDescriptionCreate failed with %d", error);
    284395        return nullptr;
     396    }
    285397    return adoptCF(format);
    286398}
    287399
    288 static inline RetainPtr<CMSampleBufferRef> createAudioSampleBufferWithPacketDescriptions(CMFormatDescriptionRef format, size_t sampleCount)
    289 {
    290     CMTime startTime = CMClockGetTime(CMClockGetHostTimeClock());
    291     CMSampleBufferRef sampleBuffer = nullptr;
    292     auto error = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, NULL, false, NULL, NULL, format, sampleCount, startTime, NULL, &sampleBuffer);
    293     if (error)
    294         return nullptr;
    295     return adoptCF(sampleBuffer);
    296 }
    297 
    298 void MediaRecorderPrivateWriter::appendAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime&, size_t sampleCount)
    299 {
    300     ASSERT(m_audioInput);
    301     if ((!m_hasStartedWriting && m_videoInput) || m_isStopped)
    302         return;
     400static inline RetainPtr<CMSampleBufferRef> createAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime& time, size_t sampleCount)
     401{
    303402    auto format = createAudioFormatDescription(description);
    304403    if (!format)
    305         return;
    306     if (m_isFirstAudioSample) {
    307         if (!m_videoInput) {
    308             // audio-only recording.
    309             if (![m_writer startWriting]) {
    310                 m_isStopped = true;
    311                 return;
    312             }
    313             [m_writer startSessionAtSourceTime:CMClockGetTime(CMClockGetHostTimeClock())];
    314             m_hasStartedWriting = true;
    315         }
    316         m_isFirstAudioSample = false;
    317         RefPtr<MediaRecorderPrivateWriter> protectedThis = this;
    318         [m_audioInput requestMediaDataWhenReadyOnQueue:m_audioPullQueue usingBlock:[this, protectedThis = WTFMove(protectedThis)] {
    319             do {
    320                 if (![m_audioInput isReadyForMoreMediaData])
    321                     break;
    322                 auto locker = holdLock(m_audioLock);
    323                 if (m_audioBufferPool.isEmpty())
    324                     break;
    325                 auto buffer = m_audioBufferPool.takeFirst();
    326                 locker.unlockEarly();
    327                 [m_audioInput appendSampleBuffer:buffer.get()];
    328             } while (true);
    329             if (m_isStopped && m_audioBufferPool.isEmpty()) {
    330                 [m_audioInput markAsFinished];
    331                 m_finishWritingAudioSemaphore.signal();
    332             }
     404        return nullptr;
     405
     406    CMSampleBufferRef sampleBuffer = nullptr;
     407    auto error = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, NULL, false, NULL, NULL, format.get(), sampleCount, toCMTime(time), NULL, &sampleBuffer);
     408    if (error) {
     409        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter createAudioSampleBufferWithPacketDescriptions failed with %d", error);
     410        return nullptr;
     411    }
     412    auto buffer = adoptCF(sampleBuffer);
     413
     414    error = CMSampleBufferSetDataBufferFromAudioBufferList(buffer.get(), kCFAllocatorDefault, kCFAllocatorDefault, 0, downcast<WebAudioBufferList>(data).list());
     415    if (error) {
     416        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMSampleBufferSetDataBufferFromAudioBufferList failed with %d", error);
     417        return nullptr;
     418    }
     419    return buffer;
     420}
     421
     422void MediaRecorderPrivateWriter::appendAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime& time, size_t sampleCount)
     423{
     424    if (auto sampleBuffer = createAudioSampleBuffer(data, description, time, sampleCount))
     425        m_audioCompressor->addSampleBuffer(sampleBuffer.get());
     426}
     427
     428void MediaRecorderPrivateWriter::stopRecording()
     429{
     430    if (m_isStopped)
     431        return;
     432
     433    m_isStopped = true;
     434
     435    if (m_videoCompressor)
     436        m_videoCompressor->finish();
     437    if (m_audioCompressor)
     438        m_audioCompressor->finish();
     439
     440    if (!m_hasStartedWriting)
     441        return;
     442    ASSERT([m_writer status] == AVAssetWriterStatusWriting);
     443
     444    m_isStopping = true;
     445
     446    flushCompressedSampleBuffers([this, weakThis = makeWeakPtr(this)]() mutable {
     447        if (!weakThis)
     448            return;
     449
     450        [m_writer flush];
     451        [m_writer finishWritingWithCompletionHandler:[this, weakThis = WTFMove(weakThis)]() mutable {
     452            callOnMainThread([this, weakThis = WTFMove(weakThis)]() mutable {
     453                if (!weakThis)
     454                    return;
     455
     456                m_isStopping = false;
     457                if (m_fetchDataCompletionHandler) {
     458                    auto buffer = WTFMove(m_data);
     459                    m_fetchDataCompletionHandler(WTFMove(buffer));
     460                }
     461
     462                m_isStopped = false;
     463                m_hasStartedWriting = false;
     464                clear();
     465            });
    333466        }];
    334     }
    335 
    336     auto sampleBuffer = createAudioSampleBufferWithPacketDescriptions(format.get(), sampleCount);
    337     if (!sampleBuffer)
    338         return;
    339     auto error = CMSampleBufferSetDataBufferFromAudioBufferList(sampleBuffer.get(), kCFAllocatorDefault, kCFAllocatorDefault, 0, downcast<WebAudioBufferList>(data).list());
    340     if (error)
    341         return;
    342 
    343     auto locker = holdLock(m_audioLock);
    344     m_audioBufferPool.append(WTFMove(sampleBuffer));
    345 }
    346 
    347 void MediaRecorderPrivateWriter::stopRecording()
    348 {
    349     if (m_isStopped)
    350         return;
    351 
    352     m_isStopped = true;
    353     if (!m_hasStartedWriting)
    354         return;
    355     ASSERT([m_writer status] == AVAssetWriterStatusWriting);
    356     if (m_videoInput)
    357         m_finishWritingVideoSemaphore.wait();
    358 
    359     if (m_audioInput)
    360         m_finishWritingAudioSemaphore.wait();
    361 
    362     m_isStopping = true;
    363     [m_writer finishWritingWithCompletionHandler:[this, weakPtr = makeWeakPtr(*this)]() mutable {
    364         callOnMainThread([this, weakPtr = WTFMove(weakPtr), buffer = SharedBuffer::createWithContentsOfFile(m_path)]() mutable {
    365             if (!weakPtr)
    366                 return;
    367 
    368             m_isStopping = false;
    369             if (m_fetchDataCompletionHandler)
    370                 m_fetchDataCompletionHandler(WTFMove(buffer));
    371             else
    372                 m_data = WTFMove(buffer);
    373 
    374             m_isStopped = false;
    375             m_hasStartedWriting = false;
    376             m_isFirstAudioSample = true;
    377             clear();
    378         });
    379         m_finishWritingSemaphore.signal();
    380     }];
    381     m_finishWritingSemaphore.wait();
     467    });
    382468}
    383469
    … …  
    393479}
    394480
     481void MediaRecorderPrivateWriter::appendData(const char* data, size_t size)
     482{
     483    if (!m_data) {
     484        m_data = SharedBuffer::create(data, size);
     485        return;
     486    }
     487    m_data->append(data, size);
     488}
     489
     490void MediaRecorderPrivateWriter::appendData(Ref<SharedBuffer>&& buffer)
     491{
     492    if (!m_data) {
     493        m_data = WTFMove(buffer);
     494        return;
     495    }
     496    m_data->append(WTFMove(buffer));
     497}
     498
    395499} // namespace WebCore
    396500
    397501#endif // ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
     502#endif

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoaFileBased.mm

    r255423 r255424  
    2424 */
    2525
    26 #include "config.h"
    27 #include "MediaRecorderPrivateWriterCocoa.h"
    28 
    2926#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
    3027
    … …  
    8885    RELEASE_LOG_ERROR(Media, "Failed to load AVEncoderBitRateKey");
    8986    return @"AVEncoderBitRateKey";
    90 }
    91 
    92 RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack)
    93 {
    94     int width = 0, height = 0;
    95     if (videoTrack) {
    96         auto& settings = videoTrack->settings();
    97         width = settings.width();
    98         height = settings.height();
    99     }
    100     return create(!!audioTrack, width, height);
    10187}
    10288

  • trunk/Source/WebKit/ChangeLog

    r255421 r255424  
     12020-01-30  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        * GPUProcess/webrtc/RemoteMediaRecorder.cpp:
     9        (WebKit::RemoteMediaRecorder::create):
     10        Use new constructor.
     11
    1122020-01-30  Carlos Garcia Campos  <cgarcia@igalia.com>
    213
Note: See TracChangeset for help on using the changeset viewer.