Changeset 255345 in webkit


Ignore:
Timestamp:
Jan 29, 2020 2:02:38 AM (4 years ago)
Author:
youenn@apple.com
Message:

[Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
​https://bugs.webkit.org/show_bug.cgi?id=206582

Reviewed by Eric Carlson.

Source/WebCore:

AVAssetWriterDelegate allows to grab recorded data whenever wanted.
This delegate requires passing compressed samples to AVAssetWriter.
Implement video encoding and audio encoding in dedicated classes and use these classes before adding buffers to AVAssetWriter.
Since AVAssetWriterDelegate is Apple SDK only, keep the existing file based implementation as a fallback.

Covered by existing tests.

  • platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
  • platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:

(WebCore::AudioSampleBufferCompressor::create):
(WebCore::AudioSampleBufferCompressor::AudioSampleBufferCompressor):
(WebCore::AudioSampleBufferCompressor::~AudioSampleBufferCompressor):
(WebCore::AudioSampleBufferCompressor::initialize):
(WebCore::AudioSampleBufferCompressor::finish):
(WebCore::AudioSampleBufferCompressor::audioConverterComplexInputDataProc):
(WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
(WebCore::AudioSampleBufferCompressor::computeBufferSizeForAudioFormat):
(WebCore::AudioSampleBufferCompressor::attachPrimingTrimsIfNeeded):
(WebCore::AudioSampleBufferCompressor::gradualDecoderRefreshCount):
(WebCore::AudioSampleBufferCompressor::sampleBufferWithNumPackets):
(WebCore::AudioSampleBufferCompressor::processSampleBuffersUntilLowWaterTime):
(WebCore::AudioSampleBufferCompressor::provideSourceDataNumOutputPackets):
(WebCore::AudioSampleBufferCompressor::processSampleBuffer):
(WebCore::AudioSampleBufferCompressor::addSampleBuffer):
(WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer):
(WebCore::AudioSampleBufferCompressor::takeOutputSampleBuffer):

  • platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
  • platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:

(WebCore::MediaRecorderPrivateWriter::create):
(WebCore::MediaRecorderPrivateWriter::MediaRecorderPrivateWriter):
(WebCore::MediaRecorderPrivateWriter::initialize):
(WebCore::MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers):
(WebCore::MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendVideoSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriter::stopRecording):

  • platform/mediarecorder/cocoa/VideoSampleBufferCompressor.h:
  • platform/mediarecorder/cocoa/VideoSampleBufferCompressor.mm:

(WebCore::VideoSampleBufferCompressor::create):
(WebCore::VideoSampleBufferCompressor::VideoSampleBufferCompressor):
(WebCore::VideoSampleBufferCompressor::~VideoSampleBufferCompressor):
(WebCore::VideoSampleBufferCompressor::initialize):
(WebCore::VideoSampleBufferCompressor::finish):
(WebCore::VideoSampleBufferCompressor::videoCompressionCallback):
(WebCore::VideoSampleBufferCompressor::initCompressionSession):
(WebCore::VideoSampleBufferCompressor::processSampleBuffer):
(WebCore::VideoSampleBufferCompressor::addSampleBuffer):
(WebCore::VideoSampleBufferCompressor::getOutputSampleBuffer):
(WebCore::VideoSampleBufferCompressor::takeOutputSampleBuffer):

Source/WebCore/PAL:

Add soft link macros for VideoToolbox and AudioToolbox.

  • PAL.xcodeproj/project.pbxproj:
  • pal/cf/AudioToolboxSoftLink.cpp: Added.
  • pal/cf/AudioToolboxSoftLink.h: Added.
  • pal/cf/CoreMediaSoftLink.cpp:
  • pal/cf/CoreMediaSoftLink.h:
  • pal/cf/VideoToolboxSoftLink.cpp: Added.
  • pal/cf/VideoToolboxSoftLink.h: Added.

Source/WebKit:

  • GPUProcess/webrtc/RemoteMediaRecorder.cpp:

(WebKit::RemoteMediaRecorder::create):
Use new constructor.

LayoutTests:

  • http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html:

Remove web audio generation since there seems to be some unstability in web audio -> stream -> media recorder.
which should be fixed as follow-up specific patches.

Location:
trunk
Files:
8 added
13 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r255342 r255345  
     12020-01-29  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        * http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html:
     9        Remove web audio generation since there seems to be some unstability in web audio -> stream -> media recorder.
     10        which should be fixed as follow-up specific patches.
     11
    1122020-01-28  Carlos Garcia Campos  <cgarcia@igalia.com>
    213

  • trunk/LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html

    r254256 r255345  
    5757
    5858    async_test(t => {
    59         const ac = new AudioContext();
    60         const osc = ac.createOscillator();
    61         const dest = ac.createMediaStreamDestination();
    62         const audio = dest.stream;
    63         osc.connect(dest);
    64 
    6559        const video = createVideoStream();
    66         assert_equals(video.getAudioTracks().length, 0, "video mediastream starts with no audio track");
    67         assert_equals(audio.getAudioTracks().length, 1, "audio mediastream starts with one audio track");
    68         video.addTrack(audio.getAudioTracks()[0]);
    69         assert_equals(video.getAudioTracks().length, 1, "video mediastream starts with one audio track");
    7060        const recorder = new MediaRecorder(video);
    7161        let mode = 0;

  • trunk/Source/WebCore/ChangeLog

    r255342 r255345  
     12020-01-29  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        AVAssetWriterDelegate allows to grab recorded data whenever wanted.
     9        This delegate requires passing compressed samples to AVAssetWriter.
     10        Implement video encoding and audio encoding in dedicated classes and use these classes before adding buffers to AVAssetWriter.
     11        Since AVAssetWriterDelegate is Apple SDK only, keep the existing file based implementation as a fallback.
     12
     13        Covered by existing tests.
     14
     15        * platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
     16        * platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:
     17        (WebCore::AudioSampleBufferCompressor::create):
     18        (WebCore::AudioSampleBufferCompressor::AudioSampleBufferCompressor):
     19        (WebCore::AudioSampleBufferCompressor::~AudioSampleBufferCompressor):
     20        (WebCore::AudioSampleBufferCompressor::initialize):
     21        (WebCore::AudioSampleBufferCompressor::finish):
     22        (WebCore::AudioSampleBufferCompressor::audioConverterComplexInputDataProc):
     23        (WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
     24        (WebCore::AudioSampleBufferCompressor::computeBufferSizeForAudioFormat):
     25        (WebCore::AudioSampleBufferCompressor::attachPrimingTrimsIfNeeded):
     26        (WebCore::AudioSampleBufferCompressor::gradualDecoderRefreshCount):
     27        (WebCore::AudioSampleBufferCompressor::sampleBufferWithNumPackets):
     28        (WebCore::AudioSampleBufferCompressor::processSampleBuffersUntilLowWaterTime):
     29        (WebCore::AudioSampleBufferCompressor::provideSourceDataNumOutputPackets):
     30        (WebCore::AudioSampleBufferCompressor::processSampleBuffer):
     31        (WebCore::AudioSampleBufferCompressor::addSampleBuffer):
     32        (WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer):
     33        (WebCore::AudioSampleBufferCompressor::takeOutputSampleBuffer):
     34        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
     35        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
     36        (WebCore::MediaRecorderPrivateWriter::create):
     37        (WebCore::MediaRecorderPrivateWriter::MediaRecorderPrivateWriter):
     38        (WebCore::MediaRecorderPrivateWriter::initialize):
     39        (WebCore::MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers):
     40        (WebCore::MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers):
     41        (WebCore::MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer):
     42        (WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer):
     43        (WebCore::MediaRecorderPrivateWriter::appendVideoSampleBuffer):
     44        (WebCore::MediaRecorderPrivateWriter::appendAudioSampleBuffer):
     45        (WebCore::MediaRecorderPrivateWriter::stopRecording):
     46        * platform/mediarecorder/cocoa/VideoSampleBufferCompressor.h:
     47        * platform/mediarecorder/cocoa/VideoSampleBufferCompressor.mm:
     48        (WebCore::VideoSampleBufferCompressor::create):
     49        (WebCore::VideoSampleBufferCompressor::VideoSampleBufferCompressor):
     50        (WebCore::VideoSampleBufferCompressor::~VideoSampleBufferCompressor):
     51        (WebCore::VideoSampleBufferCompressor::initialize):
     52        (WebCore::VideoSampleBufferCompressor::finish):
     53        (WebCore::VideoSampleBufferCompressor::videoCompressionCallback):
     54        (WebCore::VideoSampleBufferCompressor::initCompressionSession):
     55        (WebCore::VideoSampleBufferCompressor::processSampleBuffer):
     56        (WebCore::VideoSampleBufferCompressor::addSampleBuffer):
     57        (WebCore::VideoSampleBufferCompressor::getOutputSampleBuffer):
     58        (WebCore::VideoSampleBufferCompressor::takeOutputSampleBuffer):
     59
    1602020-01-28  Carlos Garcia Campos  <cgarcia@igalia.com>
    261

  • trunk/Source/WebCore/PAL/ChangeLog

    r255049 r255345  
     12020-01-29  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        Add soft link macros for VideoToolbox and AudioToolbox.
     9
     10        * PAL.xcodeproj/project.pbxproj:
     11        * pal/cf/AudioToolboxSoftLink.cpp: Added.
     12        * pal/cf/AudioToolboxSoftLink.h: Added.
     13        * pal/cf/CoreMediaSoftLink.cpp:
     14        * pal/cf/CoreMediaSoftLink.h:
     15        * pal/cf/VideoToolboxSoftLink.cpp: Added.
     16        * pal/cf/VideoToolboxSoftLink.h: Added.
     17
    1182020-01-22  Darin Adler  <darin@apple.com>
    219

  • trunk/Source/WebCore/PAL/PAL.xcodeproj/project.pbxproj

    r253834 r255345  
    116116                2E1342CD215AA10A007199D2 /* UIKitSoftLink.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E1342CB215AA10A007199D2 /* UIKitSoftLink.mm */; };
    117117                31308B1420A21705003FB929 /* SystemPreviewSPI.h in Headers */ = {isa = PBXBuildFile; fileRef = 31308B1320A21705003FB929 /* SystemPreviewSPI.h */; };
     118                416E995323DAE6BE00E871CB /* AudioToolboxSoftLink.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */; };
     119                416E995423DAE6BE00E871CB /* AudioToolboxSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = 416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */; };
     120                416E995723DAEFF800E871CB /* VideoToolboxSoftLink.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */; };
     121                416E995823DAEFF800E871CB /* VideoToolboxSoftLink.h in Headers */ = {isa = PBXBuildFile; fileRef = 416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */; };
    118122                442956CD218A72DF0080DB54 /* RevealSPI.h in Headers */ = {isa = PBXBuildFile; fileRef = 442956CC218A72DE0080DB54 /* RevealSPI.h */; };
    119123                4450FC9F21F5F602004DFA56 /* QuickLookSoftLink.mm in Sources */ = {isa = PBXBuildFile; fileRef = 4450FC9D21F5F602004DFA56 /* QuickLookSoftLink.mm */; };
    … …  
    292296                31308B1320A21705003FB929 /* SystemPreviewSPI.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SystemPreviewSPI.h; sourceTree = "<group>"; };
    293297                37119A7820CCB5FF002C6DC9 /* WebKitTargetConditionals.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; path = WebKitTargetConditionals.xcconfig; sourceTree = "<group>"; };
     298                416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AudioToolboxSoftLink.cpp; sourceTree = "<group>"; };
     299                416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioToolboxSoftLink.h; sourceTree = "<group>"; };
     300                416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = VideoToolboxSoftLink.cpp; sourceTree = "<group>"; };
     301                416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoToolboxSoftLink.h; sourceTree = "<group>"; };
    294302                442956CC218A72DE0080DB54 /* RevealSPI.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RevealSPI.h; sourceTree = "<group>"; };
    295303                4450FC9D21F5F602004DFA56 /* QuickLookSoftLink.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = QuickLookSoftLink.mm; sourceTree = "<group>"; };
    … …  
    525533                        isa = PBXGroup;
    526534                        children = (
     535                                416E995123DAE6BD00E871CB /* AudioToolboxSoftLink.cpp */,
     536                                416E995223DAE6BE00E871CB /* AudioToolboxSoftLink.h */,
    527537                                0CF99CA61F738436007EE793 /* CoreMediaSoftLink.cpp */,
    528538                                0CF99CA71F738437007EE793 /* CoreMediaSoftLink.h */,
     539                                416E995523DAEFF700E871CB /* VideoToolboxSoftLink.cpp */,
     540                                416E995623DAEFF700E871CB /* VideoToolboxSoftLink.h */,
    529541                        );
    530542                        path = cf;
    … …  
    716728                                57FD318B22B35989008D0E8B /* AppSSOSoftLink.h in Headers */,
    717729                                576CA9D622B854AB0030143C /* AppSSOSPI.h in Headers */,
     730                                416E995423DAE6BE00E871CB /* AudioToolboxSoftLink.h in Headers */,
    718731                                2D02E93C2056FAA700A13797 /* AudioToolboxSPI.h in Headers */,
    719732                                572A107822B456F500F410C8 /* AuthKitSPI.h in Headers */,
    … …  
    839852                                0C5AF9221F43A4C7002EAC02 /* UIKitSPI.h in Headers */,
    840853                                0C2DA1471F3BEB4900DBC317 /* URLFormattingSPI.h in Headers */,
     854                                416E995823DAEFF800E871CB /* VideoToolboxSoftLink.h in Headers */,
    841855                                0C2DA1591F3BEB4900DBC317 /* WebFilterEvaluatorSPI.h in Headers */,
    842856                                A10826F91F576292004772AC /* WebPanel.h in Headers */,
    … …  
    931945                        files = (
    932946                                57FD318A22B3593E008D0E8B /* AppSSOSoftLink.mm in Sources */,
     947                                416E995323DAE6BE00E871CB /* AudioToolboxSoftLink.cpp in Sources */,
    933948                                077E87B1226A460200A2AFF0 /* AVFoundationSoftLink.mm in Sources */,
    934949                                0C5FFF0F1F78D9DA009EFF1A /* ClockCM.mm in Sources */,
    … …  
    957972                                A3AB6E651F3D217F009C14B1 /* SystemSleepListenerMac.mm in Sources */,
    958973                                2E1342CD215AA10A007199D2 /* UIKitSoftLink.mm in Sources */,
     974                                416E995723DAEFF800E871CB /* VideoToolboxSoftLink.cpp in Sources */,
    959975                                A10826FA1F576292004772AC /* WebPanel.mm in Sources */,
    960976                        );

  • trunk/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp

    r240332 r255345  
    4747SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferCopyDataBytes, OSStatus, (CMBlockBufferRef theSourceBuffer, size_t offsetToData, size_t dataLength, void* destination), (theSourceBuffer, offsetToData, dataLength, destination))
    4848SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferGetDataLength, size_t, (CMBlockBufferRef theBuffer), (theBuffer))
     49SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBlockBufferReplaceDataBytes, OSStatus, (const void* sourceBytes, CMBlockBufferRef destinationBuffer, size_t offsetIntoDestination, size_t dataLength), (sourceBytes, destinationBuffer, offsetIntoDestination, dataLength))
    4950SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMFormatDescriptionGetExtensions, CFDictionaryRef, (CMFormatDescriptionRef desc), (desc))
    5051SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetTypeID, CFTypeID, (void), ())
    … …  
    137138SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueGetEndPresentationTimeStamp, CMTime, (CMBufferQueueRef queue), (queue))
    138139SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
     140SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMBufferQueueMarkEndOfData, OSStatus, (CMBufferQueueRef queue), (queue))
    139141
    140142SOFT_LINK_CONSTANT_FOR_SOURCE(PAL, CoreMedia, kCMSampleAttachmentKey_DoNotDisplay, CFStringRef)
    … …  
    164166SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferSetDataReady, OSStatus, (CMSampleBufferRef sbuf), (sbuf))
    165167SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionCreate, OSStatus, (CFAllocatorRef allocator, const AudioStreamBasicDescription* asbd, size_t layoutSize, const AudioChannelLayout* layout, size_t magicCookieSize, const void* magicCookie, CFDictionaryRef extensions, CMAudioFormatDescriptionRef* outDesc), (allocator, asbd, layoutSize, layout, magicCookieSize, magicCookie, extensions, outDesc))
     168SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionGetMagicCookie, const void*, (CMAudioFormatDescriptionRef desc, size_t* sizeOut), (desc, sizeOut))
     169SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMAudioFormatDescriptionGetRichestDecodableFormat, const AudioFormatListItem *, (CMAudioFormatDescriptionRef desc), (desc))
     170
    166171SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMClockGetHostTimeClock, CMClockRef, (void), ())
    167172SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMClockGetTime, CMTime, (CMClockRef clock), (clock))

  • trunk/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h

    r240332 r255345  
    5050SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBlockBufferGetDataLength, size_t, (CMBlockBufferRef theBuffer), (theBuffer))
    5151#define CMBlockBufferGetDataLength softLink_CoreMedia_CMBlockBufferGetDataLength
     52SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBlockBufferReplaceDataBytes, OSStatus, (const void* sourceBytes, CMBlockBufferRef destinationBuffer, size_t offsetIntoDestination, size_t dataLength), (sourceBytes, destinationBuffer, offsetIntoDestination, dataLength))
     53#define CMBlockBufferReplaceDataBytes softLink_CoreMedia_CMBlockBufferReplaceDataBytes
    5254SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMFormatDescriptionGetExtensions, CFDictionaryRef, (CMFormatDescriptionRef desc), (desc))
    5355#define CMFormatDescriptionGetExtensions softLink_CoreMedia_CMFormatDescriptionGetExtensions
    … …  
    228230SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBufferQueueInstallTriggerWithIntegerThreshold, OSStatus, (CMBufferQueueRef queue, CMBufferQueueTriggerCallback triggerCallback, void* triggerRefcon, CMBufferQueueTriggerCondition triggerCondition, CMItemCount triggerThreshold, CMBufferQueueTriggerToken* triggerTokenOut), (queue, triggerCallback, triggerRefcon, triggerCondition, triggerThreshold, triggerTokenOut))
    229231#define CMBufferQueueInstallTriggerWithIntegerThreshold softLink_CoreMedia_CMBufferQueueInstallTriggerWithIntegerThreshold
     232SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMBufferQueueMarkEndOfData, OSStatus, (CMBufferQueueRef queue), (queue))
     233#define CMBufferQueueMarkEndOfData softLink_CoreMedia_CMBufferQueueMarkEndOfData
    230234
    231235SOFT_LINK_CONSTANT_FOR_HEADER(PAL, CoreMedia, kCMSampleAttachmentKey_DoNotDisplay, CFStringRef)
    … …  
    257261SOFT_LINK_CONSTANT_FOR_HEADER(PAL, CoreMedia, kCMSampleBufferConsumerNotification_BufferConsumed, CFStringRef)
    258262#define kCMSampleBufferConsumerNotification_BufferConsumed get_CoreMedia_kCMSampleBufferConsumerNotification_BufferConsumed()
     263
     264SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetMagicCookie, const void*, (CMAudioFormatDescriptionRef desc, size_t* sizeOut), (desc, sizeOut))
     265#define CMAudioFormatDescriptionGetMagicCookie softLink_CoreMedia_CMAudioFormatDescriptionGetMagicCookie
    259266SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetStreamBasicDescription, const AudioStreamBasicDescription *, (CMAudioFormatDescriptionRef desc), (desc))
    260267#define CMAudioFormatDescriptionGetStreamBasicDescription softLink_CoreMedia_CMAudioFormatDescriptionGetStreamBasicDescription
    … …  
    265272SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetNumSamples, CMItemCount, (CMSampleBufferRef sbuf), (sbuf))
    266273#define CMSampleBufferGetNumSamples softLink_CoreMedia_CMSampleBufferGetNumSamples
     274SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMAudioFormatDescriptionGetRichestDecodableFormat, const AudioFormatListItem *, (CMAudioFormatDescriptionRef desc), (desc))
     275#define CMAudioFormatDescriptionGetRichestDecodableFormat softLink_CoreMedia_CMAudioFormatDescriptionGetRichestDecodableFormat
    267276SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCopySampleBufferForRange, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CFRange sampleRange, CMSampleBufferRef* sBufOut), (allocator, sbuf, sampleRange, sBufOut))
    268277#define CMSampleBufferCopySampleBufferForRange softLink_CoreMedia_CMSampleBufferCopySampleBufferForRange

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r255257 r255345  
    10981098                416E6FE91BBD12E5000A6043 /* ReadableStreamBuiltins.h in Headers */ = {isa = PBXBuildFile; fileRef = 9B03D8061BB3110D00B764D8 /* ReadableStreamBuiltins.h */; settings = {ATTRIBUTES = (Private, ); }; };
    10991099                416E6FE91BBD12E5000A6053 /* WritableStreamBuiltins.h in Headers */ = {isa = PBXBuildFile; fileRef = 9B03D8061BB3110D00B764E8 /* WritableStreamBuiltins.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1100                416F799023D750CF00829FC1 /* AudioSampleBufferCompressor.mm in Sources */ = {isa = PBXBuildFile; fileRef = 416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */; };
    11001101                417253AB1354BBBC00360F2A /* MediaControlElements.h in Headers */ = {isa = PBXBuildFile; fileRef = 417253A91354BBBC00360F2A /* MediaControlElements.h */; };
    11011102                417612AF1E3A994000C3D81D /* LibWebRTCMediaEndpoint.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 417612AB1E3A993B00C3D81D /* LibWebRTCMediaEndpoint.cpp */; };
    … …  
    11381139                41BF204922BA7BE80004F812 /* RealtimeVideoSource.h in Headers */ = {isa = PBXBuildFile; fileRef = 41BF204022B947160004F812 /* RealtimeVideoSource.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11391140                41C760B10EDE03D300C1655F /* ScriptState.h in Headers */ = {isa = PBXBuildFile; fileRef = 41C760B00EDE03D300C1655F /* ScriptState.h */; settings = {ATTRIBUTES = (Private, ); }; };
     1141                41CD6F8C23D6E82100B16421 /* VideoSampleBufferCompressor.mm in Sources */ = {isa = PBXBuildFile; fileRef = 41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */; };
    11401142                41D015CA0F4B5C71004A662F /* ContentType.h in Headers */ = {isa = PBXBuildFile; fileRef = 41D015C80F4B5C71004A662F /* ContentType.h */; settings = {ATTRIBUTES = (Private, ); }; };
    11411143                41D129CE1F3D0EF600D15E47 /* WorkerGlobalScopeCaches.h in Headers */ = {isa = PBXBuildFile; fileRef = 41FB278D1F34C28200795487 /* WorkerGlobalScopeCaches.h */; };
    … …  
    73627364                416E0B37209BC3C2004A95D9 /* FetchIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FetchIdentifier.h; sourceTree = "<group>"; };
    73637365                416E29A5102FA962007FC14E /* WorkerReportingProxy.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WorkerReportingProxy.h; sourceTree = "<group>"; };
     7366                416F798D23D750CA00829FC1 /* AudioSampleBufferCompressor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioSampleBufferCompressor.h; sourceTree = "<group>"; };
     7367                416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = AudioSampleBufferCompressor.mm; sourceTree = "<group>"; };
     7368                416F799323D8B69C00829FC1 /* MediaRecorderPrivateWriterCocoaFileBased.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaRecorderPrivateWriterCocoaFileBased.mm; sourceTree = "<group>"; };
    73647369                4170A2E91D8C0CC000318452 /* JSDOMWrapper.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSDOMWrapper.cpp; sourceTree = "<group>"; };
    73657370                417253A81354BBBC00360F2A /* MediaControlElements.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaControlElements.cpp; sourceTree = "<group>"; };
    … …  
    74697474                41C7E1061E6A54360027B4DE /* CanvasCaptureMediaStreamTrack.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CanvasCaptureMediaStreamTrack.h; sourceTree = "<group>"; };
    74707475                41C7E1081E6AA37C0027B4DE /* CanvasCaptureMediaStreamTrack.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = CanvasCaptureMediaStreamTrack.idl; sourceTree = "<group>"; };
     7476                41CD6F8923D6E81C00B16421 /* VideoSampleBufferCompressor.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoSampleBufferCompressor.h; sourceTree = "<group>"; };
     7477                41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = VideoSampleBufferCompressor.mm; sourceTree = "<group>"; };
    74717478                41CF8BE41D46222000707DC9 /* FetchBodyConsumer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FetchBodyConsumer.cpp; sourceTree = "<group>"; };
    74727479                41CF8BE51D46222000707DC9 /* FetchBodyConsumer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FetchBodyConsumer.h; sourceTree = "<group>"; };
    … …  
    1889518902                        isa = PBXGroup;
    1889618903                        children = (
     18904                                416F798D23D750CA00829FC1 /* AudioSampleBufferCompressor.h */,
     18905                                416F798F23D750CB00829FC1 /* AudioSampleBufferCompressor.mm */,
    1889718906                                4D73F94C218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h */,
    1889818907                                4D73F94D218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.mm */,
     18908                                416F799323D8B69C00829FC1 /* MediaRecorderPrivateWriterCocoaFileBased.mm */,
     18909                                41CD6F8923D6E81C00B16421 /* VideoSampleBufferCompressor.h */,
     18910                                41CD6F8B23D6E81D00B16421 /* VideoSampleBufferCompressor.mm */,
    1889918911                        );
    1890018912                        path = cocoa;
    … …  
    3354733559                                CD0EEE0E14743F39003EAFA2 /* AudioDestinationIOS.cpp in Sources */,
    3354833560                                CD5596911475B678001D0BD0 /* AudioFileReaderIOS.cpp in Sources */,
     33561                                416F799023D750CF00829FC1 /* AudioSampleBufferCompressor.mm in Sources */,
    3354933562                                CDA79827170A279100D45C55 /* AudioSessionIOS.mm in Sources */,
    3355033563                                CD8A7BBB197735FE00CBD643 /* AudioSourceProviderAVFObjC.mm in Sources */,
    … …  
    3427934292                                3FBC4AF3189881560046EE38 /* VideoFullscreenInterfaceAVKit.mm in Sources */,
    3428034293                                52D5A18F1C54592300DE34A3 /* VideoFullscreenLayerManagerObjC.mm in Sources */,
     34294                                41CD6F8C23D6E82100B16421 /* VideoSampleBufferCompressor.mm in Sources */,
    3428134295                                BE88E0DE1715D2A200658D98 /* VideoTrack.cpp in Sources */,
    3428234296                                BE88E0E11715D2A200658D98 /* VideoTrackList.cpp in Sources */,

  • trunk/Source/WebCore/platform/graphics/cv/ImageTransferSessionVT.mm

    r254555 r255345  
    9696    };
    9797
     98
    9899    NSDictionary* pixelBufferPoolOptions = @{
    99100        (__bridge NSString *)kCVPixelBufferPoolMinimumBufferCountKey: @(6)

  • trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp

    r254194 r255345  
    3030
    3131#include "AudioStreamDescription.h"
     32#include "MediaRecorderPrivateWriterCocoa.h"
    3233#include "MediaSample.h"
    3334#include "MediaStreamPrivate.h"
    … …  
    6970        }
    7071    }
    71     auto writer = MediaRecorderPrivateWriter::create(audioTrack, videoTrack);
     72
     73    int width = 0, height = 0;
     74    if (videoTrack) {
     75        auto& settings = videoTrack->settings();
     76        width = settings.width();
     77        height = settings.height();
     78    }
     79    auto writer = MediaRecorderPrivateWriter::create(!!audioTrack, width, height);
    7280    if (!writer)
    7381        return nullptr;

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h

    r254823 r255345  
    2626
    2727#if ENABLE(MEDIA_STREAM)
     28#include "AudioStreamDescription.h"
    2829
    2930#include "SharedBuffer.h"
    … …  
    3536#include <wtf/WeakPtr.h>
    3637#include <wtf/threads/BinarySemaphore.h>
     38#include <CoreAudio/CoreAudioTypes.h>
     39#include <CoreMedia/CMTime.h>
    3740
    3841typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
     42typedef const struct opaqueCMFormatDescription* CMFormatDescriptionRef;
     43typedef struct opaqueCMBufferQueueTriggerToken *CMBufferQueueTriggerToken;
    3944
    4045OBJC_CLASS AVAssetWriter;
    4146OBJC_CLASS AVAssetWriterInput;
     47OBJC_CLASS WebAVAssetWriterDelegate;
    4248
    4349namespace WTF {
    … …  
    4753namespace WebCore {
    4854
     55class AudioSampleBufferCompressor;
    4956class AudioStreamDescription;
    5057class MediaStreamTrackPrivate;
    5158class PlatformAudioData;
     59class VideoSampleBufferCompressor;
    5260
    53 class WEBCORE_EXPORT MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter> {
     61class WEBCORE_EXPORT MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter, WeakPtrFactoryInitialization::Eager> {
    5462public:
    55     static RefPtr<MediaRecorderPrivateWriter> create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack);
    5663    static RefPtr<MediaRecorderPrivateWriter> create(bool hasAudio, int width, int height);
    5764    ~MediaRecorderPrivateWriter();
    58    
    59     bool setupWriter();
    60     bool setVideoInput(int width, int height);
    61     bool setAudioInput();
     65
    6266    void appendVideoSampleBuffer(CMSampleBufferRef);
    6367    void appendAudioSampleBuffer(const PlatformAudioData&, const AudioStreamDescription&, const WTF::MediaTime&, size_t);
    … …  
    6569    void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&)>&&);
    6670
     71#if USE(APPLE_INTERNAL_SDK)
     72    void appendData(const char*, size_t);
     73    void appendData(Ref<SharedBuffer>&&);
     74#endif
     75
    6776private:
     77#if USE(APPLE_INTERNAL_SDK)
     78    MediaRecorderPrivateWriter(bool hasAudio, bool hasVideo);
     79    void clear();
     80
     81    bool initialize();
     82
     83    static void compressedVideoOutputBufferCallback(void*, CMBufferQueueTriggerToken);
     84    static void compressedAudioOutputBufferCallback(void*, CMBufferQueueTriggerToken);
     85
     86    void startAssetWriter();
     87    void appendCompressedSampleBuffers();
     88
     89    bool appendCompressedAudioSampleBuffer();
     90    bool appendCompressedVideoSampleBuffer();
     91
     92    void processNewCompressedAudioSampleBuffers();
     93    void processNewCompressedVideoSampleBuffers();
     94
     95    void flushCompressedSampleBuffers(CompletionHandler<void()>&&);
     96    void appendEndOfVideoSampleDurationIfNeeded(CompletionHandler<void()>&&);
     97#else
    6898    MediaRecorderPrivateWriter(RetainPtr<AVAssetWriter>&&, String&& path);
    6999    void clear();
    70100
     101    bool setupWriter();
     102    bool setVideoInput(int width, int height);
     103    bool setAudioInput();
     104#endif
     105
     106    bool m_hasStartedWriting { false };
     107    bool m_isStopped { false };
     108
    71109    RetainPtr<AVAssetWriter> m_writer;
     110
     111    bool m_isStopping { false };
     112    RefPtr<SharedBuffer> m_data;
     113    CompletionHandler<void(RefPtr<SharedBuffer>&&)> m_fetchDataCompletionHandler;
     114
     115#if USE(APPLE_INTERNAL_SDK)
     116    bool m_hasAudio;
     117    bool m_hasVideo;
     118
     119    RetainPtr<CMFormatDescriptionRef> m_audioFormatDescription;
     120    std::unique_ptr<AudioSampleBufferCompressor> m_audioCompressor;
     121    RetainPtr<AVAssetWriterInput> m_audioAssetWriterInput;
     122
     123    RetainPtr<CMFormatDescriptionRef> m_videoFormatDescription;
     124    std::unique_ptr<VideoSampleBufferCompressor> m_videoCompressor;
     125    RetainPtr<AVAssetWriterInput> m_videoAssetWriterInput;
     126    CMTime m_lastVideoPresentationTime;
     127    CMTime m_lastVideoDecodingTime;
     128    bool m_hasEncodedVideoSamples { false };
     129
     130    RetainPtr<WebAVAssetWriterDelegate> m_writerDelegate;
     131#else
    72132    RetainPtr<AVAssetWriterInput> m_videoInput;
    73133    RetainPtr<AVAssetWriterInput> m_audioInput;
    … …  
    79139    BinarySemaphore m_finishWritingAudioSemaphore;
    80140    BinarySemaphore m_finishWritingVideoSemaphore;
    81     bool m_hasStartedWriting { false };
    82     bool m_isStopped { false };
    83141    bool m_isFirstAudioSample { true };
    84142    dispatch_queue_t m_audioPullQueue;
    … …  
    87145    Deque<RetainPtr<CMSampleBufferRef>> m_audioBufferPool;
    88146
    89     bool m_isStopping { false };
    90     RefPtr<SharedBuffer> m_data;
    91     CompletionHandler<void(RefPtr<SharedBuffer>&&)> m_fetchDataCompletionHandler;
     147#endif
    92148};
    93149

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm

    r254823 r255345  
    2727#include "MediaRecorderPrivateWriterCocoa.h"
    2828
     29#if !USE(APPLE_INTERNAL_SDK)
     30#include "MediaRecorderPrivateWriterCocoaFileBased.mm"
     31#else
     32
    2933#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
    3034
     35#include "AudioSampleBufferCompressor.h"
    3136#include "AudioStreamDescription.h"
    3237#include "Logging.h"
    3338#include "MediaStreamTrackPrivate.h"
     39#include "VideoSampleBufferCompressor.h"
    3440#include "WebAudioBufferList.h"
    3541#include <AVFoundation/AVAssetWriter.h>
    3642#include <AVFoundation/AVAssetWriterInput.h>
     43#include <AVFoundation/AVAssetWriter_Private.h>
     44#include <pal/avfoundation/MediaTimeAVFoundation.h>
    3745#include <pal/cf/CoreMediaSoftLink.h>
     46#include <wtf/BlockPtr.h>
    3847#include <wtf/CompletionHandler.h>
    3948#include <wtf/FileSystem.h>
     49#include <wtf/cf/TypeCastsCF.h>
    4050
    4151#import <pal/cocoa/AVFoundationSoftLink.h>
    4252
    43 #undef AVEncoderBitRateKey
    44 #define AVEncoderBitRateKey getAVEncoderBitRateKeyWithFallback()
    45 #undef AVFormatIDKey
    46 #define AVFormatIDKey getAVFormatIDKeyWithFallback()
    47 #undef AVNumberOfChannelsKey
    48 #define AVNumberOfChannelsKey getAVNumberOfChannelsKeyWithFallback()
    49 #undef AVSampleRateKey
    50 #define AVSampleRateKey getAVSampleRateKeyWithFallback()
     53@interface WebAVAssetWriterDelegate : NSObject <AVAssetWriterDelegate> {
     54    WeakPtr<WebCore::MediaRecorderPrivateWriter> m_writer;
     55}
     56
     57- (instancetype)initWithWriter:(WebCore::MediaRecorderPrivateWriter*)writer;
     58- (void)close;
     59- (void)finish;
     60
     61@end
     62
     63@implementation WebAVAssetWriterDelegate {
     64};
     65
     66- (instancetype)initWithWriter:(WebCore::MediaRecorderPrivateWriter*)writer
     67{
     68    ASSERT(isMainThread());
     69    self = [super init];
     70    if (self)
     71        self->m_writer = makeWeakPtr(writer);
     72
     73    return self;
     74}
     75
     76- (void)dealloc
     77{
     78    [super dealloc];
     79}
     80
     81- (void)assetWriter:(AVAssetWriter *)assetWriter didProduceFragmentedHeaderData:(NSData *)fragmentedHeaderData
     82{
     83    UNUSED_PARAM(assetWriter);
     84    if (!isMainThread()) {
     85        if (auto size = [fragmentedHeaderData length]) {
     86            callOnMainThread([protectedSelf = RetainPtr<WebAVAssetWriterDelegate>(self), buffer = WebCore::SharedBuffer::create(static_cast<const char*>([fragmentedHeaderData bytes]), size)]() mutable {
     87                if (protectedSelf->m_writer)
     88                    protectedSelf->m_writer->appendData(WTFMove(buffer));
     89            });
     90        }
     91        return;
     92    }
     93
     94    if (m_writer)
     95        m_writer->appendData(static_cast<const char*>([fragmentedHeaderData bytes]), [fragmentedHeaderData length]);
     96}
     97
     98- (void)assetWriter:(AVAssetWriter *)assetWriter didProduceFragmentedMediaData:(NSData *)fragmentedMediaData fragmentedMediaDataReport:(AVFragmentedMediaDataReport *)fragmentedMediaDataReport
     99{
     100    UNUSED_PARAM(assetWriter);
     101    UNUSED_PARAM(fragmentedMediaDataReport);
     102    if (!isMainThread()) {
     103        if (auto size = [fragmentedMediaData length]) {
     104            callOnMainThread([protectedSelf = RetainPtr<WebAVAssetWriterDelegate>(self), buffer = WebCore::SharedBuffer::create(static_cast<const char*>([fragmentedMediaData bytes]), size)]() mutable {
     105                if (protectedSelf->m_writer)
     106                    protectedSelf->m_writer->appendData(WTFMove(buffer));
     107            });
     108        }
     109        return;
     110    }
     111
     112    if (m_writer)
     113        m_writer->appendData(static_cast<const char*>([fragmentedMediaData bytes]), [fragmentedMediaData length]);
     114}
     115
     116- (void)close
     117{
     118    m_writer = nullptr;
     119}
     120
     121- (void)finish
     122{
     123    m_writer = nullptr;
     124}
     125
     126@end
    51127
    52128namespace WebCore {
    … …  
    54130using namespace PAL;
    55131
    56 static NSString *getAVFormatIDKeyWithFallback()
    57 {
    58     if (PAL::canLoad_AVFoundation_AVFormatIDKey())
    59         return PAL::get_AVFoundation_AVFormatIDKey();
    60 
    61     RELEASE_LOG_ERROR(Media, "Failed to load AVFormatIDKey");
    62     return @"AVFormatIDKey";
    63 }
    64 
    65 static NSString *getAVNumberOfChannelsKeyWithFallback()
    66 {
    67     if (PAL::canLoad_AVFoundation_AVNumberOfChannelsKey())
    68         return PAL::get_AVFoundation_AVNumberOfChannelsKey();
    69 
    70     RELEASE_LOG_ERROR(Media, "Failed to load AVNumberOfChannelsKey");
    71     return @"AVNumberOfChannelsKey";
    72 }
    73 
    74 static NSString *getAVSampleRateKeyWithFallback()
    75 {
    76     if (PAL::canLoad_AVFoundation_AVSampleRateKey())
    77         return PAL::get_AVFoundation_AVSampleRateKey();
    78 
    79     RELEASE_LOG_ERROR(Media, "Failed to load AVSampleRateKey");
    80     return @"AVSampleRateKey";
    81 }
    82 
    83 static NSString *getAVEncoderBitRateKeyWithFallback()
    84 {
    85     if (PAL::canLoad_AVFoundation_AVEncoderBitRateKey())
    86         return PAL::get_AVFoundation_AVEncoderBitRateKey();
    87 
    88     RELEASE_LOG_ERROR(Media, "Failed to load AVEncoderBitRateKey");
    89     return @"AVEncoderBitRateKey";
    90 }
    91 
    92 RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack)
    93 {
    94     int width = 0, height = 0;
    95     if (videoTrack) {
    96         auto& settings = videoTrack->settings();
    97         width = settings.width();
    98         height = settings.height();
    99     }
    100     return create(!!audioTrack, width, height);
    101 }
    102 
    103132RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(bool hasAudio, int width, int height)
    104133{
    105     NSString *directory = FileSystem::createTemporaryDirectory(@"videos");
    106     NSString *filename = [NSString stringWithFormat:@"/%lld.mp4", CMClockGetTime(CMClockGetHostTimeClock()).value];
    107     NSString *path = [directory stringByAppendingString:filename];
    108 
    109     NSURL *outputURL = [NSURL fileURLWithPath:path];
    110     String filePath = [path UTF8String];
     134    auto writer = adoptRef(*new MediaRecorderPrivateWriter(hasAudio, width && height));
     135    if (!writer->initialize())
     136        return nullptr;
     137    return writer;
     138}
     139
     140void MediaRecorderPrivateWriter::compressedVideoOutputBufferCallback(void *mediaRecorderPrivateWriter, CMBufferQueueTriggerToken)
     141{
     142    auto *writer = static_cast<MediaRecorderPrivateWriter*>(mediaRecorderPrivateWriter);
     143    writer->processNewCompressedVideoSampleBuffers();
     144}
     145
     146void MediaRecorderPrivateWriter::compressedAudioOutputBufferCallback(void *mediaRecorderPrivateWriter, CMBufferQueueTriggerToken)
     147{
     148    auto *writer = static_cast<MediaRecorderPrivateWriter*>(mediaRecorderPrivateWriter);
     149    writer->processNewCompressedAudioSampleBuffers();
     150}
     151
     152MediaRecorderPrivateWriter::MediaRecorderPrivateWriter(bool hasAudio, bool hasVideo)
     153    : m_hasAudio(hasAudio)
     154    , m_hasVideo(hasVideo)
     155{
     156}
     157
     158MediaRecorderPrivateWriter::~MediaRecorderPrivateWriter()
     159{
     160    clear();
     161}
     162
     163bool MediaRecorderPrivateWriter::initialize()
     164{
    111165    NSError *error = nil;
    112     auto avAssetWriter = adoptNS([PAL::allocAVAssetWriterInstance() initWithURL:outputURL fileType:AVFileTypeMPEG4 error:&error]);
     166    m_writer = adoptNS([PAL::allocAVAssetWriterInstance() initWithFileType:AVFileTypeMPEG4 error:&error]);
    113167    if (error) {
    114168        RELEASE_LOG_ERROR(MediaStream, "create AVAssetWriter instance failed with error code %ld", (long)error.code);
    115         return nullptr;
    116     }
    117 
    118     auto writer = adoptRef(*new MediaRecorderPrivateWriter(WTFMove(avAssetWriter), WTFMove(filePath)));
    119 
    120     if (hasAudio && !writer->setAudioInput())
    121         return nullptr;
    122 
    123     if (width && height) {
    124         if (!writer->setVideoInput(width, height))
    125             return nullptr;
    126     }
    127 
    128     return WTFMove(writer);
    129 }
    130 
    131 MediaRecorderPrivateWriter::MediaRecorderPrivateWriter(RetainPtr<AVAssetWriter>&& avAssetWriter, String&& filePath)
    132     : m_writer(WTFMove(avAssetWriter))
    133     , m_path(WTFMove(filePath))
    134 {
    135 }
    136 
    137 MediaRecorderPrivateWriter::~MediaRecorderPrivateWriter()
    138 {
    139     clear();
     169        return false;
     170    }
     171
     172    m_writerDelegate = adoptNS([[WebAVAssetWriterDelegate alloc] initWithWriter: this]);
     173    [m_writer.get() setDelegate:m_writerDelegate.get()];
     174
     175    if (m_hasAudio) {
     176        m_audioCompressor = AudioSampleBufferCompressor::create(compressedAudioOutputBufferCallback, this);
     177        if (!m_audioCompressor)
     178            return false;
     179    }
     180    if (m_hasVideo) {
     181        m_videoCompressor = VideoSampleBufferCompressor::create(kCMVideoCodecType_H264, compressedVideoOutputBufferCallback, this);
     182        if (!m_videoCompressor)
     183            return false;
     184    }
     185    return true;
     186}
     187
     188void MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers()
     189{
     190    ASSERT(m_hasVideo);
     191    if (!m_videoFormatDescription) {
     192        m_videoFormatDescription = CMSampleBufferGetFormatDescription(m_videoCompressor->getOutputSampleBuffer());
     193        callOnMainThread([weakThis = makeWeakPtr(this), this] {
     194            if (!weakThis)
     195                return;
     196           
     197            if (m_hasAudio && !m_audioFormatDescription)
     198                return;
     199
     200            startAssetWriter();
     201        });
     202    }
     203    if (!m_hasStartedWriting)
     204        return;
     205    appendCompressedSampleBuffers();
     206}
     207
     208void MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers()
     209{
     210    ASSERT(m_hasAudio);
     211    if (!m_audioFormatDescription) {
     212        m_audioFormatDescription = CMSampleBufferGetFormatDescription(m_audioCompressor->getOutputSampleBuffer());
     213        callOnMainThread([weakThis = makeWeakPtr(this), this] {
     214            if (!weakThis)
     215                return;
     216           
     217            if (m_hasVideo && !m_videoFormatDescription)
     218                return;
     219
     220            startAssetWriter();
     221        });
     222    }
     223    if (!m_hasStartedWriting)
     224        return;
     225    appendCompressedSampleBuffers();
     226}
     227
     228void MediaRecorderPrivateWriter::startAssetWriter()
     229{
     230    if (m_hasVideo) {
     231        m_videoAssetWriterInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeVideo outputSettings:nil sourceFormatHint:m_videoFormatDescription.get()]);
     232        [m_videoAssetWriterInput setExpectsMediaDataInRealTime:true];
     233        if (![m_writer.get() canAddInput:m_videoAssetWriterInput.get()]) {
     234            RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed canAddInput for video");
     235            return;
     236        }
     237        [m_writer.get() addInput:m_videoAssetWriterInput.get()];
     238    }
     239   
     240    if (m_hasAudio) {
     241        m_audioAssetWriterInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeAudio outputSettings:nil sourceFormatHint:m_audioFormatDescription.get()]);
     242        [m_audioAssetWriterInput setExpectsMediaDataInRealTime:true];
     243        if (![m_writer.get() canAddInput:m_audioAssetWriterInput.get()]) {
     244            RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed canAddInput for audio");
     245            return;
     246        }
     247        [m_writer.get() addInput:m_audioAssetWriterInput.get()];
     248    }
     249   
     250    if (![m_writer.get() startWriting]) {
     251        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter::startAssetWriter failed startWriting");
     252        return;
     253    }
     254
     255    [m_writer.get() startSessionAtSourceTime:kCMTimeZero];
     256
     257    appendCompressedSampleBuffers();
     258
     259    m_hasStartedWriting = true;
     260}
     261
     262bool MediaRecorderPrivateWriter::appendCompressedAudioSampleBuffer()
     263{
     264    if (!m_audioCompressor)
     265        return false;
     266
     267    if (![m_audioAssetWriterInput isReadyForMoreMediaData])
     268        return false;
     269
     270    auto buffer = m_audioCompressor->takeOutputSampleBuffer();
     271    if (!buffer)
     272        return false;
     273
     274    [m_audioAssetWriterInput.get() appendSampleBuffer:buffer.get()];
     275    return true;
     276}
     277
     278bool MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer()
     279{
     280    if (!m_videoCompressor)
     281        return false;
     282
     283    if (![m_videoAssetWriterInput isReadyForMoreMediaData])
     284        return false;
     285
     286    auto buffer = m_videoCompressor->takeOutputSampleBuffer();
     287    if (!buffer)
     288        return false;
     289
     290    m_lastVideoPresentationTime = CMSampleBufferGetPresentationTimeStamp(buffer.get());
     291    m_lastVideoDecodingTime = CMSampleBufferGetDecodeTimeStamp(buffer.get());
     292    m_hasEncodedVideoSamples = true;
     293
     294    [m_videoAssetWriterInput.get() appendSampleBuffer:buffer.get()];
     295    return true;
     296}
     297
     298void MediaRecorderPrivateWriter::appendCompressedSampleBuffers()
     299{
     300    while (appendCompressedVideoSampleBuffer() || appendCompressedAudioSampleBuffer()) { };
     301}
     302
     303static inline void appendEndsPreviousSampleDurationMarker(AVAssetWriterInput *assetWriterInput, CMTime presentationTimeStamp, CMTime decodingTimeStamp)
     304{
     305    CMSampleTimingInfo timingInfo = { kCMTimeInvalid, presentationTimeStamp, decodingTimeStamp};
     306
     307    CMSampleBufferRef buffer = NULL;
     308    auto error = CMSampleBufferCreate(kCFAllocatorDefault, NULL, true, NULL, NULL, NULL, 0, 1, &timingInfo, 0, NULL, &buffer);
     309    if (error) {
     310        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter appendEndsPreviousSampleDurationMarker failed CMSampleBufferCreate with %d", error);
     311        return;
     312    }
     313    auto sampleBuffer = adoptCF(buffer);
     314
     315    CMSetAttachment(sampleBuffer.get(), kCMSampleBufferAttachmentKey_EndsPreviousSampleDuration, kCFBooleanTrue, kCMAttachmentMode_ShouldPropagate);
     316    if (![assetWriterInput appendSampleBuffer:sampleBuffer.get()])
     317        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter appendSampleBuffer to writer input failed");
     318}
     319
     320void MediaRecorderPrivateWriter::appendEndOfVideoSampleDurationIfNeeded(CompletionHandler<void()>&& completionHandler)
     321{
     322    if (!m_hasEncodedVideoSamples) {
     323        completionHandler();
     324        return;
     325    }
     326    if ([m_videoAssetWriterInput isReadyForMoreMediaData]) {
     327        appendEndsPreviousSampleDurationMarker(m_videoAssetWriterInput.get(), m_lastVideoPresentationTime, m_lastVideoDecodingTime);
     328        completionHandler();
     329        return;
     330    }
     331
     332    callOnMainThread([this, weakThis = makeWeakPtr(this), completionHandler = WTFMove(completionHandler)]() mutable {
     333        if (!weakThis) {
     334            completionHandler();
     335            return;
     336        }
     337        appendEndOfVideoSampleDurationIfNeeded(WTFMove(completionHandler));
     338    });
     339/*
     340    auto block = makeBlockPtr([this, weakThis = makeWeakPtr(this), completionHandler = WTFMove(completionHandler)]() mutable {
     341        if (weakThis) {
     342            appendEndsPreviousSampleDurationMarker(m_videoAssetWriterInput.get(), m_lastVideoPresentationTime, m_lastVideoDecodingTime);
     343            [m_videoAssetWriterInput markAsFinished];
     344        }
     345        completionHandler();
     346    });
     347    [m_videoAssetWriterInput requestMediaDataWhenReadyOnQueue:dispatch_get_main_queue() usingBlock:block.get()];
     348*/
     349}
     350
     351void MediaRecorderPrivateWriter::flushCompressedSampleBuffers(CompletionHandler<void()>&& completionHandler)
     352{
     353    appendCompressedSampleBuffers();
     354    appendEndOfVideoSampleDurationIfNeeded(WTFMove(completionHandler));
    140355}
    141356
    142357void MediaRecorderPrivateWriter::clear()
    143358{
    144     if (m_videoInput) {
    145         m_videoInput.clear();
    146         dispatch_release(m_videoPullQueue);
    147     }
    148     if (m_audioInput) {
    149         m_audioInput.clear();
    150         dispatch_release(m_audioPullQueue);
    151     }
    152359    if (m_writer)
    153360        m_writer.clear();
    … …  
    158365}
    159366
    160 bool MediaRecorderPrivateWriter::setVideoInput(int width, int height)
    161 {
    162     ASSERT(!m_videoInput);
    163    
    164     NSDictionary *compressionProperties = @{
    165         AVVideoAverageBitRateKey : [NSNumber numberWithInt:width * height * 12],
    166         AVVideoExpectedSourceFrameRateKey : @(30),
    167         AVVideoMaxKeyFrameIntervalKey : @(120),
    168         AVVideoProfileLevelKey : AVVideoProfileLevelH264MainAutoLevel
    169     };
    170 
    171     NSDictionary *videoSettings = @{
    172         AVVideoCodecKey: AVVideoCodecH264,
    173         AVVideoWidthKey: [NSNumber numberWithInt:width],
    174         AVVideoHeightKey: [NSNumber numberWithInt:height],
    175         AVVideoCompressionPropertiesKey: compressionProperties
    176     };
    177    
    178     m_videoInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeVideo outputSettings:videoSettings sourceFormatHint:nil]);
    179     [m_videoInput setExpectsMediaDataInRealTime:true];
    180    
    181     if (![m_writer canAddInput:m_videoInput.get()]) {
    182         m_videoInput = nullptr;
    183         RELEASE_LOG_ERROR(MediaStream, "the video input is not allowed to add to the AVAssetWriter");
    184         return false;
    185     }
    186     [m_writer addInput:m_videoInput.get()];
    187     m_videoPullQueue = dispatch_queue_create("WebCoreVideoRecordingPullBufferQueue", DISPATCH_QUEUE_SERIAL);
    188     return true;
    189 }
    190 
    191 bool MediaRecorderPrivateWriter::setAudioInput()
    192 {
    193     ASSERT(!m_audioInput);
    194 
    195     NSDictionary *audioSettings = @{
    196         AVEncoderBitRateKey : @(28000),
    197         AVFormatIDKey : @(kAudioFormatMPEG4AAC),
    198         AVNumberOfChannelsKey : @(1),
    199         AVSampleRateKey : @(22050)
    200     };
    201 
    202     m_audioInput = adoptNS([PAL::allocAVAssetWriterInputInstance() initWithMediaType:AVMediaTypeAudio outputSettings:audioSettings sourceFormatHint:nil]);
    203     [m_audioInput setExpectsMediaDataInRealTime:true];
    204    
    205     if (![m_writer canAddInput:m_audioInput.get()]) {
    206         m_audioInput = nullptr;
    207         RELEASE_LOG_ERROR(MediaStream, "the audio input is not allowed to add to the AVAssetWriter");
    208         return false;
    209     }
    210     [m_writer addInput:m_audioInput.get()];
    211     m_audioPullQueue = dispatch_queue_create("WebCoreAudioRecordingPullBufferQueue", DISPATCH_QUEUE_SERIAL);
    212     return true;
    213 }
    214367
    215368static inline RetainPtr<CMSampleBufferRef> copySampleBufferWithCurrentTimeStamp(CMSampleBufferRef originalBuffer)
    … …  
    218371    CMItemCount count = 0;
    219372    CMSampleBufferGetSampleTimingInfoArray(originalBuffer, 0, nil, &count);
    220    
     373
    221374    Vector<CMSampleTimingInfo> timeInfo(count);
    222375    CMSampleBufferGetSampleTimingInfoArray(originalBuffer, count, timeInfo.data(), &count);
    223    
    224     for (CMItemCount i = 0; i < count; i++) {
     376
     377    for (auto i = 0; i < count; i++) {
    225378        timeInfo[i].decodeTimeStamp = kCMTimeInvalid;
    226379        timeInfo[i].presentationTimeStamp = startTime;
    227380    }
     381
     382    CMSampleBufferRef newBuffer = nullptr;
     383    if (auto error = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, originalBuffer, count, timeInfo.data(), &newBuffer)) {
     384        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMSampleBufferCreateCopyWithNewTiming failed with %d", error);
     385        return nullptr;
     386    }
     387    return adoptCF(newBuffer);
     388}
    228389   
    229     CMSampleBufferRef newBuffer = nullptr;
    230     auto error = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, originalBuffer, count, timeInfo.data(), &newBuffer);
    231     if (error)
    232         return nullptr;
    233     return adoptCF(newBuffer);
    234 }
    235 
    236390void MediaRecorderPrivateWriter::appendVideoSampleBuffer(CMSampleBufferRef sampleBuffer)
    237391{
    238     ASSERT(m_videoInput);
    239     if (m_isStopped)
    240         return;
    241 
    242     if (!m_hasStartedWriting) {
    243         if (![m_writer startWriting]) {
    244             m_isStopped = true;
    245             RELEASE_LOG_ERROR(MediaStream, "create AVAssetWriter instance failed with error code %ld", (long)[m_writer error]);
    246             return;
    247         }
    248         [m_writer startSessionAtSourceTime:CMClockGetTime(CMClockGetHostTimeClock())];
    249         m_hasStartedWriting = true;
    250         RefPtr<MediaRecorderPrivateWriter> protectedThis = this;
    251         [m_videoInput requestMediaDataWhenReadyOnQueue:m_videoPullQueue usingBlock:[this, protectedThis = WTFMove(protectedThis)] {
    252             do {
    253                 if (![m_videoInput isReadyForMoreMediaData])
    254                     break;
    255                 auto locker = holdLock(m_videoLock);
    256                 if (m_videoBufferPool.isEmpty())
    257                     break;
    258                 auto buffer = m_videoBufferPool.takeFirst();
    259                 locker.unlockEarly();
    260                 if (![m_videoInput appendSampleBuffer:buffer.get()])
    261                     break;
    262             } while (true);
    263             if (m_isStopped && m_videoBufferPool.isEmpty()) {
    264                 [m_videoInput markAsFinished];
    265                 m_finishWritingVideoSemaphore.signal();
    266             }
    267         }];
    268         return;
    269     }
    270     auto bufferWithCurrentTime = copySampleBufferWithCurrentTimeStamp(sampleBuffer);
    271     if (!bufferWithCurrentTime)
    272         return;
    273 
    274     auto locker = holdLock(m_videoLock);
    275     m_videoBufferPool.append(WTFMove(bufferWithCurrentTime));
     392    // FIXME: We should not set the timestamps if they are already set.
     393    if (auto bufferWithCurrentTime = copySampleBufferWithCurrentTimeStamp(sampleBuffer))
     394        m_videoCompressor->addSampleBuffer(bufferWithCurrentTime.get());
    276395}
    277396
    … …  
    281400    CMFormatDescriptionRef format = nullptr;
    282401    auto error = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, basicDescription, 0, NULL, 0, NULL, NULL, &format);
    283     if (error)
     402    if (error) {
     403        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMAudioFormatDescriptionCreate failed with %d", error);
    284404        return nullptr;
     405    }
    285406    return adoptCF(format);
    286407}
    287408
    288 static inline RetainPtr<CMSampleBufferRef> createAudioSampleBufferWithPacketDescriptions(CMFormatDescriptionRef format, size_t sampleCount)
    289 {
    290     CMTime startTime = CMClockGetTime(CMClockGetHostTimeClock());
    291     CMSampleBufferRef sampleBuffer = nullptr;
    292     auto error = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, NULL, false, NULL, NULL, format, sampleCount, startTime, NULL, &sampleBuffer);
    293     if (error)
    294         return nullptr;
    295     return adoptCF(sampleBuffer);
    296 }
    297 
    298 void MediaRecorderPrivateWriter::appendAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime&, size_t sampleCount)
    299 {
    300     ASSERT(m_audioInput);
    301     if ((!m_hasStartedWriting && m_videoInput) || m_isStopped)
    302         return;
     409static inline RetainPtr<CMSampleBufferRef> createAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime& time, size_t sampleCount)
     410{
    303411    auto format = createAudioFormatDescription(description);
    304412    if (!format)
    305         return;
    306     if (m_isFirstAudioSample) {
    307         if (!m_videoInput) {
    308             // audio-only recording.
    309             if (![m_writer startWriting]) {
    310                 m_isStopped = true;
    311                 return;
    312             }
    313             [m_writer startSessionAtSourceTime:CMClockGetTime(CMClockGetHostTimeClock())];
    314             m_hasStartedWriting = true;
    315         }
    316         m_isFirstAudioSample = false;
    317         RefPtr<MediaRecorderPrivateWriter> protectedThis = this;
    318         [m_audioInput requestMediaDataWhenReadyOnQueue:m_audioPullQueue usingBlock:[this, protectedThis = WTFMove(protectedThis)] {
    319             do {
    320                 if (![m_audioInput isReadyForMoreMediaData])
    321                     break;
    322                 auto locker = holdLock(m_audioLock);
    323                 if (m_audioBufferPool.isEmpty())
    324                     break;
    325                 auto buffer = m_audioBufferPool.takeFirst();
    326                 locker.unlockEarly();
    327                 [m_audioInput appendSampleBuffer:buffer.get()];
    328             } while (true);
    329             if (m_isStopped && m_audioBufferPool.isEmpty()) {
    330                 [m_audioInput markAsFinished];
    331                 m_finishWritingAudioSemaphore.signal();
    332             }
     413        return nullptr;
     414
     415    CMSampleBufferRef sampleBuffer = nullptr;
     416    auto error = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, NULL, false, NULL, NULL, format.get(), sampleCount, toCMTime(time), NULL, &sampleBuffer);
     417    if (error) {
     418        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter createAudioSampleBufferWithPacketDescriptions failed with %d", error);
     419        return nullptr;
     420    }
     421    auto buffer = adoptCF(sampleBuffer);
     422
     423    error = CMSampleBufferSetDataBufferFromAudioBufferList(buffer.get(), kCFAllocatorDefault, kCFAllocatorDefault, 0, downcast<WebAudioBufferList>(data).list());
     424    if (error) {
     425        RELEASE_LOG_ERROR(MediaStream, "MediaRecorderPrivateWriter CMSampleBufferSetDataBufferFromAudioBufferList failed with %d", error);
     426        return nullptr;
     427    }
     428    return buffer;
     429}
     430
     431void MediaRecorderPrivateWriter::appendAudioSampleBuffer(const PlatformAudioData& data, const AudioStreamDescription& description, const WTF::MediaTime& time, size_t sampleCount)
     432{
     433    if (auto sampleBuffer = createAudioSampleBuffer(data, description, time, sampleCount))
     434        m_audioCompressor->addSampleBuffer(sampleBuffer.get());
     435}
     436
     437void MediaRecorderPrivateWriter::stopRecording()
     438{
     439    if (m_isStopped)
     440        return;
     441
     442    m_isStopped = true;
     443
     444    if (m_videoCompressor)
     445        m_videoCompressor->finish();
     446    if (m_audioCompressor)
     447        m_audioCompressor->finish();
     448
     449    if (!m_hasStartedWriting)
     450        return;
     451    ASSERT([m_writer status] == AVAssetWriterStatusWriting);
     452
     453    m_isStopping = true;
     454
     455    flushCompressedSampleBuffers([this, weakThis = makeWeakPtr(this)]() mutable {
     456        if (!weakThis)
     457            return;
     458
     459        [m_writer flush];
     460        [m_writer finishWritingWithCompletionHandler:[this, weakThis = WTFMove(weakThis)]() mutable {
     461            callOnMainThread([this, weakThis = WTFMove(weakThis)]() mutable {
     462                if (!weakThis)
     463                    return;
     464
     465                m_isStopping = false;
     466                if (m_fetchDataCompletionHandler) {
     467                    auto buffer = WTFMove(m_data);
     468                    m_fetchDataCompletionHandler(WTFMove(buffer));
     469                }
     470
     471                m_isStopped = false;
     472                m_hasStartedWriting = false;
     473                clear();
     474            });
    333475        }];
    334     }
    335 
    336     auto sampleBuffer = createAudioSampleBufferWithPacketDescriptions(format.get(), sampleCount);
    337     if (!sampleBuffer)
    338         return;
    339     auto error = CMSampleBufferSetDataBufferFromAudioBufferList(sampleBuffer.get(), kCFAllocatorDefault, kCFAllocatorDefault, 0, downcast<WebAudioBufferList>(data).list());
    340     if (error)
    341         return;
    342 
    343     auto locker = holdLock(m_audioLock);
    344     m_audioBufferPool.append(WTFMove(sampleBuffer));
    345 }
    346 
    347 void MediaRecorderPrivateWriter::stopRecording()
    348 {
    349     if (m_isStopped)
    350         return;
    351 
    352     m_isStopped = true;
    353     if (!m_hasStartedWriting)
    354         return;
    355     ASSERT([m_writer status] == AVAssetWriterStatusWriting);
    356     if (m_videoInput)
    357         m_finishWritingVideoSemaphore.wait();
    358 
    359     if (m_audioInput)
    360         m_finishWritingAudioSemaphore.wait();
    361 
    362     m_isStopping = true;
    363     [m_writer finishWritingWithCompletionHandler:[this, weakPtr = makeWeakPtr(*this)]() mutable {
    364         callOnMainThread([this, weakPtr = WTFMove(weakPtr), buffer = SharedBuffer::createWithContentsOfFile(m_path)]() mutable {
    365             if (!weakPtr)
    366                 return;
    367 
    368             m_isStopping = false;
    369             if (m_fetchDataCompletionHandler)
    370                 m_fetchDataCompletionHandler(WTFMove(buffer));
    371             else
    372                 m_data = WTFMove(buffer);
    373 
    374             m_isStopped = false;
    375             m_hasStartedWriting = false;
    376             m_isFirstAudioSample = true;
    377             clear();
    378         });
    379         m_finishWritingSemaphore.signal();
    380     }];
    381     m_finishWritingSemaphore.wait();
     476    });
    382477}
    383478
    … …  
    393488}
    394489
     490void MediaRecorderPrivateWriter::appendData(const char* data, size_t size)
     491{
     492    if (!m_data) {
     493        m_data = SharedBuffer::create(data, size);
     494        return;
     495    }
     496    m_data->append(data, size);
     497}
     498
     499void MediaRecorderPrivateWriter::appendData(Ref<SharedBuffer>&& buffer)
     500{
     501    if (!m_data) {
     502        m_data = WTFMove(buffer);
     503        return;
     504    }
     505    m_data->append(WTFMove(buffer));
     506}
     507
    395508} // namespace WebCore
    396509
    397510#endif // ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
     511#endif

  • trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoaFileBased.mm

    r255343 r255345  
    2424 */
    2525
    26 #include "config.h"
    27 #include "MediaRecorderPrivateWriterCocoa.h"
    28 
    2926#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
    3027
    … …  
    8885    RELEASE_LOG_ERROR(Media, "Failed to load AVEncoderBitRateKey");
    8986    return @"AVEncoderBitRateKey";
    90 }
    91 
    92 RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack)
    93 {
    94     int width = 0, height = 0;
    95     if (videoTrack) {
    96         auto& settings = videoTrack->settings();
    97         width = settings.width();
    98         height = settings.height();
    99     }
    100     return create(!!audioTrack, width, height);
    10187}
    10288

  • trunk/Source/WebKit/ChangeLog

    r255343 r255345  
     12020-01-29  youenn fablet  <youenn@apple.com>
     2
     3        [Cocoa] Use AVAssetWriterDelegate to implement MediaRecorder
     4        https://bugs.webkit.org/show_bug.cgi?id=206582
     5
     6        Reviewed by Eric Carlson.
     7
     8        * GPUProcess/webrtc/RemoteMediaRecorder.cpp:
     9        (WebKit::RemoteMediaRecorder::create):
     10        Use new constructor.
     11
    1122020-01-29  Carlos Garcia Campos  <cgarcia@igalia.com>
    213
Note: See TracChangeset for help on using the changeset viewer.