⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 263844 in webkit


Ignore:
Timestamp:
Jul 2, 2020, 7:05:28 AM (6 years ago)
Author:
youenn@apple.com
Message:

getUserMedia returns OverConstrained on Jitsi
​https://bugs.webkit.org/show_bug.cgi?id=210932
<rdar://problem/64403675>

Reviewed by Philippe Normand.

Source/WebCore:

We compute the max width, max height and max frame rate across all presets.
In case a preset for the max resolution is different from the preset for the max frame rate,
we were selecting capture parameters that no preset can match.

To fix the issue, in case the frame rate constraint is not mandatory, and we do not find a preset
when starting to capture, we remove the frame rate constraint, pick the preset and choose the max frame rate from that preset.

Update mock sources to have a high resolution preset with low frame rate to enable writing a test.

Test: fast/mediastream/get-user-media-ideal-constraints.html

  • platform/mediastream/RealtimeMediaSource.cpp:

(WebCore::RealtimeMediaSource::supportsSizeAndFrameRate):

  • platform/mediastream/RealtimeVideoCaptureSource.cpp:

(WebCore::RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate):
(WebCore::RealtimeVideoCaptureSource::setSizeAndFrameRate):

  • platform/mediastream/VideoPreset.h:

(WebCore::VideoPreset::maxFrameRate const):

  • platform/mock/MockRealtimeMediaSourceCenter.cpp:

(WebCore::defaultDevices):

LayoutTests:

We bumped the max width/height for mock sources and have to update existing tests according to that.

  • fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt:
  • fast/mediastream/apply-constraints-advanced-expected.txt:
  • fast/mediastream/apply-constraints-advanced.html:
  • fast/mediastream/apply-constraints-video-expected.txt:
  • fast/mediastream/apply-constraints-video.html:
  • fast/mediastream/get-user-media-ideal-constraints-expected.txt: Added.
  • fast/mediastream/get-user-media-ideal-constraints.html: Added.
Location:
trunk
Files:
2 added
11 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r263842 r263844  
     12020-07-02  Youenn Fablet  <youenn@apple.com>
     2
     3        getUserMedia returns OverConstrained on Jitsi
     4        https://bugs.webkit.org/show_bug.cgi?id=210932
     5        <rdar://problem/64403675>
     6
     7        Reviewed by Philippe Normand.
     8
     9        We bumped the max width/height for mock sources and have to update existing tests according to that.
     10
     11        * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt:
     12        * fast/mediastream/apply-constraints-advanced-expected.txt:
     13        * fast/mediastream/apply-constraints-advanced.html:
     14        * fast/mediastream/apply-constraints-video-expected.txt:
     15        * fast/mediastream/apply-constraints-video.html:
     16        * fast/mediastream/get-user-media-ideal-constraints-expected.txt: Added.
     17        * fast/mediastream/get-user-media-ideal-constraints.html: Added.
     18
    1192020-07-02  Karl Rackler  <rackler@apple.com>
    220

  • trunk/LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt

    r252681 r263844  
    99  capabilities.facingMode = [ user ]
    1010  capabilities.frameRate = { max: 30, min: 5 }
    11   capabilities.height = { max: 720, min: 1 }
    12   capabilities.width = { max: 1280, min: 1 }
     11  capabilities.height = { max: 1440, min: 1 }
     12  capabilities.width = { max: 2560, min: 1 }
    1313
    1414audio track capabilities:

  • trunk/LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt

    r236015 r263844  
    1313PASS settings['height'] is 480
    1414
    15 ** Constraint: {"width":{"min":320},"height":{"min":240},"advanced":[{"width":1920,"height":1280}]} - advanced width and height are not supported, minimums are less than current, nothing is changed.
     15** Constraint: {"width":{"min":320},"height":{"min":240},"advanced":[{"width":3000,"height":2000}]} - advanced width and height are not supported, minimums are less than current, nothing is changed.
    1616PASS settings['width'] is 640
    1717PASS settings['height'] is 480

  • trunk/LayoutTests/fast/mediastream/apply-constraints-advanced.html

    r236015 r263844  
    1818                                    height: { min: 240 },
    1919                                    advanced: [
    20                                         { width: 1920, height: 1280 },
     20                                        { width: 3000, height: 2000 },
    2121                                    ]
    2222                                },

  • trunk/LayoutTests/fast/mediastream/apply-constraints-video-expected.txt

    r246049 r263844  
    1919PASS settings['frameRate'] is 30
    2020
    21 ** Constraint: {"width":{"exact":2000}} - the 'exact' constraint can't be satisfied, promise should reject and no settings should change.
     21** Constraint: {"width":{"exact":3000}} - the 'exact' constraint can't be satisfied, promise should reject and no settings should change.
    2222PASS Promise was rejected
    2323PASS error.constraint is "width"
    … …  
    4343
    4444** Constraint: {"width":{"min":300,"ideal":5000}} - the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen.
    45 PASS settings['width'] is 1280
    46 PASS settings['height'] is 720
     45PASS settings['width'] is 2560
     46PASS settings['height'] is 1440
    4747
    4848** Constraint: {"width":{"min":320,"ideal":640},"height":{"min":480,"ideal":720}} - 'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen.
    … …  
    5151
    5252** Constraint: {"width":5000} - ideal width is greater than track capability, should be clamped to the maximum value.
    53 PASS settings['width'] is 1280
     53PASS settings['width'] is 2560
    5454
    5555** Constraint: {"width":100,"height":100,"frameRate":4} - frameRate value is less than track capabilities, should be clamped to the minimum values.

  • trunk/LayoutTests/fast/mediastream/apply-constraints-video.html

    r246049 r263844  
    1919                {
    2020                    message: "the 'exact' constraint can't be satisfied, promise should reject and no settings should change.",
    21                     constraint: { width: { exact: 2000 } },
     21                    constraint: { width: { exact: 3000 } },
    2222                    expected: { width: 320, height: 240, frameRate: 30 },
    2323                    error: "width",
    … …  
    4343                    message: "the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen.",
    4444                    constraint: { width: {min: 300, ideal: 5000} },
    45                     expected: { width: 1280, height: 720 },
     45                    expected: { width: 2560, height: 1440 },
    4646                },
    4747                {
    … …  
    5353                    message: "ideal width is greater than track capability, should be clamped to the maximum value.",
    5454                    constraint: { width: 5000 },
    55                     expected: { width: 1280},
     55                    expected: { width: 2560},
    5656                },
    5757                {

  • trunk/Source/WebCore/ChangeLog

    r263843 r263844  
     12020-07-02  Youenn Fablet  <youenn@apple.com>
     2
     3        getUserMedia returns OverConstrained on Jitsi
     4        https://bugs.webkit.org/show_bug.cgi?id=210932
     5        <rdar://problem/64403675>
     6
     7        Reviewed by Philippe Normand.
     8
     9        We compute the max width, max height and max frame rate across all presets.
     10        In case a preset for the max resolution is different from the preset for the max frame rate,
     11        we were selecting capture parameters that no preset can match.
     12
     13        To fix the issue, in case the frame rate constraint is not mandatory, and we do not find a preset
     14        when starting to capture, we remove the frame rate constraint, pick the preset and choose the max frame rate from that preset.
     15
     16        Update mock sources to have a high resolution preset with low frame rate to enable writing a test.
     17
     18        Test: fast/mediastream/get-user-media-ideal-constraints.html
     19
     20        * platform/mediastream/RealtimeMediaSource.cpp:
     21        (WebCore::RealtimeMediaSource::supportsSizeAndFrameRate):
     22        * platform/mediastream/RealtimeVideoCaptureSource.cpp:
     23        (WebCore::RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate):
     24        (WebCore::RealtimeVideoCaptureSource::setSizeAndFrameRate):
     25        * platform/mediastream/VideoPreset.h:
     26        (WebCore::VideoPreset::maxFrameRate const):
     27        * platform/mock/MockRealtimeMediaSourceCenter.cpp:
     28        (WebCore::defaultDevices):
     29
    1302020-07-02  Carlos Garcia Campos  <cgarcia@igalia.com>
    231

  • trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp

    r263623 r263844  
    361361    // Each of the non-null values is supported individually, see if they all can be applied at the same time.
    362362    if (!supportsSizeAndFrameRate(WTFMove(width), WTFMove(height), WTFMove(frameRate))) {
     363        // Let's try without frame rate constraint if not mandatory.
     364        if (frameRateConstraint && !frameRateConstraint->isMandatory() && supportsSizeAndFrameRate(WTFMove(width), WTFMove(height), { }))
     365            return true;
     366
    363367        if (widthConstraint)
    364368            badConstraint = widthConstraint->name();

  • trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp

    r263623 r263844  
    235235}
    236236
     237static inline double frameRateFromPreset(const VideoPreset& preset, double currentFrameRate)
     238{
     239    auto minFrameRate = preset.minFrameRate();
     240    auto maxFrameRate = preset.maxFrameRate();
     241    return currentFrameRate >= minFrameRate && currentFrameRate <= maxFrameRate ? currentFrameRate : maxFrameRate;
     242}
     243
    237244Optional<RealtimeVideoCaptureSource::CaptureSizeAndFrameRate> RealtimeVideoCaptureSource::bestSupportedSizeAndFrameRate(Optional<int> requestedWidth, Optional<int> requestedHeight, Optional<double> requestedFrameRate)
    238245{
    … …  
    244251        requestedHeight = size().height();
    245252    }
    246     if (!requestedFrameRate)
    247         requestedFrameRate = frameRate();
    248 
    249     CaptureSizeAndFrameRate result;
     253
    250254    RefPtr<VideoPreset> exactSizePreset;
    251255    RefPtr<VideoPreset> aspectRatioPreset;
    … …  
    257261        const auto& presetSize = preset->size;
    258262
    259         if (!presetSupportsFrameRate(&preset.get(), requestedFrameRate.value()))
     263        if (requestedFrameRate && !presetSupportsFrameRate(&preset.get(), requestedFrameRate.value()))
    260264            continue;
    261265
    262266        if (!requestedWidth && !requestedHeight) {
    263             result.requestedFrameRate = requestedFrameRate.value();
    264             return result;
     267            exactSizePreset = preset.ptr();
     268            break;
    265269        }
    266270
    … …  
    338342    }
    339343
    340     result.requestedFrameRate = requestedFrameRate.value();
    341344    if (exactSizePreset) {
    342         result.encodingPreset = exactSizePreset;
    343         result.requestedSize = exactSizePreset->size;
    344         return result;
     345        auto size = exactSizePreset->size;
     346        auto captureFrameRate = requestedFrameRate ? *requestedFrameRate : frameRateFromPreset(*exactSizePreset, frameRate());
     347        return CaptureSizeAndFrameRate { WTFMove(exactSizePreset), size, captureFrameRate };
    345348    }
    346349
    347350    if (aspectRatioPreset) {
    348         result.encodingPreset = aspectRatioPreset;
    349         result.requestedSize = aspectRatioMatchSize;
    350         return result;
    351     }
    352 
    353     result.encodingPreset = resizePreset;
    354     result.requestedSize = resizeSize;
    355     return result;
     351        auto captureFrameRate = requestedFrameRate ? *requestedFrameRate : frameRateFromPreset(*aspectRatioPreset, frameRate());
     352        return CaptureSizeAndFrameRate { WTFMove(aspectRatioPreset), aspectRatioMatchSize, captureFrameRate };
     353    }
     354
     355    auto captureFrameRate = requestedFrameRate ? *requestedFrameRate : frameRateFromPreset(*resizePreset, frameRate());
     356    return CaptureSizeAndFrameRate { WTFMove(resizePreset), resizeSize, captureFrameRate };
    356357}
    357358
    … …  
    366367    }
    367368
    368     Optional<RealtimeVideoCaptureSource::CaptureSizeAndFrameRate> match = bestSupportedSizeAndFrameRate(width, height, frameRate);
    369     ASSERT(match);
    370     if (!match)
    371         return;
     369    auto match = bestSupportedSizeAndFrameRate(width, height, frameRate);
     370    if (!match) {
     371        match = bestSupportedSizeAndFrameRate(width, height, { });
     372        ASSERT(match);
     373        if (!match)
     374            return;
     375    }
    372376
    373377    setFrameRateWithPreset(match->requestedFrameRate, match->encodingPreset);

  • trunk/Source/WebCore/platform/mediastream/VideoPreset.h

    r263623 r263844  
    115115    VideoPresetType type;
    116116
     117    double maxFrameRate() const;
     118    double minFrameRate() const;
     119
    117120    void log()const;
    118121
    … …  
    133136}
    134137
     138inline double VideoPreset::minFrameRate() const
     139{
     140    double minFrameRate = std::numeric_limits<double>::max();
     141    for (auto& range : frameRateRanges) {
     142        if (minFrameRate > range.minimum)
     143            minFrameRate = range.minimum;
     144    }
     145    return minFrameRate;
     146}
     147
     148inline double VideoPreset::maxFrameRate() const
     149{
     150    double maxFrameRate = 0;
     151    for (auto& range : frameRateRanges) {
     152        if (maxFrameRate < range.maximum)
     153            maxFrameRate = range.maximum;
     154    }
     155    return maxFrameRate;
     156}
     157
    135158} // namespace WebCore
    136159

  • trunk/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp

    r261373 r263844  
    6060                30,
    6161                RealtimeMediaSourceSettings::VideoFacingMode::User, {
     62                    { { 2560, 1440 }, { { 10, 10 }, { 7.5, 7.5 }, { 5, 5 } } },
    6263                    { { 1280, 720 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
    6364                    { { 640, 480 },  { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } },
Note: See TracChangeset for help on using the changeset viewer.