Changeset 247229 in webkit


Ignore:
Timestamp:
Jul 8, 2019 1:51:11 PM (5 years ago)
Author:
cturner@igalia.com
Message:

[GTK] Some media fragment tests are flaky
​https://bugs.webkit.org/show_bug.cgi?id=132248

Reviewed by Eric Carlson.

These tests look like they're flakey due to a small stop time. I'm
surmising that based on the bot's load, we step over the 0.5 mark
often. When I run this locally, despite running the fragment tests
100 times, I saw no timeouts.

I have seen crashes in the flakiness dashboard, but I would prefer
for these to have separate bugs so that we can squash them rather
than allowing them to linger. Having them expected to crash means
we have no crash logs to investigate.

Updated the baselines in the LayoutTests/ directory to reflect the
new stop time delta.

  • media/media-fragments/media-fragments.js:

(pause): Increase stop delta, on simulator builds the stddev
crossed the 0.5s mark, and I suspect on GTK builds, our bots being
heavily loaded cause the majority of timeouts.

  • platform/gtk/TestExpectations: Remove expectations.
Location:
trunk/LayoutTests
Files:
33 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r247226 r247229  
     12019-07-08  Charlie Turner  <cturner@igalia.com>
     2
     3        [GTK] Some media fragment tests are flaky
     4        https://bugs.webkit.org/show_bug.cgi?id=132248
     5
     6        Reviewed by Eric Carlson.
     7
     8        These tests look like they're flakey due to a small stop time. I'm
     9        surmising that based on the bot's load, we step over the 0.5 mark
     10        often. When I run this locally, despite running the fragment tests
     11        100 times, I saw no timeouts.
     12
     13        I have seen crashes in the flakiness dashboard, but I would prefer
     14        for these to have separate bugs so that we can squash them rather
     15        than allowing them to linger. Having them expected to crash means
     16        we have no crash logs to investigate.
     17
     18        Updated the baselines in the LayoutTests/ directory to reflect the
     19        new stop time delta.
     20
     21        * media/media-fragments/media-fragments.js:
     22        (pause): Increase stop delta, on simulator builds the stddev
     23        crossed the 0.5s mark, and I suspect on GTK builds, our bots being
     24        heavily loaded cause the majority of timeouts.
     25        * platform/gtk/TestExpectations: Remove expectations.
     26
    1272019-07-08  Ryan Haddad  <ryanhaddad@apple.com>
    228

  • trunk/LayoutTests/media/media-fragments/TC0004-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0005-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0006-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0011-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0015-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0017-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0024-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0035-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0036-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0037-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0038-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0039-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0051-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0052-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0053-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0054-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0055-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0059-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0072-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0078-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0079-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0080-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0081-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0085-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0086-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0087-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0088-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0089-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0090-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/TC0091-expected.txt

    r158743 r247229  
    88RUN(video.play())
    99EVENT(pause)
    10 EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK
     10EXPECTED (video.currentTime - fragmentEndTime <= '0.75') OK
    1111END OF TEST
    1212

  • trunk/LayoutTests/media/media-fragments/media-fragments.js

    r158527 r247229  
    7878    function pause()
    7979    {
    80         const maximumStopDelta = 0.5;
     80        const maximumStopDelta = 0.75;
    8181        var delta = Math.abs(video.currentTime - fragmentEndTime).toFixed(2);
    8282        reportExpected((delta <= maximumStopDelta), ("video.currentTime - fragmentEndTime"), "<=", maximumStopDelta, delta);

  • trunk/LayoutTests/platform/gtk/TestExpectations

    r247086 r247229  
    15101510webkit.org/b/132185 editing/pasteboard/gtk/middle-button-paste.html [ Failure Pass ]
    15111511
    1512 # Some media fragment tests are flaky
    1513 webkit.org/b/132248 media/media-fragments/TC0001.html [ Timeout Pass ]
    1514 webkit.org/b/132248 media/media-fragments/TC0002.html [ Crash Pass ]
    1515 webkit.org/b/132248 media/media-fragments/TC0004.html [ Crash Pass ]
    1516 webkit.org/b/132248 media/media-fragments/TC0005.html [ Failure Crash Pass ]
    1517 webkit.org/b/132248 media/media-fragments/TC0006.html [ Failure Pass ]
    1518 webkit.org/b/132248 media/media-fragments/TC0009.html [ Failure Pass ]
    1519 webkit.org/b/132248 media/media-fragments/TC0011.html [ Failure Pass ]
    1520 webkit.org/b/132248 media/media-fragments/TC0014.html [ Failure Pass ]
    1521 webkit.org/b/132248 media/media-fragments/TC0015.html [ Failure Crash Pass ]
    1522 webkit.org/b/134574 media/media-fragments/TC0024.html [ Failure Timeout Pass ]
    1523 webkit.org/b/132248 media/media-fragments/TC0035.html [ Failure Pass ]
    1524 webkit.org/b/132248 media/media-fragments/TC0036.html [ Failure Pass ]
    1525 webkit.org/b/132248 media/media-fragments/TC0037.html [ Failure Pass ]
    1526 webkit.org/b/132248 media/media-fragments/TC0039.html [ Timeout Failure Pass ]
    1527 webkit.org/b/132248 media/media-fragments/TC0038.html [ Timeout Failure Crash Pass ]
    1528 webkit.org/b/132248 media/media-fragments/TC0044.html [ Crash Pass ]
    1529 webkit.org/b/132248 media/media-fragments/TC0051.html [ Timeout Failure Pass ]
    1530 webkit.org/b/132248 media/media-fragments/TC0052.html [ Failure Pass ]
    1531 webkit.org/b/132248 media/media-fragments/TC0053.html [ Timeout Failure Pass ]
    1532 webkit.org/b/132248 media/media-fragments/TC0054.html [ Failure Pass ]
    1533 webkit.org/b/132248 media/media-fragments/TC0055.html [ Failure Pass Timeout ]
    1534 webkit.org/b/132248 media/media-fragments/TC0058.html [ Timeout Pass ]
    1535 webkit.org/b/132248 media/media-fragments/TC0059.html [ Failure Pass ]
    1536 webkit.org/b/132248 media/media-fragments/TC0061.html [ Timeout Pass ]
    1537 webkit.org/b/132248 media/media-fragments/TC0063.html [ Failure Pass Timeout ]
    1538 webkit.org/b/143709 media/media-fragments/TC0064.html [ Timeout Pass ]
    1539 webkit.org/b/132248 media/media-fragments/TC0065.html [ Timeout Pass ]
    1540 webkit.org/b/132248 media/media-fragments/TC0066.html [ Timeout Pass ]
    1541 webkit.org/b/132248 media/media-fragments/TC0067.html [ Timeout Pass ]
    1542 webkit.org/b/132248 media/media-fragments/TC0070.html [ Timeout Pass ]
    1543 webkit.org/b/132248 media/media-fragments/TC0072.html [ Failure Pass ]
    1544 webkit.org/b/132248 media/media-fragments/TC0078.html [ Failure Crash Pass ]
    1545 webkit.org/b/132248 media/media-fragments/TC0079.html [ Failure Pass ]
    1546 webkit.org/b/132248 media/media-fragments/TC0081.html [ Timeout Pass ]
    1547 webkit.org/b/132248 media/media-fragments/TC0080.html [ Timeout Pass ]
    1548 webkit.org/b/132248 media/media-fragments/TC0082.html [ Timeout Crash Pass ]
    1549 webkit.org/b/132248 media/media-fragments/TC0085.html [ Failure Crash Pass ]
    1550 webkit.org/b/132248 media/media-fragments/TC0086.html [ Failure Timeout Pass ]
    1551 webkit.org/b/132248 media/media-fragments/TC0087.html [ Failure Timeout Crash Pass ]
    1552 webkit.org/b/132248 media/media-fragments/TC0088.html [ Failure Crash Pass ]
    1553 webkit.org/b/132248 media/media-fragments/TC0089.html [ Failure Pass ]
    1554 webkit.org/b/132248 media/media-fragments/TC0090.html [ Failure Pass ]
    1555 webkit.org/b/132248 media/media-fragments/TC0091.html [ Failure Pass ]
    1556 
    1557 
    15581512webkit.org/b/132249 http/tests/media/video-cookie.html [ Timeout Failure Pass ]
    15591513
Note: See TracChangeset for help on using the changeset viewer.