Changeset 218139 in webkit


Ignore:
Timestamp:
Jun 12, 2017, 2:35:25 PM (8 years ago)
Author:
Jonathan Bedard
Message:

Garden tests for on-device testing

Unreviewed test gardening.

The tests gardened here have not been investigated. ​https://bugs.webkit.org/show_bug.cgi?id=173275
tracks the investigation of the tests marked as failing, crashing or skipped because they timeout
when running layout tests on-device in this revision.

  • platform/ios-device/TestExpectations: Initial gardening of expectations for on-device testing.
  • platform/ios-simulator-wk2/TestExpectations: Moved simulator specific expectations from platfrom/ios-wk2.
  • platform/ios-simulator/TestExpectations: Moved simulator specific expectations from platfrom/ios.
  • platform/ios-simulator/compositing: Removed.
  • platform/ios-wk2/TestExpectations: Moved simulator specific expectation to platform/ios-simulator-wk2.
  • platform/ios/TestExpectations: Moved simulator specific expectation to platform/ios.
  • platform/ios/compositing/iframes/compositing-for-scrollable-iframe-expected.txt: Copied from

LayoutTests/platform/ios-simulator/compositing/iframes/compositing-for-scrollable-iframe-expected.txt.

Location:
trunk/LayoutTests
Files:
1 deleted
6 edited
1 copied

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r218127 r218139  
     12017-06-12  Jonathan Bedard  <jbedard@apple.com>
     2
     3        Garden tests for on-device testing
     4
     5        Unreviewed test gardening.
     6
     7        The tests gardened here have not been investigated. https://bugs.webkit.org/show_bug.cgi?id=173275
     8        tracks the investigation of the tests marked as failing, crashing or skipped because they timeout
     9        when running layout tests on-device in this revision.
     10
     11        * platform/ios-device/TestExpectations: Initial gardening of expectations for on-device testing.
     12        * platform/ios-simulator-wk2/TestExpectations: Moved simulator specific expectations from platfrom/ios-wk2.
     13        * platform/ios-simulator/TestExpectations: Moved simulator specific expectations from platfrom/ios.
     14        * platform/ios-simulator/compositing: Removed.
     15        * platform/ios-wk2/TestExpectations: Moved simulator specific expectation to platform/ios-simulator-wk2.
     16        * platform/ios/TestExpectations: Moved simulator specific expectation to platform/ios.
     17        * platform/ios/compositing/iframes/compositing-for-scrollable-iframe-expected.txt: Copied from
     18        LayoutTests/platform/ios-simulator/compositing/iframes/compositing-for-scrollable-iframe-expected.txt.
     19
    1202017-06-12  Antoine Quint  <graouts@apple.com>
    221

  • trunk/LayoutTests/platform/ios-device/TestExpectations

    r216462 r218139  
    44#
    55
    6 # https://bugs.webkit.org/show_bug.cgi?id=170772
    7 compositing/geometry/clipped-out-perspective.html [ ImageOnlyFailure ]
    8 compositing/hidpi-subpixel-transform-origin.html [ ImageOnlyFailure ]
    9 compositing/regions/position-layer-inside-region-overflow-hidden.html [ ImageOnlyFailure ]
    10 compositing/regions/position-layers-inside-region-overflow-hidden.html [ ImageOnlyFailure ]
    11 compositing/regions/position-layers-inside-regions-overflow-hidden.html [ ImageOnlyFailure ]
     6canvas/philip/tests/toDataURL.png.complexcolours.html
    127
     8compositing/canvas/accelerated-canvas-compositing-size-limit.html [ Failure ]
     9compositing/canvas/accelerated-canvas-compositing.html [ Failure ]
     10compositing/backgrounds/fixed-backgrounds.html [ ImageOnlyFailure ]
     11compositing/contents-scale/non-decomposable-matrix.html [ ImageOnlyFailure ]
     12compositing/hidpi-compositing-vs-non-compositing-check-on-testing-framework.html [ ImageOnlyFailure ]
     13compositing/regions/move-layer-from-one-region-to-another.html [ ImageOnlyFailure ]
     14
     15css3/filters/backdrop/backdrop-filter-with-reflection-add-backdrop.html [ ImageOnlyFailure ]
     16css3/filters/backdrop/backdrop-filter-with-reflection-value-change.html [ ImageOnlyFailure ]
     17css3/filters/backdrop/backdrop-filter-with-reflection.html [ ImageOnlyFailure ]
     18
     19fast/canvas/canvas-alphaImageData-behavior.html [ Failure ]
     20fast/canvas/canvas-crash.html [ Failure ]
     21fast/canvas/canvas-createPattern-fillRect-shadow.html [ Failure ]
     22fast/canvas/canvas-draw-canvas-on-canvas-shadow.html [ Failure ]
     23fast/canvas/canvas-drawImage-shadow.html [ Failure ]
     24fast/canvas/canvas-fillRect-gradient-shadow.html [ Failure ]
     25fast/canvas/canvas-getImageData.html [ Failure ]
     26fast/canvas/canvas-render-layer.html [ Failure ]
     27fast/canvas/canvas-scale-shadowBlur.html [ Failure ]
     28fast/canvas/canvas-scale-strokePath-shadow.html [ Failure ]
     29fast/canvas/rgba-parsing.html [ Failure ]
     30fast/canvas/set-colors.html [ Failure ]
     31fast/dom/Window/window-postmessage-clone-deep-array.html [ Failure ]
     32fast/forms/input-text-word-wrap.html [ Failure ]
     33fast/forms/search-rtl.html [ Failure ]
     34fast/forms/search-vertical-alignment.html [ Failure ]
     35fast/forms/searchfield-heights.html [ Failure ]
     36fast/forms/visual-hebrew-text-field.html [ Failure ]
     37fast/harness/perftests/perf-runner-compute-statistics.html [ Failure ]
     38fast/harness/perftests/runs-per-second-iterations.html [ Failure ]
     39fast/harness/perftests/runs-per-second-log.html [ Failure ]
     40fast/scrolling/ios/touch-scroll-pointer-events-none.html [ Failure ]
     41fast/scrolling/ios/touch-scroll-visibility-hidden.html [ Failure ]
     42fast/scrolling/scrolling-tree-includes-frame.html [ Failure ]
     43fast/viewport/ios/ipad/width-is-device-width.html [ Failure ]
     44
     45fast/css/object-fit/object-fit-canvas.html [ ImageOnlyFailure ]
     46fast/filter-image/filter-image-blur.html [ ImageOnlyFailure ]
     47fast/multicol/flexbox-rows.html [ ImageOnlyFailure ]
     48
     49fast/mediacapturefromelement/CanvasCaptureMediaStream-2d-events.html [ Crash ]
     50fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events.html [ Crash ]
     51fast/multicol/newmulticol/multicol-inside-multicol.html [ Crash ]
     52
     53http/tests/cache/partitioned-cache.html [ Failure ]
     54http/tests/loading/server-redirect-for-provisional-load-caching.html [ Failure ]
     55http/tests/misc/authentication-redirect-4/authentication-sent-to-redirect-same-origin-url.html [ Failure ]
     56http/tests/security/XFrameOptions/x-frame-options-deny-multiple-clients.html [ Failure ]
     57http/tests/security/cross-frame-access-callback-explicit-domain-ALLOW.html [ Failure ]
     58
     59http/tests/fetch/redirectmode-and-preload.html [ Skip ]
     60http/tests/incremental/frame-focus-before-load.html [ Skip ]
     61http/tests/preload/download_resources.html [ Skip ]
     62http/tests/preload/onerror_event.html [ Skip ]
     63http/tests/preload/onload_event.html [ Skip ]
     64http/tests/websocket/tests/hybi/multiple-connections.html [ Skip ]
     65
     66imported/blink/storage/indexeddb/blob-delete-objectstore-db.html [ Failure ]
     67imported/blink/storage/indexeddb/blob-valid-after-deletion.html [ Failure ]
     68imported/blink/storage/indexeddb/blob-valid-before-commit.html [ Failure ]
     69imported/w3c/web-platform-tests/IndexedDB/keygenerator-explicit.html [ Failure ]
     70imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm [ Failure ]
     71imported/w3c/web-platform-tests/cssom-view/Screen-pixelDepth-Screen-colorDepth001.html [ Failure ]
     72imported/w3c/web-platform-tests/cssom-view/scrolling-quirks-vs-nonquirks.html [ Failure ]
     73imported/w3c/web-platform-tests/dom/events/EventTarget-dispatchEvent.html [ Failure ]
     74imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/toDataURL.png.complexcolours.html [ Failure ]
     75
     76imported/blink/compositing/child-transform-layer-with-foreground-layer.html [ ImageOnlyFailure ]
     77imported/blink/fast/canvas/canvas-unbalanced-save.html [ ImageOnlyFailure ]
     78
     79imported/w3c/web-platform-tests/IndexedDB/idbcursor-update-exception-order.htm [ Skip ]
     80imported/w3c/web-platform-tests/XMLHttpRequest/send-redirect-bogus-sync.htm [ Skip ]
     81imported/w3c/web-platform-tests/XMLHttpRequest/send-send.htm [ Skip ]
     82imported/w3c/web-platform-tests/cssom/ttwf-cssom-document-extension.html [ Skip ]
     83imported/w3c/web-platform-tests/dom/nodes/Element-childElement-null-xhtml.xhtml [ Skip ]
     84imported/w3c/web-platform-tests/dom/nodes/getElementsByClassName-06.htm [ Skip ]
     85imported/w3c/web-platform-tests/html/dom/elements/global-attributes/id-name-specialcase.html [ Skip ]
     86imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/fallback.multiple.html [ Skip ]
     87imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/size.attributes.setAttribute.empty.html [ Skip ]
     88imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/toBlob.null.html [ Skip ]
     89imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/toDataURL.zerosize.html [ Skip ]
     90imported/w3c/web-platform-tests/html/semantics/forms/attributes-common-to-form-controls/disabled-elements-01.html [ Skip ]
     91imported/w3c/web-platform-tests/html/semantics/forms/the-option-element/option-text-spaces.html [ Skip ]
     92imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/dir.html [ Skip ]
     93imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/indeterminate.html [ Skip ]
     94imported/w3c/web-platform-tests/html/syntax/parsing/template/clearing-the-stack-back-to-a-given-context/clearing-stack-back-to-a-table-body-context.html [ Skip ]
     95imported/w3c/web-platform-tests/html/syntax/parsing/template/clearing-the-stack-back-to-a-given-context/clearing-stack-back-to-a-table-row-context.html [ Skip ]
     96imported/w3c/web-platform-tests/html/webappapis/scripting/events/event-handler-processing-algorithm.html [ Skip ]
     97imported/w3c/web-platform-tests/html/webappapis/scripting/events/onerroreventhandler.html [ Skip ]
     98imported/w3c/web-platform-tests/html/webappapis/scripting/processing-model-2/compile-error-data-url.html [ Skip ]
     99imported/w3c/web-platform-tests/html/webappapis/scripting/processing-model-2/window-onerror-with-cross-frame-event-listeners-1.html [ Skip ]
     100imported/w3c/web-platform-tests/shadow-dom/Extensions-to-Event-Interface.html [ Skip ]
     101imported/w3c/web-platform-tests/streams/byte-length-queuing-strategy.html [ Skip ]
     102imported/w3c/web-platform-tests/streams/piping/close-propagation-forward.serviceworker.https.html [ Skip ]
     103imported/w3c/web-platform-tests/streams/readable-streams/garbage-collection.serviceworker.https.html [ Skip ]
     104imported/w3c/web-platform-tests/user-timing/idlharness.html [ Skip ]
     105
     106js/dom/deep-recursion-test.html [ Failure ]
     107
     108js/stringimpl-to-jsstring-on-large-strings-2.html [ Crash ]
     109js/stringimpl-to-jsstring-on-large-strings-3.html [ Crash ]
     110
     111loader/go-back-to-different-window-size.html [ Crash ]
     112
     113media/audio-controls-rendering.html [ Failure ]
     114media/audio-controls-timeline-in-media-document.html [ Failure ]
     115media/audio-repaint.html [ Failure ]
     116media/track/track-cue-overlap-snap-to-lines-not-set.html [ Failure ]
     117media/video-canvas-createPattern.html [ Failure ]
     118media/video-canvas-drawing-output.html [ Failure ]
     119media/video-load-preload-metadata.html [ Failure ]
     120media/video-played-collapse.html [ Failure ]
     121media/video-played-reset.html [ Failure ]
     122
     123media/track/track-cues-pause-on-exit.html [ Skip ]
     124media/video-played-ranges-1.html [ Skip ]
     125
     126media/invalid-media-url-crash.html [ Crash ]
     127
     128svg/custom/pattern-reference.svg [ Crash ]
     129
     130tiled-drawing/ios/iphone7/compositing-layers-deep-color.html [ Failure ]
     131
     132quicklook/numbers.html [ ImageOnlyFailure ]
     133
     134scrollbars/overflow-scrollbar-combinations.html [ Timeout ]
     135
     136security/contentSecurityPolicy/video-with-data-url-allowed-by-media-src-star.html [ ImageOnlyFailure ]
     137
     138security/contentSecurityPolicy/video-with-data-url-allowed-by-media-src-star.html [ ImageOnlyFailure ]
     139
     140workers/bomb.html [ Skip ]
     141
     142# <rdar://problem/30353797> need to differentiated between device types
     143tiled-drawing/ios/iphone7
     144
     145# <rdar://problem/32575766>, need to handle media sandbox correctly
     146webrtc

  • trunk/LayoutTests/platform/ios-simulator-wk2/TestExpectations

    r218018 r218139  
    44#
    55
     6editing/inserting/insert-paragraph-selection-outside-contenteditable.html [ Failure ]
    67compositing/regions/composited-content-fixed-region-scrolled.html [ Failure ]
    78fast/events/offsetX-offsetY.html [ Failure ]
     9fast/hidpi/image-srcset-invalid-descriptor.html [ Failure ]
     10fast/regions/repaint/fixed-in-named-flow-scroll.html [ Failure ]
     11http/tests/cookies/js-get-and-set-http-only-cookie.html [ Failure ]
     12http/tests/security/cross-origin-worker-indexeddb-allowed.html [ Failure ]
     13http/tests/security/cross-origin-worker-indexeddb.html [ Failure ]
    814
    915fast/events/scroll-to-anchor-vertical-lr-writing-mode.html [ ImageOnlyFailure ]
    1016fast/events/scroll-to-anchor-vertical-writing-mode.html [ ImageOnlyFailure ]
     17fast/sub-pixel/sub-pixel-composited-layers.html [ ImageOnlyFailure ]
    1118
    1219webkit.org/b/152439 compositing/overflow/dynamic-composited-scrolling-status.html [ Failure ]
    1320
    1421webkit.org/b/173178 [ Release ] svg/animations/svglength-element-removed-crash.svg [ Skip ]
     22
     23# iOS simulator doesn’t support dashboard.
     24mathml/mathml-in-dashboard.html [ Failure ]
     25svg/custom/embedded-svg-allowed-in-dashboard.xml [ Failure ]
     26svg/custom/manually-parsed-embedded-svg-allowed-in-dashboard.html [ Failure ]
     27svg/custom/manually-parsed-svg-allowed-in-dashboard.html [ Failure ]
     28svg/custom/svg-allowed-in-dashboard-object.html [ Failure ]
     29http/tests/xmlhttprequest/svg-created-by-xhr-allowed-in-dashboard.html [ Failure ]

  • trunk/LayoutTests/platform/ios-simulator/TestExpectations

    r218127 r218139  
    44#
    55
     6css3/font-variant-small-caps-synthesis-coverage.html [ ImageOnlyFailure ]
     7fast/frames/flattening/scrolling-in-object.html [ ImageOnlyFailure ]
     8fast/frames/sandboxed-iframe-close-top-noclose.html [ Failure ]
    69fast/multicol/flexbox-rows.html [ Skip ]
    710workers/wasm-mem-post-message.html [ Failure ]
    … …  
    5356fast/css/sticky/sticky-top.html [ ImageOnlyFailure ]
    5457fast/css/sticky/sticky-writing-mode-horizontal-bt.html [ ImageOnlyFailure ]
     58ietestcenter/css3/namespaces/syntax-021.xml [ ImageOnlyFailure ]
     59
     60media/track/track-in-band-legacy-api.html [ Failure ]
     61media/track/video-track-alternate-groups.html [ Failure ]
     62
     63svg/filters/filter-hidden-content.svg [ Failure ]
     64
     65transforms/hittest-translated-content-off-to-infinity-and-back.html [ Pass ImageOnlyFailure ]
    5566
    5667# The modern-media-controls tests are disabled due to past flakiness and some major code changes.
    … …  
    97108
    98109webkit.org/b/156718 storage/websql/alter-to-info-table.html [ Failure ]
     110
     111# This test relies on Arial being used to draw Arabic. However, on iOS,
     112# we explicitly disallow this because this font is too slow.
     113fast/text/initial-advance-in-intermediate-run-complex.html [ ImageOnlyFailure ]

  • trunk/LayoutTests/platform/ios-wk2/TestExpectations

    r217868 r218139  
    105105scrollbars/scrolling-by-page-on-keyboard-spacebar.html [ Failure ]
    106106scrollbars/scrollbars-on-positioned-content.html [ Failure ]
    107 
    108 # iOS doesn’t support dashboard.
    109 mathml/mathml-in-dashboard.html [ Failure ]
    110 svg/custom/embedded-svg-allowed-in-dashboard.xml [ Failure ]
    111 svg/custom/manually-parsed-embedded-svg-allowed-in-dashboard.html [ Failure ]
    112 svg/custom/manually-parsed-svg-allowed-in-dashboard.html [ Failure ]
    113 svg/custom/svg-allowed-in-dashboard-object.html [ Failure ]
    114 http/tests/xmlhttprequest/svg-created-by-xhr-allowed-in-dashboard.html [ Failure ]
    115107
    116108# SVG tests that time out (these require EventSender)
    … …  
    336328# HTTP tests that fail:
    337329http/tests/appcache/load-from-appcache-defer-resume-crash.html [ Failure ]
    338 http/tests/cookies/js-get-and-set-http-only-cookie.html [ Failure ]
    339330http/tests/download/inherited-encoding-form-submission-result.html [ Failure ]
    340331http/tests/globalhistory/history-delegate-basic-refresh-redirect.html [ Failure ]
    … …  
    359350http/tests/navigation/success200-subframeload.html [ Failure ]
    360351http/tests/navigation/timerredirect-subframeload.html [ Failure ]
    361 http/tests/security/cross-origin-worker-indexeddb-allowed.html [ Failure ]
    362 http/tests/security/cross-origin-worker-indexeddb.html [ Failure ]
    363352http/tests/security/drag-drop-local-file.html [ Failure ]
    364353http/tests/security/drag-drop-same-unique-origin.html [ Failure ]
    … …  
    541530fast/regions/hover-on-child-parent-different-regions.html [ Failure ]
    542531fast/regions/overflow/overflow-3dtransformed-region.html [ Failure ]
    543 fast/regions/repaint/fixed-in-named-flow-scroll.html [ Failure ]
    544532fast/regions/repaint/repaint-absolute-pushed-to-next-region.html [ Failure ]
    545533fast/replaced/image-map-2.html [ Failure ]
    … …  
    633621fast/shadow-dom/focus-navigation-across-slots.html [ Failure ]
    634622fast/shadow-dom/negative-tabindex-on-shadow-host.html [ Failure ]
    635 fast/sub-pixel/sub-pixel-composited-layers.html [ ImageOnlyFailure ]
    636623fast/text/word-space-between-inlines.html [ ImageOnlyFailure ]
    637624fast/css/beforeSelectorOnCodeElement.html [ Failure ]
    … …  
    649636fast/forms/targeted-frame-submission.html [ Failure ]
    650637fast/frames/onlyCommentInIFrame.html [ Failure ]
    651 fast/hidpi/image-srcset-invalid-descriptor.html [ Failure ]
    652638fast/html/listing.html [ Failure ]
    653639fast/invalid/junk-data.xml [ Failure ]
    … …  
    872858# CSS transform tests that fail:
    873859transforms/2d/transform-fixed-container.html [ Failure ]
    874 transforms/hittest-translated-content-off-to-infinity-and-back.html [ ImageOnlyFailure ]
    875860
    876861# Editing tests that fail:
    … …  
    961946editing/inserting/insert-div-007.html [ Failure ]
    962947editing/inserting/insert-div-008.html [ Failure ]
    963 editing/inserting/insert-paragraph-selection-outside-contenteditable.html [ Failure ]
    964948editing/inserting/insert-space-in-empty-doc.html [ Failure ]
    965949editing/inserting/insert-tab-001.html [ Failure ]

  • trunk/LayoutTests/platform/ios/TestExpectations

    r217937 r218139  
    728728ietestcenter/css3/multicolumn/column-width-applies-to-012.htm [ ImageOnlyFailure ]
    729729ietestcenter/css3/multicolumn/column-width-applies-to-015.htm [ ImageOnlyFailure ]
    730 ietestcenter/css3/namespaces/syntax-021.xml [ ImageOnlyFailure ]
    731730ietestcenter/css3/valuesandunits/units-000.htm [ ImageOnlyFailure ]
    732731
    … …  
    888887svg/filters/feImage-filterUnits-userSpaceOnUse-primitiveUnits-objectBoundingBox.svg [ Failure ]
    889888svg/filters/feImage-filterUnits-userSpaceOnUse-primitiveUnits-userSpaceOnUse.svg [ Failure ]
    890 svg/filters/filter-hidden-content.svg [ Failure ]
    891889svg/filters/filter-on-filter-for-text.svg [ Failure ]
    892890svg/filters/filter-on-tspan.svg [ Failure ]
    … …  
    14561454fast/frames/content-opacity-2.html [ Failure ]
    14571455fast/frames/flattening/frameset-flattening-subframesets.html [ Failure ]
    1458 fast/frames/flattening/scrolling-in-object.html [ ImageOnlyFailure ]
    14591456fast/frames/frame-set-rotation-hit.html [ Failure ]
    14601457fast/frames/frame-set-scaling-hit.html [ Failure ]
    … …  
    14661463fast/frames/paint-iframe-background.html [ Failure ]
    14671464fast/frames/sandboxed-iframe-about-blank.html [ Failure ]
    1468 fast/frames/sandboxed-iframe-close-top-noclose.html [ Failure ]
    14691465fast/frames/sandboxed-iframe-navigation-allowed.html [ Failure ]
    14701466fast/frames/valid.html [ Failure ]
    … …  
    24722468
    24732469# Uncategorized reftest failures, uncovered once we made reftests work. Needs triage.
    2474 css3/font-variant-small-caps-synthesis-coverage.html [ ImageOnlyFailure ]
    24752470fast/backgrounds/background-image-large-float-intrinsic-ratio.html [ ImageOnlyFailure ]
    24762471fast/block/positioning/fixed-container-with-relative-parent.html [ ImageOnlyFailure ]
    … …  
    26672662media/video-volume.html [ Failure ]
    26682663
    2669 media/track/track-in-band-legacy-api.html [ Failure ]
    2670 media/track/video-track-alternate-groups.html [ Failure ]
    2671 
    26722664media/W3C/audio/canPlayType/canPlayType_supported_but_no_codecs_parameter_1.html [ Failure ]
    26732665
    … …  
    27472739fast/visual-viewport/client-coordinates-relative-to-layout-viewport.html [ Skip ]
    27482740
    2749 # This test relies on Arial being used to draw Arabic. However, on iOS,
    2750 # we explicitly disallow this because this font is too slow.
    2751 fast/text/initial-advance-in-intermediate-run-complex.html [ ImageOnlyFailure ]
    2752 
    27532741# On iOS we do not visually highlight a programmatic selection
    27542742fast/selectors/040.html
Note: See TracChangeset for help on using the changeset viewer.