wiki:ImprovingInteropwithWPT

Version 4 (modified by Jon Davis, 12 months ago) (diff)

--

Improving Interop with WPT

by Philip Jägenstedt & Stephen McGruer

Slides

  • Why does interop matter?
    • Users, developers, browser engines
    • Users
      • Go to website on favorite browser, but broken
      • Go find a browser it’s meant to work on
      • All the things we do for interop make our lives easier
    • Developers
      • MDN ran largest survey ever
      • Top pain points cross-browser compat/interop/testing
      • What to do?
  • A shared test suite, for all the web
    • 20 commits from WebKit
    • Believe it is infrastructure
  • High Level View
    • Chrome + Firefox + Safari
    • 0/3 ~ 5K tests
    • 1/3 ~ 6K tests
    • 2/3 ~ 6K tests
    • 3/3 ~ 20K tests
    • Moving to target the browser specific failures (2/3)
    • Interop is steady
  • Browser specific failures in WPT
    • Gap is growing
    • Methodology for data for this presentation
    • WPT splits into tests and sub-tests
      • Sub-tests are within the test file
      • Some tests with too many subtests
      • Encoding tests have absurd
      • Normalized to test level
  • Runs on Azure pipelines
    • STP runs every 3 hours, takes < 2 hours
    • Edge Dev and Canary too
    • Thanks Microsoft and @mustjab
  • Runs on Taskcluster
    • Chrome + Firefox on every commit
    • WebKitGTK ally
    • Thanks Mozilla and @jgraham
  • Room for Improvement
  • Epiphany / WebKit
    • Also run subtests on Linux for a second opinion
  • WebKit on Linux
    • Google partnering with Igalia
    • WebKitGTK stable+nightly runs
    • Fixing noise in the data
  • 2-way Sync
    • Script for WebKit exists
  • 2-way Sync in Chromium
    • Update impl+tests in single review
    • Use of WPT by Chromium devs up ~220% (*j.mp/wpt-yes)
    • ~55% of impl+test changes now use WPT
      • Chromium & Gecko
        • Automatic import
        • Automatic export AFTER changes land
      • WebKit
        • Manual import (?)
          • Yes it is
        • Manual (?) export BEFORE changes land
    • Room for improvement / collaboration?
    • Issue on export PRs?
  • A Wishlist
    • Collaborate on wit.fyi
      • Wish there was a way to browser-specific failures in the same way computed for the graph
    • Running web-driver to internal builds
    • Collaborate on running WPT on iOS
    • Collaborate on MDN survey: developers say interop is a big thing
  • WPT Right approach
    • Importing all the tests could slow us down
    • Maybe keep them separate?
    • New tests added don’t look like regressions
    • WPT: Tracking WPT as a sub-module is not compatible
  • Please don't scrape wpt.fyi, use the API