Changes between Initial Version and Version 1 of WebKitAccessibilityPerformance


Ignore:
Timestamp:
Oct 12, 2018 3:37:47 PM (4 years ago)
Author:
Jon Davis
Comment:

Notes from 2018 WebKit Contributors Meeting

Legend:

Unmodified
Added
Removed
Modified
  • WebKitAccessibilityPerformance

    v1 v1  
     1= WebKit Accessibility Performance =
     2''by Nan Wang''
     3
     4- Problems
     5    - “Busy” text in this state entire web page is unresponsive
     6    - Heavy JS with VoiceOver, can increase CPU utilization (near 100%)
     7    - VoiceOver indicator can have a wrong indicator, may indicate wrong activation point (no click event)
     8- WebKit Accessibility Overview
     9    - Accessibility Flow
     10    - VoiceOver example on iOS
     11        - User gasps on the screen at (x, y)
     12        - VoiceOver hit-tests on the web page
     13        - WebCore creates AX tree
     14        - VoiceOver visits the AX node
     15        - VoiceOver announces the element
     16    - Why an AX tree?
     17        - Imagine a web form, several form elements
     18        - From AX API, less info, less nodes
     19    - Build Accessibility Tree
     20        - DOM Tree & Style Sheets > Render Tree > Layout > Accessibility Tree
     21    - Accessibility in WebCore
     22        - Lives in WebProcess
     23- Current Challenges
     24    - Accessibility Tree Generation
     25        - constant accessibility tree validation
     26        - Possible re-layout
     27        - Slow performance
     28    - Many IPC calls
     29        - Accessibility frame retrieval
     30            - Sometimes retrieves coordinates for frames over and over
     31    - Sandboxes Web Process
     32        - Limitations for the system Accessibility API
     33        - Assistive tech are accessing the web process
     34        - Other browsers have different approaches
     35- Solutions
     36    - Unblock the Main Thread
     37        - Reduce DOM interaction
     38            - Average attribute fetching time is 0.05 ms
     39            - On macOS constantly fetching, could be huge impact on main thread blocking
     40            - 10x speedup with caching
     41        - Define Tree Generation Timing
     42            - Reduce accessibility tree validations
     43                - 2000 updateBackingStore() calls when a user taps on an element on iOS
     44                - 0.02ms avg
     45                - 0.2 ms when a re-layout is triggered
     46        - Reduce IPC Calls
     47            - 0.2 ms when retrieving a frame on an iPhone 8 Plus
     48            - 20x speedup with frame caching
     49    - Conclusion: Cache!
     50        - Need to be careful to keep physical footprint low
     51    - Static Accessibility Trees
     52        - Store Accessibility Information in some sort of data structure
     53        - class AccessibilityStaticObject proposal
     54        - A lot of possible attributes
     55            - 50 ARIA attributes
     56            - 90 Attributes exposed on iOS (including ARIA)
     57            - More on macOS
     58        - Storing all values could be wasteful
     59        - Store a Compact Data Structure
     60        - About 20 attributes for each major data type
     61            - Use HashMap?
     62        - Accessibility Tree Generation
     63            - Class to wrap a one-time snapshot of the accessibility tree up front
     64            - `class AccessibilityStaticObject`
     65        - Incremental Tree Updates
     66            - Class to represent an atomic change to an AccessibilityTree
     67            - `class AccessibilityTreeUpdate`
     68            - Used to apply to snapshot tree
     69            - Individual actions (operation=[Add|Remove])
     70            - `oldNode=<id>`
     71            - `newNode=<id>`
     72        - Bounding Boxes
     73            - Accessibility bounding boxes are normally screen coordinates
     74        - Hit-Testing
     75            - Hit-testing onto the AccessibilityStaticObject instead of the RenderObject
     76            - Depth-first traversal of the tree in reverse pre-order to find the element
     77    - Future Work
     78        - Parameterized attributes
     79            - E.g. text bounding box
     80            - May require blocking main thread to query DOM tree
     81            - Should cache enough data
     82            - Maybe split text into inline text boxes
     83            - A sequence of same style characters within the same line
     84            - Store relative frame of each character
     85        - Pass a serialized tree to UI Process
     86            - Pro: break the limitations for system Accessibility APIs
     87            - Con: Security concerns
     88- Implement Static Accessibility Trees
     89    - Off main thread computation
     90    - Separate AX tree generation
     91    - Reduce IPC calls by caching frames