| | 1 | = WebKit Accessibility Performance = |
| | 2 | ''by Nan Wang'' |
| | 3 | |
| | 4 | - Problems |
| | 5 | - “Busy” text in this state entire web page is unresponsive |
| | 6 | - Heavy JS with VoiceOver, can increase CPU utilization (near 100%) |
| | 7 | - VoiceOver indicator can have a wrong indicator, may indicate wrong activation point (no click event) |
| | 8 | - WebKit Accessibility Overview |
| | 9 | - Accessibility Flow |
| | 10 | - VoiceOver example on iOS |
| | 11 | - User gasps on the screen at (x, y) |
| | 12 | - VoiceOver hit-tests on the web page |
| | 13 | - WebCore creates AX tree |
| | 14 | - VoiceOver visits the AX node |
| | 15 | - VoiceOver announces the element |
| | 16 | - Why an AX tree? |
| | 17 | - Imagine a web form, several form elements |
| | 18 | - From AX API, less info, less nodes |
| | 19 | - Build Accessibility Tree |
| | 20 | - DOM Tree & Style Sheets > Render Tree > Layout > Accessibility Tree |
| | 21 | - Accessibility in WebCore |
| | 22 | - Lives in WebProcess |
| | 23 | - Current Challenges |
| | 24 | - Accessibility Tree Generation |
| | 25 | - constant accessibility tree validation |
| | 26 | - Possible re-layout |
| | 27 | - Slow performance |
| | 28 | - Many IPC calls |
| | 29 | - Accessibility frame retrieval |
| | 30 | - Sometimes retrieves coordinates for frames over and over |
| | 31 | - Sandboxes Web Process |
| | 32 | - Limitations for the system Accessibility API |
| | 33 | - Assistive tech are accessing the web process |
| | 34 | - Other browsers have different approaches |
| | 35 | - Solutions |
| | 36 | - Unblock the Main Thread |
| | 37 | - Reduce DOM interaction |
| | 38 | - Average attribute fetching time is 0.05 ms |
| | 39 | - On macOS constantly fetching, could be huge impact on main thread blocking |
| | 40 | - 10x speedup with caching |
| | 41 | - Define Tree Generation Timing |
| | 42 | - Reduce accessibility tree validations |
| | 43 | - 2000 updateBackingStore() calls when a user taps on an element on iOS |
| | 44 | - 0.02ms avg |
| | 45 | - 0.2 ms when a re-layout is triggered |
| | 46 | - Reduce IPC Calls |
| | 47 | - 0.2 ms when retrieving a frame on an iPhone 8 Plus |
| | 48 | - 20x speedup with frame caching |
| | 49 | - Conclusion: Cache! |
| | 50 | - Need to be careful to keep physical footprint low |
| | 51 | - Static Accessibility Trees |
| | 52 | - Store Accessibility Information in some sort of data structure |
| | 53 | - class AccessibilityStaticObject proposal |
| | 54 | - A lot of possible attributes |
| | 55 | - 50 ARIA attributes |
| | 56 | - 90 Attributes exposed on iOS (including ARIA) |
| | 57 | - More on macOS |
| | 58 | - Storing all values could be wasteful |
| | 59 | - Store a Compact Data Structure |
| | 60 | - About 20 attributes for each major data type |
| | 61 | - Use HashMap? |
| | 62 | - Accessibility Tree Generation |
| | 63 | - Class to wrap a one-time snapshot of the accessibility tree up front |
| | 64 | - `class AccessibilityStaticObject` |
| | 65 | - Incremental Tree Updates |
| | 66 | - Class to represent an atomic change to an AccessibilityTree |
| | 67 | - `class AccessibilityTreeUpdate` |
| | 68 | - Used to apply to snapshot tree |
| | 69 | - Individual actions (operation=[Add|Remove]) |
| | 70 | - `oldNode=<id>` |
| | 71 | - `newNode=<id>` |
| | 72 | - Bounding Boxes |
| | 73 | - Accessibility bounding boxes are normally screen coordinates |
| | 74 | - Hit-Testing |
| | 75 | - Hit-testing onto the AccessibilityStaticObject instead of the RenderObject |
| | 76 | - Depth-first traversal of the tree in reverse pre-order to find the element |
| | 77 | - Future Work |
| | 78 | - Parameterized attributes |
| | 79 | - E.g. text bounding box |
| | 80 | - May require blocking main thread to query DOM tree |
| | 81 | - Should cache enough data |
| | 82 | - Maybe split text into inline text boxes |
| | 83 | - A sequence of same style characters within the same line |
| | 84 | - Store relative frame of each character |
| | 85 | - Pass a serialized tree to UI Process |
| | 86 | - Pro: break the limitations for system Accessibility APIs |
| | 87 | - Con: Security concerns |
| | 88 | - Implement Static Accessibility Trees |
| | 89 | - Off main thread computation |
| | 90 | - Separate AX tree generation |
| | 91 | - Reduce IPC calls by caching frames |
