Changes between Initial Version and Version 1 of GPUProcess


Ignore:
Timestamp:
Jan 5, 2021 3:20:25 PM (18 months ago)
Author:
Jon Davis
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • GPUProcess

    v1 v1  
     1== GPU Process ==
     2''by Wensen Hseih (Apple)'' [https://www.icloud.com/keynote/0pbB8lOJKz5Z_r1o3niIMxJuA#gpup Slide Deck]
     3
     4High level summary of the goals of GPU process, a basic walkthrough of the architecture, and the current status of the project.
     5
     6Hello, and welcome! Mostly covers the what, why, and how, and will discuss active areas of development at the end.
     7
     8— What?
     9
     10''Current:'' [architecture diagram] 1 UI process, 1 network process, multiple web content processes
     11
     12Add a new process for graphics and media playback
     13
     14— Why?
     15
     16Security. Web Process has privs to talk to GPU and kernel. If Web Process is compromised, attacker gets access to these privs
     17
     18Move it out of process so we can vet priviliged access and terminate the web process if it tries to do anything suspicious
     19
     20— How?
     21
     22ImageBuffer — currently wraps e.g. CGContext
     23
     24In GPUP world, it instead owns a DisplayList WebCore GraphicsContext
     25
     26ImageBuffer now has a GPUP counterpart, which owns the real CGContext
     27
     28<many class names>
     29
     30— How to draw a path (for example)
     31
     32something in Web Process calls e.g. GraphicsContext::strokePath
     33
     34we don’t have a platformContext(), but instead a GraphicsContextImpl (m_impl), which is a DisplayListRecorder
     35
     36builds up a DisplayList, builds up items that represent GraphicsContext operations that we’ll later apply to the real context in the GPU process
     37
     38in the strokePath case, we record all the things you need to replay the strokePath() in the GPU process
     39
     40(the path, etc.)
     41
     42then, in the GPU process, we’re back at the beginning (GraphicsContext::strokePath); this time, we don’t have a m_impl, but we DO have a PlatformContext (CGContextRef), so we go back to the platform implementation
     43
     44(and stroke the path on the CGContext)
     45
     46send all sorts of display list items (corresponding to GraphicsContext items)
     47
     48JER---
     49
     50No slides, going to talk about what we need to do to move Media out of process.
     51
     52Normal media playback is already out of process (straight files, HLS playlists)
     53
     54Next few weeks, same thing for MediaSource backed media elements.
     55
     56Will continue to keep in-process model working for ports that haven’t adopted the GPU process.
     57
     58Moving the parts of MSE that deal with samples into SourceBufferPrivate
     59
     60''Benefits are like Wenson mentioned:'' limiting the surface area risk for things that take over the Web Content process
     61
     62Limiting risk + severity of RCE in the WP
     63
     64Completed early stages, performance seems to be (generally) on-par (small regression)
     65
     66Work is straightforward, just keep pushing through and do in a way that other ports can adopt in the future
     67
     68WENSON---
     69
     70Remaining challenges and ongoing work
     71
     72PERFORMANCE
     73
     74Measuring with MotionMark
     75
     76Much of the regression is overhead from serializing and coordinating graphics commands between the WP + GPU process
     77
     78When we first enabled GPUP for canvas about a year ago, we were 40-60% regressed on the canvas subtests
     79
     80landed some recent changes in trunk that mitigate this by storing DisplayList items in shared memory (in a segmented ring buffer) so the web process can write while the GPU process is reading
     81
     82''Four main areas to complete:'' 2d canvas, dom rendering, webgl, media
     83
     84Have separate switches for each
     85
     86Once we turn them all on, we can eliminate IOKit access overall
     87
     88And get the security benefits
     89
     90Security benefits only hold if the code in the GPU process is robust as well
     91
     92Suppose we have a compromised WP
     93
     94Should never be able to crash the GPU process by sending e.g. malicious IPC
     95
     96Need to be robust, even terminating the Web Content process if we see anything fishy (OOB access, etc.)
     97
     98== Questions & Comments ==
     99
     100Ken R.: Can I ask about synchronization; have you thought about the synchronization e.g. media painting into canvas
     101
     102''Wenson:'' not flushed out yet (have the ability to paint the current frame, but need to e.g. be able to reference the correct time when the painting was requested)
     103
     104''Simon:'' There are cases right now where you have to synchronously IPC (e.g. readPixels). And canvas-to-canvas painting needs to be sure that you’ve flushed the source before copying
     105
     106When you know that a canvas is going to paint into another canvas, you push a flush identifier into the source and know you have. to wait for the dest to ask for it
     107
     108We know we have to not break any existing behavior
     109
     110''Wenson:'' Currently in the GPUP, display list playback is currently on the main thread, which makes synchronization much easier. Once we move things to separate threads, it will be trickier and need more logic
     111
     112''Ken:'' ''One real gotcha:'' Google Maps; 2D canvas on top of WebGL. HTML spec guarantees that all work done in a RAF shows up on screen at the same time. Make sure to give that thought
     113
     114''Simon:'' Pretty sure we already aren’t synchronized here. Kimmo has been doing some changes where the dest buffer management in WebGL is going to be more similar to how we manage front/back buffers for DOM rendering
     115
     116Think we’ll end up doing a better job of synchronization with the GPUP
     117
     118Hopefully that won’t be an issue
     119
     120(everyone concurs that there probably aren’t WPTs that will catch this)
     121
     122We’ve been focusing our testing on motionmark, but would be great if people have examples of complex feature rich 2d canvases that can be looked at for performance
     123
     124''Ken:'' Anything about the progress for remoting WebGL?
     125
     126''Simon:'' Kimmo is making progress; WebGLLayer is going to turn into one of our PlatformLayer things. that doesn’t actually have a CALayer, and front/back buffer management will move down into GraphicsContextGL. Very active development
     127
     128No attempt yet to optimize sending GL commands (Wenson’s mechanism is all for DisplayList, not sure if we will share that for GL yet)
     129
     130''Wenson:'' Performance will be a huge hurdle
     131
     132Think blink has a similar setup, but abstract at the GL layer
     133
     134Some advantages to having our DisplayList items as an abstraction (can encapsulate multiple GL commands for normal drawing), but needs to be powerful enough to handle both
     135
     136''Simon:'' perf going to be very sensitive to the type of content; will be in a good place when all of the content is flowing Web->GPU
     137
     138(will take a hit for getError, etc.)
     139
     140(have to stall the world and sync IPC to the GPUP to get the pixels for getPixels, etc.)
     141
     142''Ken:'' Blink’s started as GL and then added more terse representation
     143
     144Having both certainly adds value
     145
     146''Ken:'' a big gotcha, since you’re writing into shared memory; very important to only read that shared memory once in the GPU process
     147
     148''Wenson:'' not sure what you mean; no mechanical guards, but we do intend to only do it once
     149
     150Security is tricky,'' conservative approach now:'' we only need it right when we’re applying. Once it’s validated, it’s copied, and applied from there