Changes between Initial Version and Version 1 of GPUProcess

Jan 5, 2021 3:20:25 PM (2 years ago)
Jon Davis



  • GPUProcess

    v1 v1  
     1== GPU Process ==
     2''by Wensen Hseih (Apple)'' [ Slide Deck]
     4High level summary of the goals of GPU process, a basic walkthrough of the architecture, and the current status of the project.
     6Hello, and welcome! Mostly covers the what, why, and how, and will discuss active areas of development at the end.
     8— What?
     10''Current:'' [architecture diagram] 1 UI process, 1 network process, multiple web content processes
     12Add a new process for graphics and media playback
     14— Why?
     16Security. Web Process has privs to talk to GPU and kernel. If Web Process is compromised, attacker gets access to these privs
     18Move it out of process so we can vet priviliged access and terminate the web process if it tries to do anything suspicious
     20— How?
     22ImageBuffer — currently wraps e.g. CGContext
     24In GPUP world, it instead owns a DisplayList WebCore GraphicsContext
     26ImageBuffer now has a GPUP counterpart, which owns the real CGContext
     28<many class names>
     30— How to draw a path (for example)
     32something in Web Process calls e.g. GraphicsContext::strokePath
     34we don’t have a platformContext(), but instead a GraphicsContextImpl (m_impl), which is a DisplayListRecorder
     36builds up a DisplayList, builds up items that represent GraphicsContext operations that we’ll later apply to the real context in the GPU process
     38in the strokePath case, we record all the things you need to replay the strokePath() in the GPU process
     40(the path, etc.)
     42then, in the GPU process, we’re back at the beginning (GraphicsContext::strokePath); this time, we don’t have a m_impl, but we DO have a PlatformContext (CGContextRef), so we go back to the platform implementation
     44(and stroke the path on the CGContext)
     46send all sorts of display list items (corresponding to GraphicsContext items)
     50No slides, going to talk about what we need to do to move Media out of process.
     52Normal media playback is already out of process (straight files, HLS playlists)
     54Next few weeks, same thing for MediaSource backed media elements.
     56Will continue to keep in-process model working for ports that haven’t adopted the GPU process.
     58Moving the parts of MSE that deal with samples into SourceBufferPrivate
     60''Benefits are like Wenson mentioned:'' limiting the surface area risk for things that take over the Web Content process
     62Limiting risk + severity of RCE in the WP
     64Completed early stages, performance seems to be (generally) on-par (small regression)
     66Work is straightforward, just keep pushing through and do in a way that other ports can adopt in the future
     70Remaining challenges and ongoing work
     74Measuring with MotionMark
     76Much of the regression is overhead from serializing and coordinating graphics commands between the WP + GPU process
     78When we first enabled GPUP for canvas about a year ago, we were 40-60% regressed on the canvas subtests
     80landed some recent changes in trunk that mitigate this by storing DisplayList items in shared memory (in a segmented ring buffer) so the web process can write while the GPU process is reading
     82''Four main areas to complete:'' 2d canvas, dom rendering, webgl, media
     84Have separate switches for each
     86Once we turn them all on, we can eliminate IOKit access overall
     88And get the security benefits
     90Security benefits only hold if the code in the GPU process is robust as well
     92Suppose we have a compromised WP
     94Should never be able to crash the GPU process by sending e.g. malicious IPC
     96Need to be robust, even terminating the Web Content process if we see anything fishy (OOB access, etc.)
     98== Questions & Comments ==
     100Ken R.: Can I ask about synchronization; have you thought about the synchronization e.g. media painting into canvas
     102''Wenson:'' not flushed out yet (have the ability to paint the current frame, but need to e.g. be able to reference the correct time when the painting was requested)
     104''Simon:'' There are cases right now where you have to synchronously IPC (e.g. readPixels). And canvas-to-canvas painting needs to be sure that you’ve flushed the source before copying
     106When you know that a canvas is going to paint into another canvas, you push a flush identifier into the source and know you have. to wait for the dest to ask for it
     108We know we have to not break any existing behavior
     110''Wenson:'' Currently in the GPUP, display list playback is currently on the main thread, which makes synchronization much easier. Once we move things to separate threads, it will be trickier and need more logic
     112''Ken:'' ''One real gotcha:'' Google Maps; 2D canvas on top of WebGL. HTML spec guarantees that all work done in a RAF shows up on screen at the same time. Make sure to give that thought
     114''Simon:'' Pretty sure we already aren’t synchronized here. Kimmo has been doing some changes where the dest buffer management in WebGL is going to be more similar to how we manage front/back buffers for DOM rendering
     116Think we’ll end up doing a better job of synchronization with the GPUP
     118Hopefully that won’t be an issue
     120(everyone concurs that there probably aren’t WPTs that will catch this)
     122We’ve been focusing our testing on motionmark, but would be great if people have examples of complex feature rich 2d canvases that can be looked at for performance
     124''Ken:'' Anything about the progress for remoting WebGL?
     126''Simon:'' Kimmo is making progress; WebGLLayer is going to turn into one of our PlatformLayer things. that doesn’t actually have a CALayer, and front/back buffer management will move down into GraphicsContextGL. Very active development
     128No attempt yet to optimize sending GL commands (Wenson’s mechanism is all for DisplayList, not sure if we will share that for GL yet)
     130''Wenson:'' Performance will be a huge hurdle
     132Think blink has a similar setup, but abstract at the GL layer
     134Some advantages to having our DisplayList items as an abstraction (can encapsulate multiple GL commands for normal drawing), but needs to be powerful enough to handle both
     136''Simon:'' perf going to be very sensitive to the type of content; will be in a good place when all of the content is flowing Web->GPU
     138(will take a hit for getError, etc.)
     140(have to stall the world and sync IPC to the GPUP to get the pixels for getPixels, etc.)
     142''Ken:'' Blink’s started as GL and then added more terse representation
     144Having both certainly adds value
     146''Ken:'' a big gotcha, since you’re writing into shared memory; very important to only read that shared memory once in the GPU process
     148''Wenson:'' not sure what you mean; no mechanical guards, but we do intend to only do it once
     150Security is tricky,'' conservative approach now:'' we only need it right when we’re applying. Once it’s validated, it’s copied, and applied from there