Display Lists

2015 Display List Experiment

• Recording GraphicsContext calls, and playing them back later
• Drive GraphicsContext cleanup
• Perf gains
• Per-compositing layer
• Logging

Future

• Display list mutation (some CSS animations, GIFs, scrolling)
• Repaint
• Occlusion detection, state change optimization
• Shared display list segments
• Paint-to-paint caching
  • Text

Display List Rendering

What is display list rendering?

• Reduces expensive traversal of the render tree for each tile
• Converts some paints to pure display-list operations, such as:
  • GIF animation, CSS animations, scrolling
• We can also potentially use display lists to paint off the main thread

What can we gain from display lists?

• Many optimizations we can fit in to our existing code as one-off optimizations, but display lists provide a common way of reasoning about these optimizations
• Make WK code more efficient by eliminating useless repaints, using precomputed display lists
  • Better culling (accurate extent information): we can use the display list to compute the exact bounds of what needs to be repainted, allowing us to tighten the repaint rect
  • Can optimize noops out of the painting code
  • Occlusion detection and clipping
  • Can reorder operations for better batching
• Reuse display lists in a single painting pass (e.g. SVG images, <use>)
• Can keep display lists around between painting passes (e.g. across different tiles)
• We have opportunities to cache display lists
  • Text rendering: don’t have to recompute kerning, glyphs (?)
• We can log all the painting commands that happen
  • Developer tools implications: can use display list logs to tell which elements are the most costly to render, or perhaps scrub through the painting of a page
  • Can also use display lists to help layout testing

What are the biggest challenges?

• For painting off the main thread: CA doesn’t play well with this, GraphicsContext is non-threadsafe
• Map of RenderObject -> DisplayItem could use a lot of memory
• Measuring performance gain: often run into unexpected perf issues, weird/unexpected CG behaviors
• Testing: make sure the switch to display lists doesn’t introduce regressions

The prototype

• For a particular call into GraphicsContext, we record what the GraphicsContext does
• Packages up params of calls to GraphicsContext
• updateItemExtent(DrawingItem): This gets the local bounds of the item and maps into global coordinates taking transforms/shadows into account

Performance data

• Perf relative to non-display-list mode is similar on iPad and iMac
• Text is strange – should be much faster, but we seem to spend a lot of time waiting for CG
• In cases where perf goes down, we spend less time in web code – we seem to get throttled by CG
• Paint time dwarfs record time
• Even if we don’t see a performance gain, we might be using fewer CPU cycles, so less power

Next steps

• Contact other teams (e.g. CA). Maybe get a way from CA to draw from another thread?
• Continue investigating perf regressions vs. normal rendering
• Consider cleaning up GraphicsContext
• Form controls currently grab the platform context out of the context, so they need to be optimized to work with display lists

Other comments

• Renderer knows about the display lists it generates
• Geoff: if we can fix the 2 bugs with display lists, it would give us matching and/or better perf across the board
• Sam: may want to consider a tree structure for when we want to mutate display list operations efficiently
• Simon: maybe lower the level of diplay list recording to the platform context