wiki:InvestigatingLeaksAndBloat

Version 2 (modified by Simon Fraser, 3 years ago) (diff)

--

Investigating Leaks and Bloat

by Simon Fraser

  • Or… a bunch of half-baked memory tools
  • Or… don’t be afraid to change code to collect data
    • Don’t be scared to do this
  • Starting problem
  • “WebContent physical footprint doesn’t go back down significantly after going from nytimes.com to a simple page (about:blank)”
  • When we say memory use is too high we look at Physical Footprint metric
  • When measuring memory: Beware of caches
  • Proper steps
    • Load about:blank
    • Get physical footprint with vmmap
    • Load nytimes.com
    • Go back to about:blank
    • Fire memory warning
    • Get physical footprint with vmmap
  • “Something is not going away”
    • Leaks?
  • Tool called “leaks”: leaks <pid>
  • Adding Quick & Easy “State of the world” Data Gathering
  • Handy too on Mac
    • notifyutil -p
    • Broadcasts notifications to processes, processes can register handlers
    • Handles:
      • org.WebKit.lowMemory
      • com.apple.WebKit.fullGC
      • com.apple.WebKit.deleteAllCode
      • com.apple.WebKit.Cache.dump
      • com.apple.WebKit.showAllDocuments*
      • com.apple.WebKit.showMemoryCache*
      • com.apple.WebKit.showPageCache*
      • * added in last 6 months
  • notifyutil -p com.apple.WebKit.showAllDocuments
  • Why aren’t they leaks?
    • Leaks tool looks for rooted graphs of objects
  • This is really “Abandonment” or “leaks”
  • So what objects are hanging around?
    • Object Counters
  • Is this a JS heap issue, or a C++ ref-counting issue?
  • Which world is the issue happening in?
    • Not always easy to tell
    • Do the JS* objects go away?
    • Did all the doc nodes get destroyed?
      • notifyutil -p com.apple.WebKit.showAllDocuments
      • Look at referencingNodeCount
  • JS Heap
    • GC
    • Visist s every object that JS knows about (as JSCell*), starting at GC roots
      • JSC::Heap, JSC::SlotVisitor, visistChildren() methods
    • Unvisisted objects can be GC
    • Doesn’t care about unrooted cycles
  • GC Heap Inspector
    • Heap Snapshots in Web Inspector
    • Not quite enough info, no C++ pointer objects
  • notifyutil -p com.apple.WebKit.dumpGCHeap
  • Checked-in: https://trac.webkit.org/r235271
  • GC-related Document Leaks
    • DFG Scratch buffer
    • Exception backtraces
    • VM lastException
    • JSPerformanceObserverCallback: Strong<Function>
      • Strong references indicate GC problems
  • C++ leaks
    • Unbalanced ref()/deref(), retain reference cycles
    • refCount 1 indicates a C++ leak
  • Ref tracking
    • Collate matching ref() and deref() via “tokens”
  • Ref Token Tracking
    • Fix Ref<> and RefPtr<> to store tokens
      • All ref() and deref() via Ref<> and RefPtr<> are tracked
  • RefTracker class example
  • Call stack records the ref
  • StackShot is your friend
    • Wrapper for WTFGetBacktrace()
    • Hashable
    • Great for “accumulate all the calls stacks that did this thing”
    • Controllable stack depth
  • TrackedRefCounted<>
  • Dumping Unmatched Refs
    • RefTracker::dumpRemainingReferences()
  • RefTracker is powerful!
    • Sometimes dumps the only unmatched ref() callstack!
    • Easy to patch up manual ref()/deref() code paths by tracking RefTrackingTokens
  • webkit.org/b.186269 (reviewz plz)
    • Make it possible to track unbalanced ref()/deref()
  • RefTracker on Node
    • Give Node a RefTracker
    • Tracks Nodes, Elements, Documents
    • Now we can track Document refs
    • Hack the “com.apple.WebKit.showAllDocuments” callback to dump ref stacks for live Documents
    • You can turn it on and its not a huge perf hit
  • Making leaks not happen again
    • Used GCHeap Inspector
    • Fixed a leak
  • “Finding a regression in EWS is 1000x more efficient than finding it via a perf benchmark regression” — Simon Fraser
  • webkit.org/b/186214
  • Isn’t this like the old “world leaks”?
    • Sort of is…
    • Leaks of world-level objects
  • Checking for Leaks in Testing
    • ./Tools/Scripts/run-webkit-tests --world-leaks
  • How it Works
    • After each test
      • WKBundleGetLiveDocumentURLs()
  • Implications for run-webkit-tests
    • New failure type: “LEAK”
    • “LEAK” only relevant if you ran with --world-leaks
    • Leak checking is post-hoc
    • A test can pass, and then become a leak failure
  • Remaining Work
    • Add LEAK expectations for all platforms
    • Fix the causes of most common document leaks
    • Fix causes of false Leaks
    • Turn on by default
  • C++ Document Leaks
    • Web Animations
    • Document::removeFocusNavigationNodeofSubtree()
    • Content Filtering
    • IndexedDB
    • SVG text tests
    • WebFullScreenManager
  • C++ Document Lifetime Extenders
    • NavigationAction
    • Editor/Selection
    • ServicesOverlayController
    • DragController
  • To Do
    • When WebKitTestRunner detects a leak:
      • Dump unmatched refs when tests detect a leak
      • Dump a GC heap
    • Some auto analysis?
  • Halp
    • webkit.org/b/186214
  • Now the memory problems are solved, right?
    • Not really, still some issues in vmmap
    • MALLOC zones
    • Dirty size column is interesting, WebKit MALLOC (411 MB)
  • “Where does all the memory go?”
    • webkit.org/b/186422 Create lots of different malloc zones for easier accounting of memory use
    • Created custom MALLOC zones for WebKit
    • Big ones
    • Vector capacity
    • HashTable
    • webkit.org/b/186698 Make it possible to track all sites that waste container capacity
  • Container Capacity Tracking
    • Give the Vector/HashTable an ID (can’t track by address)
    • Record allocation with a StackShot
    • On deallocation remove the record
    • Dump living capacity:
      • Aggregate by StackShot
      • Sort by most to least wasteful
  • Consider not using as many HashTables?
    • Maybe use different types of tables?
  • Memory Reduction
    • bmalloc space efficiency
      • Maybe as good as system malloc? Problem if using both, using one is fine
    • JSC optimizations for space efficiency (even with the JIT)
    • WebCore optimizations for space efficiency
    • Review Vector and HashTable growth policies
      • Resiable hashes (Robin Hood hashing?)
    • Vector shrinking
    • Convert from HashMaps and HashSets to other data structures
    • Adjust caching policies
  • Better Low Memory Handler
    • Don’t dirty more pages than you release
      • Organize data structures to release memory without visiting pages
    • More use of purgeable memory
  • WebProcess watermark (big 2.5 GB?)
  • Space-Efficient Class Layout
    • Micro-optimization?
    • dump-class-layout -c Release WebCore FillLayer
    • Look for <Padding: n bytes> entries
    • Also look at <UNUSED BITS: n bits>
  • std::optional causes these to get big
    • There’s been work on different optional types
    • Optional with a magic value (doesn’t waste any space)
    • Patch for a set of optionals in a bit field
  • Beware of enums
    • For enums, you should specify size
  • Big Stuff
    • Don’t just use all available memory
    • Manage cache sizes
      • Cache budgets
      • JS Heap size
  • Don’t Cause Leaks
    • Be careful with JS bindings code
      • JSC::strong<> is almost always wrong
    • Be careful with holding refs to Documents, elements
      • Avoid cycles
      • Avoid extending object lifetimes
  • Don’t write floaty classes
    • Shrink vectors, avoid unused inline capacity on heap-allocated objects
    • Do you really need to use HashTable?
    • Beware enums and std::optionals
  • Look for Leaks
  • Tools for investigating Leaks
    • cCache dumping
    • GC heap inspector
    • Ref token tracking