Changes between Initial Version and Version 1 of Instrumentation and Metrics

Timestamp:

Apr 20, 2012 10:08:49 AM (12 years ago)

Author:

rniwa@webkit.org

Comment:

--

Instrumentation and Metrics

@@ +1 @@
+Instrumentation and metrics at 2012 contributor's meeting
+
+Host: morrita
+
+This session is measuring performance metrics such as the time to render, layout, and paint pages. We already have backend for mesuring such metrics in inspectors but not all metrics are available on all ports.
+
+There are other metrics we want to instrument such as which APIs are used so that some of them can be deprecated; i.e. usage metrics.
+
+Right now, we can't answer questions such as now many render objects and nodes each page has. We don't know the scale to which we need to handle. e.g. do we need to be able to scale our engine to handle 10,000? or million?
+
+On mobile side, we want to know what layout width is expected for each website. It'll be nice to know the layout width of engines across different ports.
+
+There are two kinds of instrumentations. One that developer uses to debug data and other is instrumentation emebedded in the end-user to report real-world data.
+
+Chromium exposes instrumentation mechanism for end-users. Other ports can use it as well.  WebCore platform histrogram mechanism exposes API to WebCore. The current API only works with numeric values and "keys" need to be pre-defined.
+
+Firefox implemented user-initiated actions that are taking too long; they store stacks, etc... Unknown as to whether they have been useful yet.
+
+We may also want to collect performance profiles such as ones that Inspector exposes.  They can also be used in performance tests as well like page-load-time tests (e.g. page cycler, PLT, etc...) Can it be avaialble for non-Chromium DRTs as well?
+
+Navigation Timing API (window.performance.X) exposd to JavaScript. Inspector already exposes this information so DRT already has means to obtain this value.
+
+Network timing. It's hard because of noise. While we do want to minimize the measurement bias, too much noise makes the results unusable. Network latency, etc... changes day to day. Maybe this is out of the scope for WebKit?
+
+Maybe good to distinguish ones WebKit measures and embedders measure.
+
+Developer wants more information. They want to see more metrics. Maybe there's more information we should expose via Inspector. Developers want to know the fine-grained memory usage. We need to expose information at different abstraction layers (big picture vs. details) depending on the needs and expertise of developers.
+
+Can we expose render tree information via Inspector?  For WebKit development, we want to measure the number of render objects, inline boxes, render arena, etc... Also concerned about things being too invasive and affecting hot-path.
+
+Having insights into memory layout, etc.. will be helpful especially with respect to accelearted compositing.
+
+Should we use Inspector infrastrucutre for instrumentation or should we have some WebCore-specific API?  Maybe we can introduce some abstraction so that the performance-related measurements can be doen in WebCore and then be consumed by Inspector or DRT.
+
+Inspector has frontend / backend separation. They currently have one-ot-one relationship. If we wanted to add new API, maybe we need to support one-to-many relationship (many frontends). There is already an interest to develop a new frontend for Chromium's exntension API. Chromium already has a debugger API that exposes information the Inspector backend collected.
+
+Metrics inspectors collect
+Heap memory size (V8; some work done in JSC but it's hard due to design differences)
+CPU time (V8)
+Live DOM node counts
+
+For DRT, we can detect whether we've detected any objects or not. We could add some DRT API to cleanup all intentionally leaked objects to find unintentional leaks.
+
+There needs to be a way to detect memory leaks so as to help prgoression/regression during the development. Leak bot gives high-level view but not helpful in terms of diagnosing the problem.
+
+Chromium has some user metric instrumentation mechanism. Are there other ports that measure user metrics? BlackBerry has some instrumentation mechanism but doesn't have a way to publicly access that. Firefox also collects performance metrics.