Analyzing Build Performance

To effectively reduce build times, it is first important to understand where that time is being spent. This page documents various tools and techniques used to perform detailed analysis of where the complier is spending its time.

Clang 9+

Clang 9 has introduced a new flag, ​-ftime-trace, which will generate time-profile information as a .json artifact during compilation. These artifacts include timing information on a per-header, per-function, per-template, or even per-optimization level.

Building ClangBuildAnalyzer

While per-object-file trace information is useful for uncovering what causes compiling a specific source file to take as long as it does, tracing overall build time requires summarizing those traces across the entire project. The same person who originally authored the Clang tracing patch also built a summarization tool using those per-object-file traces as input, ​ClangBuildAnalyzer. Here's how to build ClangBuildAnalyzer with Xcode on macOS:

1. git clone https://github.com/aras-p/ClangBuildAnalyzer
2. cd ClangBuildAnalyzer/projects/xcode
3. xcodebuild
4. Copy build/Release/ClangBuildAnalyzer to a location in $PATH

Building WebKit with tracing enabled

1. Clean your build directory (but make sure it still _exists_).
2. ClangBuildAnalyzer --start path/to/WebKitBuild
3. make debug ARGS='OTHER_CFLAGS=-ftime-trace OTHER_CPLUSPLUSFLAGS=-ftime-trace'

Then, when the build is complete:

4. ClangBuildAnalyzer --stop path/to/WebKitBuild path/to/output/file
5. ClangBuildAnalyzer --analyze path/to/output/file > path/to/text/file

ClangBuildAnalyzer writes a file with the current time to your build directory when run with --start. Then when run with --stop, it collects all of the trace files generated during that time window, and collates them into the output file. --analyze turns that into human-readable output. Profiling individual projects within WebKit would involve running steps 1-5 from within, e.g., the Source/WebCore directory.

Resolving Expensive Headers

ClangBuildAnalyzer will generate a list of the ten most expensive (in terms of compilation time) headers encountered during the build. For example:

*** Expensive headers:
832243 ms: /Volumes/Data/WebKit/Source/WebCore/bindings/js/JSDOMGlobalObject.h (included 246 times, avg 3383 ms), included via:
  JSMallocStatistics.o JSMallocStatistics.h JSDOMWrapper.h  (6515 ms)
  JSMemoryInfo.o JSMemoryInfo.h JSDOMWrapper.h  (6490 ms)
  JSInternalSettingsGenerated.o JSInternalSettingsGenerated.h JSDOMWrapper.h  (6473 ms)
  JSApplePayCancelEvent.cpp JSApplePayCancelEvent.h JSDOMWrapper.h  (6455 ms)
  JSInternalsSetLike.o JSInternalsSetLike.h JSDOMWrapper.h  (6435 ms)
  JSMockPageOverlay.o JSMockPageOverlay.h JSDOMWrapper.h  (6430 ms)
  ...

From this we can see that JSDOMGlobalObject.h is a very expensive header; it contributes about 3.3s of compile time, on average, to every source file which includes it. And it is included 246 times, which given the WebKit unified build system, means it is included by a majority of source files in the WebCore project. For these expensive headers, its often the case that the "expensive" header is expensive due to including other expensive headers, and one approach to make that header less expensive is to forward declare types rather than include their definitions. In cases where inline implementations of methods make forward declaration impossible, those inline definitions can be moved into a <Type>Inlines.h file, and the original declarations annotated with inline. Source files which contain references to those inline functions must include the <Type>Inlines.h file, or the compile will generate a -Wundefined-inline error.