Overview and Description of the Performance Utilities
The QtWebKit Performance git repository contains a set of host utilities, manual tests and benchmarks that should help to profile QtWebKit in a stable environment. The host utilities are responsible for mirroring real webcontent, serving it with a special webserver and providing analysis application to look at the results. The test applications allow to stress test QtWebKit and the performance tools provide benchmarks that focus on specific areas of QtWebKit.
For profiling it is crucial to control the network and the content. Everyone should use the same content to have a comparable result, everyone should have a comparable network connection (bandwidth and latency). To achieve these goals real web content is mirrored and served from a local link. The mirror and http_server host utility help to achieve this goal.
Host Utilities
The mirroring application
The mirroring application can be found in the host-tools/mirror/ directory and allows to mirror web content to a local SQLite3 database. It is possible to use the cookiejar of Arora to reuse the login cookies and mirror logged in pages of services like facebook.com, gmail.com. The SQLite3 database can be served by the http_server.
The do_mirror.sh script is included in the distribution to create clones of some famour websites. The list was collected by personal preference and the alexa.com list of popular websites. When invoking do_mirror.sh all arguments are passed on to the mirror application. This allows to use the Arora cookie jar for mirroring the whole set of pages.
The mirroring application is implemented as a QNetworkAccessManager that is creating QNetworkReply proxies and is writing out the received raw HTTP headers and the read content.
For a small run through using the mirror application please see this guide.
Running the tools:
$ cd host-tools/mirror/
# Help of mirror
$ ./mirror options [url]
-c cookies.ini Use the cookies from this file.
The cookie file is compatible with Arora.
-v Show the WebView when running
-k Keep the application running.
# Mirror the gmail login sequence... keep it running and show the page
$ ./mirror -v -k -c $HOME/.local/share/data/Arora/cookies.ini http://www.gmail.com
...
CTRL-C
# Mirror a set of webpages using cookies from Arora
$ ./do_mirror -c $HOME/.local/share/data/Arora/cookies.ini
...
# The database can be found here
$ ls crawl_db.db
crawl_db.db
# Change some content inside the db
$ ./manipulate-content.py --db crawl_db.db
# Create a hosts entry for the database.
# This should be placed in /etc/hosts of the system running the tests.
$ ./create_hosts -d ./crawl_db 10.23.23.23
10.23.23.23 www.gmail.com
10.23.23.23 mail.google.com
10.23.23.23 www.google.com
10.23.23.23 ssl.google-analytics.com
10.23.23.23 ajax.googleapis.com
10.23.23.23 clients1.google.com
10.23.23.23 www.kde.org
The http_server
The http_server application is a simple multi threaded HTTP server. It is looking up the requested URL in the specified database and then sending out the stored http header and content. Executing the http_server requires root privileges as it attempts to bind to port 80.
$ cd host-tools/http_server # Starting the server. It attempts to bind to port 80. $ sudo ./http_server ../mirror/crawl_db.db Seving crawl db: '../mirror/crawl_db.db'
The memusgatestat utility
The memusagestat utility was extracted from the glibc sourcecode. It is capable of creating a graph from a trace recorded by the libmemusage.so.
$ MEMUSAGE_OUTPUT=some.result LD_PRELOAD=/lib/libmemusage.so ./some_application $ ./memusagestat -o some.result.png some.result
Recording paint traces
In Qt 4.6 a new graphicssystem called trace was introduced. It allows to record all drawing operations into a trace buffer, store it to disk. Later it can be replayed with the qttracereplay utility of Qt.
$ ./my-qt-app -graphicssystem trace $ qttracereplay the_trace_file
Test Applications
An example launcher
The example launcher is provided as a simple testing tool to display one website. The to be displayed site can be requested on the command line.
$ cd tests/launcher $ ./launcher http://www.webkit.org
A stress testing application
The stress testing application can be found in the robustness/ directory. The application starts with one website and then picks any of the links on the website and navigates to this link. The goal of this application is to provide crashes, load failures, or creating a bad state inside WebCore.
$ cd tests/robustness
# Load various websites
$ ./robustness
("./robustness")
New target "http://www.digg.com"
New target "http://digg.com/space/Space_radiation_hits_record_high"
...
Performance tools
The loading test
The loading test can be found in the loading/ subdirectory. The goal of this benchmark is to measure and analayse the loading of web pages. The loading is waiting for the QWebPage::loadFinished signal and the webpage is not shown to avoid to paint the page as painting will be tested separately.
$ cd tests/loading
$ ./tst_loading
./tst_loading
********* Start testing of tst_Loading *********
Config: Using QTest library 4.5.2, Qt 4.5.2
PASS : tst_Loading::initTestCase()
...
RESULT : tst_Loading::load():"row34":
2,889 msec per iteration (total: 2889, iterations: 1)
PASS : tst_Loading::load()
...
PASS : tst_Loading::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of tst_Loading *********
The cycling test
The cycling test is a lot like loading but it also includes the painting done on first load. This makes it a mix of the painting and loading test and is the closest to a user typing in a url and loading.
$ cd tests/cycler $ ./tst_cycler
The painting test
The painting test can be found in the painting/ subdirectory. The goal of this benchmark is to measure the painting performance. For this the webpage is loaded and then viewport (first part of the page) is drawn repeatedly to a QPixmap. The painting is finished with a call to QApplication::syncX which should force the painting to happen.
$ cd tests/painting $ ./tst_painting ...
The scrolling test
The scrolling test can be found in the scrolling/ subdirectory. The goal of this benchmark is to measure the scrolling performance. The benchmark will load the site, wait for it to be loaded. Then it will measure the time it takes to scroll down and up. Scrolling is used as a test as most people reading a website will scroll down to read the text or search for the text. Scrolling will include painting, but depending on the content might also do further JavaScript processing. QCoreApplication::processEvents() is used after scrolling to force handling of events.
$ cd tests/scrolling $ ./tst_scrolling
