== Overview and Description of the Performance Utilities ==

The [http://gitorious.org/qtwebkit/performance QtWebKit Performance git repository] contains a set of host utilities, manual tests and benchmarks that should help to profile QtWebKit in a stable environment. The host utilities are responsible for mirroring real webcontent, serving it with a special webserver and providing analysis application to look at the results. The test applications allow to stress test QtWebKit and the performance tools provide benchmarks that focus on specific areas of QtWebKit.

For profiling it is crucial to control the network and the content. Everyone should use the same content to have a comparable result, everyone should have a comparable network connection (bandwidth and latency). To achieve these goals real web content is mirrored and served from a local link. The '''mirror''' and '''http_server''' host utility help to achieve this goal.

=== Host Utilities ===
==== The mirroring application ====

The mirroring application can be found in the '''host-tools/mirror/''' directory and allows to mirror web content to a local SQLite3 database. It is possible to use the cookiejar of Arora to reuse the login cookies and mirror logged in pages of services like ''facebook.com'', ''gmail.com''. The SQLite3 database can be served by the ''http_server''.

The ''do_mirror.sh'' script is included in the distribution to create clones of some famour websites. The list was collected by personal preference and the alexa.com list of popular websites. When invoking ''do_mirror.sh'' all arguments are passed on to the ''mirror'' application. This allows to use the Arora cookie jar for mirroring the whole set of pages.

The mirroring application is implemented as a QNetworkAccessManager that is creating QNetworkReply proxies and is writing out the received raw HTTP headers and the read content.

For a small run through using the '''mirror''' application please see [wiki:QtWebKitMirrorGuide this guide].

Running the tools:
{{{
$ cd host-tools/mirror/
# Help of mirror
$ ./mirror options [url]
        -c cookies.ini  Use the cookies from this file.
                        The cookie file is compatible with Arora.
        -v              Show the WebView when running
        -k              Keep the application running.

# Mirror the gmail login sequence... keep it running and show the page
$ ./mirror -v -k -c $HOME/.local/share/data/Arora/cookies.ini http://www.gmail.com
...
CTRL-C

# Mirror a set of webpages using cookies from Arora
$ ./do_mirror -c $HOME/.local/share/data/Arora/cookies.ini
...

# The database can be found here
$ ls crawl_db.db
crawl_db.db

# Create a hosts entry for the database.
# This should be placed in /etc/hosts of the system running the tests.
$ ./create_hosts -d ./crawl_db 10.23.23.23
10.23.23.23     www.gmail.com
10.23.23.23     mail.google.com
10.23.23.23     www.google.com
10.23.23.23     ssl.google-analytics.com
10.23.23.23     ajax.googleapis.com
10.23.23.23     clients1.google.com
10.23.23.23     www.kde.org
}}}

==== The http_server ====

The http_server application is a simple multi threaded HTTP server. It is looking up the requested URL in the specified database and then sending out the stored http header and content. Executing the ''http_server'' requires root privileges as it attempts to bind to port 80.

{{{
$ cd host-tools/http_server

# Starting the server. It attempts to bind to port 80.
$ sudo ./http_server ../mirror/crawl_db.db
Seving crawl db: '../mirror/crawl_db.db'
}}}

==== The memusgatestat utility ====

The ''memusagestat'' utility was extracted from the glibc sourcecode. It is capable of creating a graph from a trace recorded by the libmemusage.so.

=== Test Applications ===
==== An example launcher ====
The example launcher is provided as a simple testing tool to display one website. The to be displayed site can be requested on the command line.

{{{
$ cd tests/launcher
$ ./launcher http://www.webkit.org
}}}

==== A stress testing application ====
The stress testing application can be found in the '''robustness/''' directory. The application starts with one website and then picks any of the links on the website and navigates to this link. The goal of this application is to provide crashes, load failures, or creating a bad state inside WebCore.

{{{
$ cd tests/robustness
# Load various websites
$ ./robustness
("./robustness") 
New target "http://www.digg.com"
New target "http://digg.com/space/Space_radiation_hits_record_high" 
...
}}}

=== Performance tools ===
==== The loading test ====
The loading test can be found in the '''loading/''' subdirectory. The goal of this benchmark is to measure and analayse the loading of web pages. The loading is waiting for the ''QWebPage::loadFinished'' signal and the webpage is not shown to avoid to paint the page as painting will be tested separately.

{{{
$ cd tests/loading
$ ./tst_loading
./tst_loading 
********* Start testing of tst_Loading *********
Config: Using QTest library 4.5.2, Qt 4.5.2
PASS   : tst_Loading::initTestCase()
...

RESULT : tst_Loading::load():"row34":
     2,889 msec per iteration (total: 2889, iterations: 1)
PASS   : tst_Loading::load()
...

PASS   : tst_Loading::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of tst_Loading *********
}}}

==== The painting test ====
The painting test can be found in the '''painting/''' subdirectory. The goal of this benchmark is to measure the painting performance. For this the webpage is loaded and then viewport (first part of the page) is drawn repeatedly to a QPixmap. The painting is finished with a call to ''QApplication::syncX'' which should force the painting to happen.

{{{
$ cd tests/painting
$ ./tst_painting
...
}}}

==== The scrolling test ====
The scrolling test can be found in the '''scrolling/''' subdirectory. The goal of this benchmark is to measure the scrolling performance. The benchmark will load the site, wait for it to be loaded. Then it will measure the time it takes to scroll down and up. Scrolling is used as a test as most people reading a website will scroll down to read the text or search for the text. Scrolling will include painting, but depending on the content might also do further JavaScript processing. ''QCoreApplication::processEvents()'' is used after scrolling to force handling of events.

{{{
$ cd tests/scrolling
$ ./tst_scrolling
}}}