Changeset 190645 in webkit

Timestamp:

Oct 6, 2015 2:04:03 PM (9 years ago)

Author:

rniwa@webkit.org

Message:

Perf dashboard should have the capability to test local UI with production data
​https://bugs.webkit.org/show_bug.cgi?id=149834

Reviewed by Chris Dumez.

Added tools/run-with-remote-server.py which runs a local httpd server and pulls data from a remote server.

• Install.md: Added the instruction on how to use the script. Also updated the remaining instructions

for El Capitan.

• config.json: Added remote server configurations.
• public/admin/fetch-from-remote.php: Added. This script fetches JSON from the remote server specified in

config.json and caches the results in the location specified as "cacheDirectory" in config.json.
(main):
(fetch_remote):

• public/include/db.php:

(config_path): Extracted from generate_data_file.
(generate_data_file):

• tools/remote-server-relay.conf: Added. Apache 2.4 configuration file for a local http server launched by

run-with-remote-server.py.

• tools/run-with-remote-server.py: Added. Launches Apache with the right set of directives.

(main):
(abspath_from_root):

Location:

trunk/Websites/perf.webkit.org

Files:

• ChangeLog (modified) (1 diff)
• Install.md (modified) (6 diffs)
• config.json (modified) (1 diff)
• public/admin/fetch-from-remote.php (added)
• public/include/db.php (modified) (1 diff)
• tools/remote-server-relay.conf (added)
• tools/run-with-remote-server.py (added)

trunk/Websites/perf.webkit.org/ChangeLog

@@ +1 @@
+2015-10-06  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Perf dashboard should have the capability to test local UI with production data
+        https://bugs.webkit.org/show_bug.cgi?id=149834
+
+        Reviewed by Chris Dumez.
+
+        Added tools/run-with-remote-server.py which runs a local httpd server and pulls data from a remote server.
+
+        * Install.md: Added the instruction on how to use the script. Also updated the remaining instructions
+        for El Capitan.
+        * config.json: Added remote server configurations.
+        * public/admin/fetch-from-remote.php: Added. This script fetches JSON from the remote server specified in
+        config.json and caches the results in the location specified as "cacheDirectory" in config.json.
+        (main):
+        (fetch_remote):
+        * public/include/db.php:
+        (config_path): Extracted from generate_data_file.
+        (generate_data_file):
+        * tools/remote-server-relay.conf: Added. Apache 2.4 configuration file for a local http server launched by
+        run-with-remote-server.py.
+        * tools/run-with-remote-server.py: Added. Launches Apache with the right set of directives.
+        (main):
+        (abspath_from_root):
+
 2015-07-13  Ryosuke Niwa  <rniwa@webkit.org>
 

trunk/Websites/perf.webkit.org/Install.md

@@ -1 +1 @@
 # Checking Out the Code and Installing Required Applications
 
-The instructions assume you're using Mac OS X (Mavericks for Server.app case and Mountain Lion without Server.app) as the
-host server, and assume that we're installing this application at `/Volumes/Data/perf.webkit.org`.
+The instructions assume you're using Mac OS X as the host server and installing this application at `/Volumes/Data/perf.webkit.org`.
 
 You can choose between using Server.app or install the required tools separately
@@ -10 +9 @@
 3. Install Xcode with command line tools (only needed for svn)
 4. `svn co https://svn.webkit.org/repository/webkit/trunk/Websites/perf.webkit.org /Volumes/Data/perf.webkit.org`
-5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg`.
+5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg` and `mkdir -m 755 public/data/`
+
+# Testing Local UI Changes with Production Data
+
+The front end has the capability to pull data from a production server without replicating the database locally on OS X (Yosemite and later).
+To use this feature, modify `config.json`'s `remoteServer` entry so that "remoteServer.url" points to your production server,
+and "remoteServer.basicAuth" specifies the username and the password that is used by the production sever.
+
+Remove "basicAuth" entry for production servers that doesn't require a basic authentication (e.g. perf.webkit.org).
+
+```json
+{
+    "url": "http://perf.webkit.org",
+    "basicAuth": {
+        "username": "webkitten",
+        "password": "webkitten's secret password"            
+    }
+}
+```
+
+Then run `tools/run-with-remote-server.py`. This launches a httpd server on port 8080.
+
+The initial few page loads after starting the script could take as much as a few minutes depending on your production sever's configurations
+since Apache needs to start a pool of processes. Reloading the dashboards few times should bring the load time under control.
+
+The script caches remote server's responses under `public/data/remote-cache` and never revalidates them (to allow offline work).
+If you needed the latest content, delete caches stored in this directory by running `rm -rf public/data/remote-cache`.
 
 
@@ -21 +46 @@
  - Restarting httpd: `sudo apachectl restart`
 
+The apache logs are located at `/private/var/log/apache2`.
+
 ## Instructions if you're using Server.app
 
  - Enable PHP web applications
- - Go to Server Website / Store Site Files In, change it to /Volumes/Data/perf.webkit.org/public/`
+ - Go to Server Website / Store Site Files In, change it to `/Volumes/Data/perf.webkit.org/public/`
  - Go to Server Website / Edit advanced settings, enable Allow overrides using .htaccess files
+ - httpd config file is located at `/Library/Server/Web/Config/apache2/sites/0000_any_80.conf` (and/or 0000_any_`PORT#`.conf)
 
 ## Instructions if you're not using Server.app
@@ -32 +60 @@
 
      1. Change DocumentRoot to `/Volumes/Data/perf.webkit.org/public/`
-     2. Uncomment `"LoadModule php5_module libexec/apache2/libphp5.so"`
+     2. Uncomment `LoadModule php5_module libexec/apache2/libphp5.so`
+     3. Uncomment `LoadModule rewrite_module libexec/apache2/mod_rewrite.so`
+     4. Uncomment `LoadModule deflate_module libexec/apache2/mod_deflate.so`
 
  - In Mavericks and later, copy php.ini to load pdo_pgsql.so pgsql.so.
     `sudo cp /Applications/Server.app/Contents/ServerRoot/etc/php.ini /etc/`
+ - In El Capitan and later, comment out the `LockFile` directive in `/private/etc/apache2/extra/httpd-mpm.conf`
+   since the directive has been superseded by `Mutex` directive.
 
-
-## Common directives for the related apache config file
-
-  httpd config file is located at:
-
-    - With Server.app: /Library/Server/Web/Config/apache2/sites/0000_any_80.conf (and/or 0000_any_`PORT#`.conf)
-    - Without: /private/etc/apache2/httpd.conf
+## Production Configurations
 
  1. Update ServerAdmin to your email address
@@ -57 +83 @@
         php_flag zlib.output_compression on
 
-The apache logs are located at `/private/var/log/apache2`.
-
-
-# Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
+### Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
 
 By default, the application gives the administrative privilege to everyone. Anyone can add, remove, or edit tests,
-builders, and other entities in the database and may even execute arbitrary JavaScript on the server via aggregators.
+builders, and other entities in the database.
 
 We recommend protection via Digest Auth on https connection.
@@ -98 +121 @@
 `/Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user -f init-database.sql`
 
-## Making a Backup of the Database
+## Making a Backup and Restoring
 
 Run `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump -h localhost --no-owner -f <filepath> webkit-perf-db | gzip > backup.gz`
 
-To restore, setup a new database and run `gunzip backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`
+To restore, setup a new database and run
+`gunzip backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`

trunk/Websites/perf.webkit.org/config.json

@@ -14 +14 @@
         "port": 80
     },
+    "cacheDirectory": "public/data/remote-cache/",
+    "remoteServer": {
+        "httpdConfig": "tools/remote-server-relay.conf",
+        "httpdPID": "tools/remote-server-relay.pid",
+        "httpdErroLog": "tools/remote-server-relay.log",
+        "url": "http://perf.webkit.org",
+        "basicAuth": {
+            "username": "username",
+            "password": "password"
+        }
+    },
     "defaultDashboard": [[]],
     "dashboards": {}

trunk/Websites/perf.webkit.org/public/include/db.php

@@ -37 +37 @@
 }
 
+function config_path($key, $path) {
+    return CONFIG_DIR . config($key) . '/' . $path;
+}
+
 function generate_data_file($filename, $content) {
     if (!assert(ctype_alnum(str_replace(array('-', '_', '.'), '', $filename))))
         return FALSE;
-    return file_put_contents(CONFIG_DIR . config('dataDirectory') . '/' . $filename, $content);
+    return file_put_contents(config_path('dataDirectory'. $filename), $content);
 }