Changeset 263712 in webkit


Ignore:
Timestamp:
Jun 29, 2020 6:12:31 PM (4 years ago)
Author:
Tadeu Zagallo
Message:

New API benchmark
​https://bugs.webkit.org/show_bug.cgi?id=213750

Reviewed by Saam Barati.

APIBench is a new benchmark suite to measure the performance of the JavaScriptCore API. It has 4 benchmarks:

  • RichardsJS: This is imported from Octane and used as a reference so we know how fast the pure JS implementation runs.
  • RichardsSwift: a Swift port of the benchmark, also used as reference for how fast the pure Swift implementation is.
  • RichardsMostlyJS: As the name suggests, mostly the same the as the JS benchmark, but here the WorkerTask is implemented in Swift. The API is used to write to the packet being processed and talking to the scheduler.
  • RichardsMostlySwift: inverse of the above. Mostly the same code from the Swift benchark, but here the WorkerTask is implemented in JavaScript. The API is used to run the JS version of the task and marshaling the Packet by using JSExport.

The benchmark can be built and run using the api-bench script, as following:

PerformanceTests/APIBench/api-bench WebKitBuild/Release

By default, it will build each of the benchmarks and run 5 iterations.
The score is calculated as 5000 / geomean([avg of RichardsMostlySwift, avg of RichardsMostlyJS]).
The script prints the average for the references and tests and the final score:

Results for /Volumes/Data/wk2/OpenSource/WebKitBuild/Release (5 iterations)
================================================================================

References:

RichardsJS: 14ms
RichardsSwift: 29ms

Tests:

RichardsMostlyJS: 218ms
RichardsMostlySwift: 315ms

Score: 19.0804

  • APIBench/RichardsJS/RichardsJS.js: Added.

(runRichards):
(Scheduler):
(Scheduler.prototype.addIdleTask):
(Scheduler.prototype.addWorkerTask):
(Scheduler.prototype.addHandlerTask):
(Scheduler.prototype.addDeviceTask):
(Scheduler.prototype.addRunningTask):
(Scheduler.prototype.addTask):
(Scheduler.prototype.schedule):
(Scheduler.prototype.release):
(Scheduler.prototype.holdCurrent):
(Scheduler.prototype.suspendCurrent):
(Scheduler.prototype.queue):
(TaskControlBlock):
(TaskControlBlock.prototype.setRunning):
(TaskControlBlock.prototype.markAsNotHeld):
(TaskControlBlock.prototype.markAsHeld):
(TaskControlBlock.prototype.isHeldOrSuspended):
(TaskControlBlock.prototype.markAsSuspended):
(TaskControlBlock.prototype.markAsRunnable):
(TaskControlBlock.prototype.run):
(TaskControlBlock.prototype.checkPriorityAdd):
(TaskControlBlock.prototype.toString):
(IdleTask):
(IdleTask.prototype.run):
(IdleTask.prototype.toString):
(DeviceTask):
(DeviceTask.prototype.run):
(DeviceTask.prototype.toString):
(WorkerTask):
(WorkerTask.prototype.run):
(WorkerTask.prototype.toString):
(HandlerTask):
(HandlerTask.prototype.run):
(HandlerTask.prototype.toString):
(Packet):
(Packet.prototype.addTo):
(Packet.prototype.toString):

  • APIBench/RichardsMostlyJS/RichardsMostlyJS.xcodeproj/project.pbxproj: Added.
  • APIBench/RichardsMostlyJS/RichardsMostlyJS.xcodeproj/xcshareddata/xcschemes/RichardsMostlyJS.xcscheme: Added.
  • APIBench/RichardsMostlyJS/RichardsMostlyJS/main.swift: Added.

(run(_:)):
(toString):
(WorkerTask.toString):
(WorkerTask.run(_:)):

  • APIBench/RichardsMostlyJS/RichardsMostlyJS/richards.js: Added.

(runRichards):
(Scheduler):
(Scheduler.prototype.addIdleTask):
(Scheduler.prototype.addWorkerTask):
(Scheduler.prototype.addHandlerTask):
(Scheduler.prototype.addDeviceTask):
(Scheduler.prototype.addRunningTask):
(Scheduler.prototype.addTask):
(Scheduler.prototype.schedule):
(Scheduler.prototype.release):
(Scheduler.prototype.holdCurrent):
(Scheduler.prototype.suspendCurrent):
(Scheduler.prototype.queue):
(TaskControlBlock):
(TaskControlBlock.prototype.setRunning):
(TaskControlBlock.prototype.markAsNotHeld):
(TaskControlBlock.prototype.markAsHeld):
(TaskControlBlock.prototype.isHeldOrSuspended):
(TaskControlBlock.prototype.markAsSuspended):
(TaskControlBlock.prototype.markAsRunnable):
(TaskControlBlock.prototype.run):
(TaskControlBlock.prototype.checkPriorityAdd):
(TaskControlBlock.prototype.toString):
(IdleTask):
(IdleTask.prototype.run):
(IdleTask.prototype.toString):
(DeviceTask):
(DeviceTask.prototype.run):
(DeviceTask.prototype.toString):
(HandlerTask):
(HandlerTask.prototype.run):
(HandlerTask.prototype.toString):
(Packet):
(Packet.prototype.addTo):
(Packet.prototype.toString):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift.xcodeproj/project.pbxproj: Added.
  • APIBench/RichardsMostlySwift/RichardsMostlySwift.xcodeproj/xcshareddata/xcschemes/RichardsMostlySwift.xcscheme: Added.
  • APIBench/RichardsMostlySwift/RichardsMostlySwift/DeviceTask.swift: Added.

(DeviceTask.create(_:device:priority:)):
(DeviceTask.run(_:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/HandlerTask.swift: Added.

(HandlerTask.device):
(HandlerTask.create(_:device:priority:queue:)):
(HandlerTask.run(_:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/IdleTask.swift: Added.

(IdleTask.create(_:priority:)):
(IdleTask.run(_:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/Packet.swift: Added.

(Packet.addTo(_:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/Scheduler.swift: Added.

(waitCurrent):
(handle(_:)):
(Scheduler.schedule):
(Scheduler.add(_:)):
(Scheduler.release_(_:)):
(holdCurrent):
(queue(_:)):
(queueImpl(_:packet:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/Task.swift: Added.

(Task.run(_:)):
(Task.release_):
(Task.hold):
(Task.wait):
(Task.resume):
(Task.takePacket):
(Task.checkPriorityAdd(_:packet:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/WorkerTask.swift: Added.

(WorkerTask.create(_:priority:queue:)):
(WorkerTask.run(_:)):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/main.swift: Added.
  • APIBench/RichardsMostlySwift/RichardsMostlySwift/richards.js: Added.

(return.prototype.run):

  • APIBench/RichardsMostlySwift/RichardsMostlySwift/richards.swift: Added.

(runRichards):

  • APIBench/RichardsSwift/RichardsSwift.xcodeproj/project.pbxproj: Added.
  • APIBench/RichardsSwift/RichardsSwift.xcodeproj/xcshareddata/xcschemes/RichardsSwift.xcscheme: Added.
  • APIBench/RichardsSwift/RichardsSwift/DeviceTask.swift: Added.

(DeviceTask.create(_:device:priority:)):
(DeviceTask.run(_:)):

  • APIBench/RichardsSwift/RichardsSwift/HandlerTask.swift: Added.

(HandlerTask.device):
(HandlerTask.create(_:device:priority:queue:)):
(HandlerTask.run(_:)):

  • APIBench/RichardsSwift/RichardsSwift/IdleTask.swift: Added.

(IdleTask.create(_:priority:)):
(IdleTask.run(_:)):

  • APIBench/RichardsSwift/RichardsSwift/Packet.swift: Added.

(Packet.addTo(_:)):

  • APIBench/RichardsSwift/RichardsSwift/Scheduler.swift: Added.

(Scheduler.schedule):
(Scheduler.add(_:)):
(Scheduler.release(_:)):
(holdCurrent):
(waitCurrent):
(queue(_:)):
(handle(_:)):
(queueImpl(_:packet:)):

  • APIBench/RichardsSwift/RichardsSwift/Task.swift: Added.

(Task.run(_:)):
(Task.release):
(Task.hold):
(Task.wait):
(Task.resume):
(Task.takePacket):
(Task.checkPriorityAdd(_:packet:)):

  • APIBench/RichardsSwift/RichardsSwift/WorkerTask.swift: Added.

(WorkerTask.create(_:priority:queue:)):
(WorkerTask.run(_:)):

  • APIBench/RichardsSwift/RichardsSwift/main.swift: Added.
  • APIBench/RichardsSwift/RichardsSwift/richards.swift: Added.

(runRichards):

  • APIBench/api-bench: Added.
Location:
trunk/PerformanceTests
Files:
46 added
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/PerformanceTests/ChangeLog

    r261444 r263712  
     12020-06-29  Tadeu Zagallo  <tzagallo@apple.com>
     2
     3        New API benchmark
     4        https://bugs.webkit.org/show_bug.cgi?id=213750
     5
     6        Reviewed by Saam Barati.
     7
     8        APIBench is a new benchmark suite to measure the performance of the JavaScriptCore API. It has 4 benchmarks:
     9        - RichardsJS: This is imported from Octane and used as a reference so we know how fast the pure JS implementation runs.
     10        - RichardsSwift: a Swift port of the benchmark, also used as reference for how fast the pure Swift implementation is.
     11        - RichardsMostlyJS: As the name suggests, mostly the same the as the JS benchmark, but here the WorkerTask
     12          is implemented in Swift. The API is used to write to the packet being processed and talking to the scheduler.
     13        - RichardsMostlySwift: inverse of the above. Mostly the same code from the Swift benchark, but here the WorkerTask is
     14          implemented in JavaScript. The API is used to run the JS version of the task and marshaling the Packet by using JSExport.
     15
     16        The benchmark can be built and run using the api-bench script, as following:
     17
     18        PerformanceTests/APIBench/api-bench WebKitBuild/Release
     19
     20        By default, it will build each of the benchmarks and run 5 iterations.
     21        The score is calculated as 5000 / geomean([avg of RichardsMostlySwift, avg of RichardsMostlyJS]).
     22        The script prints the average for the references and tests and the final score:
     23
     24        Results for /Volumes/Data/wk2/OpenSource/WebKitBuild/Release (5 iterations)
     25        ================================================================================
     26
     27        References:
     28        ----------------------------------------
     29        RichardsJS: 14ms
     30        RichardsSwift: 29ms
     31
     32        Tests:
     33        ----------------------------------------
     34        RichardsMostlyJS: 218ms
     35        RichardsMostlySwift: 315ms
     36
     37        Score: 19.0804
     38
     39        * APIBench/RichardsJS/RichardsJS.js: Added.
     40        (runRichards):
     41        (Scheduler):
     42        (Scheduler.prototype.addIdleTask):
     43        (Scheduler.prototype.addWorkerTask):
     44        (Scheduler.prototype.addHandlerTask):
     45        (Scheduler.prototype.addDeviceTask):
     46        (Scheduler.prototype.addRunningTask):
     47        (Scheduler.prototype.addTask):
     48        (Scheduler.prototype.schedule):
     49        (Scheduler.prototype.release):
     50        (Scheduler.prototype.holdCurrent):
     51        (Scheduler.prototype.suspendCurrent):
     52        (Scheduler.prototype.queue):
     53        (TaskControlBlock):
     54        (TaskControlBlock.prototype.setRunning):
     55        (TaskControlBlock.prototype.markAsNotHeld):
     56        (TaskControlBlock.prototype.markAsHeld):
     57        (TaskControlBlock.prototype.isHeldOrSuspended):
     58        (TaskControlBlock.prototype.markAsSuspended):
     59        (TaskControlBlock.prototype.markAsRunnable):
     60        (TaskControlBlock.prototype.run):
     61        (TaskControlBlock.prototype.checkPriorityAdd):
     62        (TaskControlBlock.prototype.toString):
     63        (IdleTask):
     64        (IdleTask.prototype.run):
     65        (IdleTask.prototype.toString):
     66        (DeviceTask):
     67        (DeviceTask.prototype.run):
     68        (DeviceTask.prototype.toString):
     69        (WorkerTask):
     70        (WorkerTask.prototype.run):
     71        (WorkerTask.prototype.toString):
     72        (HandlerTask):
     73        (HandlerTask.prototype.run):
     74        (HandlerTask.prototype.toString):
     75        (Packet):
     76        (Packet.prototype.addTo):
     77        (Packet.prototype.toString):
     78        * APIBench/RichardsMostlyJS/RichardsMostlyJS.xcodeproj/project.pbxproj: Added.
     79        * APIBench/RichardsMostlyJS/RichardsMostlyJS.xcodeproj/xcshareddata/xcschemes/RichardsMostlyJS.xcscheme: Added.
     80        * APIBench/RichardsMostlyJS/RichardsMostlyJS/main.swift: Added.
     81        (run(_:)):
     82        (toString):
     83        (WorkerTask.toString):
     84        (WorkerTask.run(_:)):
     85        * APIBench/RichardsMostlyJS/RichardsMostlyJS/richards.js: Added.
     86        (runRichards):
     87        (Scheduler):
     88        (Scheduler.prototype.addIdleTask):
     89        (Scheduler.prototype.addWorkerTask):
     90        (Scheduler.prototype.addHandlerTask):
     91        (Scheduler.prototype.addDeviceTask):
     92        (Scheduler.prototype.addRunningTask):
     93        (Scheduler.prototype.addTask):
     94        (Scheduler.prototype.schedule):
     95        (Scheduler.prototype.release):
     96        (Scheduler.prototype.holdCurrent):
     97        (Scheduler.prototype.suspendCurrent):
     98        (Scheduler.prototype.queue):
     99        (TaskControlBlock):
     100        (TaskControlBlock.prototype.setRunning):
     101        (TaskControlBlock.prototype.markAsNotHeld):
     102        (TaskControlBlock.prototype.markAsHeld):
     103        (TaskControlBlock.prototype.isHeldOrSuspended):
     104        (TaskControlBlock.prototype.markAsSuspended):
     105        (TaskControlBlock.prototype.markAsRunnable):
     106        (TaskControlBlock.prototype.run):
     107        (TaskControlBlock.prototype.checkPriorityAdd):
     108        (TaskControlBlock.prototype.toString):
     109        (IdleTask):
     110        (IdleTask.prototype.run):
     111        (IdleTask.prototype.toString):
     112        (DeviceTask):
     113        (DeviceTask.prototype.run):
     114        (DeviceTask.prototype.toString):
     115        (HandlerTask):
     116        (HandlerTask.prototype.run):
     117        (HandlerTask.prototype.toString):
     118        (Packet):
     119        (Packet.prototype.addTo):
     120        (Packet.prototype.toString):
     121        * APIBench/RichardsMostlySwift/RichardsMostlySwift.xcodeproj/project.pbxproj: Added.
     122        * APIBench/RichardsMostlySwift/RichardsMostlySwift.xcodeproj/xcshareddata/xcschemes/RichardsMostlySwift.xcscheme: Added.
     123        * APIBench/RichardsMostlySwift/RichardsMostlySwift/DeviceTask.swift: Added.
     124        (DeviceTask.create(_:device:priority:)):
     125        (DeviceTask.run(_:)):
     126        * APIBench/RichardsMostlySwift/RichardsMostlySwift/HandlerTask.swift: Added.
     127        (HandlerTask.device):
     128        (HandlerTask.create(_:device:priority:queue:)):
     129        (HandlerTask.run(_:)):
     130        * APIBench/RichardsMostlySwift/RichardsMostlySwift/IdleTask.swift: Added.
     131        (IdleTask.create(_:priority:)):
     132        (IdleTask.run(_:)):
     133        * APIBench/RichardsMostlySwift/RichardsMostlySwift/Packet.swift: Added.
     134        (Packet.addTo(_:)):
     135        * APIBench/RichardsMostlySwift/RichardsMostlySwift/Scheduler.swift: Added.
     136        (waitCurrent):
     137        (handle(_:)):
     138        (Scheduler.schedule):
     139        (Scheduler.add(_:)):
     140        (Scheduler.release_(_:)):
     141        (holdCurrent):
     142        (queue(_:)):
     143        (queueImpl(_:packet:)):
     144        * APIBench/RichardsMostlySwift/RichardsMostlySwift/Task.swift: Added.
     145        (Task.run(_:)):
     146        (Task.release_):
     147        (Task.hold):
     148        (Task.wait):
     149        (Task.resume):
     150        (Task.takePacket):
     151        (Task.checkPriorityAdd(_:packet:)):
     152        * APIBench/RichardsMostlySwift/RichardsMostlySwift/WorkerTask.swift: Added.
     153        (WorkerTask.create(_:priority:queue:)):
     154        (WorkerTask.run(_:)):
     155        * APIBench/RichardsMostlySwift/RichardsMostlySwift/main.swift: Added.
     156        * APIBench/RichardsMostlySwift/RichardsMostlySwift/richards.js: Added.
     157        (return.prototype.run):
     158        * APIBench/RichardsMostlySwift/RichardsMostlySwift/richards.swift: Added.
     159        (runRichards):
     160        * APIBench/RichardsSwift/RichardsSwift.xcodeproj/project.pbxproj: Added.
     161        * APIBench/RichardsSwift/RichardsSwift.xcodeproj/xcshareddata/xcschemes/RichardsSwift.xcscheme: Added.
     162        * APIBench/RichardsSwift/RichardsSwift/DeviceTask.swift: Added.
     163        (DeviceTask.create(_:device:priority:)):
     164        (DeviceTask.run(_:)):
     165        * APIBench/RichardsSwift/RichardsSwift/HandlerTask.swift: Added.
     166        (HandlerTask.device):
     167        (HandlerTask.create(_:device:priority:queue:)):
     168        (HandlerTask.run(_:)):
     169        * APIBench/RichardsSwift/RichardsSwift/IdleTask.swift: Added.
     170        (IdleTask.create(_:priority:)):
     171        (IdleTask.run(_:)):
     172        * APIBench/RichardsSwift/RichardsSwift/Packet.swift: Added.
     173        (Packet.addTo(_:)):
     174        * APIBench/RichardsSwift/RichardsSwift/Scheduler.swift: Added.
     175        (Scheduler.schedule):
     176        (Scheduler.add(_:)):
     177        (Scheduler.release(_:)):
     178        (holdCurrent):
     179        (waitCurrent):
     180        (queue(_:)):
     181        (handle(_:)):
     182        (queueImpl(_:packet:)):
     183        * APIBench/RichardsSwift/RichardsSwift/Task.swift: Added.
     184        (Task.run(_:)):
     185        (Task.release):
     186        (Task.hold):
     187        (Task.wait):
     188        (Task.resume):
     189        (Task.takePacket):
     190        (Task.checkPriorityAdd(_:packet:)):
     191        * APIBench/RichardsSwift/RichardsSwift/WorkerTask.swift: Added.
     192        (WorkerTask.create(_:priority:queue:)):
     193        (WorkerTask.run(_:)):
     194        * APIBench/RichardsSwift/RichardsSwift/main.swift: Added.
     195        * APIBench/RichardsSwift/RichardsSwift/richards.swift: Added.
     196        (runRichards):
     197        * APIBench/api-bench: Added.
     198
    11992020-05-09  Don Olmstead  <don.olmstead@sony.com>
    2200
Note: See TracChangeset for help on using the changeset viewer.