Changeset 266208 in webkit


Ignore:
Timestamp:
Aug 26, 2020 5:10:05 PM (4 years ago)
Author:
clopez@igalia.com
Message:

[GTK][WPE] Add bots for generating nightly bundle packages
​https://bugs.webkit.org/show_bug.cgi?id=215623

Reviewed by Aakash Jain.

This adds a new type of bots for GTK and WPE that runs nightly (once per day).
The goal of this bots its to build WebKit for a specific distribution (Ubuntu LTS and LTS-1)
and generate a bundle with the result of the build. The bundles are then uploaded to a
server for consumption on other CIs like WPT.

The MiniBrowser bundle targets a specific distribution, so each one of this new bots has to
run the generate-minibrowser-bundle step. However, the JSC bundle is distro-agnostic because
for JSC is possible to bundle all the system libraries (similar to a static build).
That means that we only need to run the step generate-jsc-bundle in one of the bots.
This step for generating the JSC bundle was previously executed in the bot
'GTK Linux 64-bit Release (Build)'. This patch moves it to one of this new added bots.

  • BuildSlaveSupport/build.webkit.org-config/config.json:
  • BuildSlaveSupport/build.webkit.org-config/factories.py:

(BuildFactory):
(BuildFactory.init):
(TestFactory):
(TestFactory.init):
(BuildAndGenerateMiniBrowserBundleFactory):
(BuildAndGenerateMiniBrowserJSCBundleFactory):

  • BuildSlaveSupport/build.webkit.org-config/loadConfig.py:
  • BuildSlaveSupport/build.webkit.org-config/steps.py:

(GenerateJSCBundle):
(GenerateMiniBrowserBundle):

  • BuildSlaveSupport/build.webkit.org-config/steps_unittest.py:
Location:
trunk/Tools
Files:
6 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/config.json

    r265691 r266208  
    8989                    { "name": "gtk-linux-bot-14", "platform": "gtk" },
    9090                    { "name": "gtk-linux-bot-15", "platform": "gtk" },
     91                    { "name": "gtk-linux-bot-16", "platform": "gtk" },
     92                    { "name": "gtk-linux-bot-17", "platform": "gtk" },
    9193
    9294                    { "name": "jsconly-linux-igalia-bot-1", "platform": "jsc-only" },
    … …  
    101103                    { "name": "wpe-linux-bot-4", "platform": "wpe" },
    102104                    { "name": "wpe-linux-bot-5", "platform": "wpe" },
    103                     { "name": "wpe-linux-bot-6", "platform": "wpe" }
     105                    { "name": "wpe-linux-bot-6", "platform": "wpe" },
     106                    { "name": "wpe-linux-bot-7", "platform": "wpe" },
     107                    { "name": "wpe-linux-bot-8", "platform": "wpe" }
    104108                  ],
    105109
    … …  
    289293                    },
    290294                    {
    291                       "name": "GTK Linux 64-bit Release (Build)", "type": "BuildAndGenerateJSCBundle", "builddir": "gtk-linux-64-release",
     295                      "name": "GTK Linux 64-bit Release (Build)", "type": "Build", "builddir": "gtk-linux-64-release",
    292296                      "platform": "gtk", "configuration": "release", "architectures": ["x86_64"],
    293297                      "triggers": ["gtk-linux-64-release-tests", "gtk-linux-64-release-tests-js", "gtk-linux-64-release-tests-webdriver",
    … …  
    356360                    },
    357361                    {
     362                      "name": "GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804", "type": "BuildAndGenerateMiniBrowserBundle", "builddir": "gtk-linux-64-packaging-nigthly-ubuntu1804",
     363                      "platform": "gtk", "configuration": "release", "architectures": ["x86_64"],
     364                      "additionalArguments": ["--no-bubblewrap-sandbox"],
     365                      "slavenames": ["gtk-linux-bot-16"]
     366                    },
     367                    {
     368                      "name": "GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004", "type": "BuildAndGenerateMiniBrowserJSCBundle", "builddir": "gtk-linux-64-packaging-nigthly-ubuntu2004",
     369                      "platform": "gtk", "configuration": "release", "architectures": ["x86_64"],
     370                      "additionalArguments": ["--no-bubblewrap-sandbox"],
     371                      "slavenames": ["gtk-linux-bot-17"]
     372                    },
     373                    {
    358374                      "name": "WinCairo 64-bit WKL Release (Build)", "type": "Build", "builddir": "wincairo-wkl-release",
    359375                      "platform": "wincairo", "configuration": "release", "architectures": ["x86_64"],
    … …  
    448464                      "platform": "wpe", "configuration": "debug", "architectures": ["x86_64"],
    449465                      "slavenames": ["wpe-linux-bot-6"]
     466                    },
     467                    {
     468                      "name": "WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804", "type": "BuildAndGenerateMiniBrowserBundle", "builddir": "wpe-linux-64-packaging-nigthly-ubuntu1804",
     469                      "platform": "wpe", "configuration": "release", "architectures": ["x86_64"],
     470                      "additionalArguments": ["--no-bubblewrap-sandbox", "--cmakeargs=-DENABLE_WPE_QT_API=OFF"],
     471                      "slavenames": ["wpe-linux-bot-7"]
     472                    },
     473                    {
     474                      "name": "WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004", "type": "BuildAndGenerateMiniBrowserBundle", "builddir": "wpe-linux-64-packaging-nigthly-ubuntu2004",
     475                      "platform": "wpe", "configuration": "release", "architectures": ["x86_64"],
     476                      "additionalArguments": ["--no-bubblewrap-sandbox", "--cmakeargs=-DENABLE_WPE_QT_API=OFF"],
     477                      "slavenames": ["wpe-linux-bot-8"]
    450478                    }
    451479                  ],
    … …  
    600628                    { "type": "Triggerable", "name": "wpe-linux-64-debug-tests-js",
    601629                      "builderNames": ["WPE Linux 64-bit Debug (Tests JS)"]
     630                    },
     631                    { "type": "Nightly", "name": "NightlyScheduler", "change_filter": "trunk_filter",
     632                      "branch": "trunk", "hour": 22, "minute": 0,
     633                      "builderNames": ["GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804", "GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004",
     634                                       "WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804", "WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004"]
    602635                    }
    603636                  ]

  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/factories.py

    r263291 r266208  
    4848class BuildFactory(Factory):
    4949    ShouldRunJSCBundleStep = False
     50    ShouldRunMiniBrowserBundleStep = False
    5051
    5152    def __init__(self, platform, configuration, architectures, triggers=None, additionalArguments=None, SVNMirror=None, device_model=None):
    … …  
    5657        else:
    5758            self.addStep(CompileWebKit())
     59
     60        if self.ShouldRunJSCBundleStep:
     61            self.addStep(GenerateJSCBundle())
     62        if self.ShouldRunMiniBrowserBundleStep:
     63            self.addStep(GenerateMiniBrowserBundle())
    5864
    5965        if triggers:
    … …  
    6369                self.addStep(ArchiveMinifiedBuiltProduct())
    6470                self.addStep(UploadMinifiedBuiltProduct())
    65             if self.ShouldRunJSCBundleStep:
    66                 self.addStep(GenerateJSCBundle())
    6771            self.addStep(TransferToS3())
    6872            self.addStep(trigger.Trigger(schedulerNames=triggers))
    … …  
    7276    JSCTestClass = RunJavaScriptCoreTests
    7377    LayoutTestClass = RunWebKitTests
    74     ShouldRunJSCBundleStep = False
    7578
    7679    def getProduct(self):
    … …  
    113116            self.addStep(UploadTestResults())
    114117            self.addStep(ExtractTestResults())
    115         if self.ShouldRunJSCBundleStep:
    116             self.addStep(GenerateJSCBundle())
    117118        if platform == "gtk":
    118119            self.addStep(RunGtkAPITests())
    … …  
    180181
    181182
    182 class BuildAndNonLayoutTestAndGenerateJSCBundleFactory(BuildAndNonLayoutTestFactory):
     183class BuildAndGenerateMiniBrowserBundleFactory(BuildFactory):
     184    ShouldRunMiniBrowserBundleStep = True
     185
     186
     187class BuildAndGenerateMiniBrowserJSCBundleFactory(BuildFactory):
    183188    ShouldRunJSCBundleStep = True
     189    ShouldRunMiniBrowserBundleStep = True
    184190
    185191

  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/loadConfig.py

    r256535 r266208  
    2222
    2323from buildbot.buildslave import BuildSlave
    24 from buildbot.scheduler import AnyBranchScheduler, Triggerable
     24from buildbot.scheduler import AnyBranchScheduler, Triggerable, Nightly
    2525from buildbot.schedulers.forcesched import FixedParameter, ForceScheduler, StringParameter, BooleanParameter
    2626from buildbot.schedulers.filter import ChangeFilter

  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps.py

    r265606 r266208  
    311311    haltOnFailure = False
    312312
     313
     314class GenerateMiniBrowserBundle(shell.ShellCommand):
     315    command = ["./Tools/Scripts/generate-bundle", "--builder-name", WithProperties("%(buildername)s"),
     316               "--bundle=MiniBrowser", WithProperties("--platform=%(fullPlatform)s"),
     317               WithProperties("--%(configuration)s"), WithProperties("--revision=%(got_revision)s"),
     318               "--remote-config-file", "../../remote-minibrowser-bundle-upload-config.json"]
     319    name = "generate-minibrowser-bundle"
     320    description = ["generating minibrowser bundle"]
     321    descriptionDone = ["generated minibrowser bundle"]
     322    haltOnFailure = False
     323
     324
    313325class ExtractBuiltProduct(shell.ShellCommand):
    314326    command = ["python", "./Tools/BuildSlaveSupport/built-product-archive",

  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py

    r265691 r266208  
    576576    'GTK Linux 64-bit Debug (Tests WebDriver)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'webdriver-test'],
    577577    'GTK Linux 64-bit Debug (Tests JS)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'jscore-test', 'test262-test'],
    578     'GTK Linux 64-bit Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'archive-built-product', 'upload', 'generate-jsc-bundle', 'transfer-to-s3', 'trigger'],
     578    'GTK Linux 64-bit Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
    579579    'GTK Linux 64-bit Release (Perf)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'perf-test', 'benchmark-test'],
    580580    'GTK Linux 64-bit Release (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'layout-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand', 'API tests'],
    … …  
    601601    'WPE Linux 64-bit Debug (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'layout-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand', 'API tests', 'webdriver-test'],
    602602    'WPE Linux 64-bit Debug (Tests JS)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'download-built-product', 'extract-built-product', 'jscore-test', 'test262-test'],
     603    'GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'generate-minibrowser-bundle'],
     604    'GTK-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'generate-jsc-bundle', 'generate-minibrowser-bundle'],
     605    'WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu1804': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'generate-minibrowser-bundle'],
     606    'WPE-Linux-64bit-Release-Packaging-Nightly-Ubuntu2004': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'jhbuild', 'compile-webkit', 'generate-minibrowser-bundle'],
    603607}
    604608

  • trunk/Tools/ChangeLog

    r266190 r266208  
     12020-08-26  Carlos Alberto Lopez Perez  <clopez@igalia.com>
     2
     3        [GTK][WPE] Add bots for generating nightly bundle packages
     4        https://bugs.webkit.org/show_bug.cgi?id=215623
     5
     6        Reviewed by Aakash Jain.
     7
     8        This adds a new type of bots for GTK and WPE that runs nightly (once per day).
     9        The goal of this bots its to build WebKit for a specific distribution (Ubuntu LTS and LTS-1)
     10        and generate a bundle with the result of the build. The bundles are then uploaded to a
     11        server for consumption on other CIs like WPT.
     12
     13        The MiniBrowser bundle targets a specific distribution, so each one of this new bots has to
     14        run the generate-minibrowser-bundle step. However, the JSC bundle is distro-agnostic because
     15        for JSC is possible to bundle all the system libraries (similar to a static build).
     16        That means that we only need to run the step generate-jsc-bundle in one of the bots.
     17        This step for generating the JSC bundle was previously executed in the bot
     18        'GTK Linux 64-bit Release (Build)'. This patch moves it to one of this new added bots.
     19
     20        * BuildSlaveSupport/build.webkit.org-config/config.json:
     21        * BuildSlaveSupport/build.webkit.org-config/factories.py:
     22        (BuildFactory):
     23        (BuildFactory.__init__):
     24        (TestFactory):
     25        (TestFactory.__init__):
     26        (BuildAndGenerateMiniBrowserBundleFactory):
     27        (BuildAndGenerateMiniBrowserJSCBundleFactory):
     28        * BuildSlaveSupport/build.webkit.org-config/loadConfig.py:
     29        * BuildSlaveSupport/build.webkit.org-config/steps.py:
     30        (GenerateJSCBundle):
     31        (GenerateMiniBrowserBundle):
     32        * BuildSlaveSupport/build.webkit.org-config/steps_unittest.py:
     33
    1342020-08-26  Jonathan Bedard  <jbedard@apple.com>
    235
Note: See TracChangeset for help on using the changeset viewer.