Changeset 239813 in webkit

Timestamp:

Jan 9, 2019 6:20:13 PM (5 years ago)

Author:

Fujii Hironori

Message:

Use directory local sequential numbers for Unified Sources filenames instead of global sequential numbers for CMake
​https://bugs.webkit.org/show_bug.cgi?id=192391

Reviewed by Don Olmstead.

Unified Source Builds are using global sequential numbers for
bundle filenames UnifiedSource{sequential-number}.cpp. As the
result, every new source file added would shift the next ones and
prevent compiler caches from speeding up consecutive builds e.g.
in git-bisect sessions.

Changed it to directory local sequential numbers,
UnifiedSource-{hash-of-dir-name}-{sequential-number-within-the-dir}.cpp.

This is affecting only CMake builds which is where no
'--max-cpp-bundle-count' nor '--max-obj-c-bundle-count' options
are set. Xcode builds still use the old convention.

• Scripts/generate-unified-source-bundles.rb: Add new instance

variables @currentDirectory and @extraFiles to BundleManager.
Still use global sequential numbers if --max-cpp-bundle-count or
--max-obj-c-bundle-count is given.

Location:

trunk/Source/WTF

Files:

• ChangeLog (modified) (1 diff)
• Scripts/generate-unified-source-bundles.rb (modified) (9 diffs)

trunk/Source/WTF/ChangeLog

@@ +1 @@
+2019-01-09  Carlos Eduardo Ramalho  <cadubentzen@gmail.com> and Fujii Hironori  <Hironori.Fujii@sony.com>
+
+        Use directory local sequential numbers for Unified Sources filenames instead of global sequential numbers for CMake
+        https://bugs.webkit.org/show_bug.cgi?id=192391
+
+        Reviewed by Don Olmstead.
+
+        Unified Source Builds are using global sequential numbers for
+        bundle filenames UnifiedSource{sequential-number}.cpp. As the
+        result, every new source file added would shift the next ones and
+        prevent compiler caches from speeding up consecutive builds e.g.
+        in git-bisect sessions.
+
+        Changed it to directory local sequential numbers,
+        UnifiedSource-{hash-of-dir-name}-{sequential-number-within-the-dir}.cpp.
+
+        This is affecting only CMake builds which is where no
+        '--max-cpp-bundle-count' nor '--max-obj-c-bundle-count' options
+        are set. Xcode builds still use the old convention.
+
+        * Scripts/generate-unified-source-bundles.rb: Add new instance
+        variables @currentDirectory and @extraFiles to BundleManager.
+        Still use global sequential numbers if --max-cpp-bundle-count or
+        --max-obj-c-bundle-count is given.
+
 2019-01-09  Alex Christensen  <achristensen@webkit.org>
 

trunk/Source/WTF/Scripts/generate-unified-source-bundles.rb

@@ -22 +22 @@
 # THE POSSIBILITY OF SUCH DAMAGE.
 
+require 'digest'
 require 'fileutils'
 require 'pathname'
@@ -53 +54 @@
     puts
     puts "Generation options:"
-    puts "--max-cpp-bundle-count               Sets the limit on the number of cpp bundles that can be generated"
-    puts "--max-obj-c-bundle-count             Sets the limit on the number of Obj-C bundles that can be generated"
+    puts "--max-cpp-bundle-count               Use global sequential numbers for cpp bundle filenames and set the limit on the number"
+    puts "--max-obj-c-bundle-count             Use global sequential numbers for Obj-C bundle filenames and set the limit on the number"
     exit 1
 end
@@ -184 +185 @@
 
 class BundleManager
-    attr_reader :bundleCount, :extension, :fileCount, :currentBundleText, :maxCount
+    attr_reader :bundleCount, :extension, :fileCount, :currentBundleText, :maxCount, :extraFiles
 
     def initialize(extension, max)
@@ -192 +193 @@
         @currentBundleText = ""
         @maxCount = max
+        @extraFiles = []
+        @currentDirectory = nil
     end
 
@@ -206 +209 @@
     end
 
-    def bundleFileName(number)
-        @extension == "cpp" ? "UnifiedSource#{number}.#{extension}" : "UnifiedSource#{number}-#{extension}.#{extension}"
+    def bundleFileName()
+        id =
+            if @maxCount
+                @bundleCount.to_s
+            else
+                # The dash makes the filenames more clear when using a hash.
+                hash = Digest::SHA1.hexdigest(@currentDirectory.to_s)[0..7]
+                "-#{hash}-#{@bundleCount}"
+            end
+        @extension == "cpp" ? "UnifiedSource#{id}.#{extension}" : "UnifiedSource#{id}-#{extension}.#{extension}"
     end
 
     def flush
-        # No point in writing an empty bundle file
-        return if @currentBundleText == ""
-
         @bundleCount += 1
-        bundleFile = bundleFileName(@bundleCount)
+        bundleFile = bundleFileName
         $generatedSources << $unifiedSourceOutputPath + bundleFile
+        @extraFiles << bundleFile if @maxCount and @bundleCount > @maxCount
 
         writeFile(bundleFile, @currentBundleText)
@@ -225 +234 @@
     def flushToMax
         raise if !@maxCount
-        ((@bundleCount+1)..@maxCount).each {
-            | index |
-            writeFile(bundleFileName(index), "")
-        }
+        while @bundleCount < @maxCount
+            flush
+        end
     end
 
@@ -234 +242 @@
         path = sourceFile.path
         raise "wrong extension: #{path.extname} expected #{@extension}" unless path.extname == ".#{@extension}"
+        if (TopLevelDirectoryForPath(@currentDirectory) != TopLevelDirectoryForPath(path.dirname))
+            log("Flushing because new top level directory; old: #{@currentDirectory}, new: #{path.dirname}")
+            flush
+            @currentDirectory = path.dirname
+            @bundleCount = 0 unless @maxCount
+        end
         if @fileCount == MAX_BUNDLE_SIZE
             log("Flushing because new bundle is full (#{@fileCount} sources)")
@@ -256 +270 @@
     path = sourceFile.path
     $inputSources << sourceFile.to_s
-    if (TopLevelDirectoryForPath($currentDirectory) != TopLevelDirectoryForPath(path.dirname))
-        log("Flushing because new top level directory; old: #{$currentDirectory}, new: #{path.dirname}")
-        $bundleManagers.each_value { |x| x.flush }
-        $currentDirectory = path.dirname
-    end
 
     bundle = $bundleManagers[path.extname]
@@ -343 +352 @@
 
     manager.flushToMax
-    bundleCount = manager.bundleCount
-    extension = manager.extension
-    if bundleCount > maxCount
-        filesToAdd = ((maxCount+1)..bundleCount).map { |x| manager.bundleFileName(x) }.join(", ")
+
+    unless manager.extraFiles.empty?
+        extension = manager.extension
+        bundleCount = manager.bundleCount
+        filesToAdd = manager.extraFiles.join(", ")
         raise "number of bundles for #{extension} sources, #{bundleCount}, exceeded limit, #{maxCount}. Please add #{filesToAdd} to Xcode then update UnifiedSource#{extension.capitalize}FileCount"
     end