Changeset 58036 in webkit
- Timestamp:
- Apr 21, 2010 6:34:09 PM (14 years ago)
- Location:
- trunk/WebKitTools
- Files:
-
- 48 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/WebKitTools/ChangeLog
r58031 r58036 1 2010-04-21 Eric Seidel <eric@webkit.org> 2 3 Reviewed by Adam Barth. 4 5 REGRESSION(57531): the commit-queue still hates Tor Arne Vestbø 6 https://bugs.webkit.org/show_bug.cgi?id=37765 7 8 Third time is the charm. I've fixed all the 9 new-run-webkit-tests regressions from previous attempts. 10 11 I fixed the queue to not ignore Tor as a reviwer in r57531, 12 but instead it throws an exception every time his name is in a patch. 13 14 This fixes our Executive.run_command code to work around a Popen 15 bug http://bugs.python.org/issue5290 whereby python versions before 2.6 16 do not correctly handle unicode objects as input or output to 17 Popen.communicate. 18 19 Following the advice of: 20 http://farmdev.com/talks/unicode/ 21 I have changed all of webkitpy to use unicode() objects as strings 22 instead of str objects (which in Python 3 are renamed "bytes"). 23 24 String literals were left as "foo" instead of converting to u"foo" 25 as u"foo" is only required if the string has a non-ascii code point. 26 Python is smart about comparing str() and unicode() values and will 27 log an error to the console if the comparison is ever invalid. 28 29 Executive.run* now correctly accept and return unicode() objects. 30 I attempted to fix all the places that we call .write() to make sure we 31 encode any unicode() objects into utf-8. 32 33 I removed several uses of StringIO. StringIO should generally always be 34 passed a unicode() value. 35 36 Likewise I replaced most calls to open() with codecs.open(). 37 codecs.open() matches Python 3 open semantics in requiring an encoding 38 parameter. Files opened with codecs.open() with a unicode-compatible 39 encoding will vend unicode() objects from their read() calls, like how 40 StringIO created with a unicode() object will do. 41 42 I also deployed "with" statements wider (now that the project has 43 settled on Python 2.5) to close a bunch of file descriptor leaks. 44 45 * Scripts/webkitpy/common/checkout/api_unittest.py: 46 - Read/write utf-8 files instead of ascii. 47 - Update the tests to use test for proper unicode() handling. 48 * Scripts/webkitpy/common/checkout/changelog.py: 49 - Document that parse_latest_entry_from_file expects 50 file-like objects which return unicode strings. 51 * Scripts/webkitpy/common/checkout/changelog_unittest.py: 52 - Use unicode() strings instead of str() byte arrays. 53 - Deploy "with" to close file descriptor leaks. 54 * Scripts/webkitpy/common/checkout/commitinfo.py: 55 - Remove unneeded import. 56 * Scripts/webkitpy/common/checkout/scm.py: 57 - Remove use of str(). 58 * Scripts/webkitpy/common/checkout/scm_unittest.py: 59 - Read/write utf-8 files and use unicode() strings in testing. 60 * Scripts/webkitpy/common/config/committers.py: 61 - Use \u instead of \x to make slightly clearer what we're doing. 62 * Scripts/webkitpy/common/net/bugzilla.py: 63 - Add a new _string_contents() method and explain why 64 we have to call unicode() on the result of soup.string 65 and why it's safe to do so w/o needing to pass a codec name. 66 - Remove the (unused) support for passing a file object to add_patch_to_bug(). 67 * Scripts/webkitpy/common/net/buildbot.py: 68 - Use unicode() instead of str() when needing to coax a 69 NavigableString object into a unicode() object. 70 * Scripts/webkitpy/common/net/buildbot_unittest.py: 71 - Add a test which contains a unicode builder name. 72 * Scripts/webkitpy/common/net/statusserver.py: 73 - Remove use of str() 74 * Scripts/webkitpy/common/prettypatch.py: 75 - Write out the patch file as utf-8. 76 * Scripts/webkitpy/common/system/autoinstall.py: 77 - Write out files with a explicit encodings. 78 - Deploy "with" to close file descriptor leaks. 79 * Scripts/webkitpy/common/system/deprecated_logging.py: 80 - Write out log files as utf-8. 81 * Scripts/webkitpy/common/system/executive.py: 82 - Make run* properly take and return unicode() objects. 83 - Cleaned up input handling in run_command a little by adding 84 a _compute_input() method which can return early instead of having 85 such a long/cluttered if-block. 86 * Scripts/webkitpy/common/system/executive_unittest.py: 87 - Added a unit test to make sure we don't break Tor again! 88 * Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py: 89 - Write out the test list as utf-8. 90 * Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py: 91 - Write out json files as utf-8. 92 * Scripts/webkitpy/layout_tests/port/apache_http_server.py: 93 - Deploy "with" to close file descriptor leaks. 94 * Scripts/webkitpy/layout_tests/port/chromium.py: Add Executive.py FIXME. 95 * Scripts/webkitpy/layout_tests/port/chromium_linux.py: ditto. 96 * Scripts/webkitpy/layout_tests/port/chromium_mac.py: ditto. 97 * Scripts/webkitpy/layout_tests/port/gtk.py: ditto. 98 * Scripts/webkitpy/layout_tests/port/mac.py: ditto. 99 * Scripts/webkitpy/layout_tests/port/mac_unittest.py: 100 - Make the skipped file parsing test unicode. 101 * Scripts/webkitpy/layout_tests/port/qt.py: Add Executive.py FIXME. 102 * Scripts/webkitpy/layout_tests/port/server_process.py: ditto. 103 * Scripts/webkitpy/layout_tests/port/webkit.py: 104 - Deploy "with" to close file descriptor leaks. 105 * Scripts/webkitpy/layout_tests/port/websocket_server.py: 106 - Make explicit the encodings of log files and pid files. 107 * Scripts/webkitpy/layout_tests/rebaseline_chromium_webkit_tests.py: 108 - Make encodings explicit and deploy "with". 109 * Scripts/webkitpy/layout_tests/run_webkit_tests.py: ditto. 110 * Scripts/webkitpy/layout_tests/test_types/image_diff.py: ditto. 111 * Scripts/webkitpy/layout_tests/test_types/test_type_base.py: ditto. 112 * Scripts/webkitpy/layout_tests/test_types/text_diff.py: ditto. 113 * Scripts/webkitpy/style/filereader_unittest.py: ditto. 114 * Scripts/webkitpy/thirdparty/__init__.py: ditto. 115 * Scripts/webkitpy/tool/commands/earlywarningsystem.py: 116 - Removed extra import. 117 * Scripts/webkitpy/tool/commands/queues.py: 118 - No need to map args to strings now that run_command does. 119 - Update test results to match args changes. 120 - Document our global argument hacks. 121 * Scripts/webkitpy/tool/commands/upload.py: 122 - Pass the diff directly to add_patch_to_bug instead of creating a StringIO file wrapper. 123 * Scripts/webkitpy/tool/mocktool.py: 124 - Rename add_patch_to_bug argument to match bugzilla.py 125 * Scripts/webkitpy/tool/steps/abstractstep.py: 126 - Executive.run_* now require lists instead of strings. 127 The lack of this change was what broke webkit-patch 128 for svn users the first time this was landed. 129 * Scripts/webkitpy/tool/steps/postdiff.py: 130 - Pass the diff directly to add_patch_to_bug instead of creating a StringIO file wrapper. 131 * Scripts/webkitpy/tool/steps/postdiffforcommit.py: ditto 132 * Scripts/webkitpy/tool/steps/postdiffforrevert.py: ditto 133 * Scripts/webkitpy/tool/steps/steps_unittest.py: 134 - Fixed spurious logging seen when running test-webkitpy 135 1 136 2010-04-21 Chris Fleizach <cfleizach@apple.com> 2 137 -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/api_unittest.py
r58023 r58036 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 28 29 from __future__ import with_statement 30 31 import codecs 29 32 import os 30 33 import shutil … … 38 41 from webkitpy.thirdparty.mock import Mock 39 42 43 40 44 # FIXME: Copied from scm_unittest.py 41 def write_into_file_at_path(file_path, contents): 42 new_file = open(file_path, 'w') 43 new_file.write(contents) 44 new_file.close() 45 def write_into_file_at_path(file_path, contents, encoding="utf-8"): 46 with codecs.open(file_path, "w", encoding) as file: 47 file.write(contents) 45 48 46 49 47 _changelog1entry1 = """2010-03-25 Eric Seidel <eric@webkit.org>50 _changelog1entry1 = u"""2010-03-25 Tor Arne Vestb\u00f8 <vestbo@webkit.org> 48 51 49 52 Unreviewed build fix to un-break webkit-patch land. … … 54 57 * Scripts/webkitpy/common/checkout/api.py: import scm.CommitMessage 55 58 """ 56 _changelog1entry2 = """2010-03-25 Adam Barth <abarth@webkit.org>59 _changelog1entry2 = u"""2010-03-25 Adam Barth <abarth@webkit.org> 57 60 58 61 Reviewed by Eric Seidel. … … 63 66 * Scripts/webkitpy/common/checkout/api.py: 64 67 """ 65 _changelog1 = "\n".join([_changelog1entry1, _changelog1entry2])66 _changelog2 = """2010-03-25 Eric Seidel <eric@webkit.org>68 _changelog1 = u"\n".join([_changelog1entry1, _changelog1entry2]) 69 _changelog2 = u"""2010-03-25 Tor Arne Vestb\u00f8 <vestbo@webkit.org> 67 70 68 71 Unreviewed build fix to un-break webkit-patch land. … … 80 83 81 84 class CommitMessageForThisCommitTest(unittest.TestCase): 82 expected_commit_message = """2010-03-25 Eric Seidel <eric@webkit.org>85 expected_commit_message = u"""2010-03-25 Tor Arne Vestb\u00f8 <vestbo@webkit.org> 83 86 84 87 Unreviewed build fix to un-break webkit-patch land. … … 88 91 89 92 * Scripts/webkitpy/common/checkout/api.py: import scm.CommitMessage 90 2010-03-25 Eric Seidel <eric@webkit.org>93 2010-03-25 Tor Arne Vestb\u00f8 <vestbo@webkit.org> 91 94 92 95 Unreviewed build fix to un-break webkit-patch land. … … 138 141 commitinfo = checkout.commit_info_for_revision(4) 139 142 self.assertEqual(commitinfo.bug_id(), 36629) 140 self.assertEqual(commitinfo.author_name(), "Eric Seidel")141 self.assertEqual(commitinfo.author_email(), " eric@webkit.org")143 self.assertEqual(commitinfo.author_name(), u"Tor Arne Vestb\u00f8") 144 self.assertEqual(commitinfo.author_email(), "vestbo@webkit.org") 142 145 self.assertEqual(commitinfo.reviewer_text(), None) 143 146 self.assertEqual(commitinfo.reviewer(), None) -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/changelog.py
r58023 r58036 100 100 @staticmethod 101 101 def parse_latest_entry_from_file(changelog_file): 102 """changelog_file must be a file-like object which returns 103 unicode strings. Use codecs.open or StringIO(unicode()) 104 to pass file objects to this class.""" 102 105 date_line_regexp = re.compile(ChangeLogEntry.date_line_regexp) 103 106 entry_lines = [] 104 107 # The first line should be a date line. 105 108 first_line = changelog_file.readline() 109 assert(isinstance(first_line, unicode)) 106 110 if not date_line_regexp.match(first_line): 107 111 return None -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/changelog_unittest.py
r58023 r58036 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 28 29 import unittest 29 from __future__ import with_statement 30 31 import codecs 30 32 import os 31 33 import tempfile 34 import unittest 32 35 33 36 from StringIO import StringIO … … 53 56 54 57 # More example text than we need. Eventually we need to support parsing this all and write tests for the parsing. 55 _example_changelog = '''2009-08-17 David Kilzer <ddkilzer@apple.com>58 _example_changelog = u"""2009-08-17 Tor Arne Vestb\xf8 <vestbo@webkit.org> 56 59 57 60 <http://webkit.org/b/28393> check-webkit-style: add check for use of std::max()/std::min() instead of MAX()/MIN() … … 85 88 86 89 == Rolled over to ChangeLog-2009-06-16 == 87 ''' 90 """ 88 91 89 92 def test_latest_entry_parse(self): 90 changelog_contents = "%s\n%s" % (self._example_entry, self._example_changelog)93 changelog_contents = u"%s\n%s" % (self._example_entry, self._example_changelog) 91 94 changelog_file = StringIO(changelog_contents) 92 95 latest_entry = ChangeLog.parse_latest_entry_from_file(changelog_file) … … 98 101 99 102 @staticmethod 100 def _write_tmp_file_with_contents(contents): 103 def _write_tmp_file_with_contents(byte_array): 104 assert(isinstance(byte_array, str)) 101 105 (file_descriptor, file_path) = tempfile.mkstemp() # NamedTemporaryFile always deletes the file on close in python < 2.6 102 file = os.fdopen(file_descriptor, 'w') 103 file.write(contents) 104 file.close() 106 with os.fdopen(file_descriptor, "w") as file: 107 file.write(byte_array) 105 108 return file_path 106 109 107 110 @staticmethod 108 def _read_file_contents(file_path): 109 file = open(file_path) 110 contents = file.read() 111 file.close() 112 return contents 111 def _read_file_contents(file_path, encoding): 112 with codecs.open(file_path, "r", encoding) as file: 113 return file.read() 113 114 114 115 _new_entry_boilerplate = '''2009-08-19 Eric Seidel <eric@webkit.org> … … 122 123 123 124 def test_set_reviewer(self): 124 changelog_contents = "%s\n%s" % (self._new_entry_boilerplate, self._example_changelog)125 changelog_path = self._write_tmp_file_with_contents(changelog_contents )125 changelog_contents = u"%s\n%s" % (self._new_entry_boilerplate, self._example_changelog) 126 changelog_path = self._write_tmp_file_with_contents(changelog_contents.encode("utf-8")) 126 127 reviewer_name = 'Test Reviewer' 127 128 ChangeLog(changelog_path).set_reviewer(reviewer_name) 128 actual_contents = self._read_file_contents(changelog_path )129 actual_contents = self._read_file_contents(changelog_path, "utf-8") 129 130 expected_contents = changelog_contents.replace('NOBODY (OOPS!)', reviewer_name) 130 131 os.remove(changelog_path) … … 170 171 171 172 def _assert_update_for_revert_output(self, args, expected_entry): 172 changelog_contents = "%s\n%s" % (self._new_entry_boilerplate, self._example_changelog)173 changelog_path = self._write_tmp_file_with_contents(changelog_contents )173 changelog_contents = u"%s\n%s" % (self._new_entry_boilerplate, self._example_changelog) 174 changelog_path = self._write_tmp_file_with_contents(changelog_contents.encode("utf-8")) 174 175 changelog = ChangeLog(changelog_path) 175 176 changelog.update_for_revert(*args) -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/commitinfo.py
r58023 r58036 28 28 # 29 29 # WebKit's python module for holding information on a commit 30 31 import StringIO32 30 33 31 from webkitpy.common.checkout.changelog import view_source_url -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/scm.py
r58023 r58036 146 146 147 147 def strip_r_from_svn_revision(self, svn_revision): 148 match = re.match("^r(?P<svn_revision>\d+)", svn_revision)148 match = re.match("^r(?P<svn_revision>\d+)", unicode(svn_revision)) 149 149 if (match): 150 150 return match.group('svn_revision') … … 345 345 def changed_files_for_revision(self, revision): 346 346 # As far as I can tell svn diff --summarize output looks just like svn status output. 347 status_command = ["svn", "diff", "--summarize", "-c", str(revision)]347 status_command = ["svn", "diff", "--summarize", "-c", revision] 348 348 return self.run_status_and_extract_filenames(status_command, self._status_regexp("ACDMR")) 349 349 … … 362 362 363 363 def create_patch(self): 364 return run_command( self.script_path("svn-create-patch"), cwd=self.checkout_root, return_stderr=False)364 return run_command([self.script_path("svn-create-patch")], cwd=self.checkout_root, return_stderr=False) 365 365 366 366 def committer_email_for_revision(self, revision): 367 return run_command(["svn", "propget", "svn:author", "--revprop", "-r", str(revision)]).rstrip()367 return run_command(["svn", "propget", "svn:author", "--revprop", "-r", revision]).rstrip() 368 368 369 369 def contents_at_revision(self, path, revision): 370 370 remote_path = "%s/%s" % (self._repository_url(), path) 371 return run_command(["svn", "cat", "-r", str(revision), remote_path])371 return run_command(["svn", "cat", "-r", revision, remote_path]) 372 372 373 373 def diff_for_revision(self, revision): 374 374 # FIXME: This should probably use cwd=self.checkout_root 375 return run_command(['svn', 'diff', '-c', str(revision)])375 return run_command(['svn', 'diff', '-c', revision]) 376 376 377 377 def _repository_url(self): … … 406 406 407 407 def svn_commit_log(self, svn_revision): 408 svn_revision = self.strip_r_from_svn_revision(s tr(svn_revision))408 svn_revision = self.strip_r_from_svn_revision(svn_revision) 409 409 return run_command(['svn', 'log', '--non-interactive', '--revision', svn_revision]); 410 410 -
trunk/WebKitTools/Scripts/webkitpy/common/checkout/scm_unittest.py
r58023 r58036 28 28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 29 30 from __future__ import with_statement 31 30 32 import base64 33 import codecs 31 34 import getpass 32 35 import os … … 57 60 raise ScriptError('Failed to run "%s" exit_code: %d cwd: %s' % (args, exit_code, cwd)) 58 61 59 def write_into_file_at_path(file_path, contents): 60 file = open(file_path, 'w') 61 file.write(contents)62 file.close()63 64 def read_from_path(file_path): 65 file = open(file_path, 'r') 66 contents = file.read()67 file.close()68 return contents 62 63 def write_into_file_at_path(file_path, contents, encoding="utf-8"): 64 with codecs.open(file_path, "w", encoding) as file: 65 file.write(contents) 66 67 68 def read_from_path(file_path, encoding="utf-8"): 69 with codecs.open(file_path, "r", encoding) as file: 70 return file.read() 71 69 72 70 73 # Exists to share svn repository creation code between the git and svn tests … … 309 312 """ 310 313 self.checkout.apply_patch(self._create_patch(git_binary_addition)) 311 added = read_from_path('fizzbuzz7.gif' )314 added = read_from_path('fizzbuzz7.gif', encoding=None) 312 315 self.assertEqual(512, len(added)) 313 316 self.assertTrue(added.startswith('GIF89a')) … … 337 340 """ 338 341 self.checkout.apply_patch(self._create_patch(git_binary_modification)) 339 modified = read_from_path('fizzbuzz7.gif' )342 modified = read_from_path('fizzbuzz7.gif', encoding=None) 340 343 self.assertEqual('foobar\n', modified) 341 344 self.assertTrue('fizzbuzz7.gif' in self.scm.changed_files()) … … 519 522 patch_file = self._create_patch(patch_contents) 520 523 self.checkout.apply_patch(patch_file) 521 actual_contents = read_from_path("test_file.swf" )524 actual_contents = read_from_path("test_file.swf", encoding=None) 522 525 self.assertEqual(actual_contents, expected_contents) 523 526 … … 740 743 test_file_path = os.path.join(self.git_checkout_path, test_file_name) 741 744 file_contents = ''.join(map(chr, range(256))) 742 write_into_file_at_path(test_file_path, file_contents )745 write_into_file_at_path(test_file_path, file_contents, encoding=None) 743 746 run_command(['git', 'add', test_file_name]) 744 747 patch = scm.create_patch() … … 750 753 self._setup_webkittools_scripts_symlink(scm) 751 754 self.checkout.apply_patch(self._create_patch(patch)) 752 self.assertEqual(file_contents, read_from_path(test_file_path ))755 self.assertEqual(file_contents, read_from_path(test_file_path, encoding=None)) 753 756 754 757 # Check if we can create a patch from a local commit. 755 write_into_file_at_path(test_file_path, file_contents )758 write_into_file_at_path(test_file_path, file_contents, encoding=None) 756 759 run_command(['git', 'add', test_file_name]) 757 760 run_command(['git', 'commit', '-m', 'binary diff']) -
trunk/WebKitTools/Scripts/webkitpy/common/config/committers.py
r58023 r58036 241 241 Reviewer("Tim Omernick", "timo@apple.com"), 242 242 Reviewer("Timothy Hatcher", ["timothy@hatcher.name", "timothy@apple.com"], "xenon"), 243 Reviewer(u 'Tor Arne Vestb\xf8', "vestbo@webkit.org", "torarne"),243 Reviewer(u"Tor Arne Vestb\u00f8", "vestbo@webkit.org", "torarne"), 244 244 Reviewer("Vicki Murley", "vicki@apple.com"), 245 245 Reviewer("Xan Lopez", ["xan.lopez@gmail.com", "xan@gnome.org", "xan@webkit.org"], "xan"), -
trunk/WebKitTools/Scripts/webkitpy/common/net/bugzilla.py
r58023 r58036 33 33 import os.path 34 34 import re 35 import StringIO 35 36 import subprocess 36 37 … … 428 429 attachment[result_key] = flag['setter'] 429 430 431 def _string_contents(self, soup): 432 # WebKit's bugzilla instance uses UTF-8. 433 # BeautifulSoup always returns Unicode strings, however 434 # the .string method returns a (unicode) NavigableString. 435 # NavigableString can confuse other parts of the code, so we 436 # convert from NavigableString to a real unicode() object using unicode(). 437 return unicode(soup.string) 438 430 439 def _parse_attachment_element(self, element, bug_id): 440 431 441 attachment = {} 432 442 attachment['bug_id'] = bug_id … … 436 446 # FIXME: No need to parse out the url here. 437 447 attachment['url'] = self.attachment_url_for_id(attachment['id']) 438 attachment['name'] = unicode(element.find('desc').string)439 attachment['attacher_email'] = s tr(element.find('attacher').string)440 attachment['type'] = s tr(element.find('type').string)448 attachment['name'] = self._string_contents(element.find('desc')) 449 attachment['attacher_email'] = self._string_contents(element.find('attacher')) 450 attachment['type'] = self._string_contents(element.find('type')) 441 451 self._parse_attachment_flag( 442 452 element, 'review', attachment, 'reviewer_email') … … 449 459 bug = {} 450 460 bug["id"] = int(soup.find("bug_id").string) 451 bug["title"] = unicode(soup.find("short_desc").string)452 bug["reporter_email"] = s tr(soup.find("reporter").string)453 bug["assigned_to_email"] = s tr(soup.find("assigned_to").string)454 bug["cc_emails"] = [s tr(element.string)461 bug["title"] = self._string_contents(soup.find("short_desc")) 462 bug["reporter_email"] = self._string_contents(soup.find("reporter")) 463 bug["assigned_to_email"] = self._string_contents(soup.find("assigned_to")) 464 bug["cc_emails"] = [self._string_contents(element) 455 465 for element in soup.findAll('cc')] 456 466 bug["attachments"] = [self._parse_attachment_element(element, bug["id"]) for element in soup.findAll('attachment')] … … 549 559 def _fill_attachment_form(self, 550 560 description, 551 patch_file_object,561 diff, 552 562 comment_text=None, 553 563 mark_for_review=False, … … 569 579 else: 570 580 patch_name ="%s.patch" % timestamp() 581 582 # ClientForm expects a file-like object 583 patch_file_object = StringIO.StringIO(diff.encode("utf-8")) 571 584 self.browser.add_file(patch_file_object, 572 585 "text/plain", … … 576 589 def add_patch_to_bug(self, 577 590 bug_id, 578 patch_file_object,591 diff, 579 592 description, 580 593 comment_text=None, … … 595 608 self.bug_server_url, bug_id)) 596 609 self.browser.select_form(name="entryform") 610 597 611 self._fill_attachment_form(description, 598 patch_file_object,612 diff, 599 613 mark_for_review=mark_for_review, 600 614 mark_for_commit_queue=mark_for_commit_queue, … … 629 643 bug_description, 630 644 component=None, 631 patch_file_object=None,645 diff=None, 632 646 patch_description=None, 633 647 cc=None, … … 654 668 self.browser["cc"] = cc 655 669 if blocked: 656 self.browser["blocked"] = str(blocked)670 self.browser["blocked"] = unicode(blocked) 657 671 self.browser["short_desc"] = bug_title 658 672 self.browser["comment"] = bug_description 659 673 660 if patch_file_object:674 if diff: 661 675 self._fill_attachment_form( 662 676 patch_description, 663 patch_file_object,677 diff, 664 678 mark_for_review=mark_for_review, 665 679 mark_for_commit_queue=mark_for_commit_queue) -
trunk/WebKitTools/Scripts/webkitpy/common/net/buildbot.py
r58023 r58036 45 45 class Builder(object): 46 46 def __init__(self, name, buildbot): 47 self._name = unicode(name)47 self._name = name 48 48 self._buildbot = buildbot 49 49 self._builds_cache = {} … … 224 224 tables = BeautifulSoup(page).findAll("table") 225 225 for table in tables: 226 table_title = table.findPreviousSibling("p").string226 table_title = unicode(table.findPreviousSibling("p").string) 227 227 if table_title not in cls.expected_keys: 228 228 # This Exception should only ever be hit if run-webkit-tests changes its results.html format. 229 raise Exception("Unhandled title: %s" % str(table_title))229 raise Exception("Unhandled title: %s" % table_title) 230 230 # We might want to translate table titles into identifiers before storing. 231 parsed_results[table_title] = [ row.find("a").stringfor row in table.findAll("tr")]231 parsed_results[table_title] = [unicode(row.find("a").string) for row in table.findAll("tr")] 232 232 233 233 return parsed_results … … 362 362 # First cell is the name 363 363 name_link = status_cells[0].find('a') 364 builder["name"] = name_link.string364 builder["name"] = unicode(name_link.string) 365 365 366 366 self._parse_last_build_cell(builder, status_cells[1]) … … 411 411 412 412 def _parse_twisted_file_row(self, file_row): 413 string_or_empty = lambda s tring: str(string) if string else""413 string_or_empty = lambda soup: unicode(soup.string) if soup.string else u"" 414 414 file_cells = file_row.findAll('td') 415 415 return { 416 "filename" : string_or_empty(file_cells[0].find("a").string),417 "size" : string_or_empty(file_cells[1].string),418 "type" : string_or_empty(file_cells[2].string),419 "encoding" : string_or_empty(file_cells[3].string),416 "filename": string_or_empty(file_cells[0].find("a")), 417 "size": string_or_empty(file_cells[1]), 418 "type": string_or_empty(file_cells[2]), 419 "encoding": string_or_empty(file_cells[3]), 420 420 } 421 421 -
trunk/WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py
r58023 r58036 52 52 def setUp(self): 53 53 self.buildbot = BuildBot() 54 self.builder = Builder( "Test Builder", self.buildbot)54 self.builder = Builder(u"Test Builder \u2661", self.buildbot) 55 55 self._install_fetch_build(lambda build_number: ["test1", "test2"]) 56 56 -
trunk/WebKitTools/Scripts/webkitpy/common/net/statusserver.py
r58023 r58036 53 53 return 54 54 if patch.bug_id(): 55 self.browser["bug_id"] = str(patch.bug_id())55 self.browser["bug_id"] = unicode(patch.bug_id()) 56 56 if patch.id(): 57 self.browser["patch_id"] = str(patch.id())57 self.browser["patch_id"] = unicode(patch.id()) 58 58 59 59 def _add_results_file(self, results_file): … … 80 80 self.browser.open(update_svn_revision_url) 81 81 self.browser.select_form(name="update_svn_revision") 82 self.browser["number"] = str(svn_revision_number)82 self.browser["number"] = unicode(svn_revision_number) 83 83 self.browser["broken_bot"] = broken_bot 84 84 return self.browser.submit().read() -
trunk/WebKitTools/Scripts/webkitpy/common/prettypatch.py
r58023 r58036 39 39 pretty_diff = self.pretty_diff(diff) 40 40 diff_file = tempfile.NamedTemporaryFile(suffix=".html") 41 diff_file.write(pretty_diff )41 diff_file.write(pretty_diff.encode("utf-8")) 42 42 diff_file.flush() 43 43 return diff_file -
trunk/WebKitTools/Scripts/webkitpy/common/system/autoinstall.py
r58023 r58036 31 31 """Support for automatically downloading Python packages from an URL.""" 32 32 33 34 from __future__ import with_statement 35 36 import codecs 33 37 import logging 34 38 import new … … 115 119 os.makedirs(path) 116 120 117 def _write_file(self, path, text ):121 def _write_file(self, path, text, encoding): 118 122 """Create a file at the given path with given text. 119 123 … … 123 127 _log.debug("Creating file...") 124 128 _log.debug(' "%s"' % path) 125 file = open(path, "w") 126 try: 129 with codecs.open(path, "w", encoding) as file: 127 130 file.write(text) 128 finally:129 file.close()130 131 131 132 def _set_up_target_dir(self, target_dir, append_to_search_path, … … 155 156 text = ("# This file is required for Python to search this " 156 157 "directory for modules.\n") 157 self._write_file(init_path, text )158 self._write_file(init_path, text, "ascii") 158 159 159 160 def _create_scratch_directory_inner(self, prefix): … … 217 218 return False 218 219 219 file = open(version_path, "r") 220 try: 220 with codecs.open(version_path, "r", "utf-8") as file: 221 221 version = file.read() 222 finally:223 file.close()224 222 225 223 return version.strip() == url.strip() … … 232 230 _log.debug(' To: "%s"' % version_path) 233 231 234 self._write_file(version_path, url )232 self._write_file(version_path, url, "utf-8") 235 233 236 234 def _extract_targz(self, path, scratch_dir): … … 285 283 286 284 try: 285 # We open this file w/o encoding, as we're reading/writing 286 # the raw byte-stream from the zip file. 287 287 outfile = open(path, 'wb') 288 288 except IOError, err: … … 385 385 self._log_transfer("Starting download...", url, target_path) 386 386 387 stream = file(target_path, "wb") 388 bytes = self._download_to_stream(url, stream) 389 stream.close() 387 with open(target_path, "wb") as stream: 388 bytes = self._download_to_stream(url, stream) 390 389 391 390 _log.debug("Downloaded %s bytes." % bytes) -
trunk/WebKitTools/Scripts/webkitpy/common/system/deprecated_logging.py
r58023 r58036 31 31 # This module is now deprecated in favor of python's built-in logging.py. 32 32 33 import codecs 33 34 import os 34 35 import sys 36 35 37 36 38 def log(string): 37 39 print >> sys.stderr, string 38 40 41 39 42 def error(string): 40 43 log("ERROR: %s" % string) 41 44 exit(1) 45 42 46 43 47 # Simple class to split output between multiple destinations … … 46 50 self.files = files 47 51 48 def write(self, string): 52 # Callers should pass an already encoded string for writing. 53 def write(self, bytes): 49 54 for file in self.files: 50 file.write(string) 55 file.write(bytes) 56 51 57 52 58 class OutputTee: … … 72 78 if log_directory and not os.path.exists(log_directory): 73 79 os.makedirs(log_directory) 74 return open(log_path, 'a+')80 return codecs.open(log_path, "a+", "utf-8") 75 81 76 82 def _tee_outputs_to_files(self, files): -
trunk/WebKitTools/Scripts/webkitpy/common/system/executive.py
r58023 r58036 89 89 90 90 def _run_command_with_teed_output(self, args, teed_output): 91 args = map(unicode, args) # Popen will throw an exception if args are non-strings (like int()) 91 92 child_process = subprocess.Popen(args, 92 93 stdout=subprocess.PIPE, … … 100 101 if output_line == "" and child_process.poll() != None: 101 102 return child_process.poll() 103 # We assume that the child process wrote to us in utf-8, 104 # so no re-encoding is necessary before writing here. 102 105 teed_output.write(output_line) 103 106 104 def run_and_throw_if_fail(self, args, quiet=False): 107 # FIXME: Remove this deprecated method and move callers to run_command. 108 # FIXME: This method is a hack to allow running command which both 109 # capture their output and print out to stdin. Useful for things 110 # like "build-webkit" where we want to display to the user that we're building 111 # but still have the output to stuff into a log file. 112 def run_and_throw_if_fail(self, args, quiet=False, decode_output=True): 105 113 # Cache the child's output locally so it can be used for error reports. 106 114 child_out_file = StringIO.StringIO() 107 115 tee_stdout = sys.stdout 108 116 if quiet: 109 dev_null = open(os.devnull, "w") 117 dev_null = open(os.devnull, "w") # FIXME: Does this need an encoding? 110 118 tee_stdout = dev_null 111 119 child_stdout = tee(child_out_file, tee_stdout) … … 116 124 child_output = child_out_file.getvalue() 117 125 child_out_file.close() 126 127 # We assume the child process output utf-8 128 if decode_output: 129 child_output = child_output.decode("utf-8") 118 130 119 131 if exit_code: … … 146 158 # using Cygwin, it worked fine. We should investigate whether 147 159 # we need this platform specific code here. 148 subprocess.call(('taskkill.exe', '/f', '/pid', str(pid)),160 subprocess.call(('taskkill.exe', '/f', '/pid', unicode(pid)), 149 161 stdin=open(os.devnull, 'r'), 150 162 stdout=subprocess.PIPE, … … 164 176 pass 165 177 166 # FIXME: This should be merged with run_and_throw_if_fail 167 178 def _compute_stdin(self, input): 179 """Returns (stdin, string_to_communicate)""" 180 if not input: 181 return (None, None) 182 if hasattr(input, "read"): # Check if the input is a file. 183 return (input, None) # Assume the file is in the right encoding. 184 185 # Popen in Python 2.5 and before does not automatically encode unicode objects. 186 # http://bugs.python.org/issue5290 187 # See https://bugs.webkit.org/show_bug.cgi?id=37528 188 # for an example of a regresion caused by passing a unicode string directly. 189 # FIXME: We may need to encode differently on different platforms. 190 if isinstance(input, unicode): 191 input = input.encode("utf-8") 192 return (subprocess.PIPE, input) 193 194 # FIXME: run_and_throw_if_fail should be merged into this method. 168 195 def run_command(self, 169 196 args, … … 172 199 error_handler=None, 173 200 return_exit_code=False, 174 return_stderr=True): 175 if hasattr(input, 'read'): # Check if the input is a file. 176 stdin = input 177 string_to_communicate = None 178 else: 179 stdin = None 180 if input: 181 stdin = subprocess.PIPE 182 # string_to_communicate seems to need to be a str for proper 183 # communication with shell commands. 184 # See https://bugs.webkit.org/show_bug.cgi?id=37528 185 # For an example of a regresion caused by passing a unicode string through. 186 string_to_communicate = str(input) 187 if return_stderr: 188 stderr = subprocess.STDOUT 189 else: 190 stderr = None 201 return_stderr=True, 202 decode_output=True): 203 args = map(unicode, args) # Popen will throw an exception if args are non-strings (like int()) 204 stdin, string_to_communicate = self._compute_stdin(input) 205 stderr = subprocess.STDOUT if return_stderr else None 191 206 192 207 process = subprocess.Popen(args, … … 196 211 cwd=cwd) 197 212 output = process.communicate(string_to_communicate)[0] 213 # run_command automatically decodes to unicode() unless explicitly told not to. 214 if decode_output: 215 output = output.decode("utf-8") 198 216 exit_code = process.wait() 199 217 -
trunk/WebKitTools/Scripts/webkitpy/common/system/executive_unittest.py
r58023 r58036 1 # Copyright (C) 20 09Google Inc. All rights reserved.1 # Copyright (C) 2010 Google Inc. All rights reserved. 2 2 # Copyright (C) 2009 Daniel Bates (dbates@intudata.com). All rights reserved. 3 3 # … … 32 32 from webkitpy.common.system.executive import Executive, run_command 33 33 34 34 35 class ExecutiveTest(unittest.TestCase): 35 36 … … 39 40 self.failUnlessRaises(OSError, run_bad_command) 40 41 41 if __name__ == '__main__': 42 unittest.main() 42 def test_run_command_with_unicode(self): 43 """Validate that it is safe to pass unicode() objects 44 to Executive.run* methods, and they will return unicode() 45 objects by default unless decode_output=False""" 46 executive = Executive() 47 unicode_tor = u"WebKit \u2661 Tor Arne Vestb\u00F8!" 48 utf8_tor = unicode_tor.encode("utf-8") 49 50 output = executive.run_command(["cat"], input=unicode_tor) 51 self.assertEquals(output, unicode_tor) 52 53 output = executive.run_command(["echo", "-n", unicode_tor]) 54 self.assertEquals(output, unicode_tor) 55 56 output = executive.run_command(["echo", "-n", unicode_tor], decode_output=False) 57 self.assertEquals(output, utf8_tor) 58 59 # FIXME: We should only have one run* method to test 60 output = executive.run_and_throw_if_fail(["echo", "-n", unicode_tor], quiet=True) 61 self.assertEquals(output, unicode_tor) 62 63 output = executive.run_and_throw_if_fail(["echo", "-n", unicode_tor], quiet=True, decode_output=False) 64 self.assertEquals(output, utf8_tor) -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/dump_render_tree_thread.py
r58023 r58036 36 36 """ 37 37 38 from __future__ import with_statement 39 40 import codecs 38 41 import copy 39 42 import logging … … 90 93 filename = os.path.splitext(filename)[0] + "-stack.txt" 91 94 port.maybe_make_directory(os.path.split(filename)[0]) 92 open(filename, "wb").write(error) # FIXME: This leaks a file handle. 95 with codecs.open(filename, "wb", "utf-8") as file: 96 file.write(error) 93 97 elif error: 94 98 _log.debug("Previous test output extra lines after dump:\n%s" % … … 291 295 tests_run_filename = os.path.join(self._options.results_directory, 292 296 "tests_run.txt") 293 tests_run_file = open(tests_run_filename, "a")297 tests_run_file = codecs.open(tests_run_filename, "a", "utf-8") 294 298 295 299 while True: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py
r58023 r58036 28 28 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 29 30 from __future__ import with_statement 31 32 import codecs 30 33 import logging 31 34 import os … … 119 122 json = self._get_json() 120 123 if json: 121 results_file = open(self._results_file_path, "w")124 results_file = codecs.open(self._results_file_path, "w", "utf-8") 122 125 results_file.write(json) 123 126 results_file.close() … … 152 155 153 156 if os.path.exists(self._results_file_path): 154 old_results_file = open(self._results_file_path, "r")155 old_results = old_results_file.read()157 with codecs.open(self._results_file_path, "r", "utf-8") as file: 158 old_results = file.read() 156 159 elif self._builder_base_url: 157 160 # Check if we have the archived JSON file on the buildbot server. -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/apache_http_server.py
r58023 r58036 30 30 """A class to start/stop the apache http server used by layout tests.""" 31 31 32 33 from __future__ import with_statement 34 35 import codecs 32 36 import logging 33 37 import optparse … … 152 156 httpd_config = self._port_obj._path_to_apache_config_file() 153 157 httpd_config_copy = os.path.join(output_dir, "httpd.conf") 154 httpd_conf = open(httpd_config).read() 158 # httpd.conf is always utf-8 according to http://archive.apache.org/gnats/10125 159 with codecs.open(httpd_config, "r", "utf-8") as httpd_config_file: 160 httpd_conf = httpd_config_file.read() 155 161 if self._is_cygwin(): 156 162 # This is a gross hack, but it lets us use the upstream .conf file … … 165 171 'ServerRoot "%s"' % self._get_cygwin_path(cygusr)) 166 172 167 f = open(httpd_config_copy, 'wb') 168 f.write(httpd_conf) 169 f.close() 173 with codecs.open(httpd_config_copy, "w", "utf-8") as file: 174 file.write(httpd_conf) 170 175 171 176 if self._is_cygwin(): -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium.py
r58023 r58036 30 30 """Chromium implementations of the Port interface.""" 31 31 32 from __future__ import with_statement 33 34 import codecs 32 35 import logging 33 36 import os … … 141 144 offset = abspath.find('third_party') 142 145 if offset == -1: 146 # FIXME: This seems like the wrong error to throw. 143 147 raise AssertionError('could not find Chromium base dir from ' + 144 148 abspath) … … 205 209 Basically this string should contain the equivalent of a 206 210 test_expectations file. See test_expectations.py for more details.""" 207 expectations_file = self.path_to_test_expectations_file() 208 return file(expectations_file, "r").read() 211 expectations_path = self.path_to_test_expectations_file() 212 with codecs.open(expectations_file, "r", "utf-8") as file: 213 return file.read() 209 214 210 215 def test_expectations_overrides(self): … … 214 219 except AssertionError: 215 220 return None 216 if os.path.exists(overrides_file): 217 return file(overrides_file, "r").read() 218 else: 221 if not os.path.exists(overrides_file): 219 222 return None 223 with codecs.open(overrides_file, "r", "utf-8") as file 224 return file.read() 220 225 221 226 def test_platform_names(self): … … 266 271 267 272 class ChromiumDriver(base.Driver): 268 """Abstract interface for t he DumpRenderTree interface."""273 """Abstract interface for test_shell.""" 269 274 270 275 def __init__(self, port, image_path, options): … … 390 395 _log.warning('stopping test driver timed out, ' 391 396 'killing it') 392 null = open(os.devnull, "w") 397 # FIXME: This should use Executive. 398 null = open(os.devnull, "w") # Does this need an encoding? 393 399 subprocess.Popen(["kill", "-9", 394 400 str(self._proc.pid)], stderr=null) -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium_linux.py
r58023 r58036 127 127 128 128 def _kill_all_process(self, process_name): 129 # FIXME: This should use Executive. 129 130 null = open(os.devnull) 130 131 subprocess.call(['killall', '-TERM', '-u', os.getenv('USER'), -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/chromium_mac.py
r58023 r58036 100 100 101 101 def _check_wdiff_install(self): 102 # FIXME: This should use Executive. 102 103 f = open(os.devnull, 'w') 103 104 rcode = 0 … … 117 118 def _kill_all_process(self, process_name): 118 119 """Kill any processes running under this name.""" 120 # FIXME: This should use Executive. 119 121 # On Mac OS X 10.6, killall has a new constraint: -SIGNALNAME or 120 122 # -SIGNALNUMBER must come first. Example problem: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/gtk.py
r58023 r58036 63 63 64 64 def _kill_all_process(self, process_name): 65 # FIXME: This should use Executive. 65 66 null = open(os.devnull) 66 67 subprocess.call(['killall', '-TERM', '-u', os.getenv('USER'), -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/http_server.py
r55603 r58036 30 30 """A class to help start/stop the lighttpd server used by layout tests.""" 31 31 32 32 from __future__ import with_statement 33 34 import codecs 33 35 import logging 34 36 import optparse … … 115 117 116 118 # Write out the config 117 f = file(base_conf_file, 'rb') 118 base_conf = f.read() 119 f.close() 120 121 f = file(out_conf_file, 'wb') 119 with codecs.open(base_conf_file, "r", "utf-8") as file: 120 base_conf = file.read() 121 122 # FIXME: This should be re-worked so that this block can 123 # use with open() instead of a manual file.close() call. 124 # lighttpd.conf files seem to be UTF-8 without BOM: 125 # http://redmine.lighttpd.net/issues/992 126 f = codecs.open(out_conf_file, "w", "utf-8") 122 127 f.write(base_conf) 123 128 -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/mac.py
r58023 r58036 134 134 # FIXME: This doesn't have anything to do with WebKit. 135 135 def _kill_all_process(self, process_name): 136 # FIXME: This should use Executive. 136 137 # On Mac OS X 10.6, killall has a new constraint: -SIGNALNAME or 137 138 # -SIGNALNUMBER must come first. Example problem: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/mac_unittest.py
r58023 r58036 41 41 self.assertEqual(relative_paths, ['LayoutTests/platform/mac-leopard/Skipped', 'LayoutTests/platform/mac/Skipped']) 42 42 43 example_skipped_file = """43 example_skipped_file = u""" 44 44 # <rdar://problem/5647952> fast/events/mouseout-on-window.html needs mac DRT to issue mouse out events 45 45 fast/events/mouseout-on-window.html -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/qt.py
r58023 r58036 64 64 65 65 def _kill_all_process(self, process_name): 66 # FIXME: This should use Executive. 66 67 null = open(os.devnull) 67 68 subprocess.call(['killall', '-TERM', '-u', os.getenv('USER'), -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/server_process.py
r58023 r58036 216 216 _log.warning('stopping %s timed out, killing it' % 217 217 self._name) 218 # FIXME: This should use Executive. 218 219 null = open(os.devnull, "w") 219 220 subprocess.Popen(["kill", "-9", -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py
r58023 r58036 30 30 """WebKit implementations of the Port interface.""" 31 31 32 33 from __future__ import with_statement 34 35 import codecs 32 36 import logging 33 37 import os … … 135 139 136 140 actual_length = os.stat(actual_filename).st_size 137 actual_file = open(actual_filename).read() 141 with open(actual_filename) as file: 142 actual_file = file.read() 138 143 expected_length = os.stat(expected_filename).st_size 139 expected_file = open(expected_filename).read() 144 with open(expected_filename) as file: 145 expected_file = file.read() 140 146 sp.write('Content-Length: %d\n%sContent-Length: %d\n%s' % 141 147 (actual_length, actual_file, expected_length, expected_file)) … … 166 172 result = False 167 173 elif output and diff_filename: 168 open(diff_filename, 'w').write(output) # FIXME: This leaks a file handle. 174 with open(diff_filename, 'w') as file: 175 file.write(output) 169 176 elif sp.timed_out: 170 177 _log.error("ImageDiff timed out on %s" % expected_filename) … … 253 260 _log.warn("Failed to open Skipped file: %s" % filename) 254 261 continue 255 skipped_file = file(filename) 256 tests_to_skip.extend(self._tests_from_skipped_file(skipped_file)) 257 skipped_file.close() 262 with codecs.open(filename, "r", "utf-8") as skipped_file: 263 tests_to_skip.extend(self._tests_from_skipped_file(skipped_file)) 258 264 return tests_to_skip 259 265 … … 261 267 # The WebKit mac port uses a combination of a test_expectations file 262 268 # and 'Skipped' files. 263 expectations_ file= self.path_to_test_expectations_file()264 expectations = file(expectations_file, "r").read()265 return expectations+ self._skips()269 expectations_path = self.path_to_test_expectations_file() 270 with codecs.open(expectations_path, "r", "utf-8") as file: 271 return file.read() + self._skips() 266 272 267 273 def _skips(self): … … 397 403 have_seen_content_type = False 398 404 actual_image_hash = None 399 output = ''405 output = u'' 400 406 image = '' 401 407 … … 410 416 have_seen_content_type = True 411 417 else: 412 output += line 418 # We expect the text content from DumpRenderTree to be UTF-8 419 output += line.decode("utf-8") 413 420 line = self._server_process.read_line(timeout) 414 421 timeout = deadline - time.time() … … 434 441 435 442 if self._image_path and len(self._image_path): 436 image_file = file(self._image_path, "wb") 437 image_file.write(image) 438 image_file.close() 443 with open(self._image_path, "wb") as image_file: 444 image_file.write(image) 439 445 return (self._server_process.crashed, 440 446 self._server_process.timed_out, -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/websocket_server.py
r58023 r58036 31 31 32 32 33 from __future__ import with_statement 34 35 import codecs 33 36 import logging 34 37 import optparse … … 152 155 153 156 output_log = os.path.join(self._output_dir, log_file_name + "-out.txt") 154 self._wsout = open(output_log, "w")157 self._wsout = codecs.open(output_log, "w", "utf-8") 155 158 156 159 python_interp = sys.executable … … 217 220 url = url + '://127.0.0.1:%d/' % self._port 218 221 if not url_is_alive(url): 219 fp = open(output_log)222 fp = codecs.open(output_log, "utf-8") 220 223 try: 221 224 for line in fp: … … 232 235 'Failed to start %s server.' % self._server_name) 233 236 if self._pidfile: 234 f = open(self._pidfile, 'w') 235 f.write("%d" % self._process.pid) 236 f.close() 237 with codecs.open(self._pidfile, "w", "ascii") as file: 238 file.write("%d" % self._process.pid) 237 239 238 240 def stop(self, force=False): … … 244 246 pid = self._process.pid 245 247 elif self._pidfile: 246 f = open(self._pidfile) 247 pid = int(f.read().strip()) 248 f.close() 248 with codecs.open(self._pidfile, "r", "ascii") as file: 249 pid = int(f.read().strip()) 249 250 250 251 if not pid: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/rebaseline_chromium_webkit_tests.py
r58023 r58036 42 42 """ 43 43 44 from __future__ import with_statement 45 46 import codecs 44 47 import copy 45 48 import logging … … 599 602 _log.info('Saving original file to "%s"', backup_file) 600 603 os.rename(path, backup_file) 601 f = open(path, "w") 602 f.write(new_expectations) 603 f.close() 604 # FIXME: What encoding are these files? 605 # Or is new_expectations always a byte array? 606 with open(path, "w") as file: 607 file.write(new_expectations) 604 608 else: 605 609 _log.info('No test was rebaselined so nothing to remove.') … … 716 720 baseline_filename, self._platform, 717 721 'old') 718 f = open(base_file, 'wb') 719 f.write(output) 720 f.close() 722 # FIXME: This assumes run_shell returns a byte array. 723 # We should be using an explicit encoding here. 724 with open(base_file, "wb") as file: 725 file.write(output) 721 726 _log.info(' Html: created old baseline file: "%s".', 722 727 base_file) … … 749 754 self._options.html_directory, baseline_filename, 750 755 self._platform, 'diff') 751 f = open(diff_file, 'wb')752 f.write(output)753 f.close()756 # FIXME: This assumes run_shell returns a byte array, not unicode() 757 with open(diff_file, 'wb') as file: 758 file.write(output) 754 759 _log.info(' Html: created baseline diff file: "%s".', 755 760 diff_file) … … 836 841 _log.debug(html) 837 842 838 f = open(self._html_file, 'w') 839 f.write(html) 840 f.close() 843 with codecs.open(self._html_file, "w", "utf-8") as file: 844 file.write(html) 841 845 842 846 _log.info('Baseline comparison html generated at "%s"', -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests.py
r58023 r58036 368 368 tests_run_filename = os.path.join(self._options.results_directory, 369 369 "tests_run.txt") 370 tests_run_file = open(tests_run_filename, "w") 371 tests_run_file.write(tests_run_msg + "\n") 372 tests_run_file.close() 370 with codecs.open(tests_run_filename, "w", "utf-8") as file: 371 file.write(tests_run_msg + "\n") 373 372 374 373 len_skip_chunk = int(len(files) * len(skipped) / … … 967 966 dashboard). 968 967 """ 969 _log.debug("Writing JSON files in %s." % 970 self._options.results_directory) 971 unexpected_file = open(os.path.join(self._options.results_directory, 972 "unexpected_results.json"), "w") 973 unexpected_file.write(simplejson.dumps(unexpected_results, 974 sort_keys=True, indent=2)) 975 unexpected_file.close() 968 results_directory = self._options.results_directory 969 _log.debug("Writing JSON files in %s." % results_directory) 970 unexpected_json_path = os.path.join(results_directory, "unexpected_results.json") 971 with codecs.open(unexpected_json_path, "w", "utf-8") as file: 972 simplejson.dump(unexpected_results, file, sort_keys=True, indent=2) 976 973 977 974 # Write a json file of the test_expectations.txt file for the layout 978 975 # tests dashboard. 979 expectations_file = open(os.path.join(self._options.results_directory, 980 "expectations.json"), "w") 976 expectations_path = os.path.join(results_directory, "expectations.json") 981 977 expectations_json = \ 982 978 self._expectations.get_expectations_json_for_all_platforms() 983 expectations_file.write("ADD_EXPECTATIONS(" + expectations_json + ");")984 expectations_file.close()979 with codecs.open(expectations_path, "w", "utf-8") as file: 980 file.write(u"ADD_EXPECTATIONS(%s);" % expectations_json) 985 981 986 982 json_layout_results_generator.JSONLayoutResultsGenerator( … … 1393 1389 out_filename = os.path.join(self._options.results_directory, 1394 1390 "results.html") 1395 out_file = open(out_filename, 'w') 1391 # FIXME: This should be re-written to prepare the string first 1392 # so that the "open" call can be wrapped in a with statement. 1393 out_file = codecs.open(out_filename, "w", "utf-8") 1396 1394 # header 1397 1395 if self._options.full_results_html: … … 1406 1404 for test_file in test_files: 1407 1405 test_failures = result_summary.failures.get(test_file, []) 1408 out_file.write( "<p><a href='%s'>%s</a><br />\n"1406 out_file.write(u"<p><a href='%s'>%s</a><br />\n" 1409 1407 % (self._port.filename_to_uri(test_file), 1410 1408 self._port.relative_test_filename(test_file))) 1411 1409 for failure in test_failures: 1412 out_file.write( " %s<br/>"1410 out_file.write(u" %s<br/>" 1413 1411 % failure.result_html_output( 1414 1412 self._port.relative_test_filename(test_file))) … … 1417 1415 # footer 1418 1416 out_file.write("</body></html>\n") 1417 out_file.close() 1419 1418 return True 1420 1419 … … 1433 1432 tests = [] 1434 1433 for file in files: 1435 for line in open(file): 1434 # FIXME: This could be cleaner using a list comprehension. 1435 for line in codecs.open(file, "r", "utf-8"): 1436 1436 line = test_expectations.strip_comments(line) 1437 1437 if line: … … 1508 1508 unexpected_results_filename = os.path.join( 1509 1509 options.results_directory, "unexpected_results.json") 1510 f = file(unexpected_results_filename) 1511 results = simplejson.load(f) 1512 f.close() 1510 with open(unexpected_results_filename, "r", "utf-8") as file: 1511 results = simplejson.load(file) 1513 1512 last_unexpected_results = results['tests'].keys() 1514 1513 if options.print_unexpected_results: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/test_types/image_diff.py
r58023 r58036 35 35 """ 36 36 37 from __future__ import with_statement 38 39 import codecs 37 40 import errno 38 41 import logging … … 79 82 checksum: value of the actual checksum result 80 83 """ 81 png_file = open(png_path, "rb") 82 png_data = png_file.read() 83 png_file.close() 84 with open(png_path, "rb") as png_file: 85 png_data = png_file.read() 84 86 self._save_baseline_data(filename, png_data, ".png") 85 87 self._save_baseline_data(filename, checksum, ".checksum") … … 141 143 _log.debug('Using %s' % expected_png_file) 142 144 145 # FIXME: We repeat this pattern often, we should share code. 143 146 try: 144 expected_hash = open(expected_hash_file, "r").read() 147 with codecs.open(expected_hash_file, "r", "ascii") as file: 148 expected_hash = file.read() 145 149 except IOError, e: 146 150 if errno.ENOENT != e.errno: -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/test_types/test_type_base.py
r58023 r58036 33 33 """ 34 34 35 from __future__ import with_statement 36 37 import codecs 35 38 import cgi 36 39 import errno … … 151 154 raise NotImplemented 152 155 153 def _write_into_file_at_path(self, file_path, contents): 154 file = open(file_path, "wb") 155 file.write(contents) 156 file.close() 156 def _write_into_file_at_path(self, file_path, byte_array): 157 """This method assumes that byte_array is already encoded 158 into the right format.""" 159 with open(file_path, "wb") as file: 160 file.write(byte_array) 157 161 158 162 def write_output_files(self, port, filename, file_type, … … 189 193 diff = port.diff_text(expected, output, expected_filename, actual_filename) 190 194 diff_filename = self.output_filename(filename, self.FILENAME_SUFFIX_DIFF + file_type) 191 self._write_into_file_at_path(diff_filename, diff )195 self._write_into_file_at_path(diff_filename, diff.encode("utf-8")) 192 196 193 197 # Shell out to wdiff to get colored inline diffs. 194 198 wdiff = port.wdiff_text(expected_filename, actual_filename) 195 199 wdiff_filename = self.output_filename(filename, self.FILENAME_SUFFIX_WDIFF) 196 self._write_into_file_at_path(wdiff_filename, wdiff )200 self._write_into_file_at_path(wdiff_filename, wdiff.encode("utf-8")) 197 201 198 202 # Use WebKit's PrettyPatch.rb to get an HTML diff. 199 203 pretty_patch = port.pretty_patch_text(diff_filename) 200 204 pretty_patch_filename = self.output_filename(filename, self.FILENAME_SUFFIX_PRETTY_PATCH) 201 self._write_into_file_at_path(pretty_patch_filename, pretty_patch )205 self._write_into_file_at_path(pretty_patch_filename, pretty_patch.encode("utf-8")) -
trunk/WebKitTools/Scripts/webkitpy/layout_tests/test_types/text_diff.py
r58023 r58036 34 34 """ 35 35 36 from __future__ import with_statement 37 38 import codecs 36 39 import errno 37 40 import logging … … 71 74 72 75 def get_normalized_text(self, filename): 76 # FIXME: We repeat this pattern often, we should share code. 73 77 try: 74 text = open(filename).read() 78 with codecs.open(filename, "r", "utf-8") as file: 79 text = file.read() 75 80 except IOError, e: 76 81 if errno.ENOENT != e.errno: -
trunk/WebKitTools/Scripts/webkitpy/style/filereader_unittest.py
r58023 r58036 23 23 """Contains unit tests for filereader.py.""" 24 24 25 from __future__ import with_statement 26 27 import codecs 25 28 import os 26 29 import shutil … … 68 71 shutil.rmtree(self._temp_dir) 69 72 70 def _create_file(self, rel_path, text ):73 def _create_file(self, rel_path, text, encoding="utf-8"): 71 74 """Create a file with given text and return the path to the file.""" 75 # FIXME: There are better/more secure APIs for creatin tmp file paths. 72 76 file_path = os.path.join(self._temp_dir, rel_path) 73 74 file = open(file_path, 'w') 75 file.write(text) 76 file.close() 77 77 with codecs.open(file_path, "w", encoding) as file: 78 file.write(text) 78 79 return file_path 79 80 -
trunk/WebKitTools/Scripts/webkitpy/thirdparty/__init__.py
r58023 r58036 25 25 """Autoinstalls third-party code required by WebKit.""" 26 26 27 from __future__ import with_statement 28 29 import codecs 27 30 import os 28 31 … … 89 92 readme_path = os.path.join(autoinstalled_dir, "README") 90 93 if not os.path.exists(readme_path): 91 file = open(readme_path, "w") 92 try: 94 with codecs.open(readme_path, "w", "ascii") as file: 93 95 file.write("This directory is auto-generated by WebKit and is " 94 96 "safe to delete.\nIt contains needed third-party Python " 95 97 "packages automatically downloaded from the web.") 96 finally:97 file.close() -
trunk/WebKitTools/Scripts/webkitpy/tool/commands/earlywarningsystem.py
r58023 r58036 26 26 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 29 from StringIO import StringIO30 28 31 29 from webkitpy.tool.commands.queues import AbstractReviewQueue -
trunk/WebKitTools/Scripts/webkitpy/tool/commands/queues.py
r58023 r58036 72 72 webkit_patch_args = [self.tool.path()] 73 73 # FIXME: This is a hack, we should have a more general way to pass global options. 74 # FIXME: We must always pass global options and their value in one argument 75 # because our global option code looks for the first argument which does 76 # not begin with "-" and assumes that is the command name. 74 77 webkit_patch_args += ["--status-host=%s" % self.tool.status_server.host] 75 webkit_patch_args += map(str,args)78 webkit_patch_args.extend(args) 76 79 return self.tool.executive.run_and_throw_if_fail(webkit_patch_args) 77 80 … … 124 127 message = "Error: %s" % message 125 128 output = script_error.message_with_output(output_limit=1024*1024) # 1MB 126 return tool.status_server.update_status(cls.name, message, state["patch"], StringIO(output)) 129 # We pre-encode the string to a byte array before passing it 130 # to status_server, because ClientForm (part of mechanize) 131 # wants a file-like object with pre-encoded data. 132 return tool.status_server.update_status(cls.name, message, state["patch"], StringIO(output.encode("utf-8"))) 127 133 128 134 -
trunk/WebKitTools/Scripts/webkitpy/tool/commands/queues_unittest.py
r58023 r58036 76 76 77 77 queue.run_webkit_patch(run_args) 78 expected_run_args = ["echo", "--status-host=example.com"] + map(str, run_args)78 expected_run_args = ["echo", "--status-host=example.com"] + run_args 79 79 tool.executive.run_and_throw_if_fail.assert_called_with(expected_run_args) 80 80 … … 151 151 1 patch in commit-queue [106] 152 152 """, 153 "process_work_item": "MOCK run_and_throw_if_fail: ['echo', '--status-host=example.com', 'land-attachment', '--force-clean', '--build', '--test', '--non-interactive', '--ignore-builders', '--build-style=both', '--quiet', '76543']\n",153 "process_work_item": "MOCK run_and_throw_if_fail: ['echo', '--status-host=example.com', 'land-attachment', '--force-clean', '--build', '--test', '--non-interactive', '--ignore-builders', '--build-style=both', '--quiet', 76543]\n", 154 154 } 155 155 self.assert_queue_outputs(CommitQueue(), tool=tool, work_item=rollout_patch, expected_stderr=expected_stderr) -
trunk/WebKitTools/Scripts/webkitpy/tool/commands/upload.py
r58023 r58036 31 31 import os 32 32 import re 33 import StringIO34 33 import sys 35 34 … … 261 260 return comment_text 262 261 263 def _diff_file_for_commit(self, tool, commit_id):264 diff = tool.scm().create_patch_from_local_commit(commit_id)265 return StringIO.StringIO(diff) # add_patch_to_bug expects a file-like object266 267 262 def execute(self, options, args, tool): 268 263 commit_ids = tool.scm().commit_ids_from_commitish_arguments(args) … … 285 280 have_obsoleted_patches.add(bug_id) 286 281 287 diff _file = self._diff_file_for_commit(tool,commit_id)282 diff = tool.scm().create_patch_from_local_commit(commit_id) 288 283 description = options.description or commit_message.description(lstrip=True, strip_url=True) 289 284 comment_text = self._comment_text_for_commit(options, commit_message, tool, commit_id) 290 tool.bugs.add_patch_to_bug(bug_id, diff _file, description, comment_text, mark_for_review=options.review, mark_for_commit_queue=options.request_commit)285 tool.bugs.add_patch_to_bug(bug_id, diff, description, comment_text, mark_for_review=options.review, mark_for_commit_queue=options.request_commit) 291 286 292 287 … … 405 400 406 401 diff = tool.scm().create_patch_from_local_commit(commit_id) 407 diff_file = StringIO.StringIO(diff) # create_bug expects a file-like object 408 bug_id = tool.bugs.create_bug(bug_title, comment_text, options.component, diff_file, "Patch", cc=options.cc, mark_for_review=options.review, mark_for_commit_queue=options.request_commit) 402 bug_id = tool.bugs.create_bug(bug_title, comment_text, options.component, diff, "Patch", cc=options.cc, mark_for_review=options.review, mark_for_commit_queue=options.request_commit) 409 403 410 404 if bug_id and len(commit_ids) > 1: … … 425 419 426 420 diff = tool.scm().create_patch() 427 diff_file = StringIO.StringIO(diff) # create_bug expects a file-like object 428 bug_id = tool.bugs.create_bug(bug_title, comment_text, options.component, diff_file, "Patch", cc=options.cc, mark_for_review=options.review, mark_for_commit_queue=options.request_commit) 421 bug_id = tool.bugs.create_bug(bug_title, comment_text, options.component, diff, "Patch", cc=options.cc, mark_for_review=options.review, mark_for_commit_queue=options.request_commit) 429 422 430 423 def prompt_for_bug_title_and_comment(self): -
trunk/WebKitTools/Scripts/webkitpy/tool/mocktool.py
r58023 r58036 261 261 bug_description, 262 262 component=None, 263 patch_file_object=None,263 diff=None, 264 264 patch_description=None, 265 265 cc=None, … … 303 303 def add_patch_to_bug(self, 304 304 bug_id, 305 patch_file_object,305 diff, 306 306 description, 307 307 comment_text=None, -
trunk/WebKitTools/Scripts/webkitpy/tool/steps/abstractstep.py
r58023 r58036 40 40 log("Running %s" % script_name) 41 41 # FIXME: This should use self.port() 42 self._tool.executive.run_and_throw_if_fail( port.script_path(script_name), quiet)42 self._tool.executive.run_and_throw_if_fail([port.script_path(script_name)], quiet) 43 43 44 44 # FIXME: The port should live on the tool. -
trunk/WebKitTools/Scripts/webkitpy/tool/steps/postdiff.py
r58023 r58036 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 28 29 import StringIO30 31 29 from webkitpy.tool.steps.abstractstep import AbstractStep 32 30 from webkitpy.tool.steps.options import Options … … 45 43 def run(self, state): 46 44 diff = self.cached_lookup(state, "diff") 47 diff_file = StringIO.StringIO(diff) # add_patch_to_bug expects a file-like object48 45 description = self._options.description or "Patch" 49 46 comment_text = None … … 53 50 if codereview_issue: 54 51 description += "-%s" % state["codereview_issue"] 55 self._tool.bugs.add_patch_to_bug(state["bug_id"], diff _file, description, comment_text=comment_text, mark_for_review=self._options.review, mark_for_commit_queue=self._options.request_commit)52 self._tool.bugs.add_patch_to_bug(state["bug_id"], diff, description, comment_text=comment_text, mark_for_review=self._options.review, mark_for_commit_queue=self._options.request_commit) 56 53 if self._options.open_bug: 57 54 self._tool.user.open_url(self._tool.bugs.bug_url_for_bug_id(state["bug_id"])) -
trunk/WebKitTools/Scripts/webkitpy/tool/steps/postdiffforcommit.py
r58023 r58036 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 28 29 import StringIO30 31 29 from webkitpy.tool.steps.abstractstep import AbstractStep 32 30 … … 36 34 self._tool.bugs.add_patch_to_bug( 37 35 state["bug_id"], 38 StringIO.StringIO(self.cached_lookup(state, "diff")),36 self.cached_lookup(state, "diff"), 39 37 "Patch for landing", 40 38 mark_for_review=False, -
trunk/WebKitTools/Scripts/webkitpy/tool/steps/postdiffforrevert.py
r58023 r58036 27 27 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 28 29 import StringIO30 31 29 from webkitpy.common.net.bugzilla import Attachment 32 30 from webkitpy.tool.steps.abstractstep import AbstractStep … … 45 43 self._tool.bugs.add_patch_to_bug( 46 44 state["bug_id"], 47 StringIO.StringIO(self.cached_lookup(state, "diff")),45 self.cached_lookup(state, "diff"), 48 46 "%s%s" % (Attachment.rollout_preamble, state["revision"]), 49 47 comment_text=comment_text, -
trunk/WebKitTools/Scripts/webkitpy/tool/steps/steps_unittest.py
r58023 r58036 49 49 options = Mock() 50 50 options.update = True 51 self._run_step(Update, options) 51 expected_stderr = "Updating working directory\n" 52 OutputCapture().assert_outputs(self, self._run_step, [Update, options], expected_stderr=expected_stderr) 52 53 53 54 def test_prompt_for_bug_or_title_step(self):
Note: See TracChangeset
for help on using the changeset viewer.