Changeset 73991 in webkit


Ignore:
Timestamp:
Dec 13, 2010 5:42:35 PM (13 years ago)
Author:
eric@webkit.org
Message:

2010-12-13 Eric Seidel <eric@webkit.org>

Reviewed by Adam Barth.

FlakyTestReporter doesn't understand bots running from multiple email addresses
https://bugs.webkit.org/show_bug.cgi?id=50960

This explains at least one of the dupes of:
https://bugs.webkit.org/show_bug.cgi?id=50863
that we saw filed by the commit-queue this morning.
I think the other one was explained by my previous fix to result counting code.

Since this is really hard to test with a unit test, instead I
create a new (possibly useful in the future) command
which given a layout test path will return you
the one bug which our tools would assume it the flaky test bug.
If some other script wants to use bug-for-test we'll
need to extend it with some options like --create-if-missing or similar.

  • Scripts/webkitpy/common/net/bugzilla/bug.py:
  • Scripts/webkitpy/common/net/bugzilla/bugzilla.py:
  • Scripts/webkitpy/tool/bot/flakytestreporter.py:
  • Scripts/webkitpy/tool/commands/init.py:
  • Scripts/webkitpy/tool/commands/bugfortest.py: Added.
Location:
trunk/WebKitTools
Files:
1 added
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/WebKitTools/ChangeLog

    r73985 r73991  
     12010-12-13  Eric Seidel  <eric@webkit.org>
     2
     3        Reviewed by Adam Barth.
     4
     5        FlakyTestReporter doesn't understand bots running from multiple email addresses
     6        https://bugs.webkit.org/show_bug.cgi?id=50960
     7
     8        This explains at least one of the dupes of:
     9        https://bugs.webkit.org/show_bug.cgi?id=50863
     10        that we saw filed by the commit-queue this morning.
     11        I think the other one was explained by my previous fix to result counting code.
     12
     13        Since this is really hard to test with a unit test, instead I
     14        create a new (possibly useful in the future) command
     15        which given a layout test path will return you
     16        the one bug which our tools would assume it the flaky test bug.
     17        If some other script wants to use bug-for-test we'll
     18        need to extend it with some options like --create-if-missing or similar.
     19
     20        * Scripts/webkitpy/common/net/bugzilla/bug.py:
     21        * Scripts/webkitpy/common/net/bugzilla/bugzilla.py:
     22        * Scripts/webkitpy/tool/bot/flakytestreporter.py:
     23        * Scripts/webkitpy/tool/commands/__init__.py:
     24        * Scripts/webkitpy/tool/commands/bugfortest.py: Added.
     25
    1262010-12-13  Mihai Parparita  <mihaip@chromium.org>
    227
  • trunk/WebKitTools/Scripts/webkitpy/common/net/bugzilla/bug.py

    r73823 r73991  
    4646    def title(self):
    4747        return self.bug_dictionary["title"]
     48
     49    def reporter_email(self):
     50        return self.bug_dictionary["reporter_email"]
    4851
    4952    def assigned_to_email(self):
  • trunk/WebKitTools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py

    r73950 r73991  
    149149        return self._fetch_bugs_from_advanced_query(quicksearch_url)
    150150
     151    # Currently this returns all bugs across all components.
     152    # In the future we may wish to extend this API to construct more restricted searches.
    151153    def fetch_bugs_matching_search(self, search_string, author_email=None):
    152154        query = "buglist.cgi?query_format=advanced"
  • trunk/WebKitTools/Scripts/webkitpy/tool/bot/flakytestreporter.py

    r73838 r73991  
    5555        return self._tool.bugs.username
    5656
     57    # FIXME: This should move into common.config
     58    _bot_emails = set([
     59        "commit-queue@webkit.org",  # commit-queue
     60        "eseidel@chromium.org",  # old commit-queue
     61        "webkit.review.bot@gmail.com",  # style-queue, sheriff-bot, CrLx/Gtk EWS
     62        "buildbot@hotmail.com",  # Win EWS
     63        # Mac EWS currently uses eric@webkit.org, but that's not normally a bot
     64    ])
     65
    5766    def _lookup_bug_for_flaky_test(self, flaky_test):
    58         bot_email = self._bugzilla_email()
    59         bugs = self._tool.bugs.queries.fetch_bugs_matching_search(search_string=flaky_test, author_email=bot_email)
     67        bugs = self._tool.bugs.queries.fetch_bugs_matching_search(search_string=flaky_test)
     68        if not bugs:
     69            return None
     70        # Match any bugs which are from known bots or the email this bot is using.
     71        allowed_emails = self._bot_emails | set([self._bugzilla_email])
     72        bugs = filter(lambda bug: bug.reporter_email() in allowed_emails, bugs)
    6073        if not bugs:
    6174            return None
    6275        if len(bugs) > 1:
    63             _log.warn("Found %s %s matching '%s' from the %s (%s), using the first." % (pluralize('bug', len(bugs)), bugs, flaky_test, self._bot_name, bot_email))
     76            # FIXME: There are probably heuristics we could use for finding
     77            # the right bug instead of the first, like open vs. closed.
     78            _log.warn("Found %s %s matching '%s' filed by a bot, using the first." % (pluralize('bug', len(bugs)), [bug.id() for bug in bugs], flaky_test))
    6479        return bugs[0]
    6580
  • trunk/WebKitTools/Scripts/webkitpy/tool/commands/__init__.py

    r73688 r73991  
    22
    33from webkitpy.tool.commands.bugsearch import BugSearch
     4from webkitpy.tool.commands.bugfortest import BugForTest
    45from webkitpy.tool.commands.download import *
    56from webkitpy.tool.commands.earlywarningsystem import *
  • trunk/WebKitTools/Scripts/webkitpy/tool/mocktool.py

    r73951 r73991  
    146146# This matches one of Bug.unassigned_emails
    147147_unassigned_email = "webkit-unassigned@lists.webkit.org"
     148# This is needed for the FlakyTestReporter to believe the bug
     149# was filed by one of the webkitpy bots.
     150_commit_queue_email = "commit-queue@webkit.org"
    148151
    149152
     
    155158    "title": "Bug with two r+'d and cq+'d patches, one of which has an "
    156159             "invalid commit-queue setter.",
     160    "reporter_email": "foo@foo.com",
    157161    "assigned_to_email": _unassigned_email,
    158162    "attachments": [_patch1, _patch2],
     
    164168    "id": 75,
    165169    "title": "Bug with a patch needing review.",
     170    "reporter_email": "foo@foo.com",
    166171    "assigned_to_email": "foo@foo.com",
    167172    "attachments": [_patch3],
     
    173178    "id": 76,
    174179    "title": "The third bug",
     180    "reporter_email": "foo@foo.com",
    175181    "assigned_to_email": _unassigned_email,
    176182    "attachments": [_patch7],
     
    182188    "id": 77,
    183189    "title": "The fourth bug",
     190    "reporter_email": "foo@foo.com",
    184191    "assigned_to_email": "foo@foo.com",
    185192    "attachments": [_patch4, _patch5, _patch6],
     
    191198    "id": 78,
    192199    "title": "The fifth bug",
     200    "reporter_email": _commit_queue_email,
    193201    "assigned_to_email": "foo@foo.com",
    194202    "attachments": [],
Note: See TracChangeset for help on using the changeset viewer.