Home | History | Annotate | only in /external/chromium_org/media/tools/layout_tests
Up to higher level directory
NameDateSize
bug.py16-Dec-20141.8K
graph/16-Dec-2014
layouttest_analyzer.py16-Dec-201421.9K
layouttest_analyzer_helpers.py16-Dec-201422.5K
layouttest_analyzer_helpers_unittest.py16-Dec-20149K
layouttest_analyzer_runner.py16-Dec-20146.5K
layouttests.py16-Dec-20148.7K
layouttests_unittest.py16-Dec-20141.9K
README16-Dec-20143.5K
result/16-Dec-2014
test_data/16-Dec-2014
test_expectations.py16-Dec-20144K
test_expectations_history.py16-Dec-20145K
test_expectations_history_unittest.py16-Dec-20142.5K
test_expectations_unittest.py16-Dec-20141.7K
testname/16-Dec-2014
trend_graph.py16-Dec-20143.2K
trend_graph_unittest.py16-Dec-20141.3K

README

      1 Layout test analyzer script.
      2 
      3 * Prerequisite:
      4 
      5 This script requires PySVN (http://pysvn.tigris.org/) to be installed on the
      6 machine.
      7 
      8 * How to execute
      9 
     10 python layouttest_analyzer.py (Please run with '-h' for available command-line
     11                                options)
     12 
     13 Example: python layouttest_analyzer.py -r imasaki (a] chromium.org 
     14 -t /var/www/analyzer/graph.html
     15 
     16 * File/Directory structure
     17 
     18 layout_tests/
     19   README: this file.
     20   *.py: python scripts.
     21   tmp/: temp files.
     22   test_data/: data for unit tests.
     23   graph/: the default location for graph files.
     24   anno/: the default location for bug annotation files.
     25   result/: the default location for the analyzer results.
     26   testname/: the default location for the test group definition files in CSV.
     27 
     28 * Execution overview.
     29 
     30 1) using PySVN, the script collects layout test of interest
     31 (specified in command-line parameter and testnames/*.csv file) from
     32 Webkit SVN repository as well as test description. Then, it gets the
     33 latest test expectation file and does the simple parsing. The
     34 parsed test expectation is joined with the layout tests using test name
     35 as its join key. The tests are classified into 'whole', 'skip',
     36 'nonskip' test groups. 'Whole' contains all tests, 'skip' contains
     37 tests that are skipped as specified in the test expectation file, 'nonskip'
     38 tests are in the test expectation file and not skipped (these are tests
     39 that need attention).
     40 
     41 2) the script reads the previous latest analyzer results, then,
     42 compares it with current results for each test group ('whole', 'skip',
     43 and 'nonskip'). It also looks into the SVN test expectation history
     44 diff for the time period for the test names of interest.
     45 
     46 3) all obtained information is sent out to a email list specified
     47 in the parameters in pretty HTML format.
     48 
     49 4) the trend graph is updated using the current results.
     50 
     51 * Components
     52 
     53 ** LayoutTests:
     54 
     55 A class to store test names in layout tests. The test names (including
     56 regular expression patterns) are read from a CSV file and used for
     57 getting layout test names from Webkit SVN as well as test description.
     58 
     59 ** TestExpectations:
     60 
     61 A class to model the content of test expectation file for analysis.
     62 The location of the test expectations file can be found in
     63 |TEST_EXPECTATIONS_LOCATIONS|. It is necessary to parse this
     64 file and store meaningful information for the analysis (joining with
     65 existing layout tests using a test name).  Instance variable
     66 |all_test_expectation_info| is used.  A test name such as
     67 'media/video-source-type.html' is used for the key to store
     68 information. However, a test name can appear multiple times in the
     69 test expectation file. So, the map should keep all the occurrence
     70 information. For example, the current test expectation file has the
     71 following two entries: 
     72 
     73 BUGWK58587 LINUX DEBUG GPU : media/video-zoom.html = IMAGE 
     74 BUGCR86714 MAC GPU : media/video-zoom.html = CRASH IMAGE 
     75 
     76 In this case, all_test_expectation_info['media/video-zoom.html'] will
     77 have a list with two elements, each of which is the map of the test
     78 expectation information.
     79 
     80 ** TestExpectationsHistory:
     81 
     82 A class to represent history of the test expectation file. The history
     83 is obtained by calling PySVN.log()/diff() APIs using the specified time
     84 period.
     85 
     86 ** TrendGraph: 
     87 
     88 A class to manage trend graph which is using Google Visualization
     89   APIs. Google Visualization API
     90   (http://code.google.com/apis/chart/interactive/docs/gallery/annotatedtimeline.html)
     91   is used to present the historical analyzer result. Currently, data
     92   is directly written to JavaScript file using file in-place
     93   replacement for simplicity.
     94