1 <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> 2 <html> 3 <head> 4 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 5 <title>org.eclipse.test.performance.ui readme</title> 6 </head> 7 8 <p><b><font size="+2">README</font></b><br> </p><p>last updated: October 26, 2005<br> 9 </p><p><b>Introduction</b><br> <br> This readme describes how to create performance 10 charts and tables similar to those available on the eclipse platform download 11 pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b> 12 stored in this project. Although the program is currently geared to generating 13 results for the Eclipse Platform builds, the program may work for you given the 14 following conditions:</p><ul> <li>The "eclipse.perf.config" system property 15 used to run tests used three keys config, build and jvm <ul> <li>for example, 16 "eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06". 17 <br> <br> </li></ul></li><li>Values for the "build" key begin with either an 'I' 18 or 'N' with the exception of the reference builds (for example, 3.0). <ul> <li>The 19 line graphs plot the reference build, all builds starting with 'I' and the seven 20 'N' builds preceding the current build.</li><li>This can be overriden with the use of the "-current.prefix" parameter where you can specify a comma-separated list of prefixes for build ids to include in the main line graph.<br> <br> </li></ul></li><li>Scenario names 21 all contain a prefix "<component>.test" <ul> <li>For example, 22 it is assumed that the scenario "org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()" 23 belongs to the "org.eclipse.help" component. </li><li>Unique scenario 24 prefixes are derived from all the scenarios in the database and are used to group 25 results.</li></ul></li></ul><p><br> For purposes of this document, the term "<a href="images/descriptions.html#fp">fingerprint</a>" 26 will be used to refer to a bar graph which represents the performance summary 27 at a global or component level for each test configuration.<br> <br> The term 28 "<a href="images/descriptions.html#sst">scenario status table</a>" will 29 be used to refer to a table of scenarios with green checks and red x indicators 30 below a component fingerprint. This table gives an overall view of the status 31 for all scenarios for a component for all configurations.<br> </p><p>The term 32 "<a href="images/descriptions.html#sr">scenario results</a>" will be 33 used to refer to a web page comparing current performance measurements to the 34 baseline measurements for one scenario on one configuration. The web page displays 35 the raw data for the current and baseline measurements and also displays line 36 graphs showing the measured values over the course of builds between the current 37 and baseline.</p><p><br> <b>Requirements</b> </p><ul> <li>Windows or Linux (x86) 38 operating system</li><li>Cloudscape or Derby plug-in</li><li>org.eclipse.releng.basebuilder</li><li>1.4 39 level jvm installed and on path</li></ul><p><b>Setup</b><br> </p><ul> <li>Checkout 40 org.eclipse.releng.basebuilder from cvs repository dev.eclipse.org:/home/eclipse 41 (HEAD should be fine).</li><li>Create a Cloudscape or derby plug-in as described 42 <A HREF="http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html?rev=HEAD&content-type=text/html#cloudscape">here</A>.</li><li>Set 43 the environment variable "dbloc" to the location of the database containing 44 the performance data<br> For example:<br> <ul> <li> export dbloc=net://minsky.ottawa.ibm.com 45 (Linux)</li><li>set dbloc==net://minsky.ottawa.ibm.com (Windows)<br> </li></ul></li></ul><p><b>Running 46 the program</b></p><p>From a terminal or dos prompt,<br> </p><OL><LI>Set the environment 47 variable "dbloc" to the location of the database containing the performance 48 data<BR> For example:<UL><LI>export dbloc=net://minsky.ottawa.ibm.com (Linux)</LI><LI>set 49 dbloc==net://minsky.ottawa.ibm.com (Windows)<BR><BR></LI></UL></LI><LI>cd 50 to org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR></LI><LI>If 51 running on Linux, execute "chmod 755 genresults"</LI><LI>Execute the 52 following command:<b><br> <br> genresults -baseline <baseline build id> 53 -current <current build id> -config <comma-separated list of configs> 54 -jvm <jvm name> -output <output dir> [-config.properties <properties>] 55 [highlight <buildid patterns>] [scenario.pattern <scenario prefix 56 patterns>] [-fingerprints][-scenarioresults][-current.prefix <comma-separated list of build id prefixes>][-baseline.prefix <comma-separated list of build id prefixes>]</b></LI></OL><br> Parameter descriptions: 57 <blockquote> <p>-baseline <baseline build id. A previous value used in the 58 "build" key in the eclipse.perf.config system property or the eclipse.perf.assertAgainst 59 property. > <br> -current <current build id. The value used in the "build" 60 key in the eclipse.perf.config system property.><br> -config <a comma-separated 61 list of names of configurations for which to produce results. The values used 62 should be the same as values specified for the "config" key in the eclipse.perf.config 63 system property.><br> -jvm <jvm description. The value used in the "jvm" 64 key in the eclipse.perf.config system property.><br> -output <path to an 65 output directory for fingerprints and scenarios for each configuration specified 66 in -config parameter. Fingerprint gifs and html files are generated in the directory 67 specified, configuration specific subdirectories are created to store scenario 68 results and line graphs.></p></blockquote><p><br> Optional arguments:</p><blockquote> 69 <p>-config.properties <semi-colon separated list of: config, alternate config 70 description. The value should be quoted if there are spaces in the value.><BR>of builds which match any prefix in this list. When omitted, a magenta line is draw on the graph which represents the baseline value specified in the -baseline parameter.><br> -highlight <Comma-separated 71 list of build Id prefixes(excluding 'I' and 'N'), which are used to add and highlight 72 the most recent matching buildId on line graphs."><br> -scenario.pattern 73 <Scenario prefix pattern used to query performance results database. Can be 74 used to generate results for a single component or subset of scenarios.><br> 75 -fingerprints or -scenarioresults <use one or the other to generate fingerprints 76 or scenario results only. Not specifying either will execute both.><BR>-baseline.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.assertAgainst property or eclipse.perf.config system property. Values used to generate a second line graph representing changes in repeated baseline test runs>.<BR>-current.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.config system property. Values here used to override default of "N,I". Used to select build id's to include in main line graph.></p></blockquote> 77 </html>
