1 page.title=Testing from Other IDEs 2 parent.title=Testing 3 parent.link=index.html 4 @jd:body 5 6 <div id="qv-wrapper"> 7 <div id="qv"> 8 <h2>In this document</h2> 9 <ol> 10 <li> 11 <a href="#CreateTestProjectCommand">Working with Test Projects</a> 12 <ol> 13 <li> 14 <a href="#CreateTestProject">Creating a test project</a> 15 </li> 16 <li> 17 <a href="#UpdateTestProject">Updating a test project</a> 18 </li> 19 </ol> 20 </li> 21 <li> 22 <a href="#CreateTestApp">Creating a Test Package</a> 23 </li> 24 <li> 25 <a href="#RunTestsCommand">Running Tests</a> 26 <ol> 27 <li> 28 <a href="#RunTestsAnt">Quick build and run with Ant</a> 29 </li> 30 <li> 31 <a href="#RunTestsDevice">Running tests on a device or emulator</a> 32 </li> 33 </ol> 34 </li> 35 <li> 36 <a href="#AMSyntax">Using the Instrument Command</a> 37 <ol> 38 <li> 39 <a href="#AMOptionsSyntax">Instrument options</a> 40 </li> 41 <li> 42 <a href="#RunTestExamples">Instrument examples</a> 43 </li> 44 </ol> 45 </li> 46 </ol> 47 <h2>See Also</h2> 48 <ol> 49 <li> 50 <a href="{@docRoot}tools/testing/testing_android.html"> 51 Testing Fundamentals</a> 52 </li> 53 <li> 54 <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> 55 </li> 56 </ol> 57 </div> 58 </div> 59 <p> 60 This document describes how to create and run tests directly from the command line. 61 You can use the techniques described here if you are developing in an IDE other than Eclipse 62 or if you prefer to work from the command line. This document assumes that you already know how 63 to create a Android application in your programming environment. Before you start this 64 document, you should read the topic 65 <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, 66 which provides an overview of Android testing. 67 </p> 68 <p> 69 If you are developing in Eclipse with ADT, you can set up and run your tests 70 directly in Eclipse. For more information, please read 71 <a href="{@docRoot}tools/testing/testing_eclipse.html"> 72 Testing from Eclipse with ADT</a>. 73 </p> 74 <h2 id="CreateTestProjectCommand">Working with Test Projects</h2> 75 <p> 76 You use the <code>android</code> tool to create test projects. 77 You also use <code>android</code> to convert existing test code into an Android test project, 78 or to add the <code>test</code> Ant target to an existing Android test project. 79 These operations are described in more detail in the section <a href="#UpdateTestProject"> 80 Updating a test project</a>. The <code>test</code> target is described in 81 <a href="#RunTestsAnt">Quick build and run with Ant</a>. 82 </p> 83 <h3 id="CreateTestProject">Creating a test project</h3> 84 <p> 85 To create a test project with the <code>android</code> tool, enter: 86 </p> 87 <pre> 88 android create test-project -m <main_path> -n <project_name> -p <test_path> 89 </pre> 90 <p> 91 You must supply all the flags. The following table explains them in detail: 92 </p> 93 <table> 94 <tr> 95 <th>Flag</th> 96 <th>Value</th> 97 <th>Description</th> 98 </tr> 99 <tr> 100 <td><code>-m, --main</code></td> 101 <td> 102 Path to the project of the application under test, relative to the test package 103 directory. 104 </td> 105 <td> 106 For example, if the application under test is in <code>source/HelloAndroid</code>, and 107 you want to create the test project in <code>source/HelloAndroidTest</code>, then the 108 value of <code>--main</code> should be <code>../HelloAndroid</code>. 109 <p> 110 To learn more about choosing the location of test projects, please read 111 <a href="{@docRoot}tools/testing/testing_android.html#TestProjects"> 112 Testing Fundamentals</a>. 113 </p> 114 </td> 115 </tr> 116 <tr> 117 <td><code>-n, --name</code></td> 118 <td>Name that you want to give the test project.</td> 119 <td> </td> 120 </tr> 121 <tr> 122 <td><code>-p, --path</code></td> 123 <td>Directory in which you want to create the new test project.</td> 124 <td> 125 The <code>android</code> tool creates the test project files and directory structure 126 in this directory. If the directory does not exist, <code>android</code> creates it. 127 </td> 128 </tr> 129 </table> 130 <p> 131 If the operation is successful, <code>android</code> lists to STDOUT the names of the files 132 and directories it has created. 133 </p> 134 <p> 135 This creates a new test project with the appropriate directories and build files. The directory 136 structure and build file contents are identical to those in a regular Android application 137 project. They are described in detail in the topic 138 <a href="{@docRoot}tools/projects/index.html">Managing Projects</a>. 139 </p> 140 <p> 141 The operation also creates an <code>AndroidManifest.xml</code> file with instrumentation 142 information. When you run the test, Android uses this information to load the application you 143 are testing and control it with instrumentation. 144 </p> 145 <p> 146 For example, suppose you create a project in the directory <code>~/source/HelloAndroid</code>, 147 with the package name <code>com.example.helloandroid</code>, 148 and the activity name <code>HelloAndroid</code>. You can to create the test for this in 149 <code>~/source/HelloAndroidTest</code>. To do so, you enter: 150 </p> 151 <pre> 152 $ cd ~/source 153 $ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest 154 </pre> 155 <p> 156 This creates a directory called <code>~/src/HelloAndroidTest</code>. In the new directory you 157 see the file <code>AndroidManifest.xml</code>. This file contains the following 158 instrumentation-related elements and attributes: 159 </p> 160 <ul> 161 <li> 162 <code><application></code>: to contain the 163 <code><uses-library></code> element. 164 </li> 165 <li> 166 <code><uses-library android:name="android.test.runner"</code>: 167 specifies this testing application uses the <code>android.test.runner</code> library. 168 </li> 169 <li> 170 <code><instrumentation></code>: contains attributes that control Android 171 instrumentation. The attributes are: 172 <ul> 173 <li> 174 <code>android:name="android.test.InstrumentationTestRunner"</code>: 175 {@link android.test.InstrumentationTestRunner} runs test cases. It extends both 176 JUnit test case runner classes and Android instrumentation classes. 177 </li> 178 <li> 179 <code>android:targetPackage="com.example.helloandroid"</code>: specifies 180 that the tests in HelloAndroidTest should be run against the application with the 181 <em>Android</em> package name <code>com.example.helloandroid</code>. 182 </li> 183 <li> 184 <code>android:label="Tests for .HelloAndroid"</code>: specifies a 185 user-readable label for the instrumentation class. By default, 186 the <code>android</code> tool gives it the value "Tests for " plus 187 the name of the main Activity of the application under test. 188 </li> 189 </ul> 190 </li> 191 </ul> 192 <h3 id="UpdateTestProject">Updating a test project</h3> 193 <p> 194 You use the <code>android</code> tool when you need to change the path to the 195 project of the application under test. If you are changing an existing test project created in 196 Eclipse with ADT so that you can also build and run it from the command line, you must use the 197 "create" operation. See the section <a href="#CreateTestProject">Creating a test project</a>. 198 </p> 199 <p class="note"> 200 <strong>Note:</strong> If you change the Android package name of the application under test, 201 you must <em>manually</em> change the value of the <code><android:targetPackage></code> 202 attribute within the <code>AndroidManifest.xml</code> file of the test package. 203 Running <code>android update test-project</code> does not do this. 204 </p> 205 <p> 206 To update a test project with the <code>android</code> tool, enter: 207 </p> 208 <pre>android update test-project -m <main_path> -p <test_path></pre> 209 210 <table> 211 <tr> 212 <th>Flag</th> 213 <th>Value</th> 214 <th>Description</th> 215 </tr> 216 <tr> 217 <td><code>-m, --main</code></td> 218 <td>The path to the project of the application under test, relative to the test project</td> 219 <td> 220 For example, if the application under test is in <code>source/HelloAndroid</code>, and 221 the test project is in <code>source/HelloAndroidTest</code>, then the value for 222 <code>--main</code> is <code>../HelloAndroid</code>. 223 </td> 224 </tr> 225 <tr> 226 <td><code>-p, --path</code></td> 227 <td>The of the test project.</td> 228 <td> 229 For example, if the test project is in <code>source/HelloAndroidTest</code>, then the 230 value for <code>--path</code> is <code>HelloAndroidTest</code>. 231 </td> 232 </tr> 233 </table> 234 <p> 235 If the operation is successful, <code>android</code> lists to STDOUT the names of the files 236 and directories it has created. 237 </p> 238 <h2 id="CreateTestApp">Creating a Test Package</h2> 239 <p> 240 Once you have created a test project, you populate it with a test package. 241 The application does not require an {@link android.app.Activity Activity}, 242 although you can define one if you wish. Although your test package can 243 combine Activities, Android test class extensions, JUnit extensions, or 244 ordinary classes, you should extend one of the Android test classes or JUnit classes, 245 because these provide the best testing features. 246 </p> 247 <p> 248 If you run your tests with {@link android.test.InstrumentationTestRunner} 249 (or a related test runner), then it will run all the methods in each class. You can modify 250 this behavior by using the {@link junit.framework.TestSuite TestSuite} class. 251 </p> 252 253 <p> 254 To create a test package, start with one of Android's test classes in the Java package 255 {@link android.test android.test}. These extend the JUnit 256 {@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test 257 classes also provide instrumentation for testing. 258 </p> 259 <p> 260 For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to 261 override the <code>setUp()</code> and <code>tearDown()</code> methods: 262 </p> 263 <ul> 264 <li> 265 <code>setUp()</code>: This method is invoked before any of the test methods in the class. 266 Use it to set up the environment for the test. You can use <code>setUp()</code> 267 to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>. 268 You can then use this intent to start the Activity under test. 269 <p class="note"> 270 <strong>Note:</strong> If you override this method, call 271 <code>super.setUp()</code> as the first statement in your code. 272 </p> 273 </li> 274 <li> 275 <code>tearDown()</code>: This method is invoked after all the test methods in the class. Use 276 it to do garbage collection and re-setting before moving on to the next set of tests. 277 <p class="note"><strong>Note:</strong> If you override this method, you must call 278 <code>super.tearDown()</code> as the <em>last</em> statement in your code.</p> 279 </li> 280 </ul> 281 <p> 282 Another useful convention is to add the method <code>testPreConditions()</code> to your test 283 class. Use this method to test that the application under test is initialized correctly. If this 284 test fails, you know that that the initial conditions were in error. When this happens, further 285 test results are suspect, regardless of whether or not the tests succeeded. 286 </p> 287 <p> 288 To learn more about creating test packages, see the topic <a 289 href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, 290 which provides an overview of Android testing. If you prefer to follow a tutorial, 291 try the <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing</a> 292 tutorial, which leads you through the creation of tests for an actual Android application. 293 </p> 294 <h2 id="RunTestsCommand">Running Tests</h2> 295 <p> 296 You run tests from the command line, either with Ant or with an 297 <a href="{@docRoot}tools/help/adb.html"> 298 Android Debug Bridge (adb)</a> shell. 299 </p> 300 <h3 id="RunTestsAnt">Quick build and run with Ant</h3> 301 <p> 302 You can use Ant to run all the tests in your test project, using the target 303 <code>test</code>, which is created automatically when you create a test project with 304 the <code>android</code> tool. 305 </p> 306 <p> 307 This target re-builds your main project and test project if necessary, installs the test 308 application to the current AVD or device, and then runs all the test classes in the test 309 application. The results are directed to <code>STDOUT</code>. 310 </p> 311 <p> 312 You can update an existing test project to use this feature. To do this, use the 313 <code>android</code> tool with the <code>update test-project</code> option. This is described 314 in the section <a href="#UpdateTestProject">Updating a test project</a>. 315 </p> 316 <h3 id="RunTestsDevice">Running tests on a device or emulator</h3> 317 <p> 318 When you run tests from the command line with 319 <a href="{@docRoot}tools/help/adb.html"> 320 Android Debug Bridge (adb)</a>, you get more options for choosing the tests 321 to run than with any other method. You can select individual test methods, filter tests 322 according to their annotation, or specify testing options. Since the test run is controlled 323 entirely from a command line, you can customize your testing with shell scripts in various ways. 324 </p> 325 <p> 326 To run a test from the command line, you run <code>adb shell</code> to start a command-line 327 shell on your device or emulator, and then in the shell run the <code>am instrument</code> 328 command. You control <code>am</code> and your tests with command-line flags. 329 </p> 330 <p> 331 As a shortcut, you can start an <code>adb</code> shell, call <code>am instrument</code>, and 332 specify command-line flags all on one input line. The shell opens on the device or emulator, 333 runs your tests, produces output, and then returns to the command line on your computer. 334 </p> 335 <p> 336 To run a test with <code>am instrument</code>: 337 </p> 338 <ol> 339 <li> 340 If necessary, rebuild your main application and test package. 341 </li> 342 <li> 343 Install your test package and main application Android package files 344 (<code>.apk</code> files) to your current Android device or emulator</li> 345 <li> 346 At the command line, enter: 347 <pre> 348 $ adb shell am instrument -w <test_package_name>/<runner_class> 349 </pre> 350 <p> 351 where <code><test_package_name></code> is the Android package name of your test 352 application, and <code><runner_class></code> is the name of the Android test 353 runner class you are using. The Android package name is the value of the 354 <code>package</code> attribute of the <code>manifest</code> element in the manifest file 355 (<code>AndroidManifest.xml</code>) of your test package. The Android test runner 356 class is usually {@link android.test.InstrumentationTestRunner}. 357 </p> 358 <p> 359 Your test results appear in <code>STDOUT</code>. 360 </p> 361 </li> 362 </ol> 363 <p> 364 This operation starts an <code>adb</code> shell, then runs <code>am instrument</code> 365 with the specified parameters. This particular form of the command will run all of the tests 366 in your test package. You can control this behavior with flags that you pass to 367 <code>am instrument</code>. These flags are described in the next section. 368 </p> 369 <h2 id="AMSyntax">Using the am instrument Command</h2> 370 <p> 371 The general syntax of the <code>am instrument</code> command is: 372 </p> 373 <pre> 374 am instrument [flags] <test_package>/<runner_class> 375 </pre> 376 <p> 377 The main input parameters to <code>am instrument</code> are described in the following table: 378 </p> 379 <table> 380 <tr> 381 <th> 382 Parameter 383 </th> 384 <th> 385 Value 386 </th> 387 <th> 388 Description 389 </th> 390 </tr> 391 <tr> 392 <td> 393 <code><test_package></code> 394 </td> 395 <td> 396 The Android package name of the test package. 397 </td> 398 <td> 399 The value of the <code>package</code> attribute of the <code>manifest</code> 400 element in the test package's manifest file. 401 </td> 402 </tr> 403 <tr> 404 <td> 405 <code><runner_class></code> 406 </td> 407 <td> 408 The class name of the instrumented test runner you are using. 409 </td> 410 <td> 411 This is usually {@link android.test.InstrumentationTestRunner}. 412 </td> 413 </tr> 414 </table> 415 <p> 416 The flags for <code>am instrument</code> are described in the following table: 417 </p> 418 <table> 419 <tr> 420 <th> 421 Flag 422 </th> 423 <th> 424 Value 425 </th> 426 <th> 427 Description 428 </th> 429 </tr> 430 <tr> 431 <td> 432 <code>-w</code> 433 </td> 434 <td> 435 (none) 436 </td> 437 <td> 438 Forces <code>am instrument</code> to wait until the instrumentation terminates 439 before terminating itself. The net effect is to keep the shell open until the tests 440 have finished. This flag is not required, but if you do not use it, you will not 441 see the results of your tests. 442 </td> 443 </tr> 444 <tr> 445 <td> 446 <code>-r</code> 447 </td> 448 <td> 449 (none) 450 </td> 451 <td> 452 Outputs results in raw format. Use this flag when you want to collect 453 performance measurements, so that they are not formatted as test results. This flag is 454 designed for use with the flag <code>-e perf true</code> (documented in the section 455 <a href="#AMOptionsSyntax">Instrument options</a>). 456 </td> 457 </tr> 458 <tr> 459 <td> 460 <code>-e</code> 461 </td> 462 <td> 463 <test_options> 464 </td> 465 <td> 466 Provides testing options as key-value pairs. The 467 <code>am instrument</code> tool passes these to the specified instrumentation class 468 via its <code>onCreate()</code> method. You can specify multiple occurrences of 469 <code>-e <test_options></code>. The keys and values are described in the 470 section <a href="#AMOptionsSyntax">am instrument options</a>. 471 <p> 472 The only instrumentation class that uses these key-value pairs is 473 {@link android.test.InstrumentationTestRunner} (or a subclass). Using them with 474 any other class has no effect. 475 </p> 476 </td> 477 </tr> 478 </table> 479 480 <h3 id="AMOptionsSyntax">am instrument options</h3> 481 <p> 482 The <code>am instrument</code> tool passes testing options to 483 <code>InstrumentationTestRunner</code> or a subclass in the form of key-value pairs, 484 using the <code>-e</code> flag, with this syntax: 485 </p> 486 <pre> 487 -e <key> <value> 488 </pre> 489 <p> 490 Some keys accept multiple values. You specify multiple values in a comma-separated list. 491 For example, this invocation of <code>InstrumentationTestRunner</code> provides multiple 492 values for the <code>package</code> key: 493 </p> 494 <pre> 495 $ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \ 496 > com.android.test/android.test.InstrumentationTestRunner 497 </pre> 498 <p> 499 The following table describes the key-value pairs and their result. Please review the 500 <strong>Usage Notes</strong> following the table. 501 </p> 502 <table> 503 <tr> 504 <th>Key</th> 505 <th>Value</th> 506 <th>Description</th> 507 </tr> 508 <tr> 509 <td> 510 <code>package</code> 511 </td> 512 <td> 513 <Java_package_name> 514 </td> 515 <td> 516 The fully-qualified <em>Java</em> package name for one of the packages in the test 517 application. Any test case class that uses this package name is executed. Notice that 518 this is not an <em>Android</em> package name; a test package has a single 519 Android package name but may have several Java packages within it. 520 </td> 521 </tr> 522 <tr> 523 <td rowspan="2"><code>class</code></td> 524 <td><class_name></td> 525 <td> 526 The fully-qualified Java class name for one of the test case classes. Only this test 527 case class is executed. 528 </td> 529 </tr> 530 <tr> 531 <td><class_name><strong>#</strong>method name</td> 532 <td> 533 A fully-qualified test case class name, and one of its methods. Only this method is 534 executed. Note the hash mark (#) between the class name and the method name. 535 </td> 536 </tr> 537 <tr> 538 <td><code>func</code></td> 539 <td><code>true</code></td> 540 <td> 541 Runs all test classes that extend {@link android.test.InstrumentationTestCase}. 542 </td> 543 </tr> 544 <tr> 545 <td><code>unit</code></td> 546 <td><code>true</code></td> 547 <td> 548 Runs all test classes that do <em>not</em> extend either 549 {@link android.test.InstrumentationTestCase} or 550 {@link android.test.PerformanceTestCase}. 551 </td> 552 </tr> 553 <tr> 554 <td><code>size</code></td> 555 <td> 556 [<code>small</code> | <code>medium</code> | <code>large</code>] 557 </td> 558 <td> 559 Runs a test method annotated by size. The annotations are <code>@SmallTest</code>, 560 <code>@MediumTest</code>, and <code>@LargeTest</code>. 561 </td> 562 </tr> 563 <tr> 564 <td><code>perf</code></td> 565 <td><code>true</code></td> 566 <td> 567 Runs all test classes that implement {@link android.test.PerformanceTestCase}. 568 When you use this option, also specify the <code>-r</code> flag for 569 <code>am instrument</code>, so that the output is kept in raw format and not 570 re-formatted as test results. 571 </td> 572 </tr> 573 <tr> 574 <td><code>debug</code></td> 575 <td><code>true</code></td> 576 <td> 577 Runs tests in debug mode. 578 </td> 579 </tr> 580 <tr> 581 <td><code>log</code></td> 582 <td><code>true</code></td> 583 <td> 584 Loads and logs all specified tests, but does not run them. The test 585 information appears in <code>STDOUT</code>. Use this to verify combinations of other 586 filters and test specifications. 587 </td> 588 </tr> 589 <tr> 590 <td><code>emma</code></td> 591 <td><code>true</code></td> 592 <td> 593 Runs an EMMA code coverage analysis and writes the output to 594 <code>/data//coverage.ec</code> on the device. To override the file location, use the 595 <code>coverageFile</code> key that is described in the following entry. 596 <p class="note"> 597 <strong>Note:</strong> This option requires an EMMA-instrumented build of the test 598 application, which you can generate with the <code>coverage</code> target. 599 </p> 600 </td> 601 </tr> 602 <tr> 603 <td><code>coverageFile</code></td> 604 <td><code><filename></code></td> 605 <td> 606 Overrides the default location of the EMMA coverage file on the device. Specify this 607 value as a path and filename in UNIX format. The default filename is described in the 608 entry for the <code>emma</code> key. 609 </td> 610 </tr> 611 </table> 612 <strong><code>-e</code> Flag Usage Notes</strong> 613 <ul> 614 <li> 615 <code>am instrument</code> invokes 616 {@link android.test.InstrumentationTestRunner#onCreate(Bundle)} 617 with a {@link android.os.Bundle} containing the key-value pairs. 618 </li> 619 <li> 620 The <code>package</code> key takes precedence over the <code>class</code> key. If you 621 specifiy a package, and then separately specify a class within that package, Android 622 will run all the tests in the package and ignore the <code>class</code> key. 623 </li> 624 <li> 625 The <code>func</code> key and <code>unit</code> key are mutually exclusive. 626 </li> 627 </ul> 628 <h3 id="RunTestExamples">Usage examples</h3> 629 <p> 630 The following sections provide examples of using <code>am instrument</code> to run tests. 631 They are based on the following structure:</p> 632 <ul> 633 <li> 634 The test package has the Android package name <code>com.android.demo.app.tests</code> 635 </li> 636 <li> 637 There are three test classes: 638 <ul> 639 <li> 640 <code>UnitTests</code>, which contains the methods 641 <code>testPermissions</code> and <code>testSaveState</code>. 642 </li> 643 <li> 644 <code>FunctionTests</code>, which contains the methods 645 <code>testCamera</code>, <code>testXVGA</code>, and <code>testHardKeyboard</code>. 646 </li> 647 <li> 648 <code>IntegrationTests</code>, 649 which contains the method <code>testActivityProvider</code>. 650 </li> 651 </ul> 652 </li> 653 <li> 654 The test runner is {@link android.test.InstrumentationTestRunner}. 655 </li> 656 </ul> 657 <h4>Running the entire test package</h4> 658 <p> 659 To run all of the test classes in the test package, enter: 660 </p> 661 <pre> 662 $ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner 663 </pre> 664 <h4>Running all tests in a test case class</h4> 665 <p> 666 To run all of the tests in the class <code>UnitTests</code>, enter: 667 </p> 668 <pre> 669 $ adb shell am instrument -w \ 670 > -e class com.android.demo.app.tests.UnitTests \ 671 > com.android.demo.app.tests/android.test.InstrumentationTestRunner 672 </pre> 673 <p> 674 <code>am instrument</code> gets the value of the <code>-e</code> flag, detects the 675 <code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class. 676 </p> 677 <h4>Selecting a subset of tests</h4> 678 <p> 679 To run all of the tests in <code>UnitTests</code>, and the <code>testCamera</code> method in 680 <code>FunctionTests</code>, enter: 681 </p> 682 <pre> 683 $ adb shell am instrument -w \ 684 > -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \ 685 > com.android.demo.app.tests/android.test.InstrumentationTestRunner 686 </pre> 687 <p> 688 You can find more examples of the command in the documentation for 689 {@link android.test.InstrumentationTestRunner}. 690 </p> 691
