Home | History | Annotate | Download | only in dashboard 
      1 # VTS Dashboard
      2 
      3 ## Introduction
      4 
      5 The VTS Dashboard displays the summarized results of the Multi Device Tests along with graphs.
      6 
      7 ## Installation
      8 
      9 ### Steps to run locally:
     10 
     11 1. Google App Engine uses Java 8. Install Java 8 before running running locally:
     12    'sudo apt install openjdk-8-jdk'
     13 
     14    To use java 8:
     15    Copy the following lines in ~/.bashrc :
     16 
     17 ```
     18     function setup_jdk() {
     19       # Remove the current JDK from PATH
     20       if [ -n "$JAVA_HOME" ] ; then
     21         PATH=${PATH/$JAVA_HOME\/bin:/}
     22       fi
     23       export JAVA_HOME=$1
     24       export PATH=$JAVA_HOME/bin:$PATH
     25     }
     26 
     27     function use_java8() {
     28     #  setup_jdk /usr/java/jre1.8.0_73
     29       setup_jdk /usr/lib/jvm/java-8-openjdk-amd64
     30     }
     31 
     32     Then from cmd:
     33     $ use_java8
     34 ```
     35 
     36 2. Maven is used for build. Install Maven 3.3.9:
     37    Download maven from:
     38    https://maven.apache.org/download.cgi
     39 
     40    Steps to Install Maven:
     41    1) Unzip the Binary tar:
     42       tar -zxf apache-maven-3.3.3-bin.tar.gz
     43 
     44    2) Move the application directory to /usr/local
     45       sudo cp -R apache-maven-3.3.3 /usr/local
     46 
     47    3) Make a soft link in /usr/bin for universal access of mvn
     48       sudo ln -s /usr/local/apache-maven-3.3.3/bin/mvn /usr/bin/mvn
     49 
     50    4) Verify maven installation:
     51       $ mvn -v
     52 
     53       The output should resemble this:
     54 
     55       Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
     56       Maven home: /opt/apache-maven-3.3.9
     57       Java version: 1.8.0_45-internal, vendor: Oracle Corporation
     58       Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
     59       Default locale: en_US, platform encoding: UTF-8
     60       OS name: "linux", version: "3.13.0-88-generic", arch: "amd64", family: "unix"
     61 
     62 3. Install Google Cloud SDK. Follow the instructions listed on official source:
     63    https://cloud.google.com/sdk/docs/quickstart-linux
     64 
     65    The default location where the application searches for a google-cloud-sdk is:
     66    /usr/local/share/google/google-cloud-sdk
     67 
     68    Therefore move the extracted folder to this location: /usr/local/share/google/
     69 
     70    Otherwise, to have a custom location, specify the location of
     71    google-cloud-sdk in test/vti/dashboard/pom.xml by putting the configuration:
     72 
     73 ```
     74    <configuration>
     75      <gcloud_directory>PATH/TO/GCLOUD_DIRECTORY</gcloud_directory>
     76    </configuration>
     77 ```
     78    within the 'com.google.appengine' plugin tag :
     79 
     80 ## To run GAE on local machine:
     81 
     82 $ cd test/vti/dashboard
     83 $ mvn appengine:devserver
     84 
     85 ## To deploy to Google App Engine
     86 
     87 $ cd test/vti/dashboard
     88 $ mvn appengine:update
     89 
     90 visit https://<YOUR-PROJECT-NAME>.appspot.com
     91 
     92 ## Update config file through gcloud command
     93 
     94 You can deploy or update GAE's a config file without deploying the whole project.
     95 The next commands show how to do it.
     96 
     97 ```
     98 gcloud app deploy --project=<YOUR-PROJECT-NAME> cron.xml
     99 gcloud app deploy --project=<YOUR-PROJECT-NAME> queue.xml
    100 gcloud app deploy --project=<YOUR-PROJECT-NAME> datastore-indexes.xml
    101 ```
    102 
    103 ## Test Data
    104 
    105 ### Purpose
    106 
    107 When you start your local GAE server, you will see empty page as the local datastore do not have any data.
    108 So we need to put some sample data into local datastore so that developers are able to continue to
    109 develop new features or fix bugs. Thus, we developed the next two test APIs, which are only available
    110 in your local dev environment.
    111 
    112 ```
    113 http://127.0.0.1:8080/api/test_data/report
    114 http://127.0.0.1:8080/api/test_data/plan
    115 ```
    116 
    117 ### How to set test data on json files for generating mock data on local dev server
    118 
    119 If you want to generate some mock data for your local development, you need to set some fake data
    120 on json files under the testdata folder. However, you need to abide by some rules in doing this,
    121 otherwise you will end up with errors from the mock data dev API.
    122 
    123 First, in test-plan-report-data.json, you need to set the same number of data under "testCaseNames"
    124 and "results". For example, if you put 5 elements of data in "testCaseNames", you should put the same
    125 number of data under "results".
    126 
    127 ```json
    128 ........
    129 "testCaseRunList": [
    130   {
    131     "testCaseNames": [
    132       "stdatomic.atomic_exchange_64bit",
    133       "stdatomic.atomic_compare_exchange_64bit",
    134       "stdatomic.atomic_exchange_64bit",
    135       "stdatomic.atomic_compare_exchange_64bit",
    136       "stdatomic.atomic_exchange_64bit",
    137       "stdatomic.atomic_compare_exchange_64bit"
    138     ],
    139     "results": [
    140       2,
    141       2,
    142       2,
    143       2,
    144       2,
    145       2
    146     ]
    147   }
    148 ],
    149 ........
    150 ```
    151 
    152 Second, in test-report-data.json file, you need to make sure that "testModules" should have
    153 the "testName"'s value under "testRunList" and the "testTimes" should have the "startTimestamp"'s value
    154 in the test-report-data.json file.
    155 
    156 test-report-data.json
    157 ```json
    158 ......
    159   "testRunList": [
    160     {
    161       "testName": "BionicUnitTests", <- "testModules" should be copied from here
    162       "type": 1,
    163       "startTimestamp":1515562811, <- "testTimes" should be copied from here
    164 ......
    165     {
    166       "testName": "CpuProfilingTest", <- "testModules" should be copied from here
    167       "type": 2,
    168       "startTimestamp":1515562811, <- "testTimes" should be copied from here
    169 ......
    170 ```
    171 
    172 test-plan-report-data.json
    173 ```json
    174 ......
    175   {
    176     "testPlanName": "vts-serving-staging-fuzz",
    177     "testModules": ["BionicUnitTests", "CpuProfilingTest"],
    178     "testTimes": [1515562811, 1515562811]
    179   },
    180   {
    181     "testPlanName": "vts-serving-staging-hal-conventional",
    182     "testModules": ["BionicUnitTests", "CpuProfilingTest"],
    183     "testTimes": [1515562811, 1515562811]
    184   }
    185 ......
    186 ```
    187 "testModules" and "testTimes"'s elements order is also matter.
    188 
    189 ### Command to generate mock data through API
    190 
    191 The next two commands will generate mock data in your local dev datastore.
    192 The execution order of the commands is very important, otherwise you can't find some of the data in your local datastore.
    193 Thus, please execute the below command as I wrote in order.
    194 
    195 ```
    196 curl -d @testdata/test-report-data.json -m 30 -X POST http://127.0.0.1:8080/api/test_data/report -H "Content-Type: application/json" --verbose
    197 curl -d @testdata/test-plan-report-data.json -m 30 -X POST http://127.0.0.1:8080/api/test_data/plan -H "Content-Type: application/json" --verbose
    198 ```
    199 
    200 ## Monitoring
    201 
    202 The following steps list how to create a monitoring service for the VTS Dashboard.
    203 
    204 ### Create a Stackdriver account
    205 
    206 1. Go to Google Cloud Platform Console:
    207    http://console.developers.google.com
    208 
    209 2. In the Google Cloud Platform Console, select Stackdriver > Monitoring.
    210    If your project is not in a Stackdriver account you'll see a message to
    211    create a new project.
    212 
    213 3. Click Create new Stackdriver account and then Continue.
    214 
    215 4. With your project shown, click Create account.
    216 
    217 5. In the page, "Add Google Cloud Platform projects to monitor", click Continue to skip ahead.
    218 
    219 6. In the page, "Monitor AWS accounts", click Done to skip ahead.
    220 
    221 7. In a few seconds you see the following message:
    222    "Finished Initial collection"
    223    Click Launch Monitoring.
    224 
    225 8. In the page, "Get reports by email", click No reports and Continue.
    226 
    227 9. You will see your Stackdriver account dashboard.
    228    Close the "Welcome to Stackdriver" banner if you don't need it.
    229 
    230 ### Steps to create an uptime check and an alerting policy
    231 
    232 1. Go to Stack Monitoring console:
    233    https://app.google.stackdriver.com/
    234 
    235 2. Go to Alerting > Uptime Checks in the top menu and then click Add Uptime Check.
    236    You see the New Uptime Check panel.
    237 
    238 3. Fill in the following fields for the uptime check:
    239 
    240     Check type: HTTP
    241     Resource Type: Instance
    242     Applies To: Single, lamp-1-vm
    243     Leave the other fields with their default values.
    244 
    245 4. Click Test to verify your uptime check is working.
    246 
    247 5. Click Save. After you click on save you'll see a panel to
    248    'Create Alerting Policy'
    249 
    250 6. Fill out the configuration for notifications and click save policy.
    251 
    252 ### Test the check and alert
    253 
    254 This procedure can take up to fifteen minutes.
    255 
    256 To test the check and alert, go to the VM Instances page, select your instance, and click Stop from the top menu.
    257 You'll have to wait up to five minutes for the next uptime check to fail. The alert and notification don't happen until the next failure occurs.
    258 
    259 To correct the "problem," return to the VM Instances page, select your instance, and click Start from the top menu.
    260