1 # Admin Tasks for the Chrome Performance Dashboard 2 3 ## "Dashboard is down" check list 4 5 - Is app engine up? If not, just have to sit tight. You can check the status 6 here:[https://code.google.com/status/appengine](https://code.google.com/status/appengine) 7 - Check the[main app engine dashboard page](https://console.developers.google.com/appengine?project=chromeperf&moduleId=default). 8 - Are we over quota? 9 - Look at the error rates on the dashboard. 10 - Check the task queues. 11 - Test data not showing up 12 - Check [/new\_tests](https://chromeperf.appspot.com/new_tests). 13 - Search the logs 14 - Is the test internal-only and the user is logged out? 15 16 ## Scheduled downtime 17 18 If it's necessary at some point to have scheduled downtime, announce 19 it ahead of time. At least 2 days before the downtime (ideally more), 20 announce in these ways: 21 22 1. Use[/set\_warning\_message](https://chromeperf.appspot.com/set_warning_message)to 23 put a warning message on the dashboard itself. 24 2. Send an email to any Chromium perf sheriffs who will be affected, 25 or all perf sheriffs (`perf-sheriffs (a] chromium.org`). 26 3. Send an email to `chrome-perf-dashboard-announce (a] google.com`. 27 28 If possible, it's probably best to schedule it for Saturday, when usage 29 is likely to be relatively low. 30 31 ## Routine tasks 32 33 There are several routine tasks to do to set up the dashboard for a 34 user. The official process for this is to file bugs on crbug.com 35 with labels: 36 37 - `Performance-Dashboard-IPWhitelist` 38 - `Performance-Dashboard-BotWhitelist` 39 - `Performance-Dashboard-MonitoringRequest` 40 41 ### Editing sheriff rotations 42 43 You can view, create and edit sheriff rotations 44 at[/edit\_sheriffs](https://chromeperf.appspot.com/edit_sheriffs). 45 46 #### Adding a new sheriff 47 48 Its fine to add a new sheriff rotation any time a team wants alerts 49 to go to a new email address. Its fine to make a temporary sheriff 50 rotation for monitoring new tests before they are stable. Here are the 51 fields that need to be filled out: 52 53 - **Name**: This is the name of the sheriff 54 rotation. It will be listed in the drop-down 55 at[/alerts](https://chromeperf.appspot.com/alerts). 56 - **Rotation URL**: Some sheriff rotations have a URL for specifying 57 the email of the sheriff. For example, the Chromium Perf Sheriff URL 58 is[http://chromium-build.appspot.com/p/chromium/sheriff\_perf.js](http://chromium-build.appspot.com/p/chromium/sheriff_perf.js). 59 Most sheriff rotations dont have a URL, and if not its fine to leave 60 this blank and just specify an email address. 61 - **Notification Email**: 62 This is usually a mailing list that alerts should go to. However, 63 theres nothing stopping it from being an individuals email 64 account. It must be specified if there is no Rotation URL, but its 65 optional otherwise. 66 - **Days before alerting on missing data**: 67 Number of days before "stoppage alerts" are made; -1 for no alerts. 68 - **Internal-only**: If the tests this sheriff is monitoring are internal-only, 69 or the name of the sheriff rotation is sensitive, please 70 set this to "Yes". If set to "Yes", the sheriff rotation will only 71 show up on the alerts page for users logged in with google.com accounts. 72 - **Summarize Email**: By default, the perf dashboard sends one email 73 for each alert, as soon as it gets the alert. If that will add up to 74 too much mail, setting this to "Yes" will switch to a daily summary. 75 76 #### Monitoring tests 77 78 After creating a sheriffing rotation, you need to add the individual 79 tests to monitor. You do this by clicking on "Set a sheriff for a 80 group of tests". It asks for a pattern. Patterns match test paths, 81 which are of the form "Master/Bot/test-suite/graph/trace". You can replace 82 any part of the test path with a `*` for a wildcard. 83 84 The dashboard will list the matching tests before allowing you to apply 85 the pattern, so youll be able to check if the pattern is correct. 86 87 To remove a pattern, click "Remove a sheriff from a group of tests". 88 89 If you want to keep alerting on most of the tests in a pattern and 90 just disable alerting on a few noisy ones, you can add the "Disable 91 Alerting" anomaly threshold config to the noisy tests (see "Modify 92 anomaly threshold configs" below). 93 94 ### Setting up alert threshold configs 95 96 The default alert thresholds should work reasonably well for most test 97 data, but there are some graphs for which it may not be correct. If 98 there are invalid alerts, or the dashboard is not sending alerts when 99 you expect them, you may want to modify an alert threshold config. 100 101 To edit alert threshold configs, go 102 to[/edit\_anomaly\_configs](https://chromeperf.appspot.com/edit_anomaly_configs). 103 Add a new config with a descriptive name and a JSON mapping of parameters 104 to values. 105 106 ### Anomaly config debugger page 107 108 Start off by using the anomaly threshold debugging 109 page:[/debug\_alert](https://chromeperf.appspot.com/debug_alert). The 110 page shows the segmentation of the data that was given by the anomaly 111 finding algorithm. Based on the documentation, change the config 112 parameters to get the alerts where you want them. 113 114 ### Automatically applying labels to bugs 115 116 The dashboard can automatically apply labels to bugs filed on alerts, 117 based on which test triggered the alert. This is useful for flagging 118 the relevant teams attention. For example, the dashboard automatically 119 applies the label "Cr-Blink-JavaScript" to dromaeo regressions, 120 which cuts down on a lot of CC-ing by hand. 121 122 To make a label automatically applied to a bug, go 123 to[/edit\_sheriffs](https://chromeperf.appspot.com/edit_sheriffs)and 124 click "Set a bug lable to automatically apply to a group of 125 tests". Then type in a pattern as described in "Edit Sheriff 126 Rotations -> Monitoring Tests" section above, and type in the bug 127 label. Youll see a list of tests the label will be applied to before 128 you confirm. 129 130 To remove a label, go 131 to[/edit\_sheriffs](https://chromeperf.appspot.com/edit_sheriffs)and 132 click "Remove a bug label that automatically applies to a group of 133 tests". 134 135 ### Migrating and renaming data 136 137 When a test name changes, it is possible to migrate 138 the existing test data to use the new name. You 139 can do this by entering a pattern for the test name 140 at[/migrate\_test\_names](https://chromeperf.appspot.com/migrate_test_names). 141 142 ### Whitelisting senders of data 143 144 There are two types of whitelisting on the perf dashboard: 145 146 The IP whitelist is a list of IP addresses of machines which 147 are allowed to post data to /add\_point. This is to prevent 148 /add\_point from being spammed. You can add a bot to the IP whitelist 149 at[/ip\_whitelist](https://chromeperf.appspot.com/ip_whitelist). If 150 youre seeing 403 errors on your buildbots, the IPs to add are likely 151 already in the logs. Note that if you are seeing 500 errors, those are 152 not related to IP whitelisting. They are usually caused by an error in 153 the JSON data sent by the buildbot. If you cant tell by looking at 154 the JSON data what is going wrong, the easiest thing to do is to add a 155 unit test with the JSON to `add_point_test.py` and debug it from there. 156 157 The bot whitelist is a list of bot names which are publicly visible. If a 158 bot is not on the list, users must be logged into google.com accounts to 159 see the data for that bot. You can add or remove a bot from the whitelist 160 at[/bot\_whitelist](https://chromeperf.appspot.com/bot_whitelist), 161 and make a bots existing data publicly visible (or internal\_only) 162 at[/change\_internal\_only](https://chromeperf.appspot.com/change_internal_only). 163
