Home | History | Annotate | Download | only in glbench 
      1 GLBench runs a bunch of OpenGL or OpenGL ES performance tests and writes 
      2 performance numbers to stdout and resulting images to a directory for 
      3 verification.
      4 
      5 For the test to pass the performance numbers have to be better than a predefined 
      6 threshold, while the resulting images have to be found in a repository of 
      7 reference images. As the image name encodes the raw pixel MD5 this can be 
      8 done as a simple file existence check. If we ever get too much pixel 
      9 variation using a tool like perceptualdiff to waive small differences 
     10 should be acceptable.
     11 
     12 
     13 Executable options
     14 ==================
     15 
     16 ./glbench [-save [-outdir=<directory>]]
     17 
     18 
     19 Example
     20 =======
     21 
     22 ./glbench -save -outdir=img
     23 # board_id: NVIDIA Corporation - Quadro FX 380/PCI/SSE2
     24 swap_swap                    =   214.77 us           [swap_swap.pixmd5-20dbc406b95e214a799a6a7f9c700d2f.png]
     25 clear_color                  =  4448.28 mpixels_sec  [clear_color.pixmd5-e3609de1022a164fe240a562c69367de.png]
     26 clear_depth                  = 10199.76 mpixels_sec  [clear_depth.pixmd5-e3609de1022a164fe240a562c69367de.png]
     27 clear_colordepth             =  3250.57 mpixels_sec  [clear_colordepth.pixmd5-e3609de1022a164fe240a562c69367de.png]
     28 clear_depthstencil           = 26447.22 mpixels_sec  [clear_depthstencil.pixmd5-e3609de1022a164fe240a562c69367de.png]
     29 [...]
     30 
     31 ls img
     32 clear_color.pixmd5-e3609de122a164fe240a562c69367de.png
     33 clear_colordepth.pixmd5-e3609de122a164fe240a562c69367de.png
     34 clear_colordepthstencil.pixmd5-e3609de122a164fe240a562c69367de.png
     35 compositing.pixmd5-7d02a16a7ac15cd6cbbc5c786f1.png
     36 [...]
     37 
     38 
     39 Running from the autotest harness
     40 =================================
     41 
     42 The autotest script graphics_GLBench.py will 
     43 0) run glbench -save
     44 1) first try to identify known buggy images by searching in 
     45        deps/glbench/glbench_knownbad_images.txt
     46 2) then identify good images by searching in 
     47        deps/glbench/glbench_reference_images.txt
     48 3) if it is unable to find the test image it will do a second lookup at
     49        http://commondatastorage.googleapis.com/chromeos-localmirror/distfiles/glbench_*
     50 4) TODO(ihf) use perceptualdiff to do a fuzzy compare
     51 5) raise an error if the image is completely unknown
     52 6) report performance numbers back to the harness
     53 
     54 
     55 Handling of reference images
     56 ============================
     57 
     58 Good reference images themselves are located at       ../glbench-images/glbench_reference_images/
     59 Images that have outstanding defects and an open bug filed are at ../glbench-images/glbench_knownbad_images/chromium-bug-NNNNN/
     60 When that bug is closed the directory should be moved to ../glbench-images/glbench_fixedbad_images/chromium-bug-NNNNN/
     61 
     62 To push out new reference images place them in the appropriate 
     63 directories (create a new bug if needed) and run 
     64     > update_glbench_image_filelists.sh 
     65 
     66 to update the image filelists in deps/glbench/glbench_reference_images.txt etc.
     67