Home | History | Annotate | only in /external/deqp-deps/glslang
Up to higher level directory
NameDateSize
.appveyor.yml22-Oct-20202.4K
.clang-format22-Oct-2020355
.gitattributes22-Oct-2020523
.travis.yml22-Oct-20204.1K
Android.bp22-Oct-20204K
BUILD.gn22-Oct-20205.6K
build_overrides/22-Oct-2020
ChooseMSVCCRT.cmake22-Oct-20203.8K
CMakeLists.txt22-Oct-20204.7K
CODE_OF_CONDUCT.md22-Oct-2020280
External/22-Oct-2020
glslang/22-Oct-2020
gtests/22-Oct-2020
hlsl/22-Oct-2020
known_good.json22-Oct-2020501
known_good_khr.json22-Oct-2020464
LICENSE22-Oct-20205.3K
LICENSE.txt22-Oct-20205.3K
make-revision22-Oct-2020174
METADATA22-Oct-2020298
MODULE_LICENSE_BSD22-Oct-20200
ndk_test/22-Oct-2020
NOTICE22-Oct-20205.3K
OGLCompilersDLL/22-Oct-2020
OWNERS22-Oct-202039
README-spirv-remap.txt22-Oct-20205K
README.md22-Oct-202012.2K
SPIRV/22-Oct-2020
StandAlone/22-Oct-2020
Test/22-Oct-2020
update_glslang_sources.py22-Oct-20205K

README-spirv-remap.txt

      1 
      2 VERSION
      3 --------------------------------------------------------------------------------
      4 spirv-remap 0.97
      5 
      6 INTRO:
      7 --------------------------------------------------------------------------------
      8 spirv-remap is a utility to improve compression of SPIR-V binary files via
      9 entropy reduction, plus optional stripping of debug information and
     10 load/store optimization.  It transforms SPIR-V to SPIR-V, remapping IDs.  The
     11 resulting modules have an increased ID range (IDs are not as tightly packed
     12 around zero), but will compress better when multiple modules are compressed
     13 together, since compressor's dictionary can find better cross module
     14 commonality.
     15 
     16 Remapping is accomplished via canonicalization.  Thus, modules can be
     17 compressed one at a time with no loss of quality relative to operating on
     18 many modules at once.  The command line tool operates on multiple modules
     19 only in the trivial repetition sense, for ease of use.  The remapper API
     20 only accepts a single module at a time.
     21 
     22 There are two modes of use: command line, and a C++11 API.  Both are
     23 described below.
     24 
     25 spirv-remap is currently in an alpha state.  Although there are no known
     26 remapping defects, it has only been exercised on one real world game shader
     27 workload.
     28 
     29 
     30 FEEDBACK
     31 --------------------------------------------------------------------------------
     32 Report defects, enhancements requests, code improvements, etc to:
     33    spvremapper (a] lunarg.com
     34 
     35 
     36 COMMAND LINE USAGE:
     37 --------------------------------------------------------------------------------
     38 Examples are given with a verbosity of one (-v), but more verbosity can be
     39 had via -vv, -vvv, etc, or an integer parameter to --verbose, such as
     40 "--verbose 4".  With no verbosity, the command is silent and returns 0 on
     41 success, and a positive integer error on failure.
     42 
     43 Pre-built binaries for several OSs are available.  Examples presented are
     44 for Linux.  Command line arguments can be provided in any order.
     45 
     46 1. Basic ID remapping
     47 
     48 Perform ID remapping on all shaders in "*.spv", writing new files with
     49 the same basenames to /tmp/out_dir.
     50 
     51   spirv-remap -v --map all --input *.spv --output /tmp/out_dir
     52 
     53 2. Perform all possible size reductions
     54 
     55   spirv-remap-linux-64 -v --do-everything --input *.spv --output /tmp/out_dir
     56 
     57 Note that --do-everything is a synonym for:
     58 
     59   --map all --dce all --opt all --strip all
     60 
     61 API USAGE:
     62 --------------------------------------------------------------------------------
     63 
     64 The public interface to the remapper is defined in SPIRV/SPVRemapper.h as follows:
     65 
     66 namespace spv {
     67 
     68 class spirvbin_t
     69 {
     70 public:
     71    enum Options { ... };
     72    spirvbin_t(int verbose = 0);  // construct
     73 
     74    // remap an existing binary in memory
     75    void remap(std::vector<std::uint32_t>& spv, std::uint32_t opts = DO_EVERYTHING);
     76 
     77    // Type for error/log handler functions
     78    typedef std::function<void(const std::string&)> errorfn_t;
     79    typedef std::function<void(const std::string&)> logfn_t;
     80 
     81    // Register error/log handling functions (can be c/c++ fn, lambda fn, or functor)
     82    static void registerErrorHandler(errorfn_t handler) { errorHandler = handler; }
     83    static void registerLogHandler(logfn_t handler)     { logHandler   = handler; }
     84 };
     85 
     86 } // namespace spv
     87 
     88 The class definition is in SPVRemapper.cpp.
     89 
     90 remap() accepts an std::vector of SPIR-V words, modifies them per the
     91 request given in 'opts', and leaves the 'spv' container with the result.
     92 It is safe to instantiate one spirvbin_t per thread and process a different
     93 SPIR-V in each.
     94 
     95 The "opts" parameter to remap() accepts a bit mask of desired remapping
     96 options.  See REMAPPING AND OPTIMIZATION OPTIONS.
     97 
     98 On error, the function supplied to registerErrorHandler() will be invoked.
     99 This can be a standard C/C++ function, a lambda function, or a functor.
    100 The default handler simply calls exit(5); The error handler is a static
    101 member, so need only be set up once, not once per spirvbin_t instance.
    102 
    103 Log messages are supplied to registerLogHandler().  By default, log
    104 messages are eaten silently.  The log handler is also a static member.
    105 
    106 BUILD DEPENDENCIES:
    107 --------------------------------------------------------------------------------
    108  1. C++11 compatible compiler
    109  2. cmake
    110  3. glslang
    111 
    112 
    113 BUILDING
    114 --------------------------------------------------------------------------------
    115 The standalone remapper is built along side glslangValidator through its
    116 normal build process.
    117 
    118 
    119 REMAPPING AND OPTIMIZATION OPTIONS
    120 --------------------------------------------------------------------------------
    121 API:
    122    These are bits defined under spv::spirvbin_t::, and can be
    123    bitwise or-ed together as desired.
    124 
    125    MAP_TYPES      = canonicalize type IDs
    126    MAP_NAMES      = canonicalize named data
    127    MAP_FUNCS      = canonicalize function bodies
    128    DCE_FUNCS      = remove dead functions
    129    DCE_VARS       = remove dead variables
    130    DCE_TYPES      = remove dead types
    131    OPT_LOADSTORE  = optimize unneeded load/stores
    132    MAP_ALL        = (MAP_TYPES | MAP_NAMES | MAP_FUNCS)
    133    DCE_ALL        = (DCE_FUNCS | DCE_VARS | DCE_TYPES)
    134    OPT_ALL        = (OPT_LOADSTORE)
    135    ALL_BUT_STRIP  = (MAP_ALL | DCE_ALL | OPT_ALL)
    136    DO_EVERYTHING  = (STRIP | ALL_BUT_STRIP)
    137 
    138 

README.md

      1 Also see the Khronos landing page for glslang as a reference front end:
      2 
      3 https://www.khronos.org/opengles/sdk/tools/Reference-Compiler/
      4 
      5 The above page includes where to get binaries, and is kept up to date
      6 regarding the feature level of glslang.
      7 
      8 glslang
      9 =======
     10 
     11 [![Build Status](https://travis-ci.org/KhronosGroup/glslang.svg?branch=master)](https://travis-ci.org/KhronosGroup/glslang)
     12 [![Build status](https://ci.appveyor.com/api/projects/status/q6fi9cb0qnhkla68/branch/master?svg=true)](https://ci.appveyor.com/project/Khronoswebmaster/glslang/branch/master)
     13 
     14 An OpenGL and OpenGL ES shader front end and validator.
     15 
     16 There are several components:
     17 
     18 1. A GLSL/ESSL front-end for reference validation and translation of GLSL/ESSL into an AST.
     19 
     20 2. An HLSL front-end for translation of a broad generic HLL into the AST. See [issue 362](https://github.com/KhronosGroup/glslang/issues/362) and [issue 701](https://github.com/KhronosGroup/glslang/issues/701) for current status.
     21 
     22 3. A SPIR-V back end for translating the AST to SPIR-V.
     23 
     24 4. A standalone wrapper, `glslangValidator`, that can be used as a command-line tool for the above.
     25 
     26 How to add a feature protected by a version/extension/stage/profile:  See the
     27 comment in `glslang/MachineIndependent/Versions.cpp`.
     28 
     29 Tasks waiting to be done are documented as GitHub issues.
     30 
     31 Execution of Standalone Wrapper
     32 -------------------------------
     33 
     34 To use the standalone binary form, execute `glslangValidator`, and it will print
     35 a usage statement.  Basic operation is to give it a file containing a shader,
     36 and it will print out warnings/errors and optionally an AST.
     37 
     38 The applied stage-specific rules are based on the file extension:
     39 * `.vert` for a vertex shader
     40 * `.tesc` for a tessellation control shader
     41 * `.tese` for a tessellation evaluation shader
     42 * `.geom` for a geometry shader
     43 * `.frag` for a fragment shader
     44 * `.comp` for a compute shader
     45 
     46 There is also a non-shader extension
     47 * `.conf` for a configuration file of limits, see usage statement for example
     48 
     49 Building
     50 --------
     51 
     52 Instead of building manually, you can also download the binaries for your
     53 platform directly from the [master-tot release][master-tot-release] on GitHub.
     54 Those binaries are automatically uploaded by the buildbots after successful
     55 testing and they always reflect the current top of the tree of the master
     56 branch.
     57 
     58 ### Dependencies
     59 
     60 * A C++11 compiler.
     61   (For MSVS: 2015 is recommended, 2013 is fully supported/tested, and 2010 support is attempted, but not tested.)
     62 * [CMake][cmake]: for generating compilation targets.
     63 * make: _Linux_, ninja is an alternative, if configured.
     64 * [Python 2.7][python]: for executing SPIRV-Tools scripts. (Optional if not using SPIRV-Tools.)
     65 * [bison][bison]: _optional_, but needed when changing the grammar (glslang.y).
     66 * [googletest][googletest]: _optional_, but should use if making any changes to glslang.
     67 
     68 ### Build steps
     69 
     70 The following steps assume a Bash shell. On Windows, that could be the Git Bash
     71 shell or some other shell of your choosing.
     72 
     73 #### 1) Check-Out this project 
     74 
     75 ```bash
     76 cd <parent of where you want glslang to be>
     77 git clone https://github.com/KhronosGroup/glslang.git
     78 ```
     79 
     80 #### 2) Check-Out External Projects
     81 
     82 ```bash
     83 cd <the directory glslang was cloned to, "External" will be a subdirectory>
     84 git clone https://github.com/google/googletest.git External/googletest
     85 ```
     86 
     87 If you want to use googletest with Visual Studio 2013, you also need to check out an older version:
     88 
     89 ```bash
     90 # to use googletest with Visual Studio 2013
     91 cd External/googletest
     92 git checkout 440527a61e1c91188195f7de212c63c77e8f0a45
     93 cd ../..
     94 ```
     95 
     96 If you wish to assure that SPIR-V generated from HLSL is legal for Vulkan,
     97 or wish to invoke -Os to reduce SPIR-V size from HLSL or GLSL, install
     98 spirv-tools with this:
     99 
    100 ```bash
    101 ./update_glslang_sources.py
    102 ```
    103 
    104 #### 3) Configure
    105 
    106 Assume the source directory is `$SOURCE_DIR` and the build directory is
    107 `$BUILD_DIR`. First ensure the build directory exists, then navigate to it:
    108 
    109 ```bash
    110 mkdir -p $BUILD_DIR
    111 cd $BUILD_DIR
    112 ```
    113 
    114 For building on Linux:
    115 
    116 ```bash
    117 cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX="$(pwd)/install" $SOURCE_DIR
    118 # "Release" (for CMAKE_BUILD_TYPE) could also be "Debug" or "RelWithDebInfo"
    119 ```
    120 
    121 For building on Windows:
    122 
    123 ```bash
    124 cmake $SOURCE_DIR -DCMAKE_INSTALL_PREFIX="$(pwd)/install"
    125 # The CMAKE_INSTALL_PREFIX part is for testing (explained later).
    126 ```
    127 
    128 The CMake GUI also works for Windows (version 3.4.1 tested).
    129 
    130 Also, consider using `git config --global core.fileMode false` (or with `--local`) on Windows
    131 to prevent the addition of execution permission on files.
    132 
    133 #### 4) Build and Install
    134 
    135 ```bash
    136 # for Linux:
    137 make -j4 install
    138 
    139 # for Windows:
    140 cmake --build . --config Release --target install
    141 # "Release" (for --config) could also be "Debug", "MinSizeRel", or "RelWithDebInfo"
    142 ```
    143 
    144 If using MSVC, after running CMake to configure, use the
    145 Configuration Manager to check the `INSTALL` project.
    146 
    147 ### If you need to change the GLSL grammar
    148 
    149 The grammar in `glslang/MachineIndependent/glslang.y` has to be recompiled with
    150 bison if it changes, the output files are committed to the repo to avoid every
    151 developer needing to have bison configured to compile the project when grammar
    152 changes are quite infrequent. For windows you can get binaries from
    153 [GnuWin32][bison-gnu-win32].
    154 
    155 The command to rebuild is:
    156 
    157 ```bash
    158 bison --defines=MachineIndependent/glslang_tab.cpp.h
    159       -t MachineIndependent/glslang.y
    160       -o MachineIndependent/glslang_tab.cpp
    161 ```
    162 
    163 The above command is also available in the bash script at
    164 `glslang/updateGrammar`.
    165 
    166 Testing
    167 -------
    168 
    169 Right now, there are two test harnesses existing in glslang: one is [Google
    170 Test](gtests/), one is the [`runtests` script](Test/runtests). The former
    171 runs unit tests and single-shader single-threaded integration tests, while
    172 the latter runs multiple-shader linking tests and multi-threaded tests.
    173 
    174 ### Running tests
    175 
    176 The [`runtests` script](Test/runtests) requires compiled binaries to be
    177 installed into `$BUILD_DIR/install`. Please make sure you have supplied the
    178 correct configuration to CMake (using `-DCMAKE_INSTALL_PREFIX`) when building;
    179 otherwise, you may want to modify the path in the `runtests` script.
    180 
    181 Running Google Test-backed tests:
    182 
    183 ```bash
    184 cd $BUILD_DIR
    185 
    186 # for Linux:
    187 ctest
    188 
    189 # for Windows:
    190 ctest -C {Debug|Release|RelWithDebInfo|MinSizeRel}
    191 
    192 # or, run the test binary directly
    193 # (which gives more fine-grained control like filtering):
    194 <dir-to-glslangtests-in-build-dir>/glslangtests
    195 ```
    196 
    197 Running `runtests` script-backed tests:
    198 
    199 ```bash
    200 cd $SOURCE_DIR/Test && ./runtests
    201 ```
    202 
    203 ### Contributing tests
    204 
    205 Test results should always be included with a pull request that modifies
    206 functionality.
    207 
    208 If you are writing unit tests, please use the Google Test framework and
    209 place the tests under the `gtests/` directory.
    210 
    211 Integration tests are placed in the `Test/` directory. It contains test input
    212 and a subdirectory `baseResults/` that contains the expected results of the
    213 tests.  Both the tests and `baseResults/` are under source-code control.
    214 
    215 Google Test runs those integration tests by reading the test input, compiling
    216 them, and then compare against the expected results in `baseResults/`. The
    217 integration tests to run via Google Test is registered in various
    218 `gtests/*.FromFile.cpp` source files. `glslangtests` provides a command-line
    219 option `--update-mode`, which, if supplied, will overwrite the golden files
    220 under the `baseResults/` directory with real output from that invocation.
    221 For more information, please check `gtests/` directory's
    222 [README](gtests/README.md).
    223 
    224 For the `runtests` script, it will generate current results in the
    225 `localResults/` directory and `diff` them against the `baseResults/`.
    226 When you want to update the tracked test results, they need to be
    227 copied from `localResults/` to `baseResults/`.  This can be done by
    228 the `bump` shell script.
    229 
    230 You can add your own private list of tests, not tracked publicly, by using
    231 `localtestlist` to list non-tracked tests.  This is automatically read
    232 by `runtests` and included in the `diff` and `bump` process.
    233 
    234 Programmatic Interfaces
    235 -----------------------
    236 
    237 Another piece of software can programmatically translate shaders to an AST
    238 using one of two different interfaces:
    239 * A new C++ class-oriented interface, or
    240 * The original C functional interface
    241 
    242 The `main()` in `StandAlone/StandAlone.cpp` shows examples using both styles.
    243 
    244 ### C++ Class Interface (new, preferred)
    245 
    246 This interface is in roughly the last 1/3 of `ShaderLang.h`.  It is in the
    247 glslang namespace and contains the following.
    248 
    249 ```cxx
    250 const char* GetEsslVersionString();
    251 const char* GetGlslVersionString();
    252 bool InitializeProcess();
    253 void FinalizeProcess();
    254 
    255 class TShader
    256     setStrings(...);
    257     setEnvInput(EShSourceHlsl or EShSourceGlsl, stage,  EShClientVulkan or EShClientOpenGL, 100);
    258     setEnvClient(EShClientVulkan or EShClientOpenGL, EShTargetVulkan_1_0 or EShTargetVulkan_1_1 or EShTargetOpenGL_450);
    259     setEnvTarget(EShTargetSpv, EShTargetSpv_1_0 or EShTargetSpv_1_3);
    260     bool parse(...);
    261     const char* getInfoLog();
    262 
    263 class TProgram
    264     void addShader(...);
    265     bool link(...);
    266     const char* getInfoLog();
    267     Reflection queries
    268 ```
    269 
    270 See `ShaderLang.h` and the usage of it in `StandAlone/StandAlone.cpp` for more
    271 details.
    272 
    273 ### C Functional Interface (orignal)
    274 
    275 This interface is in roughly the first 2/3 of `ShaderLang.h`, and referred to
    276 as the `Sh*()` interface, as all the entry points start `Sh`.
    277 
    278 The `Sh*()` interface takes a "compiler" call-back object, which it calls after
    279 building call back that is passed the AST and can then execute a backend on it.
    280 
    281 The following is a simplified resulting run-time call stack:
    282 
    283 ```c
    284 ShCompile(shader, compiler) -> compiler(AST) -> <back end>
    285 ```
    286 
    287 In practice, `ShCompile()` takes shader strings, default version, and
    288 warning/error and other options for controlling compilation.
    289 
    290 Basic Internal Operation
    291 ------------------------
    292 
    293 * Initial lexical analysis is done by the preprocessor in
    294   `MachineIndependent/Preprocessor`, and then refined by a GLSL scanner
    295   in `MachineIndependent/Scan.cpp`.  There is currently no use of flex.
    296 
    297 * Code is parsed using bison on `MachineIndependent/glslang.y` with the
    298   aid of a symbol table and an AST.  The symbol table is not passed on to
    299   the back-end; the intermediate representation stands on its own.
    300   The tree is built by the grammar productions, many of which are
    301   offloaded into `ParseHelper.cpp`, and by `Intermediate.cpp`.
    302 
    303 * The intermediate representation is very high-level, and represented
    304   as an in-memory tree.   This serves to lose no information from the
    305   original program, and to have efficient transfer of the result from
    306   parsing to the back-end.  In the AST, constants are propogated and
    307   folded, and a very small amount of dead code is eliminated.
    308 
    309   To aid linking and reflection, the last top-level branch in the AST
    310   lists all global symbols.
    311 
    312 * The primary algorithm of the back-end compiler is to traverse the
    313   tree (high-level intermediate representation), and create an internal
    314   object code representation.  There is an example of how to do this
    315   in `MachineIndependent/intermOut.cpp`.
    316 
    317 * Reduction of the tree to a linear byte-code style low-level intermediate
    318   representation is likely a good way to generate fully optimized code.
    319 
    320 * There is currently some dead old-style linker-type code still lying around.
    321 
    322 * Memory pool: parsing uses types derived from C++ `std` types, using a
    323   custom allocator that puts them in a memory pool.  This makes allocation
    324   of individual container/contents just few cycles and deallocation free.
    325   This pool is popped after the AST is made and processed.
    326 
    327   The use is simple: if you are going to call `new`, there are three cases:
    328 
    329   - the object comes from the pool (its base class has the macro
    330     `POOL_ALLOCATOR_NEW_DELETE` in it) and you do not have to call `delete`
    331 
    332   - it is a `TString`, in which case call `NewPoolTString()`, which gets
    333     it from the pool, and there is no corresponding `delete`
    334 
    335   - the object does not come from the pool, and you have to do normal
    336     C++ memory management of what you `new`
    337 
    338 
    339 [cmake]: https://cmake.org/
    340 [python]: https://www.python.org/
    341 [bison]: https://www.gnu.org/software/bison/
    342 [googletest]: https://github.com/google/googletest
    343 [bison-gnu-win32]: http://gnuwin32.sourceforge.net/packages/bison.htm
    344 [master-tot-release]: https://github.com/KhronosGroup/glslang/releases/tag/master-tot
    345