1 [The "BSD license"] 2 Copyright (c) 2010 Terence Parr 3 Maven Plugin - Copyright (c) 2009 Jim Idle 4 5 All rights reserved. 6 7 Redistribution and use in source and binary forms, with or without 8 modification, are permitted provided that the following conditions 9 are met: 10 1. Redistributions of source code must retain the above copyright 11 notice, this list of conditions and the following disclaimer. 12 2. Redistributions in binary form must reproduce the above copyright 13 notice, this list of conditions and the following disclaimer in the 14 documentation and/or other materials provided with the distribution. 15 3. The name of the author may not be used to endorse or promote products 16 derived from this software without specific prior written permission. 17 18 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR 19 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 20 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 21 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 22 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 23 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 27 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 29 ============================================================================ 30 31 This file contains the build instructions for the ANTLR toolset as 32 of version 3.1.3 and beyond. 33 34 The ANTLR toolset must be built using the Maven build system as 35 this build system updates the version numbers and controls the 36 whole build process. However, if you just want the latest build 37 and do not care to learn anything about Maven, then visit the 'target' 38 directories (for jars) under the depot mirror root here: 39 40 http://antlr.org/depot 41 42 If you are looking for the latest released version of ANTLR, then 43 visit the downloads page on the main antlr.org website. 44 45 These instructions are mainly for the ANTLR development team, 46 though you are free to build ANTLR yourself of course. 47 48 Source code Structure 49 ----------------------- 50 51 The main development branch of ANTLR is stored within the Perforce SCM at: 52 53 //depot/code/antlr/main/... 54 55 release branches are stored in Perforce like so: 56 57 //depot/code/antlr/release-3.1.3/... 58 59 In this top level directory, you will find a master build file for 60 Maven called pom.xml and you will also note that there are a number of 61 subdirectories: 62 63 tool - The ANTLR tool itself 64 runtime/Java - The ANTLR Java runtime 65 runtime/X - The runtime for language target X 66 gunit - The grammar test tool 67 antlr3-maven-plugin - The plugin tool for Maven allowing Maven 68 projects to process ANTLR grammars. 69 70 Each of these sub-directories also contains a file pom.xml that 71 controls the build of each sub-component (or module in Maven 72 parlance). 73 74 Build Parameters 75 ----------------- 76 77 Alongside each pom.xml (other than for the antlr3-maven-plugin), you 78 will see that there is a file called antlr.config. This file is called 79 a filter and should contain a set of key/value pairs in the same 80 manner as Java properties files: 81 82 antlr.something="Some config thang!" 83 84 When the build of any component happens, any values in the 85 antlr.config for the master build file and any values in the 86 antlr.config file for each component are made available to the 87 build. This is mainly used by the resource processor, which will 88 filter any file it finds under: src/main/resources/** and replace any 89 references such as ${antlr.something} with the actual value at the 90 time of the build. 91 92 Building 93 -------- 94 95 Building ANTLR is trivial, assuming that you have loaded Maven version 96 3.0.3 or better on to your build system and installed it as explained 97 here: 98 99 http://maven.apache.org/download.html 100 101 Note that the ANTLR toolset will ONLY build with version 3.0.3 of Maven 102 as of release 3.4. 103 104 If you are unfamiliar with Maven (and even if you are), the best 105 resource for learning about it is The Definitive Guide: 106 107 http://www.sonatype.com/books/maven-book/reference/public-book.html 108 109 The instructions here assume that Maven is installed and working correctly. 110 111 If this is the first time you have built the ANTLR toolset, you will 112 possibly need to install the master pom in your local repository 113 (however the build may be able to locate this in the ANTLR snapshot or 114 release repository). If you try to build sub-modules on their own (as 115 in run the mvn command in the sub directory for that tool, such as 116 runtime/Java), and you receive a message that maven cannot find the 117 master pom, then execute this in the main (or release) directory: 118 119 mvn -N install 120 121 This command will install the master build pom in your local maven 122 repository (it's ~/.m2 on UNIX) and individual builds of sub-modules 123 will now work correctly. 124 125 To build then, simply cd into the master build directory 126 (e.g. $P4ROOT//code/antlr/main) and type: 127 128 mvn -Dmaven.test.skip=true 129 130 Assuming that everything is correctly installed and synchronized, then 131 ANTLR will build and skip any unit tests in the modules (the ANTLR 132 tool tests can take a long time). 133 134 This command will build each of the tools in the correct order and 135 will create the jar artifacts of all the components in your local 136 development Maven repository (which takes precedence over remote 137 repositories by default). At the end of the build you should see: 138 139 [INFO] ------------------------------------------------------------------------ 140 [INFO] Reactor Summary: 141 [INFO] ------------------------------------------------------------------------ 142 [INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s] 143 [INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s] 144 [INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s] 145 [INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s] 146 [INFO] ANTLR gUnit ........................................... SUCCESS [1.566s] 147 [INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s] 148 [INFO] ------------------------------------------------------------------------ 149 [INFO] ------------------------------------------------------------------------ 150 [INFO] BUILD SUCCESSFUL 151 [INFO] ------------------------------------------------------------------------ 152 [INFO] Total time: 11 seconds 153 154 However, unless you are using Maven exclusively in your projects, you 155 will most likely want to build the ANTLR Uber Jar, which is an 156 executable jar containing all the components that ANTLR needs to build 157 and run parsers (note that at runtime, you need only the runtime 158 components you use, such as the Java runtime and say stringtemplate). 159 160 Because the Uber jar is not something we want to deploy to Maven 161 repositories it is built with a special invocation of Maven: 162 163 mvn -Dmaven.test.skip=true package assembly:assembly 164 165 Note that Maven will appear to build everything twice, which is a 166 quirk of how it calculates the dependencies and makes sure it has 167 everything packaged up so it can build the uber-jar assembly. 168 169 Somewhere in the build output (towards the end), you will find a line 170 like this: 171 172 [INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar 173 174 This is the executable jar that you need and you can either copy it 175 somewhere or, like me, you can create this script (assuming UNIX) 176 somewhere in your PATH: 177 178 #! /bin/bash 179 java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar $* 180 181 Version Numbering 182 ------------------- 183 184 The first and Golden rule is that any pom files stored under the main 185 branch of the toolset should never be modified to contain a release 186 version number. They should always contain a.b.c-SNAPSHOT 187 (e.g. 3.1.3-SNAPSHOT). Only release branches should have their pom 188 version numbers set to a release version. You can release as many 189 SNAPSHOTS as you like, but only one release version. However, release 190 versions may be updated with a patch level: 3.1.3-1, 3.1.3-2 and so 191 on. 192 193 Fortunately, Maven helps us with the version numbering in a number of 194 ways. Firstly, the pom.xml files for the various modules do not 195 specify a version of the artifacts themselves. They pick up their 196 version number from the master build pom. However, there is a catch, 197 because they need to know what version of the parent pom they inherit 198 from and so they DO mention the version number. However, this does 199 prevent accidentally releasing different versions of sub-modules than 200 the master pom describes. 201 202 Fortunately once again, Maven has a neat way of helping us change the 203 version. All you need do is check out all the pom.xml files from 204 perforce, then modify the <version>a.b.c-SNAPSHOT</version> in the 205 master pom. When the version number is correct in the master pom, you 206 make sure your working directory is the location of the master pom and 207 type: 208 209 mvn versions:update-child-modules 210 211 This command will then update the child pom.xml files to reflect the 212 version number defined in the master pom.xml. 213 214 There is unfortunately one last catch here though and that is that the 215 antlr3-maven-plugin and the gunit-maven-plugin are not able to use the 216 parent pom. The reason for this is subtle but makes sense as doing so 217 would create a circular dependency between the ANTLR tool (which uses 218 the plugin to build its own grammar files), and the plugins (which 219 uses the tool to build grammar files and gunit to test). 220 221 This catch-22 situation means that the pom.xml file in the 222 antlr3-maven-plugin directory and the one in the gunit-maven-plugin 223 directory MUST be updated manually (or we must write a script to do 224 this). 225 226 Finally, we need to remember that because the tool is dependent on the 227 antlr3-maven-plugin and the plugin is itself dependent on the 228 tool, that we must manually update the versions of each that they 229 reference. So, when we bump the version of the toolset to say 230 3.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and 231 the gunit-maven-plugin pom.xml to reference that version of the antlr 232 tool. The tool itself is always built with the prior released version 233 of the plugin, so when we release we must change the main branch of 234 the plugin to use the newly released version of the plugin. This is 235 covered in the release checklist. 236 237 Deploying 238 ---------- 239 240 Deploying the tools at the current version is relatively easy, but to 241 deploy to the ANTLR repositories (snapshot or release) you must have 242 been granted access to the Sonatype OSS repositories' ANTLR login. 243 Few people will have this access of course. 244 245 Next, because we do not publish access information for antlr.org, you 246 will need to configure the repository server names locally. You do 247 this by creating (or adding to) the file: 248 249 ~/.m2/settings.xml 250 251 Which should look like this: 252 253 <?xml version="1.0" encoding="UTF-8"?> 254 <settings> 255 <servers> 256 <server> 257 <id>sonatype-nexus-snapshots</id> 258 <username>xxxxxxx</username> 259 <password>xxxxxxx</password> 260 </server> 261 <server> 262 <id>sonatype-nexus-staging</id> 263 <username>xxxxxxx</username> 264 <password>xxxxxxx</password> 265 </server> 266 </servers> 267 </settings> 268 269 When this configuration is in place, you will be able to deploy the components, 270 either individually or from the master directory: 271 272 mvn -Dmaven.test.skip=true -Ddeplot deploy 273 274 You will then see lots of information about checking existing version 275 information and so on, and the components will be deployed once you 276 supply the ANTLR public key passphrase to sign the jars. 277 278 Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT 279 then deployment will always be to the development snapshot 280 directory. When the artifacts are versioned with a release version 281 then deployment will be to the release stahinh repository, which 282 will then be mirrored around the world if closed and release. 283 The sonatype documentation should be consulted. 284 285 Release Checklist 286 ------------------ 287 288 Here is the procedure to use to make a release of ANTLR. Note that we 289 should really use the mvn release:release command, but the perforce 290 plugin for Maven is not commercial quality and I want to rewrite it. 291 292 For this checklist, let's assume that the current development version 293 of ANTLR is 3.1.3-SNAPSHOT. This means that it will probably (but not 294 necessarily) become release version 3.1.3 and that the development 295 version will bump to 3.1.4-SNAPSHOT. 296 297 0) Run a build of the main branch and check that it is builds and 298 passes as many tests as you want it to. 299 300 1) First make a branch from main into the target release 301 directory. Then submit this to perforce. You could change versions 302 numbers before submitting, but doing that in separate stages will 303 keep things sane; 304 305 --- Use main development branch from here --- 306 307 2) Before we deploy the release, we want to update the versions of the 308 development branch, so we don't deploy what is now the new release 309 as an older snapshot (this is not super important, but procedure is 310 good right?). 311 312 Check out all the pom.xml files (and if you are using any 313 antlr.config parameters that must change, then do that too). 314 315 3) Edit the master pom.xml in the main directory and change the version from 316 3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT. 317 318 4) Edit the pom.xml file for antlr3-maven-plugin under the main 319 directory and change the version from 3.1.3-SNAPSHOT to 320 3.1.4-SNAPSHOT. Do the same for the pom.xml in the 321 gunit-maven-plugin directory. 322 323 Update the pom.xml for the archetype manually too. 324 325 5) Now (from the main directory), run the command: 326 327 mvn versions:update-child-modules 328 329 You should see: 330 331 [INFO] [versions:update-child-modules] 332 [INFO] Module: gunit 333 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 334 [INFO] Module: runtime/Java 335 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 336 [INFO] Module: tool 337 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 338 339 6) Run a build of the main branch: 340 341 mvn -Dmaven.test.skip=true 342 343 All should be good. 344 345 7) Submit the pom changes of the main branch to perforce. 346 347 8) Deploy the new snapshot as a placeholder for the next release. It 348 will go to the snapshot repository of course: 349 350 mvn -N deploy 351 mvn -Dmaven.test.skip=true deploy 352 353 9) You are now finished with the main development branch and should change 354 working directories to the release branch you made earlier. 355 356 --- Use release branch from here --- 357 358 10) Check out all the pom.xml files in the release branch (and if you are 359 using any antlr.config parameters that must change, then do that too). 360 361 11) Edit the master pom.xml in the release-3.1.3 directory and change 362 the version from 3.1.3-SNAPSHOT to 3.1.3. 363 364 12) Edit the pom.xml file for antlr3-maven-plugin under the 365 release-3.1.3 directory and change the version from 3.1.3-SNAPSHOT 366 to 3.1.3. Also change the version of the tool that the this 367 pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we are now 368 releasing the plugin of course and it needs to reference the 369 version we are about to release. You will find this reference in 370 the dependencies section of the antlr3-maven-plugin pom.xml. Also 371 change the version references in the pom for gunit-maven-plugin. 372 373 13) Now (from the release-3.1.3 directory), run the command: 374 375 mvn versions:update-child-modules 376 377 You should see: 378 379 [INFO] [versions:update-child-modules] 380 [INFO] Module: gunit 381 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 382 now org.antlr:antlr-master:3.1.3 383 [INFO] Module: runtime/Java 384 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 385 now org.antlr:antlr-master:3.1.3 386 [INFO] Module: tool 387 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 388 now org.antlr:antlr-master:3.1.3 389 390 14) Run a build of the release-3.1.3 branch: 391 392 mvn # Note I am letting unit tests run here! 393 394 All should be good, or as good as it gets ;-) 395 396 15) Submit the pom changes of the release-3.1.3 branch to perforce. 397 398 16) Deploy the new release (this is it guys, make sure you are happy): 399 400 mvn -N deploy 401 mvn -Dmaven.test.skip=true deploy 402 403 Note that we must skip the tests as Maven will not let you 404 deploy releases that fail any junit tests. 405 406 17) The final step is that we must update the main branch pom.xml for 407 the tool to reference the newly release version of the 408 antlr3-maven-plugin. This is because each release of ANTLR is 409 built with the prior release of ANTLR, and we have just released 410 a new version. Edit the pom.xml for the tool (main/tool/pom.xml) 411 under the main (that's the MAIN branch, not the release branch) 412 and find the dependency reference to the antlr plugin. If you 413 just released say 3.1.3, then the tool should now reference 414 version 3.1.3 of the plugin. Having done this, you should 415 probably rebuild the main branch and let it run the junit 416 tests. Later, I will automate this dependency update as mvn can 417 do this for us. 418 419 18) Having deployed the release to maven, you will want to create the 420 uber jar for the new release, to make it downloadable from the 421 antlr.org website. This is a repeat of the earlier described step 422 to build the uber jar: 423 424 mvn -Dmaven.test.skip=true package assembly:assembly 425 426 MAven will produce the uber jar in the target directory: 427 428 antlr-master-3.1.3-completejar.jar 429 430 And this is the complete jar that can be downloaded from the web site. You 431 may wish to produce an md5 checksum to go with the jar: 432 433 md5sum target/antlr-master-3.1.3-completejar.jar 434 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx target/antlr-master-3.1.4-SNAPSHOT-completejar.jar 435 436 The command you just ran will also produce a second jar: 437 438 antlr-master-3.1.3-src.jar 439 440 This is the source code for everythign you just deployed and can 441 be unjarred and built from scratch using the very procedures 442 described here, which means you will now be reading this 443 BUILD.txt file for ever. 444 445 19) Reward anyone around you with good beer. 446 447 448 Miscellany 449 ----------- 450 451 It was a little tricky to get all the interdependencies correct 452 because ANTLR builds itself using itself and the maven plugin 453 references the ANTLR Tool as well. Hence the maven tool is not a child 454 project of the master pom.xml file, even though it is built by it. 455 456 An observant person will not that when the assembly:assembly phase is 457 run, that it invokes the build of the ANTLR tool using the version of 458 the Maven plugin that it has just built, and this results in the 459 plugin using the version of ANTLR tool that it has just built. This is 460 safe because everything will already be up to date and so we package 461 up the version of the tool that we expect, but the Maven plugin we 462 deploy will use the correct version of ANTLR, even though there is 463 technically a circular dependency. 464 465 The master pom.xml does give us a way to cause the build of the ANTLR 466 tool to use itself to build itself. This is because in 467 dependencyManagement in the master pom.xml, we can reference the 468 current version of the Tool and the Maven plugin, even though in the 469 pom.xml for the tool itself refers to the previous version of the 470 plugin. 471 472 What happens is that if we first cd into the tool and maven 473 directories and build ANTLR, it will build itself with the prior 474 version and this will deploy locally (.m2). We can then clean build 475 from the master pom and when ANTLR asks for the prior version of the 476 tool, the master pom.xml will override it and build with the interim 477 versions we just built manually. 478 479 However, strictly speaking, we need a third build where we rebuild the 480 tool again with the version of the tool that was built with itself and 481 not deploy the version that was built by the version of itself that 482 was built by a prior version of itself. I decided that this was not 483 particularly useful and complicates things too much. Building with a 484 prior version of the tool is fine and if there was ever a need to, we 485 could release twice in quick succession. 486 487 I have occasionally seen the MAven reactor screw up (or perhaps it is 488 the ANTLR tool) when building. If this happens you will see an ANTLR 489 Panic - cannot find en.stg message. If this happens to you, then just 490 rerun the build and it will eventually work. 491 492 Jim Idle - March 2009 493 494
