Home | History | Annotate | Download | only in Atari
      1 (Message inbox:32)
      2 Date:          Mon, 03 Jul 89 21:15:32 CET
      3 From:  V61%DHDURZ1.BITNET (a] lbl.gov
      4 Subject:       Flex, bug fix, improvments, patches for Minix & TOS
      5 To:  vern (a] lbl-csam.arpa
      6 
      7 At first I have to thank you for your wonderful program. I had ported the
      8 old version to OS9,TOS (Atari ST) and Minix and the new version 2.1 Beta
      9 to Minix and TOS.
     10      
     11 While porting and using flex I detected a bug and made some improvements.
     12 I have included a  shared, compressed and uuencoded file contaning all cdiffs
     13 and additional files (Sorry, but I'm on EBCDIC-Bitnet) and a short discussion
     14 of the changes. Even some of the TOS specific changes might be of general
     15 interest !
     16      
     17 I posted these cdiffs to the minix discussion group, but I think it's up
     18 to you to post them to the unix-sources group. If you plan to post even
     19 the TOS compiler specific patches please contact me because there might be
     20 further compiler (P.D.) additions. If you have an interest I could also
     21 port the new version to OS9 -- this is a little bit more difficult, because
     22 OS9 uses CR as end of line character (the EOL char. is coded into the
     23 initscan.c tables,...). It is necessary to change all occurences of '\n' to
     24 macros and variables and it's useful to add a new -n options (see commented
     25 line in main.c)
     26      
     27      
     28      
     29 The changes:                                                  (1.7.89 RAL)
     30      
     31  - Bug fix: The original flex didn't like trailing spaces in exclusive start
     32      condition lists ! If you add an trailing space to line 68 in scan.l
     33      
     34      "%x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE "
     35      
     36      you get a misleading error message:
     37      
     38      "Syntax error at line 69: bad start condition list"
     39      
     40      This bug can either be fixed in parse.y or in scan.l . I have chosen the
     41      last because there the fix is minimal: Just change the rule (line 128)
     42      
     43       "\n"    to   "{OPTWS}\n"
     44      
     45  - Enhancements:
     46     - new option "-?" that provides some help information about the other
     47       flags (main.c)
     48     - new option "-aTMPPATH" that allows a redefinition of the standard
     49       path for the temporary file (this might be rather large >200k if
     50       F,f options are selected). (main.c, flexdef.h (l.376))
     51     - hexdump of illegal characters -- this proved to be a useful debugging
     52       tool especialy if invisible control characters occur which weren't
     53       covered by the rules. (scan.l  fprintf statement line 129,...)
     54      
     55  - Patches due to TOS
     56     - General: TOS uses CR,LF as end of line char., Flex wants only a single
     57       LF as EOL char. Therefore all I/O must be translated using f* calls.
     58       This is done everywhere besides the YY_INPUT macro (flex.skel (scan.c),
     59       line 31) that uses a low level 'read'. This should be definitly changed
     60       to fread, so that all I/O calls occur on the same level.
     61     - the short action_file_name has been "flexXXXXXX.tmp", but that's too
     62       much for TOS,MSDOS ! I changed it to "flexXXXX.tmp" in main.c (patch
     63       covered by the -a option additions)
     64     - some compilers don't like external names that are ambiguous within
     65       the first 8 characters. I defined macros that change all these long
     66       external names to names that are unique within the first 8 characters.
     67       Just define SHORT_EXTERNAL_NAMES to use this feature (flexdef.h)
     68     - some statement changes that some compiler don't like:
     69       typedef *xxx[]  -> typedef **xxx       (flexdef.h.,l.308)
     70       "/*" -> '/','*' within a comment in (scan.l, l.209)
     71     - changed short "lexyy.c" to "lex_yy.c" what's more similar to the unix
     72       "lex.yy.c" (main.c).
     73     - a few further really compiler dependent changes provided with
     74       #ifdef ATARI && LATTICE res. TURBO braces.
     75      
     76  - Additional TOS only files
     77     - Makefile.tos: Common makefile for all TOS compilers. If you add further
     78       ones please email me the new makefile.
     79     - flex.lnk: Lattice - GST linker argument extension file
     80     - flex.tlk: Turbo linker argument extension file
     81      
     82      
     83 Additional remarks:
     84      
     85 I didn't add a new initscan.c (= flex -ist scan.l). The current one is good
     86 enough for a first compilation. With this first version of flex you can
     87 rebuild your own scan.c and the final flex version !
     88      
     89 Minix ST :
     90   - I had to "chmem =70000 cv" (>50000) to get flex linked
     91   - 'memset' (PC 1.3, EFTH40,...) is necessary
     92   - chmem =90000 flex may be sufficient
     93      
     94 Minix PC :
     95   It should be possible to port Flex to Minix PC. The current sizes of flex
     96   are:
     97                         Minix ST (ACK)    Lattice (TOS)    Turbo (TOS)
     98      
     99        size               75300             83305             57957
    100        compilation time      22'               15'             3'40"
    101        flex -is scan.l     1'49"               43"               30"
    102      
    103   The Minix ST size includes the bad generated code using only a subset of
    104   the 68000 commands, long addresses only and a huge relocation table.
    105   Therefore the PC size will be <64 k ! More serious is the fact that I had
    106   to chmem =90000 flex to get scan.l converted to scan.c . But I never saw
    107   a more complex lex source than scan.l -- so it should be possible to
    108   reduce some array sizes without limitation for all day usage.
    109      
    110   No one volunteered yet for a Minix PC port -- but if someone will try it
    111   I would provide him with a new scan.c and some hints.
    112      
    113 TOS:
    114   Don't forget to adapt the flexskel path within flexdef.h !
    115      
    116      
    117 Bitnet:  V61@DHDURZ1                               Ronald Lamprecht
    118 UUCP:    ...!unido!DHDURZ1.bitnet!V61              Theoretische Physik
    119 ARPAnet: V61%DHDURZ1.BITNET (a] CUNYVM.CUNY.EDU       (Heidelberg, West Germany)
    120 (Message inbox:36)
    121 Date:          Wed, 05 Jul 89 21:16:07 CET
    122 From:  V61%DHDURZ1.BITNET (a] csa2.lbl.gov
    123 Subject:       Re: What is TOS
    124 To:  vern (a] lbl-csam.arpa
    125 
    126 TOS is the name of the Atari ST operating system that is very similar
    127 to MSDOS (Both use CR,LF as end of line character). Therefore I have
    128 been astonished that no EOL convertion porblems occur within MSDOS.
    129      
    130 I have been aware of the double buffering when changing read to fread and  I
    131 accept your argument of a possible slow down. But if you integrate the other
    132 Atari - TOS changes, please insert a
    133         #ifdef ATARI
    134               fread ....
    135         #else
    136               read  ....
    137         #endif
    138 in flex.skel .
    139      
    140 Bitnet:  V61@DHDURZ1                               Ronald Lamprecht
    141 UUCP:    ...!unido!DHDURZ1.bitnet!V61              Theoretische Physik
    142 ARPAnet: V61%DHDURZ1.BITNET (a] CUNYVM.CUNY.EDU       (Heidelberg, West Germany)
    143 
    144 
    145 
    146 
    147 echo x - Makefile_cdiff
    148 sed '/^X/s///' > Makefile_cdiff << '/'
    149 X*** Src_2.1/Makefile	Thu Jun 28 00:06:42 1989
    150 X--- Makefile	Thu Jul  3 02:12:48 1989
    151 X***************
    152 X*** 5,10 ****
    153 X--- 5,11 ----
    154 X  # Porting considerations:
    155 X  #
    156 X  #    For System V Unix machines, add -DSYS_V to CFLAGS.
    157 X+ #    For Minix (ST), add -DSYS_V to CFLAGS
    158 X  #    For Vax/VMS, add -DSYS_V to CFLAGS.
    159 X  #    For MS-DOS, add "-DMS_DOS -DSYS_V" to CFLAGS.  Create \tmp if not present.
    160 X  #         You will also want to rename flex.skel to something with a three
    161 X***************
    162 X*** 21,28 ****
    163 X  SKELETON_DIR = /usr/local/lib
    164 X  SKELETON_FILE = flex.skel
    165 X  SKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\"
    166 X! CFLAGS = -O
    167 X! LDFLAGS = -s
    168 X  
    169 X  FLEX_FLAGS =
    170 X  FLEX = ./flex
    171 X--- 22,29 ----
    172 X  SKELETON_DIR = /usr/local/lib
    173 X  SKELETON_FILE = flex.skel
    174 X  SKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\"
    175 X! CFLAGS = -O -DSYS_V
    176 X! LDFLAGS = 
    177 X  
    178 X  FLEX_FLAGS =
    179 X  FLEX = ./flex
    180 X***************
    181 X*** 57,63 ****
    182 X  	yylex.c
    183 X  
    184 X  flex : $(FLEXOBJS)
    185 X! 	$(CC) $(CFLAGS) -o flex $(LDFLAGS) $(FLEXOBJS)
    186 X  
    187 X  first_flex:
    188 X  	cp initscan.c scan.c
    189 X--- 58,65 ----
    190 X  	yylex.c
    191 X  
    192 X  flex : $(FLEXOBJS)
    193 X! 	$(CC) $(CFLAGS) -o flex $(FLEXOBJS) $(LDFLAGS)
    194 X! 	chmem =150000 flex
    195 X  
    196 X  first_flex:
    197 X  	cp initscan.c scan.c
    198 /
    199 echo x - flex.skel_cdif
    200 sed '/^X/s///' > flex.skel_cdif << '/'
    201 X*** Src_2.1/flex.skel	Thu Jun 28 00:19:20 1989
    202 X--- flex.skel	Thu Jul  2 22:18:31 1989
    203 X***************
    204 X*** 28,34 ****
    205 X   * is returned in "result".
    206 X   */
    207 X  #define YY_INPUT(buf,result,max_size) \
    208 X! 	if ( (result = read( fileno(yyin), buf, max_size )) < 0 ) \
    209 X  	    YY_FATAL_ERROR( "read() in flex scanner failed" );
    210 X  #define YY_NULL 0
    211 X  #define yyterminate() return ( YY_NULL )
    212 X--- 28,34 ----
    213 X   * is returned in "result".
    214 X   */
    215 X  #define YY_INPUT(buf,result,max_size) \
    216 X! 	if ( (result = fread( buf,1, max_size, yyin )) < 0 ) \
    217 X  	    YY_FATAL_ERROR( "read() in flex scanner failed" );
    218 X  #define YY_NULL 0
    219 X  #define yyterminate() return ( YY_NULL )
    220 /
    221 echo x - flexdef.h_cdif
    222 sed '/^X/s///' > flexdef.h_cdif << '/'
    223 X*** Src_2.1/flexdef.h	Thu Jun 28 00:43:27 1989
    224 X--- flexdef.h	Thu Jul  3 02:45:50 1989
    225 X***************
    226 X*** 26,31 ****
    227 X--- 26,40 ----
    228 X  
    229 X  /* @(#) $Header: flexdef.h,v 2.0 89/06/20 15:49:50 vern Locked $ (LBL) */
    230 X  
    231 X+ #ifdef ATARI
    232 X+ #define SYS_V
    233 X+ #define abs(x) ((x) < 0 ? -(x) : (x))
    234 X+ #define SHORT_FILE_NAMES
    235 X+ #ifdef TURBO
    236 X+ #define SHORT_EXTERNAL_NAMES
    237 X+ #endif
    238 X+ #endif
    239 X+ 
    240 X  #ifndef FILE
    241 X  #include <stdio.h>
    242 X  #endif
    243 X***************
    244 X*** 41,47 ****
    245 X  #endif
    246 X  
    247 X  #ifndef VMS
    248 X! char *memset();
    249 X  #else
    250 X  /* memset is needed for old versions of the VMS C runtime library */
    251 X  #define memset(s, c, n) \
    252 X--- 50,58 ----
    253 X  #endif
    254 X  
    255 X  #ifndef VMS
    256 X! #ifndef ATARI && TURBO
    257 X! char *memset();
    258 X! #endif
    259 X  #else
    260 X  /* memset is needed for old versions of the VMS C runtime library */
    261 X  #define memset(s, c, n) \
    262 X***************
    263 X*** 81,91 ****
    264 X--- 92,129 ----
    265 X  #define true 1
    266 X  #define false 0
    267 X  
    268 X+ #ifdef ATARI
    269 X+ #define DEFAULT_SKELETON_FILE "D:\\include\\flexskel"
    270 X+ #endif
    271 X+ 
    272 X  
    273 X  #ifndef DEFAULT_SKELETON_FILE
    274 X  #define DEFAULT_SKELETON_FILE "flex.skel"
    275 X  #endif
    276 X  
    277 X+ #ifdef SHORT_EXTERNAL_NAMES
    278 X+ /* avoid long external names that are ambiguous within the first 8 characters */
    279 X+ #define current_mns                 c__mns
    280 X+ #define current_max_rules           c__max_rules
    281 X+ #define current_max_state_type      c__max_state_type
    282 X+ #define current_max_scs             c__max_scs
    283 X+ #define current_max_dfa_size        c__max__size
    284 X+ #define current_max_xpairs          c__max_xpairs
    285 X+ #define current_max_template_xpairs c__max_template_xpairs
    286 X+ #define current_max_dfas            c__max_dfas
    287 X+ #define current_maxccls             c__maxccles
    288 X+ #define current_max_ccl_tbl_size    c__max_ccl_tbl_size
    289 X+ #define indent_puts                 ind_puts
    290 X+ #define indent_put2s                ind_put2s
    291 X+ #define gen_next_compressed_state   gen_n_comressed_state
    292 X+ #define gen_next_match              gen_n_match
    293 X+ #define gen_next_state              gen_n_state
    294 X+ #define variable_trailing_context_rules  var_tr_context_rules
    295 X+ #define variable_trailing_rule           var_tr_rule
    296 X+ #define backtrack_report            backtr_report
    297 X+ #define backtrack_file              backtr_file
    298 X+ #endif
    299 X+ 
    300 X  /* special chk[] values marking the slots taking by end-of-buffer and action
    301 X   * numbers
    302 X   */
    303 X***************
    304 X*** 305,311 ****
    305 X      int int_val;
    306 X      } ;
    307 X  
    308 X! typedef struct hash_entry *hash_table[];
    309 X  
    310 X  #define NAME_TABLE_HASH_SIZE 101
    311 X  #define START_COND_HASH_SIZE 101
    312 X--- 343,349 ----
    313 X      int int_val;
    314 X      } ;
    315 X  
    316 X! typedef struct hash_entry **hash_table;
    317 X  
    318 X  #define NAME_TABLE_HASH_SIZE 101
    319 X  #define START_COND_HASH_SIZE 101
    320 X***************
    321 X*** 372,378 ****
    322 X  extern int datapos, dataline, linenum;
    323 X  extern FILE *skelfile, *yyin, *temp_action_file, *backtrack_file;
    324 X  extern char *infilename;
    325 X! extern char action_file_name[];
    326 X  
    327 X  
    328 X  /* variables for stack of states having only one out-transition:
    329 X--- 410,416 ----
    330 X  extern int datapos, dataline, linenum;
    331 X  extern FILE *skelfile, *yyin, *temp_action_file, *backtrack_file;
    332 X  extern char *infilename;
    333 X! extern char *action_file_name;
    334 X  
    335 X  
    336 X  /* variables for stack of states having only one out-transition:
    337 /
    338 echo x - main.c_cdiff
    339 sed '/^X/s///' > main.c_cdiff << '/'
    340 X*** Src_2.1/main.c	Thu Jun 28 00:30:39 1989
    341 X--- main.c	Thu Jul  3 02:27:47 1989
    342 X***************
    343 X*** 81,96 ****
    344 X  FILE *temp_action_file;
    345 X  FILE *backtrack_file;
    346 X  int end_of_buffer_state;
    347 X! #ifndef SHORT_FILE_NAMES
    348 X! char action_file_name[] = "/tmp/flexXXXXXX";
    349 X! #else
    350 X! char action_file_name[] = "flexXXXXXX.tmp";
    351 X! #endif
    352 X! 
    353 X  #ifndef SHORT_FILE_NAMES
    354 X  static char outfile[] = "lex.yy.c";
    355 X  #else
    356 X! static char outfile[] = "lexyy.c";
    357 X  #endif
    358 X  static int outfile_created = 0;
    359 X  
    360 X--- 81,91 ----
    361 X  FILE *temp_action_file;
    362 X  FILE *backtrack_file;
    363 X  int end_of_buffer_state;
    364 X! char *action_file_name;
    365 X  #ifndef SHORT_FILE_NAMES
    366 X  static char outfile[] = "lex.yy.c";
    367 X  #else
    368 X! static char outfile[] = "lex_yy.c";
    369 X  #endif
    370 X  static int outfile_created = 0;
    371 X  
    372 X***************
    373 X*** 328,333 ****
    374 X--- 323,329 ----
    375 X      {
    376 X      int i, sawcmpflag, use_stdout;
    377 X      char *arg, *skelname = NULL, *flex_gettime(), clower(), *mktemp();
    378 X+     char *tmp_action =(char *)0, *malloc();
    379 X  
    380 X      printstats = syntaxerror = trace = spprdflt = interactive = caseins = false;
    381 X      backtrack_report = performance_report = ddebug = fulltbl = fullspd = false;
    382 X***************
    383 X*** 349,354 ****
    384 X--- 345,355 ----
    385 X  	for ( i = 1; arg[i] != '\0'; ++i )
    386 X  	    switch ( arg[i] )
    387 X  		{
    388 X+ 		case 'a':
    389 X+ 		    if ( i != 1 )
    390 X+ 			flexerror( "-a flag must be given separately" );
    391 X+ 		    tmp_action = &arg[i+1];
    392 X+ 		    goto get_next_arg;
    393 X  		case 'b':
    394 X  		    backtrack_report = true;
    395 X  		    break;
    396 X***************
    397 X*** 445,452 ****
    398 X  		    printstats = true;
    399 X  		    break;
    400 X  
    401 X! 		default:
    402 X! 		    lerrif( "unknown flag %c", (int) arg[i] );
    403 X  		    break;
    404 X  		}
    405 X  
    406 X--- 446,458 ----
    407 X  		    printstats = true;
    408 X  		    break;
    409 X  
    410 X! 		case '?':
    411 X! 		    flexinfo(0);
    412 X! 		    break;
    413 X! 
    414 X! 		default:
    415 X! 		    fprintf(stderr,"flex : unknown flag %c\n", (int) arg[i] );
    416 X! 		    flexinfo(1);
    417 X  		    break;
    418 X  		}
    419 X  
    420 X***************
    421 X*** 454,459 ****
    422 X--- 460,493 ----
    423 X  	;
    424 X  	}
    425 X  
    426 X+ 
    427 X+ /* if you change the default tmp file names don't forget to change the
    428 X+    initialization for i, too !
    429 X+    
    430 X+ */
    431 X+ #ifndef SHORT_FILE_NAMES
    432 X+     i = 10;
    433 X+     if (!tmp_action) i += 5;
    434 X+ #else
    435 X+     i = 12;
    436 X+ #endif
    437 X+     if (tmp_action) 
    438 X+         i += strlen(tmp_action) + 1;
    439 X+     if((action_file_name = malloc(i+1)) == (char *)0)
    440 X+         flexerror("No memory for action_file_name");
    441 X+     *action_file_name = (char) NULL;
    442 X+     if (tmp_action) 
    443 X+         strcat(action_file_name,tmp_action);
    444 X+ #ifndef SHORT_FILE_NAMES
    445 X+     else
    446 X+         strcat(action_file_name,"/tmp");
    447 X+     strcat(action_file_name,"/flexXXXXXX");
    448 X+ #else
    449 X+     if (tmp_action)
    450 X+         strcat(action_file_name,"/");
    451 X+     strcat(action_file_name,"flexXXXX.tmp");
    452 X+ #endif
    453 X+     
    454 X      if ( (fulltbl || fullspd) && usemecs )
    455 X  	flexerror( "full table and -cm don't make sense together" );
    456 X  
    457 X***************
    458 X*** 520,526 ****
    459 X      if ( (skelfile = fopen( skelname, "r" )) == NULL )
    460 X  	lerrsf( "can't open skeleton file %s", skelname );
    461 X  
    462 X!     (void) mktemp( action_file_name );
    463 X  
    464 X      if ( (temp_action_file = fopen( action_file_name, "w" )) == NULL )
    465 X  	lerrsf( "can't open temporary action file %s", action_file_name );
    466 X--- 554,562 ----
    467 X      if ( (skelfile = fopen( skelname, "r" )) == NULL )
    468 X  	lerrsf( "can't open skeleton file %s", skelname );
    469 X  
    470 X! #ifndef ATARI && LATTICE
    471 X!     (void) mktemp( action_file_name );
    472 X! #endif
    473 X  
    474 X      if ( (temp_action_file = fopen( action_file_name, "w" )) == NULL )
    475 X  	lerrsf( "can't open temporary action file %s", action_file_name );
    476 X***************
    477 X*** 566,571 ****
    478 X--- 602,640 ----
    479 X      }
    480 X  
    481 X  
    482 X+ flexinfo(status)
    483 X+ int status;
    484 X+ {
    485 X+    fprintf(stderr,"Syntax  :  FLEX  inp_file\n");
    486 X+    fprintf(stderr,"Function: fast lexical analyzer generator  V%s\n",flex_version);
    487 X+    fprintf(stderr,"Options : a dir_path  : directory path for temporary files\n");
    488 X+    fprintf(stderr,"	- b  : generate backtracking information to lex.backtrack\n");
    489 X+    fprintf(stderr,"	- c  : compressed table, no equiv., no meta equiv.classes\n");
    490 X+    fprintf(stderr,"	   e : equivalence classes\n");
    491 X+    fprintf(stderr,"	   F : fast table\n");
    492 X+    fprintf(stderr,"	  |f : full table\n");
    493 X+    fprintf(stderr,"	  |m : meta equivalence classes\n");
    494 X+    fprintf(stderr,"	- d  : generate debugging scanner\n");
    495 X+    fprintf(stderr,"	- F  : fast table\n");
    496 X+    fprintf(stderr,"	- f  : full (not compressed) table\n");
    497 X+    fprintf(stderr,"	- I  : generate interactive scanner\n");
    498 X+    fprintf(stderr,"	- i  : generate case-insensitive scanner\n");
    499 X+    fprintf(stderr,"	- L  : supress #line directives\n");
    500 X+ /*   fprintf(stderr,"	- n hexnum : generate scanner using <hexnum> as newline char.\n");*/
    501 X+    fprintf(stderr,"	- p  : generate performance report to stderr\n");
    502 X+    fprintf(stderr,"	- S skeleton_path : file path for skeleton file\n");
    503 X+    fprintf(stderr,"	- s  : suppress echo of unmatched scanner input to stdout\n");
    504 X+    fprintf(stderr,"	- T  : run flex in trace mode\n");
    505 X+ #ifdef ATARI
    506 X+    fprintf(stderr,"	- t  : place result on stdout instead of lex_yy.c\n");
    507 X+ #else
    508 X+    fprintf(stderr,"	- t  : place result on stdout instead of lex.yy.c\n");
    509 X+ #endif
    510 X+    fprintf(stderr,"	- v  : print statistics of generated scanner\n");
    511 X+    fprintf(stderr,"     default =  -cem\n");
    512 X+    exit(status);
    513 X+ }
    514 X+ 
    515 X  /* readin - read in the rules section of the input file(s)
    516 X   *
    517 X   * synopsis
    518 /
    519 echo x - scan.l_cdiff
    520 sed '/^X/s///' > scan.l_cdiff << '/'
    521 X*** Src_2.1/scan.l	Thu Jun 30 19:42:00 1989
    522 X--- scan.l	Thu Jul  2 22:19:26 1989
    523 X***************
    524 X*** 125,132 ****
    525 X  
    526 X  {SCNAME}		RETURNNAME;
    527 X  ^{OPTWS}\n		++linenum; /* allows blank lines in section 1 */
    528 X! \n			++linenum; return ( '\n' );
    529 X! .			synerr( "illegal character" ); BEGIN(RECOVER);
    530 X  
    531 X  
    532 X  <C_COMMENT>"*/"		ECHO; BEGIN(0);
    533 X--- 125,136 ----
    534 X  
    535 X  {SCNAME}		RETURNNAME;
    536 X  ^{OPTWS}\n		++linenum; /* allows blank lines in section 1 */
    537 X! {OPTWS}\n		++linenum; return ( '\n' );
    538 X! .			{
    539 X! 			synerr( "illegal character" );
    540 X! 			fprintf(stderr,"Char : $%x\n",yytext[yyleng-1]);
    541 X! 			BEGIN(RECOVER);
    542 X! 			}
    543 X  
    544 X  
    545 X  <C_COMMENT>"*/"		ECHO; BEGIN(0);
    546 X***************
    547 X*** 206,212 ****
    548 X  <SECT2>^{OPTWS}\n	++linenum; /* allow blank lines in section 2 */
    549 X  
    550 X  	/* this horrible mess of a rule matches indented lines which
    551 X! 	 * do not contain "/*".  We need to make the distinction because
    552 X  	 * otherwise this rule will be taken instead of the rule which
    553 X  	 * matches the beginning of comments like this one
    554 X  	 */
    555 X--- 210,216 ----
    556 X  <SECT2>^{OPTWS}\n	++linenum; /* allow blank lines in section 2 */
    557 X  
    558 X  	/* this horrible mess of a rule matches indented lines which
    559 X! 	 * do not contain '/','*'.  We need to make the distinction because
    560 X  	 * otherwise this rule will be taken instead of the rule which
    561 X  	 * matches the beginning of comments like this one
    562 X  	 */
    563 /
    564 echo x - Makefile.tos
    565 sed '/^X/s///' > Makefile.tos << '/'
    566 X# make file for "flex" tool
    567 X
    568 X# @(#) $Header: Makefile,v 2.3 89/06/20 17:27:12 vern Exp $ (LBL)
    569 X
    570 X# Porting considerations:
    571 X#
    572 X#    For System V Unix machines, add -DSYS_V to CFLAGS.
    573 X#    For Vax/VMS, add -DSYS_V to CFLAGS.
    574 X#    For MS-DOS, add "-DMS_DOS -DSYS_V" to CFLAGS.  Create \tmp if not present.
    575 X#         You will also want to rename flex.skel to something with a three
    576 X#         character extension, change SKELETON_FILE below appropriately,
    577 X#    For Amiga, add "-DAMIGA -DSYS_V" to CFLAGS.
    578 X#
    579 X# A long time ago, flex was successfully built using Microsoft C and
    580 X# the following options: /AL, /stack:10000, -LARGE, -Ml, -Mt128, -DSYS_V
    581 X
    582 X
    583 X# the first time around use "make first_flex"
    584 X
    585 X# The following definitions must be set according to your compiler -
    586 X# examples for a Lattice Compiler with GST assembler and TURBO C with
    587 X# assembler are provided below and must just be updated (don't forget to
    588 X# update the linker argument extension files (*.lnk,*.tlk), too) :
    589 X#
    590 X#CCPATH  = path to compiler directory without trailing \
    591 X#CHPATH  = path to header files without trailing \
    592 X#CC      = filename of the compiler
    593 X#CFLAGS  = compiler option flags
    594 X#CIEXT   = extension of C sources that should be used for input filenames
    595 X#ASMPATH = path to assembler directory without trailing \
    596 X#ASM     = filename of the assembler
    597 X#AFLAGS  = assembler option flags
    598 X#AIEXT   = extension of assembler sources that should be used for assembler
    599 X#          input filenames
    600 X#AEXT    = general assembler filename extension
    601 X#LNKPATH = path to linker directory without trailing \
    602 X#LINK    = filename of the linker
    603 X#LFLAG0  = first option (full pathname of C startupcode) 
    604 X#LFLAG1  = further options + option flag for argument extension filename
    605 X#LFLAG2  = further options + option flag for output-filename 
    606 X#LNKEXT  = extension of linker argument extension file
    607 X#OIEXT   = extension of objects that should be used for linker input files
    608 X#OEXT    = general object file extension
    609 X
    610 X# Lattice definitions
    611 XCCPATH  = d:\latt
    612 XCHPATH  = d:\latt\include
    613 XCC      = lc.ttp
    614 XCFLAGS  = -h -n -cw -cc -i$(CHPATH)\ -g$(CCPATH)\ -dLATTICE -dATARI
    615 XCIEXT   =
    616 XASMPATH = d:\gst
    617 XASM     = assem.ttp
    618 XAFLAGS  = -nolist -errors errors.out
    619 XAIEXT   =
    620 XAEXT    = .asm
    621 XLNKPATH = d:\gst
    622 XLINK    = ld.ttp
    623 XLFLAG0  =
    624 XLFLAG1  = -with 
    625 XLFLAG2  = -nolist -sec -mem 200 -prog 
    626 XLNKEXT  = .lnk
    627 XOIEXT   =
    628 XOEXT    = .bin
    629 X
    630 X# Turbo definitions
    631 X#CCPATH  = d:\turbo
    632 X#CHPATH  = d:\turbo\include
    633 X#CC      = tcc.prg
    634 X#CFLAGS  = -GJMPRZ -H=$(CHPATH)\ -w- -DTURBO -DATARI
    635 X#CIEXT   = .c
    636 X#ASMPATH = d:\turbo
    637 X#ASM     = mas.prg
    638 X#AFLAGS  =
    639 X#AIEXT   = .s
    640 X#AEXT    = .s
    641 X#LNKPATH = d:\turbo
    642 X#LINK    = tlink.ttp
    643 X#LFLAG0  = $(LNKPATH)\lib\tcstart.o
    644 X#LFLAG1  = -I=
    645 X#LFLAG2  = -O=
    646 X#LNKEXT  = .tlk
    647 X#OIEXT   = .o
    648 X#OEXT    = .o
    649 X
    650 X# Other definitions
    651 X# (not used for Atari because of short argument string - defined in flexdef.h
    652 X
    653 XSKELETON_DIR = /usr/lib
    654 XSKELETON_FILE = flex.skel
    655 XSKELFLAGS = -DDEFAULT_SKELETON_FILE=\"$(SKELETON_DIR)/$(SKELETON_FILE)\"
    656 X
    657 X
    658 XFLEX       = d:\etc\flex.ttp
    659 XFLEX_FLAGS = -is
    660 XYACC       = d:\etc\yacc.ttp
    661 XRENAME     = d:\bin\rename
    662 XDEL        = d:\bin\del
    663 X
    664 X# Internal definitions
    665 XLNK     = $(LNKPATH)\$(LINK)
    666 X
    667 XFLEXOBJS = \
    668 X	ccl$(OEXT) \
    669 X	dfa$(OEXT) \
    670 X	ecs$(OEXT) \
    671 X	gen$(OEXT) \
    672 X	main$(OEXT) \
    673 X	misc$(OEXT) \
    674 X	nfa$(OEXT) \
    675 X	parse$(OEXT) \
    676 X	scan$(OEXT) \
    677 X	sym$(OEXT) \
    678 X	tblcmp$(OEXT) \
    679 X	yylex$(OEXT)
    680 X
    681 XFLEX_C_SOURCES = \
    682 X	ccl.c \
    683 X	dfa.c \
    684 X	ecs.c \
    685 X	gen.c \
    686 X	main.c \
    687 X	misc.c \
    688 X	nfa.c \
    689 X	parse.c \
    690 X	scan.c \
    691 X	sym.c \
    692 X	tblcmp.c \
    693 X	yylex.c
    694 X
    695 Xflex : $(FLEXOBJS)
    696 X	 $(LNK) $(LFLAG0) $(LFLAG1)flex$(LNKEXT) $(LFLAG2)flex.ttp
    697 X
    698 Xfirst_flex:
    699 X	cp initscan.c scan.c
    700 X	make $(MFLAGS) flex
    701 X
    702 Xparse.h parse.c : parse.y
    703 X	$(YACC)   -d parse.y
    704 X	$(DEL)    parse.c
    705 X	$(RENAME) y_tab.c parse.c
    706 X	$(DEL)    parse.h
    707 X	$(RENAME) y_tab.h parse.h
    708 X
    709 Xscan.c : scan.l
    710 X	$(FLEX)   $(FLEX_FLAGS) scan.l
    711 X	$(RENAME) lex_yy.c scan.c
    712 X
    713 Xscan$(OEXT) : scan.c parse.h flexdef.h
    714 X	 $(CCPATH)\$(CC) $(CFLAGS) scan$(CIEXT)
    715 X
    716 Xmain$(OEXT) : main.c flexdef.h
    717 X	 $(CCPATH)\$(CC) $(CFLAGS)  main$(CIEXT)
    718 X
    719 Xccl$(OEXT) : ccl.c flexdef.h 
    720 X	 $(CCPATH)\$(CC) $(CFLAGS)  ccl$(CIEXT)
    721 X
    722 Xdfa$(OEXT) : dfa.c flexdef.h
    723 X	 $(CCPATH)\$(CC) $(CFLAGS)  dfa$(CIEXT)
    724 X
    725 Xecs$(OEXT) : ecs.c flexdef.h
    726 X	 $(CCPATH)\$(CC) $(CFLAGS)  ecs$(CIEXT)
    727 X
    728 Xgen$(OEXT) : gen.c flexdef.h
    729 X	 $(CCPATH)\$(CC) $(CFLAGS)  gen$(CIEXT)
    730 X
    731 Xmisc$(OEXT) : misc.c flexdef.h
    732 X	 $(CCPATH)\$(CC) $(CFLAGS)  misc$(CIEXT)
    733 X
    734 Xnfa$(OEXT) : nfa.c flexdef.h
    735 X	 $(CCPATH)\$(CC) $(CFLAGS)  nfa$(CIEXT)
    736 X
    737 Xparse$(OEXT) : parse.c flexdef.h
    738 X	 $(CCPATH)\$(CC) $(CFLAGS)  parse$(CIEXT)
    739 X
    740 Xsym$(OEXT) : sym.c flexdef.h
    741 X	 $(CCPATH)\$(CC) $(CFLAGS)  sym$(CIEXT)
    742 X
    743 Xtblcmp$(OEXT) : tblcmp.c flexdef.h
    744 X	 $(CCPATH)\$(CC) $(CFLAGS)  tblcmp$(CIEXT)
    745 X
    746 Xyylex$(OEXT) : yylex.c flexdef.h 
    747 X	 $(CCPATH)\$(CC) $(CFLAGS)  yylex$(CIEXT)
    748 X
    749 Xflex.man : flex.1
    750 X	nroff -man flex.1 >flex.man
    751 X
    752 Xlint : $(FLEX_C_SOURCES)
    753 X	lint $(FLEX_C_SOURCES) > flex.lint
    754 X
    755 Xdistrib :
    756 X	mv scan.c initscan.c
    757 X	chmod 444 initscan.c
    758 X	$(MAKE) $(MFLAGS) clean
    759 X
    760 Xclean :
    761 X	rm -f core errs flex *$(OEXT) parse.c *.lint parse.h flex.man tags
    762 X
    763 Xtags :
    764 X	ctags $(FLEX_C_SOURCES)
    765 X
    766 Xvms :	flex.man
    767 X	$(MAKE) $(MFLAGS) distrib
    768 X
    769 Xtest :
    770 X	$(FLEX) $(FLEX_FLAGS) scan.l
    771 X	$(RENAME) lex_yy.c scan.ctest
    772 X	cmp scan.c scan.ctest
    773 X
    774 /
    775 echo x - Readme2
    776 sed '/^X/s///' > Readme2 << '/'
    777 XThe changes:                                                  (1.7.89 RAL)
    778 X
    779 X - Bug fix: The original flex didn't like trailing spaces in exclusive start
    780 X     condition lists ! If you add an trailing space to line 68 in scan.l
    781 X     
    782 X     "%x SECT2 SECT2PROLOG SECT3 CODEBLOCK PICKUPDEF SC CARETISBOL NUM QUOTE "
    783 X     
    784 X     you get a misleading error message: 
    785 X     
    786 X     "Syntax error at line 69: bad start condition list"
    787 X     
    788 X     This bug can either be fixed in parse.y or in scan.l . I have chosen the
    789 X     last because there the fix is minimal: Just change the rule (line 128)
    790 X     
    791 X      "\n"    to   "{OPTWS}\n"
    792 X      
    793 X - Enhancements:
    794 X    - new option "-?" that provides some help information about the other
    795 X      flags (main.c)
    796 X    - new option "-aTMPPATH" that allows a redefinition of the standard
    797 X      path for the temporary file (this might be rather large >200k if 
    798 X      F,f options are selected). (main.c, flexdef.h (l.376))
    799 X    - hexdump of illegal characters -- this proved to be a useful debugging
    800 X      tool especialy if invisible control characters occur which weren't
    801 X      covered by the rules. (scan.l  fprintf statement line 129,...)
    802 X      
    803 X - Patches due to TOS
    804 X    - General: TOS uses CR,LF as end of line char., Flex wants only a single
    805 X      LF as EOL char. Therefore all I/O must be translated using f* calls.
    806 X      This is done everywhere besides the YY_INPUT macro (flex.skel (scan.c),
    807 X      line 31) that uses a low level 'read'. This should be definitly changed
    808 X      to fread, so that all I/O calls occur on the same level.
    809 X    - the short action_file_name has been "flexXXXXXX.tmp", but that's too 
    810 X      much for TOS,MSDOS ! I changed it to "flexXXXX.tmp" in main.c (patch
    811 X      covered by the -a option additions)
    812 X    - some compilers don't like external names that are ambiguous within
    813 X      the first 8 characters. I defined macros that change all these long
    814 X      external names to names that are unique within the first 8 characters.
    815 X      Just define SHORT_EXTERNAL_NAMES to use this feature (flexdef.h)
    816 X    - some statement changes that some compiler don't like:
    817 X      typedef *xxx[]  -> typedef **xxx       (flexdef.h.,l.308)
    818 X      "/*" -> '/','*' within a comment in (scan.l, l.209)
    819 X    - changed short "lexyy.c" to "lex_yy.c" what's more similar to the unix
    820 X      "lex.yy.c" (main.c).
    821 X    - a few further really compiler dependent changes provided with
    822 X      #ifdef ATARI && LATTICE res. TURBO braces.
    823 X      
    824 X - Additional TOS only files
    825 X    - Makefile.tos: Common makefile for all TOS compilers. If you add further
    826 X      ones please email me the new makefile.
    827 X    - flex.lnk: Lattice - GST linker argument extension file
    828 X    - flex.tlk: Turbo linker argument extension file
    829 X      
    830 X    
    831 XAdditional remarks:
    832 X 
    833 XI didn't add a new initscan.c (= flex -ist scan.l). The current one is good
    834 Xenough for a first compilation. With this first version of flex you can
    835 Xrebuild your own scan.c and the final flex version !
    836 X
    837 XMinix ST :
    838 X  - I had to "chmem =70000 cv" (>50000) to get flex linked
    839 X  - 'memset' (PC 1.3, EFTH40,...) is necessary
    840 X  - chmem =90000 flex may be sufficient
    841 X  
    842 XMinix PC :
    843 X  It should be possible to port Flex to Minix PC. The current sizes of flex
    844 X  are:
    845 X                        Minix ST (ACK)    Lattice (TOS)    Turbo (TOS)
    846 X       
    847 X       size               75300             83305             57957
    848 X       compilation time      22'               15'             3'40"
    849 X       flex -is scan.l     1'49"               43"               30"
    850 X       
    851 X  The Minix ST size includes the bad generated code using only a subset of
    852 X  the 68000 commands, long addresses only and a huge relocation table.
    853 X  Therefore the PC size will be <64 k ! More serious is the fact that I had
    854 X  to chmem =90000 flex to get scan.l converted to scan.c . But I never saw
    855 X  a more complex lex source than scan.l -- so it should be possible to
    856 X  reduce some array sizes without limitation for all day usage.
    857 X  
    858 X  No one volunteered yet for a Minix PC port -- but if someone will try it
    859 X  I would provide him with a new scan.c and some hints.
    860 X  
    861 XTOS: 
    862 X  Don't forget to adapt the flexskel path within flexdef.h !
    863 X
    864 X
    865 /
    866 echo x - flex.lnk
    867 sed '/^X/s///' > flex.lnk << '/'
    868 X*
    869 X*
    870 X*  linker control file for flex.ttp
    871 X*
    872 X*
    873 X*
    874 XINPUT   d:\latt\lib\startup.bin
    875 X*
    876 XINPUT ccl.bin
    877 XINPUT dfa.bin
    878 XINPUT ecs.bin
    879 XINPUT gen.bin
    880 XINPUT misc.bin
    881 XINPUT nfa.bin
    882 XINPUT parse.bin
    883 XINPUT sym.bin
    884 XINPUT tblcmp.bin
    885 XINPUT main.bin
    886 XINPUT yylex.bin
    887 XINPUT scan.bin
    888 X*
    889 XLIBRARY d:\latt\lib\clib.bin
    890 X
    891 /
    892 echo x - flex.tlk
    893 sed '/^X/s///' > flex.tlk << '/'
    894 Xccl.o
    895 Xdfa.o
    896 Xecs.o
    897 Xgen.o
    898 Xmisc.o
    899 Xnfa.o
    900 Xparse.o
    901 Xsym.o
    902 Xtblcmp.o
    903 Xyylex.o
    904 Xmain.o
    905 Xscan.o
    906 Xd:\turbo\lib\tcstdlib.lib   ; standard lib
    907 Xd:\turbo\lib\tcextlib.lib   ; extended lib
    908 Xd:\turbo\lib\tctoslib.lib   ; TOS lib
    909 Xd:\turbo\lib\tcgemlib.lib   ; AES and VDI lib
    910 X-S=200000
    911 /
    912