1 page.title=SONiVOX JETCreator User Manual 2 @jd:body 3 4 5 6 <p>Content Authoring Application for the JET Interactive Music Engine</p> 7 8 9 <h1>1 Introduction</h1> 10 11 <h2>1.1 Overview</h2> 12 13 <p>This document contains the user guidelines 14 for the SONiVOX JET Creator, an authoring application for creating and 15 auditioning JET files. JET is an interactive music player for small embedded 16 devices, including the those running the Android platform. It allows applications to 17 include interactive music soundtracks, in MIDI 18 format, that respond in real-time to game play events and user interaction.</p> 19 20 21 <p>JET works in conjunction with SONiVOX's 22 Embedded Audio Synthesizer (EAS) which is the MIDI 23 playback device for Android. Both the 24 JET and EAS engines are integrated into the Android embedded platform through the 25 {@link android.media.JetPlayer} class, as well 26 as inherent in the JET Creator application. As such, the JET content author can 27 be sure that the playback will sound exactly the same in both the JET Creator 28 and the final Android application playing back on Android mobile devices.</p> 29 30 31 <p>In addition to the graphical user 32 interface, there are two main functionalities taking place in JET Creator. The 33 first involves gathering all the source data (MIDI 34 files and DLS file), adding JET's real-time attributes and building a JET 35 (.jet) file that the Android application will use. The second functionality 36 involves auditioning the interactive playback elements as they will take place 37 in the Android application.</p> 38 39 40 <p>The JET Creator application is written in 41 the Python programming language, therefore you need to have the current version 42 of Python and WXWidgets installed. There is both a Mac and Windows version.</p> 43 44 45 <h2>1.2 Abbreviations and Common Terms</h2> 46 47 <p>It is important to use a common set of 48 terms to minimize confusion. Since JET uses MIDI 49 in a unique way, normal industry terms may not always suffice. Here is the 50 definition of terms as they are used in this document and in the JET Creator 51 application:</p> 52 53 54 <p><i>Channel</i>: MIDI data associated with a specific MIDI 55 channel. Standard MIDI allows for 16 channels of MIDI 56 data each of which are typically associated with a specific instrument. </p> 57 58 59 60 <p><i>Controller</i>: A MIDI event consisting of a 61 channel number, controller number, and a controller value. The MIDI 62 spec associates many controller numbers with 63 specific functions, such as volume, expression, sustain pedal, etc. JET also 64 uses controller events as a means of embedding special control information in a 65 MIDI sequence to provide for audio synchronization.</p> 66 67 68 69 <p><i>DAW</i>: Digital Audio Workstation. A common term for MIDI 70 and audio sequencing applications such as Logic, SONAR, Cubase and others. </p> 71 72 73 74 <p><i>EAS:</i> Embedded MIDI Synthesizer. The 75 name of the SONiVOX MIDI synthesizer engine.</p> 76 77 78 79 <p><i>JET</i>: Jet Interactive Engine. The name of the SONiVOX JET interactive 80 music engine.</p> 81 82 83 84 <p><i>M/B/T</i>: Measures, Beats and Ticks</p> 85 86 87 88 <p><i>Segment</i>: A musical section such as a chorus or verse that is a component of 89 the overall composition. In JET, a segment can be an entire MIDI file or a 90 derived from a portion of a MIDI file.</p> 91 92 93 94 <p><i>SMF-0</i>: Standard MIDI File Type 0, a MIDI file that contains a single 95 track, but may be made up of multiple channels of MIDI 96 data.</p> 97 98 99 100 <p><i>SMF-1</i>: Standard MIDI File Type 1, a MIDI file that contains a one more 101 tracks, and each track may in turn be made up of one or more channels of MIDI 102 data. By convention, each channel is stored on a 103 separate track in an SMF-1 file. However, it is possible to have multiple MIDI 104 channels on a single track, or multiple tracks that contain data for the same MIDI 105 channel.</p> 106 107 108 109 <p><i>Track</i>: A single track in a DAW containing a timed sequence of events. Be careful not to confuse Tracks with 110 Channels. A MIDI file may contain many tracks with several tracks utilizing the 111 same MIDI channel. </p> 112 113 114 115 116 117 <h1>2 The JET Interactive Music Concept</h1> 118 119 <p>Interactive music can be defined as music 120 that changes in real-time according to non-predictable events such as user 121 interaction or game play events. In this way, interactive music is much more 122 engaging as it has the ability to match the energy and mood of a game much 123 closer than a pre-composed composition that never changes. In some applications 124 and games, interactive music is central to the game play. Guitar Hero is one 125 such popular game. When the end user successfully captures the musical notes 126 coming down the fret board, the music adapts itself and simultaneously keeps 127 score of successes and failures. JET allows for these types of music driven 128 games as well.</p> 129 130 131 132 <p>There are several methods for making and 133 controlling interactive music and JET is one such method. This section 134 describes the features of JET and how they might be used in a game or software 135 application. It also describes how JET can be used to save memory in small 136 footprint devices such as Android enabled mobile handsets.</p> 137 138 <h3>2.1.1 Data Compression</h3> 139 140 <p>JET supports a flexible music format that 141 can be used to create extended musical sequences with a minimal amount of data. 142 A musical composition is broken up into segments that can be sequenced to 143 create a longer piece. The sequencing can be fixed at the time the music file 144 is authored, or it can be created dynamically under program control.</p> 145 146 <h3>2.1.2 Linear Music Example</h3> 147 148 <p> 149 <img border=0 width=575 height=393 src="{@docRoot}images/jet/linear_music.png"> 150 <br>Figure 1: Linear Music Piece</p> 151 152 <p>This diagram shows how musical segments are 153 stored. Each segment is authored as a separate MIDI 154 file. A post-processing tool combines the files into a single container file. 155 Each segment can contain alternate music tracks that can be muted or un-muted 156 to create additional interest. An example might be a brass accent in the chorus 157 that is played only the last time through. Also, segments can be transposed up 158 or down.</p> 159 160 161 <p>The bottom part of the diagram shows how 162 the musical segments can be recombined to create a linear music piece. In this 163 example, the bridge might end with a half-step key modulation and the remaining 164 segments could be transposed up a half-step to match.</p> 165 166 <h3>2.1.3 Non-linear Music Example</h3> 167 168 <p> 169 <img border=0 width=576 height=389 170 src="{@docRoot}images/jet/nonlinear_music.png"> 171 <br>Figure 2: Non-linear music piece</p> 172 173 174 <p>In this diagram, we see a non-linear music 175 piece. The scenario is a first-person-shooter (FPS) and JET is providing the 176 background music. The intro plays as the level is loading and then transitions 177 under program control to the Searching segment. This segment is repeated indefinitely, 178 perhaps with small variations (using the mute/un-mute feature) until activity 179 in the game dictates a change.</p> 180 181 182 183 <p>As the player nears a monster lair, the 184 program starts a synchronized transition to the Danger segment, increasing the 185 tension level in the audio. As the player draws closer to the lair, additional 186 tracks are un-muted to increase the tension.</p> 187 188 189 190 <p>As the player enters into combat with the 191 monster, the program starts a synchronized transition to the Combat segment. 192 The segment repeats indefinitely as the combat continues. A Bonus Hit 193 temporarily un-mutes a decorative track that notifies the player of a 194 successful attack, and similarly, another track is temporarily un-muted to 195 signify when the player receives Special Damage.</p> 196 197 198 199 <p>At the end of combat, the music transitions 200 to a victory or defeat segment based on the outcome of battle.</p> 201 202 <h3>2.1.4 Mute/Un-mute Synchronization</h3> 203 204 <p>JET can also synchronize the muting and 205 un-muting of tracks to events in the music. For example, in the FPS game, it would 206 probably be desirable to place the musical events relating to bonuses and 207 damage as close to the actual game event as possible. However, simply un-muting 208 a track at the moment the game event occurs might result in a music clip 209 starting in the middle. Alternatively, a clip could be started from the 210 beginning, but then it wouldn't be synchronized with the other music tracks.</p> 211 212 213 <p>However, with the JET sync engine, a clip 214 can be started at the next opportune moment and maintain synchronization. This 215 can be accomplished by placing a number of short music clips on a decorative 216 track. A MIDI event in the stream signifies 217 the start of a clip and a second event signifies the end of a clip. When the 218 application calls the JET clip function, the next clip in the track is allowed 219 to play fully synchronized to the music. Optionally, the track can be 220 automatically muted by a second MIDI event.</p> 221 222 223 <p> 224 <img border=0 width=576 height=155 225 src="{@docRoot}images/jet/sync_muteunmute.png"> 226 <br>Figure 3: Synchronized Mute/Unmute</p> 227 228 229 <h2>2.2 Audio Synchronization</h2> 230 231 <p>JET provides an audio synchronization API 232 that allows game play to be synchronized to events in the audio. The mechanism 233 relies on data embedded in the MIDI file at 234 the time the content is authored. When the JET engine senses an event during 235 playback it generates a callback into the application program. The timing of 236 the callback can be adjusted to compensate for any latency in the audio 237 playback system so that audio and video can be synchronized. The diagram below 238 shows an example of a simple music game that involves pressing the left and 239 right arrows in time with the music.</p> 240 241 <p><img border=0 width=576 height=134 242 src="{@docRoot}images/jet/music_game.png"> 243 <br>Figure 4: Music Game</p> 244 245 246 247 <p>The arrows represent events in the music sequence 248 where game events need to be synchronized. In this case, the blue arrow 249 represents a time where the player is supposed to press the left button, and 250 the red arrow is for the right button. The yellow arrow tells the game engine 251 that the sequence is complete. The player is allowed a certain time window 252 before and after the event to press the appropriate key.</p> 253 254 255 256 <p>If an event is received and the player has 257 not pressed a button, a timer is set to half the length of the window. If the 258 player presses the button before the timer expires, the game registers a 259 success, and if not, the game registers a failure. </p> 260 261 262 263 <p>If the player presses the button before the 264 event is received, a timer is set to half the length of the window. If an event 265 is received before the timer expires, the game registers a success, and if not, 266 the game registers a failure. Game play might also include bonuses for getting 267 close to the timing of the actual event. </p> 268 269 270 271 <h1>3 JET Content Authoring Overview</h1> 272 273 <p>To author JET files and hear them playback 274 interactively, the content author will work in two applications which are 275 designed to work together smoothly. The first is application is any 276 off-the-shelf MIDI sequencing application that 277 supports VST (for PC) or AU (for Mac) plugins. Here the author will compose 278 their MIDI music files using the plugin as the 279 synthesizer device. The second application is the JET Creator application. Here 280 the author will import their MIDI music files 281 (and optionally a DLS2 soundset) and setup the conditions for interactive 282 playback within the JET enabled game. Optionally the content author may create 283 a custom set of DLS instruments using an instrument editor that supports the 284 DLS Level 2 format. One such application is Awave from MJSoft. </p> 285 286 <p>Please see the JET Content Authoring Guidelines</i> documentation for additional 287 details on content authoring.</p> 288 289 290 291 <h1>4 Installing and Launching JET Creator</h1> 292 293 <p>JET Creator is a python language 294 application, therefore, you must have Python and wxPython installed on your 295 machine. </p> 296 297 298 <p>JetCreator was created and tested with:</p> 299 300 <p>Python Version 2.5.4</p> 301 302 <p>wxPython Version 2.8.7.1</p> 303 304 305 <p>These can be downloaded here:</p> 306 307 308 309 <p>PC:</p> 310 <ul> 311 <li>http://www.python.org/download/releases/2.5.4/</li> 312 313 <li>http://www.wxpython.org/download.php</li> 314 </ul> 315 316 317 <p>MAC:</p> 318 <ul> 319 <li>http://wiki.python.org/moin/MacPython/Leopard</li> 320 321 <li>http://www.wxpython.org/download.php</li> 322 </ul> 323 324 325 <p>After installing Python and wxPython, 326 simply unzip or copy all the files in the JET Creator application directory to 327 a folder on your hard drive.</p> 328 329 330 <p>To launch JET Creator go to a command 331 prompt and set the directory to where you've installed Python. Next run python 332 with the command:</p> 333 334 <p><pre>python jetcreator.py</pre></p> 335 336 337 338 339 340 <h1>5 Using JET Creator</h1> 341 342 343 344 <h2>5.1 File Types</h2> 345 346 <p>There are a few different file types 347 associated with JET Creator.</p> 348 349 350 351 <p>.jtc JET 352 Creator project file. This file contains all the information associated with a 353 JET Creator project. When you Save or Save-as out of JET Creator, this file 354 type is saved.</p> 355 356 357 358 <p>.jet JET 359 File. This output file is automatically generated from JET Creator whenever you 360 save your JET Creator project. This is the file that bundles all JET assets 361 together into a single file that the Android application will use. Give this 362 file to the Android application developer.</p> 363 364 365 366 <p>.mid File. This is the standard MIDI 367 type 1 file that JET Creator will use to make segments.</p> 368 369 370 371 <p>.seg Segment 372 File. This is a JET Segment file. It has the same name as the MIDI 373 file which it references but contains additional Segment information.</p> 374 375 376 377 <p>.zip Zip 378 Archive file. When you Export a JET Archive, a zip file is created that 379 contains all the assets (files) necessary for JET Creator. Use this to transfer 380 JET Creator projects to other people.</p> 381 382 383 384 <h2>5.2 Open Dialog</h2> 385 386 <p>When 387 you first launch JET Creator you are presented with an open dialog like the 388 following.</p> 389 390 391 392 <p><img border=0 width=450 height=285 393 src="{@docRoot}images/jet/jc_open_dlg.png" 394 </p> 395 396 397 398 399 400 <p> <b>Open</b> will open an existing .jtc (JET Creator file) file. Use the browser 401 button to browse to the directory where you have saved your .jtc file.</p> 402 403 404 405 <p> <b>New</b> will create a new .jtc file.</p> 406 407 408 409 <p> <b>Import</b> will import a JET Archive (.zip) file.</p> 410 411 412 413 <p> <b>Cancel</b> will cancel the dialog and exit the application.</p> 414 415 416 417 418 419 <h1>5 Main Window </h1> 420 421 <p>The main window of the JET Creator 422 application looks like the picture below. There are three main sections from 423 top to bottom: segment view, event view, and timeline. </p> 424 425 426 427 <p>The segment view section displays a list of 428 the current segments, which MIDI file and 429 (optionally) DLS2 file each segment is derived from. It also shows each 430 segments start and stop time and each segments quantize, transpose, repeat and 431 mute flag settings.</p> 432 433 434 435 <p>Just below the Segment view is the event 436 view. The event view section displays all events associated with a given 437 segment. Events only display when the segment they are assigned to is 438 highlighted. Each event displays its type, start and end points, track and midi 439 channel assignment, and its event ID.</p> 440 441 442 443 <p>Just below the Event view is the timeline 444 display. The timeline shows how many measures a given segment is as well as any 445 events associated with that segment. The timeline changes to display the 446 currently selected or playing segment. You can trigger an event in this window 447 while the segment is play by simply clicking on the event in the timeline 448 display.</p> 449 450 451 <p><img border=0 width=554 height=378 452 src="{@docRoot}images/jet/jc_main_wnd.png"> 453 <br><i>JET 454 Creator Main Window<o:p></o:p></i></p> 455 456 457 <p>The buttons along the left side of main 458 window do the following:</p> 459 460 <p>Add: 461 Displays the segment or event window for adding a new segment or event</p> 462 463 <p>Revise: 464 Displays the segment or event window for updating an existing segment or event</p> 465 466 <p>Delete: 467 Deletes the selected segment or event (will ask for confirmation)</p> 468 469 <p>Move: 470 Displays the move window which allows you to move selected segments or events 471 in time</p> 472 473 <p>Queue All: Queue's 474 (selects) all segments for playback</p> 475 476 <p>Dequeue All: Dequeues 477 (deselects) all segments</p> 478 479 <p>Play: 480 Starts playback of all queued segments. This button changes to Stop if any 481 segments are playing</p> 482 483 <p>Audition: 484 Displays the Audition window (see below)</p> 485 486 487 488 <h2>5.1 Segment Window</h2> 489 490 <p>The segment window is where a given 491 segment's attributes are assigned and auditioned, as shown in the picture 492 below. The left side of the window displays the segments attributes that are 493 stored in the JET file. The right side of the window allows the author to set 494 mute flags, repeat and transpose settings and audition the segment as it will 495 play in the JET game.</p> 496 497 498 499 <p><b>Note</b>: the audition attributes (mute flags, repeat and transpose) are <i 500 >not</i> stored in the JET content file 501 (.jet) but rather are defined by the game or application itself. In programming 502 language, these settings correspond directly with the API calls to the JET 503 engine. By including them here, the JET content author can simulate how the 504 segment will respond to the applications API commands during game play.</p> 505 506 507 508 <p><img border=0 width=553 height=295 509 src="{@docRoot}images/jet/jc_seg_wnd.png" > 510 </p> 511 512 <p>The segment parameters do the following:</p> 513 514 <ul> 515 <li>Segment Name - Sets 516 the name of the segment</li> 517 518 <li>MIDI File - 519 The name and location of the MIDI file from which 520 the segment is derived. The button to the immediate right will bring up a 521 browser for locating a midi file on the hard drive.</li> 522 523 <li>DLS File - 524 The name and location of the DLS2 file, if any, that the MIDI 525 file uses for that segment.</li> 526 527 <li>Starting M/B/T - 528 Starting measure, beat and tick of the segment</li> 529 530 <li>Ending M/B/T - 531 Ending measure, beat and tick of the segment</li> 532 533 <li>Quantize - 534 Quantize value for quantizing the current segment during playback</li> 535 536 </ul> 537 538 <p>The audition fields are as follows:</p> 539 540 <ul> 541 <li>Track Mutes - 542 Shows the MIDI <b>tracks</b> (not channels) 543 in the MIDI file. Clicking on a track's 544 checkbox will mute that track. </li> 545 546 <li>Channel - 547 Displays the MIDI channel assigned to each 548 track</li> 549 550 <li>Name - 551 Displays the track name meta event (if present) for each track</li> 552 553 <li>Repeat - 554 Indicates the number of times a segment should repeat during playback</li> 555 556 <li>Transpose - 557 Indicates the transposition in semi-tones or half-steps a segment should 558 transpose during playback</li> 559 560 <li>To the right of the Audition window are a few additional buttons. 561 These do as follows:</li> 562 563 <li>OK - 564 Selecting OK confirms all segment settings and closes the segment window</li> 565 566 <li>Cancel - 567 Selecting Cancel cancels any changes and closes the segment window</li> 568 569 <li>Replicate - 570 Displays the Replicate Segment window for entering multiple segments at once. 571 See below.</li> 572 573 <li>Play/Stop Segment - Starts 574 or Stops playback of the segment using the segment attributes assigned. </li> 575 576 <li>Play/Stop MIDI File - 577 Starts or Stops playback of the MIDI file 578 which the segment is assigned to. </li> 579 580 <li>Pause/Resume - 581 Pauses or Resumes playback.</li> 582 583 </ul> 584 585 586 587 <h2>5.2 Event Window</a></h2> 588 589 <p>The event window is where a given segment's 590 event attributes are assigned and auditioned, as shown in the picture below. To 591 add an event to a segment, the author must first select the segment which will 592 contain the event, then select the Add button. This will bring up the Event 593 window.</p> 594 595 596 597 <p> 598 <img border=0 width=554 height=294 599 src="{@docRoot}images/jet/jc_event_wnd.png"></p> 600 601 602 603 604 605 <p>There are two main sections to the event 606 window. The segment section on the left side of the event window is for display 607 only. It shows what the segment attributes are for the given segment. The Event 608 section, on the right side, is where events can be assigned. The following 609 parameters are available:</p> 610 611 612 613 <p>Event Name - 614 Assigns a name to an event</p> 615 616 <p>Event Type - 617 Selects which type of event to assign.</p> 618 619 <p>Starting M/B/T - 620 Sets the starting measure, beat, and tick for the event</p> 621 622 <p>Ending M/B/T - 623 Sets the ending measure, beat, and tick for the event, if applicable</p> 624 625 <p>Track - 626 Sets which track in the given segment the event will apply to</p> 627 628 <p>Channel - 629 Sets which MIDI channel the event will apply 630 to. The MIDI channel should match the MIDI 631 channel of the track</p> 632 633 <p>Event ID - 634 Sets the event ID for the event. Multiple events can be assigned to the same 635 segment and therefore the Event ID is used to identify them</p> 636 637 638 639 <p>To the right of the Audition window are a few additional buttons. 640 These do as follows:</p> 641 642 <p> 643 644 <p>OK - 645 Selecting OK confirms all event settings and closes the event window</p> 646 647 <p>Cancel - 648 Selecting Cancel cancels any changes and closes the event window</p> 649 650 <p>Replicate - 651 Displays the Replicate Event window for entering multiple events at once. See 652 below.</p> 653 654 <p>Play/Stop - 655 Starts or Stops playback of the segment using the segment attributes assigned. 656 While the segment is playing, events can be triggered and auditioned.</p> 657 658 <p>Trigger - 659 Triggers the event assigned. This replicates the API command that the JET game 660 will use to trigger the event, therefore giving the content author a method for 661 auditioning the behaviour of the event.</p> 662 663 <p>Mute/UnMute - 664 Mute/UnMute will mute or unmute the track that the event is assigned to</p> 665 666 <p>Pause/Resume - 667 Pauses or Resumes playback.</p> 668 669 670 671 <p>To audition the behaviour of an event, you 672 can select the Play button. This will initiate playback. The trigger button 673 will send the trigger event when pressed. This is equivalent to selecting the 674 green trigger event in the timeline. </p> 675 676 677 678 <p>Note: Trigger events are meant to unmute a 679 single track of a segment when triggered, then mute that track at the end of 680 the trigger segment. Therefore you should make sure the mute flag is set to 681 mute the track that a trigger event will be unmuting when receiving a trigger event. 682 </p> 683 684 685 686 <p>Please read Section <i>6 Under The Hood</i> 687 below for details on how trigger events work and behave. </p> 688 689 690 691 <h2>5.3 Replicate Windows</h2> 692 693 <p>Often in creating JET files, you'll need to 694 create tens or even hundreds of events. You may also need to move events. The 695 Replicate and Move windows allow for this. There are two Replicate windows for 696 creating multiple segments or events. They look like the following:</p> 697 698 699 700 <p><img border=0 width=395 height=419 701 src="{@docRoot}images/jet/jc_rep_wnd.png" ></p> 702 703 <p>Replicate Segment Window</p> 704 705 706 <p><img border=0 width=398 height=425 707 src="{@docRoot}images/jet/jc_repseg_wnd.png"></p> 708 709 710 711 <p>Replicate Event Window</p> 712 713 714 715 <p>Both Replicate windows function the same. 716 After creating an initial segment or event, you can select the Replicate 717 button. The parameters are as follows:</p> 718 719 720 721 <p>Name Prefix - 722 Sets the prefix for the name of each segment or event created</p> 723 724 <p>Starting M/B/T - 725 Sets the starting time for the first segment or event</p> 726 727 <p>Increment M/B/T - 728 Sets the time between segments or events created. </p> 729 730 <p>Number - 731 Sets the number of segments or events you wish to create. If the number 732 overflows the length of the MIDI file (for 733 segments) or segment (for events), those objects will not be created.</p> 734 735 <p>Preview - 736 Preview allows you to examine the objects created before saying OK to insert 737 them.</p> 738 739 740 741 742 743 <h2>5.4 Move Windows</h2> 744 745 <p>The Move function acts similarly to the 746 Replicate function in that it allows you to edit multiple segments or events at 747 one time, in this case move them in time. Like Replicate, there are two Move 748 windows, one for Segments and one for Events. The windows look like the 749 following:</p> 750 751 752 753 <p><img border=0 width=400 height=424 754 src="{@docRoot}images/jet/jc_moveseg_wnd.png"></p> 755 756 757 758 <p>Move Event Window</p> 759 760 761 762 <p>To use Move, first select the segments or 763 events you wish to move in time, then click the Move button. The parameters are 764 as follows:</p> 765 766 767 768 <p>Starting M/B/T - 769 Sets the starting time for the first segment or event</p> 770 771 <p>Increment M/B/T - 772 Sets the time in M/B/T you wish to move the objects by. </p> 773 774 <p>Preview - 775 Preview allows you to examine the objects created before saying OK to move 776 them.</p> 777 778 779 780 781 782 <h2>5.5 Audition Window</a></h2> 783 784 <p>Clicking the Audition button in the main 785 window of the JET Creator application will open the Audition window. This is 786 where the content author or application programmer can simulate the interactive 787 playback as it may occur in the mobile application or game itself.</p> 788 789 790 791 <p><img border=0 width=554 height=370 792 src="{@docRoot}images/jet/jc_audition_wnd.png"></p> 793 794 795 796 <p>JET Audition Window</p> 797 798 799 800 801 802 <p>There are four main sections to the 803 audition window. The left most section displays the available segments and 804 their length in seconds. The middle section displays a running list of what 805 segments are queued for playback and what their playback status is. The far 806 right section displays the mute flags for the currently playing segment. The 807 timeline section at the bottom is the same as in the main window. It displays 808 the currently playing segment as well as a visual display of any event triggers 809 associated with that segment. </p> 810 811 812 813 <p>The Audition window allows you to queue up 814 any segment in any order for playback. To do this simply select the segment you 815 wish to cue and hit Queue. That segment will appear in the queue window and 816 start playing (if it is the first segment). Subsequently you can select any 817 other segment or segments and cue them up for playback. As the segments 818 complete playback, the next segment in the queue will begin playing. As is the 819 other windows of JET Creator, you can mute, unmute, trigger event clips, etc. 820 in realtime as each segment is playing back.</p> 821 822 823 824 <p>Specifically the buttons behave as follows:</p> 825 826 827 828 <p>Queue - 829 loads the selected segment into the queue and starts playback</p> 830 831 <p>Cancel and Queue - 832 cancels the currently playing segment before queueing the selected segment for 833 playback</p> 834 835 <p>Cancel Current - 836 cancels the currently playing segment in the queue and begins playback of the 837 next segment</p> 838 839 <p>Stop - 840 stops playback of all queued segments</p> 841 842 <p>Mute All - 843 mutes all tracks in the current segment</p> 844 845 <p>Mute None - 846 unmutes all tracks in the current segment</p> 847 848 <p>Original Mutes - 849 sets the original mute flags for the current segment</p> 850 851 852 853 <p>The combination of these playback options 854 allows an author or application programmer to audition any behaviour an 855 interactive music application may encounter. </p> 856 857 858 859 860 861 <h2>5.6 JET Creator Menus </h2> 862 863 <p>The JET Creator menus provide access to 864 many of the parameters in the main window plus a few additional parameters.</p> 865 866 <h3>5.6.1 File Menu</h3> 867 868 <p>The File Menu contains the following 869 elements:</p> 870 871 872 873 <p>New - 874 Creates a new JET Creator file (.jtc)</p> 875 876 <p>Open - 877 Opens an existing JET Creator file</p> 878 879 <p>Save - 880 Saves the currently opened JET Creator file</p> 881 882 <p>Save As - 883 Saves the currently opened JET Creator file to a new file</p> 884 885 <p>Import Project - Imports a JET Creator archive (.zip)</p> 886 887 <p>Export Project - Exports a JET Creator archive (.zip)</p> 888 889 <p>Exit - 890 Exits the application</p> 891 892 893 894 <h3>5.6.2 Edit Menu</h3> 895 896 <p>The Edit Menu contains the following 897 elements:</p> 898 899 900 901 <p>Undo - 902 Undo will undo the last edit made</p> 903 904 <p>Redo - 905 Redo will redo the last undo</p> 906 907 <p>Cut - 908 Copy selected parameter into clipboard and Delete selection</p> 909 910 <p>Copy - 911 Copy selected parameter into clipboard and keep selection</p> 912 913 <p>Paste - 914 Paste selected parameter</p> 915 916 917 918 <h3>5.6.3 JET</h3> 919 920 <p>The Edit Menu contains the following 921 elements:</p> 922 923 924 925 <p>Properties - 926 Brings up the JET Creator priorities window. This window allows you to set the 927 following conditions for a given JET file:</p> 928 929 <p>Copyright Info - Contains copyright info to be inserted into JET file</p> 930 931 <p>Chase Controllers - Option to chase controllers (on/off). This should usually 932 be ON.</p> 933 934 <p>Delete Empty Tracks - Deletes any empty MIDI tracks</p> 935 936 937 938 <h3>5.6.4 Segments</h3> 939 940 <p>The Segments Menu contains the following 941 elements:</p> 942 943 944 945 <p>Add Segment - 946 Brings up the Segment window</p> 947 948 <p>Update Segment - Updates segment attributes</p> 949 950 <p>Delete Segment - Deletes the current segment from the 951 Segment List</p> 952 953 954 955 <h3>5.6.5 Help</h3> 956 957 <p>The Help Menu will contain at least the 958 following elements:</p> 959 960 961 962 <p>JET Creator Help - will launch PDF help document or go to on-line help</p> 963 964 <p>About - 965 JET Creator version number, SONiVOX info</p> 966 967 968 969 970 971 <h1>6 Trigger Events Explained</h1> 972 973 <p>Breaking a MIDI 974 file into individual (non-linear) segments and queueing up those segments for 975 playback in a game based on events within the game is one way JET music files are 976 interactive. Trigger events are an additional method for interactive playback. 977 Both would be used together in an interactive game or application.</p> 978 979 980 981 <p>Trigger events allow for the following:</p> 982 983 984 <ol> 985 <li><span 986 lang=EN-IE>Tracks <i>within</i> a MIDI segment can be turned on or off based on game 987 events. For example the composer could author two drum tracks, one fast 988 and one slow. If the action in a game is fast, the fast drum track could 989 play. If the action in the game is slow, the slow drum track can play.</li> 990 <li><span 991 lang=EN-IE>User actions can be compared to trigger events which are 992 pre-inserted into a music file at musically correct places. Based on the 993 results, scoring or other game actions can take place. </li> 994 <li><span 995 lang=EN-IE>Musical transitions between levels or action sequences can be 996 synchronized to be musically seamless.</li> 997 </ol> 998 999 1000 1001 <p>Under the hood, JET uses standard MIDI CC 1002 events to accomplish these actions and to synchronize audio. The controllers 1003 used by JET are among those not defined for specific use by the specification. The specific controller definitions 1004 are as follows:</p> 1005 1006 1007 1008 <p> Controllers 1009 80-83 Reserved for use by 1010 application</p> 1011 1012 <p> Controller 1013 102 JET event marker</p> 1014 1015 <p> Controller 1016 103 JET clip marker</p> 1017 1018 <p> Controllers 1019 104-119 Reserved for future use</p> 1020 1021 1022 1023 <h2>6.1 JET Clip Marker (CC103)</h2> 1024 1025 <p>Controller 103 is reserved for marking 1026 clips in a MIDI track that can be triggered by 1027 the JET_TriggerClip API call. The clip ID is encoded in the low 6 bits of the 1028 controller value. Bit 6 is set to one to indicate the start of a clip, and set 1029 to zero to indicate the end of a clip.</p> 1030 1031 1032 1033 <p>For example, to identify a clip with a clip 1034 ID of 1, the author inserts a MIDI controller 1035 event with controller=103 and value=65 at the start of the clip and another 1036 event with controller=103 and value=1 at the end of the clip. When the 1037 JET_TriggerClip() function is called with a clip ID of 1, the track will be 1038 un-muted when the controller value 65 is encountered and muted again when the 1039 controller value 1 is encountered.</p> 1040 1041 1042 1043 <p><img border=0 width=492 height=367 1044 src="{@docRoot}images/jet/clip_marker.png"></p> 1045 1046 <p>Figure 5: Synchronized Clip</p> 1047 1048 1049 1050 <p>In the figure above, if the 1051 JET_TriggerClip() function is called prior to the first controller event, Track 1052 3 will be un-muted when the first controller event occurs, the first clip will 1053 play, and the track will be muted when the second controller event occurs. If 1054 the JET_TriggerClip() function is called after the first controller event has 1055 occurred, Track 3 will be un-muted when the third controller event occurs, the 1056 second clip will play, and the track will be muted again when the fourth 1057 controller event occurs.</p> 1058 1059 1060 1061 <p><b>Note:</b> Normally, the track containing the clip is muted by the application 1062 when the segment is initially queued by the call to JET_QueueSegment(). If it 1063 is not muted, the clip will always play until Jet_TriggerClip() has been called 1064 with the clip ID.</p> 1065 1066 1067 1068 <h2>6.2 JET Event Marker (CC102)</h2> 1069 1070 <p>Controller 102 is reserved for marking 1071 events in the MIDI streams that are specific 1072 to JET functionality. Currently, the only defined value is 0, which marks the 1073 end of a segment for timing purposes. </p> 1074 1075 1076 1077 <p>Normally, JET starts playback of the next 1078 segment (or repeats the current segment) when the MIDI 1079 end-of-track meta-event is encountered. Some MIDI 1080 authoring tools make it difficult to place the end-of-track marker accurately, 1081 resulting in synchronization problems when segments are joined together.</p> 1082 1083 1084 1085 <p>To avoid this problem, the author can place 1086 a JET end-of-segment marker (controller=102, value=0) at the point where the 1087 segment is to be looped. When the end-of-segment marker is encountered, the 1088 next segment will be triggered, or if the current segment is looped, playback 1089 will resume at the start of the segment.</p> 1090 1091 1092 1093 <p>The end-of-segment marker can also be used 1094 to allow for completion of a musical figure beyond the end of measure that 1095 marks the start of the next segment. For example, the content author might 1096 create a 4-bar segment with a drum fill that ends on beat 1 of the 5<sup>th</sup> 1097 bar, a bar beyond the natural end of the segment. By placing an end-of-segment 1098 marker at the end of the 4<sup>th</sup> bar, the next segment will be 1099 triggered, but the drum fill will continue in parallel with the next segment 1100 providing musical continuity.</p> 1101 1102 1103 1104 <p><img border=0 width=467 height=185 1105 src="{@docRoot}images/jet/event_marker.png"></p> 1106 1107 <p>Figure 6: End-of-segment Marker</p> 1108 1109 <h2>6.3 Application Controllers (CC80-83)</h2> 1110 1111 <p>The application may use controllers in this 1112 range for its own purposes. When a controller in this range is encountered, the 1113 event is entered into an event queue that can be queried by the application. 1114 Some possible uses include synchronizing video events with audio and marking a 1115 point in a MIDI segment to queue up the next 1116 segment. The range of controllers monitored by the application can be modified 1117 by the application during initialization.</p> 1118 1119 <h1>7 JET Creator Guidelines</h1> 1120 1121 <p></p> 1122 1123 <h2>7.1 Order of Tasks</h2> 1124 1125 <p>As with all projects, its best to discuss and design the interactive music scheme with the game designer and programmer before beginning your composition. An outline and/or specification can go a long way in saving you from having to redo things after the game is in place.</p> 1126 1127 <p>In general youll want to first write your music in your DAW of choice the way youre used to composing, then break up the final MIDI file as needed for the application. Next, move to JET Creator and create all of your music segments in the order easiest to preview them when played in order. Finally, add the JET Events to control the segments via the Android game and Audition them as needed in JET Creator. Finally, save the project in JET Creator and hand off the .jet file to the programmer to integrate it in the game. After previewing there will likely be changes to the MIDI file(s) and JET Creator attributes. </p> 1128 1129 <h2>7.2 Conserving Memory</h2> 1130 1131 <p>If youre trying to conserve memory, compose as few MIDI files as possible, and create several segments from that MIDI file. For example a 12 bar MIDI file with three sections of 4 bars, A, B, C, can create a much longer song. Simply create multiple segments that reference the one MIDI file, then order them however you like. For example, A, A, B, A, C, A, B, A, A would create a 36 bar song. Use JET to add repeats, transpose segments, and interactively mute and unmute tracks to keep it even more interesting.</p> 1132 1133 <h2>7.3 Replicate</h2> 1134 1135 <p>To make adding segments or events faster, use the Replicate command. Replicate can add multiple segments or events at one time and uses an offset parameter and prefix naming convention to keep things easy to read. The MOVE command is also useful for moving multiple events by a set number of measures, beats or ticks.</p> 1136 1137 <h2>7.4 Interactive Options</h2> 1138 1139 <p>There are several interactive audio concepts possible in JET. Below are a few examples although we hope developers will come up with others we havent thought of! These are:</p> 1140 1141 <h3>7.4.1 Multiple Segment Triggering</h3> 1142 1143 <p>In this method the application is triggering specific segments based on events in the game. For example a hallway with lots of fighting might trigger segment 1 and a hallway with no fighting might trigger segment 2. Using JET TriggerClips in conjunction with this method creates even more diversity.</p> 1144 1145 <h3>7.4.2 Mute Arrays</h3> 1146 1147 <p>In this method the application is triggering mute and unmute events to specific tracks in a single MIDI sequence. For example a hallway with lots of fighting might play MIDI tracks 1-16 and a hallway with no fighting might play the same midi file but mute tracks 9-16. Using JET TriggerClips in conjunction with this method creates even more diversity.</p> 1148 1149 <h3>7.4.3 Music Driven Gameplay</h3> 1150 1151 <p>Music driven gaming is similar to what Guitar Hero and JETBOY have done in that the music content determines how graphic events are displayed. The application then queries the user response to the graphic events and interactively modifies the music in response. In this method the game is utilizing JET Application Events, MIDI controllers that are embedded in the MIDI file and read by the game in real-time. Based on the user response, multiple segment triggering and/or mute arrays can be set.</p> 1152 1153