mod2inty.php - convert Soundtracker like MODs to Intellivision

[carlsson]

As most of you in this part of the forum know, there is a set of music routines by Arnauld Chevallier that can be used to playback tracker music.

DZ-Jay has previously posted a How To-guide on how to manually put together your music data.

Some have asked for a tool to simplify the process. While this is no one click solution, I have written a PHP script that will, assuming the input file is of the proper formatting, output something that almost will assemble.

Get the PHP script here: mod2inty.php

First, compose some music in a tracker of your choice. I am an Amiga user since the early 1990's, so for me the choice has always been OctaMED. However, you might find that other trackers like Protracker will work too. What you need is an option to print to file your music data. That means you can't take a .MOD file as it is, you need a text representation of the content of the file.

Obviously when you compose, you need to think ahead as regarding to the Intellivision sound limitations. That means three channels, of which one (channel A) may be mixed with noise for drum effects. Most trackers will let you work with four channels, but the fourth channel is no good, unless you plan to manually mix two channels at a later stage.

The printed file will contain some headers, like instrument table, volumes, playlist and so on. You need to edit that away with your favorite text editor. In my case, it means Emacs but YMMV. Save the playlist for later, because you will want to set the patterns in the right order.

What you should have is a file like this: mmix.txt

Now open a command prompt which has access to the PHP CLI executable. That is freely downloadable elsewhere, for various operating systems if you don't have it. For those who prefer other scripting languages, I leave it as an exercise to the user to convert the script.

Use the command "php mod2inty.php printedfile.txt 4" in case your song has 4 tracks. As noted above, only the first three are used anyway so you can use a lower number. Note that all the output will appear on standard output so you might want to use > to capture it into a file.

This script tries to handle the following:

• Split each track inside each block to a subpattern, with support for long (sustained) notes
• Handle the effect command "C00" to silence notes, insert rest
• Identify volume changes on actual played notes, not fade out while note is playing (MOD volumes = $00 - $3F)
• Identify arpeggio effect commands "0XX" and create matching pitch effects and instruments
• Identify identical subpatterns and omit duplicates
• Save the configuration of subpatterns for each block in the MOD

What it doesn't do:

• Identify what should be drums, if those are used, since MOD's use drum samples at any position
• Identify the playlist order of patterns, this is what you need to fix afterwards
• Identify different instruments using the same arpeggio effects (i.e. those would all be converted to the same instrument)
• Handle a lot of other effect commands for slide, portamento, vibrato etc. Some of these are supported in the music routines, most aren't

Obviously there may be awful bugs, and in particular if your input format doesn't match the example, results may be highly unpredictable.

 

The output is a music file with instruments & envelopes stolen borrowed from Arnauld's demo song. You should of course edit those to your liking.

However, the file won't assemble automatically because for every instrument imported from the MOD file, an X is added in front of the instrument number, and for every instrument automatically generated due to a pitch effect, a Z is added in front of the new instrument number. This is for your sanity, so you make sure to go through the output and make adjustments as required. Since X and Z only appear in front of the instrument numbers, those are easy to remove or replace with your text editor. Also remember to adjust the order of patterns.

Hopefully someone will understand something of this, and perhaps even have use of it.

Good luck!

[Vincehood]

Hello,

sounds interesting. I don't know much about this but is this process possible to apply to a MIDI file used as input. If yes, are the tracker you mention suitable for such case as well?

Thanks

/V

[carlsson]

There are tools to generate text output from MIDI. It looks like this:

 

0 On ch=2 n=33 v=127
96 Off ch=1 n=57 v=127
192 On ch=1 n=57 v=127
192 Off ch=2 n=33 v=127
192 On ch=2 n=45 v=127
288 Off ch=1 n=57 v=127
384 Pb ch=1 v=8221
384 On ch=1 n=64 v=127
384 Off ch=2 n=45 v=127
576 Off ch=1 n=64 v=127
576 Pb ch=1 v=8222
576 On ch=1 n=61 v=127
576 On ch=2 n=33 v=127
768 Off ch=2 n=33 v=127
768 Pb ch=2 v=8222
768 On ch=2 n=37 v=127
960 Off ch=2 n=37 v=127
960 On ch=2 n=40 v=127
1152 Off ch=1 n=61 v=127
1152 On ch=1 n=56 v=127
1152 Off ch=2 n=40 v=127
... and so on

The first value is number of ticks in the selected time base, n is the note number and v is volume or possibly velocity. Thus very human unreadable. My above tool would not grok anything of this, but I'm sure someone could write a different script.

 

There may be tools to convert from MIDI to MOD (as I may have mentioned in the ABC thread) but I don't know how good the results will be. The Modplug Tracker among others handle both type of formats. One would have to experiment a little and be prepared that results might take more efforts than expected.

 

I believe OctaMED on the Amiga can only use a MIDI interface for playback or in later versions export to a MIDI file, not import it directly.

[decle]

Hello,

sounds interesting. I don't know much about this but is this process possible to apply to a MIDI file used as input. If yes, are the tracker you mention suitable for such case as well?

Thanks

/V

 

Hey,

 

I used MIDIs as the start point for the music in StudioVision. As Carlsson says using them is a bit fiddly, but at the time I did not know anything about MOD files. My advice would be to stick with the MOD tools that Carlsson, DZ-Jay and Arnauld have put to together as they are better and will be able to provide you with more help.

 

In using MIDIs I was after lots of simple music and I'm not a musician at all. So I cobbled together an automated process that would take existing piano MIDIs and converted them to INTV assembly language using MIDICSV. This process requires that the source music and what you are after are 2 voices, which seems to fit some piano music quite well.

 

The process automatically converts

• from a suitable source MIDI: http://www.trachtman.org/MIDI/Joplin/entrtanr.mid (taken from Warren Trachtman's excellent website)
• to a stripped out 2 voice version: stripped.zip
• to a DECLE representation: tune.asm
• to a really simple integrated player: musicPlayer.rom

While I'm happy to share this stuff, if I were you I would stick to using MODs as they are much more flexible, Arnauld's player is much better and many more people know and understand how the whole thing works.

 

 

Hope this helps

 

decle

[carlsson]

The more solutions, the better! In the other end, someone suggested that one should implement the Vortex tracker format on the Intellivision. If I understand correctly, that is a dump of AY registers over a time base which means the player routine will be very simple, but the music data will take up a fair chunk of ROM space in the final product.

 

How did you strip down the MIDI song to two channels, through an automated tool or by hand? I understand that MIDICSV is used to get the comma separated list of notes that you convert to DECLEs.

 

One of the reasons I work with MODs, other than it is a format I've been using for ~25 years, is that it is intended for computer music with limited resources. MIDI can handle up to 16 channels (or more if you have multiple sound cards) and each channel may contain chords or several notes at the same time as long as they don't overlap in duration. MODs usually are restricted to 4 channels, one note per channel. On e.g. the Amiga or PC, you would simulate chords by using rich samples, or the arpeggio command which my converter tries to capture. When it comes to volume changes, I suppose both formats are equally well suited. Eventually when the player supports vibratos, glissandos and so, those are simple commands in the MOD, while I don't have a clue how a glissando is represented in MIDI and definitely vibrato even less as it is a performance instruction more than part of the notation.

 

In the other end of the scope, we have MusicXML which is developed in order to get a proper, portable format to represent sheet music. MIDI is mostly note data for electronic instruments to playback and e.g. spells out triplets as they likely will be performed, spells out staccato as very short notes with rests inbetween, spells out crescendos as velocity changes per note, spells out repeats as duplicated music data and a lot more. When I'm importing MIDI to my music notation software, I usually have to fix all this with macros or by hand. Some files are better coded so the music software can detect which notation is really intended. Obviously MOD would be an even worse choice to get music notation from as it works with note on and off per tick, various effect bytes and depends on the sampled sounds you load to the instruments. Though I remember that at least OctaMED had a music notation input mode, but I think that one dumbed down the UI a lot compared to the traditional roll.

[decle]

The more solutions, the better!

Whilst I appreciate the sentiment and I recognise it is the open source way, I'm not sure I agree. I would much rather effort was focused on developing and supporting a primary tool that fulfills 90% of the need as simply and elegantly as possible, like we have with Jzintv. Therefore, in posting I was trying to suggest that I acted out of naivity and that whilst MIDIs and tools like those I developed could be used, I was hoping to demonstrate that they are limited and inflexible and I think it would be better to put the effort into the MOD based approach which seems to be a much better fit. Perhaps I missed the mark.

 

The risk with putting all the effort into one tool is that it becomes bloated and hard to use in an effort to cover 100% of the requirements. Here, I think there is a need for strong curators and a POSIX ethos. This is why, whilst I hope that my patch to Jzintv to enable more GRAM might be taken forward, I will not push this agenda. I do not know what Joe's ambitions for Jzintv are. I recognise that enhancing it to cover hardware that does not exist is a niche requirement and may not be a direction he wants to take the tool. This is a view I would totally respect. The great thing about open source is that if this is the case I can make the patch available and if people want to use it they can. I can do my thing without diluting the focus of the core tool.

 

In the other end, someone suggested that one should implement the Vortex tracker format on the Intellivision. If I understand correctly, that is a dump of AY registers over a time base which means the player routine will be very simple, but the music data will take up a fair chunk of ROM space in the final product.

 

I took a quick look at AY players in trying to work out how I was going to do StudioVision music. I think these are trying to solve a different problem and I think they are probably the wrong tool for the job.

 

How did you strip down the MIDI song to two channels, through an automated tool or by hand? I understand that MIDICSV is used to get the comma separated list of notes that you convert to DECLEs.

 

That is correct. The CSV file is processed automatically by a small Perl script that strips the highest note from each of the first two channels in the MIDI file. There are various fudges in there to handle overlaps, very small rests etc. This generates both the tune.asm and another CSV file which is then converted back to stripped.mid.

 

One of the reasons I work with MODs, other than it is a format I've been using for ~25 years, is that it is intended for computer music with limited resources. MIDI can handle up to 16 channels (or more if you have multiple sound cards) and each channel may contain chords or several notes at the same time as long as they don't overlap in duration. MODs usually are restricted to 4 channels, one note per channel. On e.g. the Amiga or PC, you would simulate chords by using rich samples, or the arpeggio command which my converter tries to capture. When it comes to volume changes, I suppose both formats are equally well suited. Eventually when the player supports vibratos, glissandos and so, those are simple commands in the MOD, while I don't have a clue how a glissando is represented in MIDI and definitely vibrato even less as it is a performance instruction more than part of the notation.

 

Excellent, this is all the reasons why my approach is not a good one. Like AY players, I now think MIDIs are the wrong tool for this particular job. Starting over I would definitely try harder to work contribute to an automated process for MODs. My primary reason for looking at MIDIs at the time was that as a non-musician it provided a vast catalogue of potential source music and I knew just enough about the format to see how I could convert them into something playable. In practice the constraints of the process I developed means that most of of this music catalogue cannot be used because it is too sophisticated. It took me ages to find a Star Wars MIDI I could convert.

 

As far as I can tell the only thing that using MIDI might give over MODs is that some pieces are live recordings have dynamics and tempo changes which don't seem to be in most chip tunes that are authored on a computer. Lots of Warren Trachtman's MIDIs have this. I think this gives the music a more authentic, natural feel that I like. I worked hard to keep this aspect of the music through the automatic conversion process I put together. This is definitely something I'd look to try to keep going forward and I'm sure it is something that could be modelled in MODs if the source music had it. I guess the question is how much MOD music is sourced from live performances.

[carlsson]

There actually are commands to change tempo in the middle of a MOD. Various trackers use different commands though, with different ranges of granularity. Some composers use those commands a lot, others not so much. But if the (Intellivision) player routine doesn't have support for such tempo changes, the data needs to be adjusted with note lengths to compensate for tempo changes. Sometimes it could be nearly impossible.

 

The already tracked music in AY format has the benefit that it is designed to match the sound chip 1:1. Since it would just read and set register values without needing to look up notes from a table, keep track of durations, probably patterns and so on, it can be very fast I imagine. On the Intellivision with up to 84 kB of ROM space with the right memory map, it might be lesser of a problem than on the ZX Spectrum, Amstrad CPC or Vectrex. However I must admit I haven't studied it in detail, so perhaps it has better properties than I believe. For someone composing music, you could use an existing cross platform AY tracker and save the song in this format with all forms of effects translated as register settings.

 

I'll agree that if the existing tools and methods are difficult, perhaps it is better to ask for help from someone who is capable of handling them instead of inventing new methods. For the same reason, I'm happily accepting help with in particular graphics as I know I am lacking a lot in that department, and even take programming tips.

And yes, an emulator supporting non-existing hardware is a bit like vaporware. For many years, the xvic emulator in the VICE package accidentally supported that the VIC-I chip could access the 3K expansion memory slot, which led to some people writing programs for a such configuration. In the real world, the only way to do that is to piggyback solder memory chips inside the VIC-20 computer, as memory expansion on the cartridge bus is only available to the CPU. That behavior was later corrected in VICE, and I don't know if there is a setting or demand for the previous function.

[+DZ-Jay]

I agree with some of the comments from decle, but I disagree with the final conclusions. I think there are competing motivations at play here that we need to separate: on the one hand, there is an entire generation brought up with arcane "programmer-think" applications to write music, designed by engineers, who are used to this sort of environment and feel comfortable writing music in it. Plus, they have an interest in being able to translate the existing canon of work into our platform.

 

On the other hand, there is a need for a more natural interface for actual musicians and traditional music works -- I'm talking about the existing historical library of musical pieces in the public domain and professionally (or traditionally) trained musicians, not chiptune artists. That is not intended to denigrate either in the least; it's just a fact that the two may not always coincide, notwithstanding their talent and experience.

 

The MOD format is not a data-exchange format. It may have grown into that role due to historical accident, but it is merely the technical data structure to facilitate execution of a particular type of music player. The MIDI format was designed specifically to be a data-exchange format and to be device agnostic; but it was intended to convey the capabilities of music synthesizers, which are rich in features and capabilities outside the reach of most micro-computers.

 

If you compare them, the MOD tells you want the player needs to do as it plays, including how to define the instruments and all other effects; while the MIDI specifies only what general events transpired (including some performance-specific qualities), but leaves the implementation to the target player. This is on purpose: it's the electronic, synthesizer-friendly version of a music score.

 

All that said, I don't think either is a perfect fit for the Intellivision, although they both can be made to work.

 

Think about our current discussion for a second: When people talk about transcoding music data from MIDI into MOD format in order to transcode it later into a specific player format, it shows that the MIDI format was inadequate to begin with. So why use MIDI? Because a user-friendly, musician-focus software package allows the creation of music and exporting to MIDI. If the tool is apt for the user and it proves useful to most people, why not then write a MIDI player, or at least a converter from MIDI to an optimized internal tracker format?

 

Rather than changing the focus into an interoperability engineering problem, why not solve the original problem of supporting the musician?

 

Personally, I don't find the MOD format useful for composing music, unless you're stuck in the past using Fast Tracker on an Amiga computer. There are better music composition tools out there and there are some industry standards available. The limitations can be managed by the artist in the same way that graphic artists employ limited colour palettes when they use, say, Photoshop to make Intellivision artwork. For example, the Intellivision supports only three channels. I use Logic Pro to compose a song. I can ignore the fact that it supports 64 channels and just use the first three. Big deal, I can handle it.

 

Again, my focus here is in modern composition, not getting my favourite tunes from Micro X into the Intellivision. That should be a separate effort altogether.

 

-dZ.

Edited January 25, 2017 by DZ-Jay

[carlsson]

Indeed now when I think about it, what I am using is a textual representation of the MOD that is reasonably human readable for some of us. I haven't looked into how the binary format is constructed. I'm sure MIDI can have a similar textual representation that is mostly human readable too, just that none of the examples we posted so far points in that direction.

 

In the other thread we discussed the ABC notation, which is a highly human readable textual representation of music, just that if one would try to compose music directly in that notation, it would be just as arcane as FastTracker is, if not more. However as an intermediate format to either playback as it is (new routine required), or convert to one of the existing formats (Arnauld's tracker, IntyBASIC MUSIC) it would be rather efficient.

 

As it turns out, there are tools like Maestro! to convert MIDI to ABC. I suppose if you compose music with the intended target in mind - number of voices, octave range etc - it will work out.

https://github.com/digero/maestro/releases

 

Actually you might be able to get the source code and make a branch that uses part of the conversion engine but outputs data in an existing Intellivision format instead of the ABC format.

[decle]

Totally agree with DZ

 

So am I right to think that there might be 3 separate requirements:

• A format that efficiently represents music within the capabilities of the Inty to play, and a player that will render it efficiently. MOD or some Inty derivative such as we have with Arnaud's player might be suitable. Carlsson's script is a good step in this direction.
• A tool that will take the output of a musician friendly authoring tool like LogicPro and generate a playable representation from it (midi2mod2inty?).
• A tool that will take MIDI representations (or something similar for which there is a large catalogue of existing material which is not copyright encumbered), strip it down to a level that the Inty might be able to play and generate a playable representation, for the music scuffers out there like me. Presumably this if the starting point is MIDI the same mechanism could be used by musicians and scuffers?

So are we really after something that will take MIDI and generate something that will integrate with Carlsson's work to give something end to end?

[carlsson]

After analyzing the data format, using DZ-Jay's excellent How To-guide, Arnauld's routine (if that is the one we eventually want to use and perhaps contribute to) was not terribly difficult to understand, so I think a tool that converts MIDI data in whichever format, doesn't really need to take the step over a MOD-like textual representation. However if someone right now, right here has songs in MIDI format they desperately need to include in their upcoming Intellivision production, I see three solutions:

 

1. Code the music by hand, which may be cumbersome, whether you're using the Arnauld or IntyBASIC approach

2. Try to find a MIDI2MOD tool + my script, cross your fingers and hope nothing will explode

3. Try Decle's approach with the CSV representation, using his separate routine

 

It should be noted that one more difference between MOD and MIDI is the logic structure: MODs consist of blocks of N channels and M ticks per block. Those blocks are joined in any order, and you can repeat the same block multiple times. MIDI is a continuous stream of music data. No repeat or block markers as far as I know, but I'll be the first to admit I'm wrong about this. If the target music format is one that consists of blocks and playlists, depending on the type of music you've composed, you might save some space by working in a format that has this subdivision. If however you write ever changing symphonies where there are not three seconds of music that appears exactly the same later on, the block structure is pointless and just a lot of overhead.

[+DZ-Jay]

In the other thread we discussed the ABC notation, which is a highly human readable textual representation of music, just that if one would try to compose music directly in that notation, it would be just as arcane as FastTracker is, if not more. .

I very strongly disagree. The ABC notation is an ASCII representation of a traditional music score in sheet music. This notation has been used for hundreds of years by musicians all across the world and across cultures, to great effect, to compose music.

 

I am not a musician (I'm not even a very good programmer), but I I'm used to reading musical scores in sheet music since I was a little kid. I think in notes and musical cues, and have a lot of trouble imagining music as a set of volume and frequency change events over an arbitrary time scale measured in powers of 2.

 

Most musicians would expect the computer to do that for them.

[+DZ-Jay]

Totally agree with DZ

 

So am I right to think that there might be 3 separate requirements:

• A format that efficiently represents music within the capabilities of the Inty to play, and a player that will render it efficiently. MOD or some Inty derivative such as we have with Arnaud's player might be suitable. Carlsson's script is a good step in this direction.
• A tool that will take the output of a musician friendly authoring tool like LogicPro and generate a playable representation from it (midi2mod2inty?).
• A tool that will take MIDI representations (or something similar for which there is a large catalogue of existing material which is not copyright encumbered), strip it down to a level that the Inty might be able to play and generate a playable representation, for the music scuffers out there like me. Presumably this if the starting point is MIDI the same mechanism could be used by musicians and scuffers?

So are we really after something that will take MIDI and generate something that will integrate with Carlsson's work to give something end to end?

I once again caution not to treat this solely as an interoperability engineering problem. The focus would then go on building tools to transcode from one format to the other, when the problem is actually, how do we get music from thought to sound chip?

 

Perhaps all the effort in building these intermediary tools could go into implementing a more suitable tracker that can process an optimized data format, and an encoder for whatever interface format people want.

 

dZ.

[carlsson]

For comparison, I've written music with traditional scoring since I was 9 years old or so and still do. I have no problem with stave notation. It is the textual representation that I would have trouble with, in particular if I want to have two parts, say a bass line and a melody. In a music program like NoteWorthy Composer (best $49 I have ever spent!), I just set up two staves which are lined up below eachother and I see directly how these will sync. If I was to compose with ABC syntax, I would have to keep in my mind how one part will turn out compared to the other, because they don't visually line up as they'd do both with traditional scoring, and coincidentally with the MOD tracker.

 

As a way to convey a melody line in a context where graphics are not possible, it fills a purpose, or possibly as an intermediate way to store music in a mostly human readable form. But there is no point to argue about what we like or dislike, there are enough people elsewhere doing that.

Edited January 25, 2017 by carlsson

[+DZ-Jay]

After analyzing the data format, using DZ-Jay's excellent How To-guide, Arnauld's routine (if that is the one we eventually want to use and perhaps contribute to) was not terribly difficult to understand, so I think a tool that converts MIDI data in whichever format, doesn't really need to take the step over a MOD-like textual representation. However if someone right now, right here has songs in MIDI format they desperately need to include in their upcoming Intellivision production, I see three solutions:

 

1. Code the music by hand, which may be cumbersome, whether you're using the Arnauld or IntyBASIC approach

2. Try to find a MIDI2MOD tool + my script, cross your fingers and hope nothing will explode

3. Try Decle's approach with the CSV representation, using his separate routine

 

It should be noted that one more difference between MOD and MIDI is the logic structure: MODs consist of blocks of N channels and M ticks per block. Those blocks are joined in any order, and you can repeat the same block multiple times. MIDI is a continuous stream of music data. No repeat or block markers as far as I know, but I'll be the first to admit I'm wrong about this. If the target music format is one that consists of blocks and playlists, depending on the type of music you've composed, you might save some space by working in a format that has this subdivision. If however you write ever changing symphonies where there are not three seconds of music that appears exactly the same later on, the block structure is pointless and just a lot of overhead.

Like I mentioned, MIDI Is a data exchange format. It is intended for a synthesizer to translate it into it's own internal representation in order to execute it.

 

MOD, on the other hand, is a tracker data storage format: it is the actual song data required by the tracker to play.

 

One is a transient format for interoperability, the other is specific to a particular type of music player: those based on the Fast Tracker model.

 

Fast Tracker was designed at a later era than the Intellivision. The MOD format includes support for sampled sounds and instruments and many features supported by the fabulous Amiga sound processor. Although adequate, it may not be the most appropriate for the PSG and the Intellivision.

 

It seems rather strange that back in the day, trackers and data formats were built to support the specific features and limitations of each platform; while now -- with all the collective experience and advancements we have -- we're obsessed in forcing an old tool into a platform for which it was never intended.

 

dZ.

[carlsson]

I'm not obsessed about the MOD format, it is just that out of the already existing formats that I am familiar with, it was the one that most closely resembled the format of the music routine I wanted to use (and at least some people in the other thread specifically asked for). Also it is the way of doing computer music for systems with limited capacity I've used for roughly 20 years so it is the method I know works for me. Of course your milage may vary and people with other experience will find this convoluted.

 

The question is what constitutes the ideal music format on the Intellivision, and then we would work from there to what is required to get the most desired source format to work. From my background with chip music on other formats, I think the block and playlist based routine is in great dominance, but perhaps other systems such as C64, Amiga, ZX Spectrum, NES etc are more demo oriented than the Intellivision is, so it necessarily isn't ideal for game music of a different, more free flowing kind.

Edited January 25, 2017 by carlsson

[+DZ-Jay]

Personally, I don't care about the data format. What I don't want is to be forced to think like a computer and treat an internal technical format as the human interface.

 

I think it is important to separate the two: a tracker data structure is a physical artifact private to itself, optimized for its own requirements and limitations.

 

dZ.