Fixing Ecco Jr. GEMS soundtrack

[Moduvator]

Ecco Jr. is the only Ecco game of the series that uses stock unmodified GEMS sound driver, whereas previous two 16-bit games had it modified by Novotrade to better suit their needs. The overall topic of GEMS is quite deep and I have learned a lot about it whilst working on this little project -- perhaps more than I needed!



The Problem
Anyone who has played the game is likely aware there are two music tracks in the original Ecco Jr. that have this oddly musical glitching to them.
Octopus Passage:

Bay of Scattered Songs:


Octopus Passage in particular does not sound unlike something straight out of Doctor Who episode! Given that was the way those songs sound in the game personally I always thought that was the composer's original intention, however I started to doubt this when someone has played the music through a standalone GEMS Player that takes all the data from a ROM and plays it back, essentially emulating the entire Mega Drive sound subsystem, using data blocks from a game as well as its Z80 driver. The music did not glitch there and I have quickly become curious as to what exactly is causing that in the first place. I have tried using the GemsScan tool that tries to locate stock driver initialization subroutines inside a ROM, but that got me nowhere so I switched to something else.

The Analysis
One day I decided to try again and dig a little into disassembly of the game ROM, inspired by my recent success analyzing some segments of the original 16-bit series. Sure enough: very quickly I was able to find EXACTLY the regular sound driver functions, straight from the original SEGA's development kit. It soon became obvious why Gems Scan did not work, for it relies on detecting this exact sequence of exactly these instructions to find the "load pointers" segment:

Code: Select all

2009     move.l	A1, D0
9088     sub.l	A0, D0
5340     subq.w	#1, D0

With Ecco Jr., however, the very last instruction uses subq.l, doing long word access, meaning the opcode we're looking for is $5380 which is why the detection fails. I have modified the software to reflect this and am including the updated version along with the source code in the end of this post. This was enough to keep the program happy and spit out the data properly:

Code: Select all

GEMS loader found at 0168D6 (Method 1a).
Driver:         008000, Driver End: 009886 (Size 1886)
Pointer loader found at 0169C8 (Method 1)
Instruments:    009886 (guessed size: 00D91)
Envelopes:      00A617 (guessed size: 00061)
Sequences:      00A678 (guessed size: 0B0EF)
Samples:        015767 (guessed size: EA899)

The tool will produce the following 5 files containing the data it has extracted from the ROM:

• 00 Driver.bin - Z80 driver. I have checked it against several other games such as Aladdin, CoolSpot, Sylvester & Tweety, etc, and concluded it's the same across the board.
• 01 Instruments.bin - Instrument patches in GEMS internal format. This includes both FM as well as PSG.
• 02 Envelopes.bin - Modulation envelopes to be used within songs
• 03 Sequences.bin - Song files, or 'sequences' in GEMS terminology. These contain commands for the driver: either musical notes or instructions what to do with them.
• 04 Samples.bin - Digital audio samples to be played back by the engine. Usually 8-bit Unsigned PCM.

At this point we are ready to start the experiments. Of course, the first thing that I did was to play the music back using gems2rom utility, only to be immediately disappointed as both tracks were glitching there still. Undeterred, I went with Valley Bell's music player next that gave me the advantage of not having to assemble a ROM and loading it into an emulator or putting it on a flashcart every time I wanted to test a change. To my surprise this had a partial effect of Octopus Passage (number 12 in the bank) playing back OK, whilst Bay of Scattered Songs (number 8) would still glitch.

I compared the driver binary extracted from the ROM and having satisfied myself it's exactly the same as in a few other Mega Drive games, I decided to try and replace the Envelopes file next. This one contains the so-called Modulator Bank, essentially a space efficient method to create reusable pitch bend patterns without having to record a string of pitch commands instead. This did not cure Bay of Scattered Songs, however the glitching was now different, so that was the moment I realized I must be on the right way!

The Solution
I have then experimented with giving the player a zero length Modulation Bank altogether and this finally had the desired effect, however I have quickly learned most of the sound effects are now broken, since they are relying on those, unlike music tracks, none of which appears to need them... EXCEPT the affected two! A Modulator Bank would start with a table of 2-byte pointers to the actual modulators after that table which can have arbitrary length. A song would then call it using a modulation instruction that has command code of 0x62 with the next byte containing a pointer index inside that table.
Now, if we use gems_split to look at what's happening inside Octopus Passage...

Code: Select all

channel_1:
 delay 0
 volume 12
 retrigger 0
 priority 25
 delay 1152
 patch patch_48
 loop $7F
 delay 384
 nop
 duration 95
 delay 24
 note $2D
 delay 72
 modulation modulation_1B
 delay 24
 note $24
 delay 72
 modulation modulation_1B
 duration 169
 delay 24
 note $2B
 delay 72
 modulation modulation_1B
 delay 24
 modulation modulation_1C
 ...

1B and 1C would translate into 27 and 28 decimal correspondingly. The only problem is... Ecco Jr. only has 7 entries in that bank. To make matters worse, GEMS driver does NOT perform any out of bounds checks if a sequence requests a record that isn't there and will blindly jump to an index and try to interpret garbage it finds there as valid data. Which is exactly what's happening in our case!
I speculate that perhaps those two songs were written with a different GEMS module loaded that had a much larger bank and then sequences got imported into the game without anyone noticing, composer's original intention like being to use modulations to add vibrato to those notes. Given the oddly musical nature of the glitching, this was likely how it managed to fly under QA's radar where people went "Sounds OK to me, not my job to judge the music!".

In order to fix this I chose the path of the least resistance. There is a NOP (0x63) command in GEMS, but it's one byte whereas every modulation takes two. This means I had to pad both offending tracks with equal number of zero bytes for every modulation command replaced to keep all the offsets the same.

The Results
And the final effect? 30 years later after SEGA accidentally releasing Ecco Jr. with two glitched tracks we can finally listen them almost the way composer originally intended! It would be absolutely fabulous if we could get a hold of Andy Armer to explain us the way it was meant to be played, but alas

Octopus Passage:

Bay of Scattered Songs:

I have successfully re-integrated the fixed sequence banks into the ROM files they came from and can confirm the game runs fine on original hardware which the two tracks above were recorded from using a YM2612. The result of this are patches one can use if they so will to fix their games just as well! As before I elected to go with UPS which will automatically do a CRC check and will not let you patch a wrong version.

Download Patches:

• ECCO_JR_FEB_1995_Deglitched.zip

  (532 Bytes) Downloaded 5 times

• ECCO_JR_MAR_1995_Deglitched.zip

  (539 Bytes) Downloaded 5 times

Tools and Links:

• gems2rom - software to create "jukebox" ROMS from GEMS data files
• GemsPlay standalone GEMS player for Windows by Valley Bell
• GemsScan by Valley Bell that I had to modify

  GemsScan-mod.zip

  (12.17 KiB) Downloaded 1 time

• Gems_Split from toolkit by r57shell

[Sid Vicious]

Impressive work, I must say. I like the corrected tracks.

I will tell about your work to my friends. Hope, they'll like it.