(Link to AcmlmWiki) Offline: thank ||bass
Register | Login
Views: 13,040,846
 Main | Memberlist | Active users | Calendar | Chat | Online users
Ranks | FAQ | ACS | Stats | Color Chart | Search | Photo album 		05-01-24 12:58 AM
0 users currently in ROM Hacking.
Acmlm's Board - I3 Archive - ROM Hacking - RLE compression for SMK
  
User name:
Password:
Reply:
 
Options: - -
Quik-Attach:
Preview for more options
Max size 1.00 MB, types: png, gif, jpg, txt, zip, rar, tar, gz, 7z, ace, mp3, ogg, mid, ips, bz2, lzh, psd

UserPost
Stifu
Posts: 635/647		 The ones that are like An but with a larger value for n are the F4-F7 ones...
I forgot to make that clear.
HyperHacker
Posts: 4995/5072		 Yeah, I see now they're not quite the same as An (relative vs absolute offset), so they must be special cases.
Stifu
Posts: 634/647		 I did test everything thoroughly (but will do so more later too)...
And I did understand your binary thing in the big lines (I did study binary stuff too :p), but didn't know how I could change/optimize my program using the info you gave me, and just saw it as a simpler way to look at things.

Unless I'm mistaken, the FC-FE commands are NOT based on the other commands before it, unlike the other En and Fn commands... So at the moment I think they're exceptions, like FF, as you said.
HyperHacker
Posts: 4994/5072		 You just need to brush up on your Binary. Since the command byte is actually cccnnnnn, commands "E0-FE" are all one command, #7, which just executes another command with a higher counter value.

Actually, since command #7 uses the high 3 bits of N as a command #, the bytes FC-FE shouldn't even be valid. They'd have the high 3 bits set, executing command #7 again. I'm not exactly sure what this would do... how much have you tested this? Could be a special-case check like for FF.
Stifu
Posts: 633/647		 Hmmm ? You mean the "Reverse-Bit Copy" thing, or something else ? I didn't quite understand that, even re-reading your PM, I can't see something along the lines of what FC-FE do... but I admit some parts kinda confused me.
Whatever, you've still been helpful.

Now to work on the graphic interface of the editor...
HyperHacker
Posts: 4988/5072		 I'd call it something like LZ77, Inverse Bits. Subtracting the value from 0xFF should invert its bits.

BTW, my PM should have explained what FC-FE did.
Stifu
Posts: 632/647		 Alright, I made a bit of progress...

0xAn: 3 bytes, 0xAn P1 P2. Lays n(+1) bytes. Those bytes are calculated this way: (FF - byte found at P2P1 of the decompressed track)

I hope the explanation is clear enough. You just have to substract from FF what's already been decompressed at an earlier location.
If the bytes from the decompressed track are like 00 00 07 22 33, then the 0xAn command will produce FF FF F8 DD CC...

How would you call that command, HH ?

Anyway...

0xF4: 4 bytes, 0xF4 P1 P2 P3. Works like 0xAn, with P1 replacing n.
0xF5: 4 bytes, same as 0xF4, but with P1+256
0xF6: 4 bytes, same as 0xF4, but with P1+512
0xF7: 4 bytes, same as 0xF4, but with P1+768

The only commands we still need to figure out are FC, FD and FE...

Edit: Decompression 100% complete !

0xFC: 3 bytes, FC NN P1. Lays NN(+1) bytes. Byte calculation: FF - byte at (CurrentLocation - P1) of decompressed track
0xFD: 3 bytes, same as 0xFC, but with NN+256
0xFE: 3 bytes, same as 0xFC, but with NN+512
midwife
Posts: 7/8		 quite ( a lot ) of time to test each command ... Stifu made a great Job ! gratz !
GlitchCog
Posts: 29/29		 Awesome. I was just looking through the tracks that are already there to figure it out, so I wouldn't have gotten some that weren't used until I could step back and look at the overall patterns. Thanks.
Stifu
Posts: 625/647
Originally posted by GlitchCog
I guess E5 is just an additional 256 blocks. That's what I've got it set up to do for now, anyhow.

Exactly. I'll just post our updated notes although they aren't complete yet...

Super Mario Kart - Track RLE format:

Px = Parameter x

Commands:

0x0n: X bytes, 0x0n etc. The next n(+1) bytes are not compressed. 02 F0 F4 63 produce F0 F4 63
0x1n: X bytes, same as 0x0n, but with n+16
0x2n: 2 bytes, 0x2n P1. Repeats P1 n(+1) times. 22 F0 produce F0 F0 F0
0x3n: 2 bytes, same as 0x2n, but with n+16
0x4n: 3 bytes, 0x4n P1 P2. Repeats P1 and P2 alternately until n(+1) bytes are laid. 43 22 23 produce 22 23 22 23
0x5n: 3 bytes, same as 0x4n, but with n+16
0x6n: 2 bytes, 0x6n P1. Lays n(+1) bytes, starting from P1 and incrementing it by 1 each time. 62 03 produce 03 04 05
0x7n: 2 bytes, same as 0x6n, but with n+16
0x8n: 3 bytes, 0x8n P1 P2. "Block copy": copies n(+1) bytes from P2P1 location of decompressed track. 80 50 01 means "copy 81 bytes starting from location 0150 of the decompressed track"
0x9n: 3 bytes, same as 0x8n, but with n+16
???0xAn: 2 bytes, 0xAn P1. "Block copy": repeats previous(?) data ranges until n(+1)+?? (?? = multiple de 16?) bytes are laid
0xBn: 2 bytes, same as 0xAn, but with n+16
0xCn: 2 bytes, 0xCn P1. "Block copy": copies n(+1) bytes from (currentLocation - P1) of decompressed track
0xDn: 2 bytes, same as 0xCn, but with n+16


"Super" versions of the commands from 0x0n to 0x7n:
===================================================
0xE0: X bytes, 0xE0 P1 etc. The following P1(+1) bytes are not compressed. E0 01 44 07 produce 44 07
0xE1: X bytes, same as 0xE0, but with P1+256
0xE2: X bytes, same as 0xE0, but with P1+512
0xE3: X bytes, same as 0xE0, but with P1+768
0xE4: 3 bytes, 0xE4 P1 P2. P2 is repeated P1(+1) times. E4 03 88 produce 88 88 88 88
0xE5: 3 bytes, same as 0xE4, but with P1+256
0xE6: 3 bytes, same as 0xE4, but with P1+512
0xE7: 3 bytes, same as 0xE4, but with P1+768
0xE8: 4 bytes, 0xE8 P1 P2 P3. Repeats P2 and P3 alternately until P1(+1) bytes are laid. E8 04 00 01 produce 00 01 00 01 00
0xE9: 4 bytes, same as 0xE8, but with P1+256
0xEA: 4 bytes, same as 0xE8, but with P1+512
0xEB: 4 bytes, same as 0xE8, but with P1+768
0XEC: 3 bytes, 0xEC P1 P2. Lays P1(+1) bytes, starting from P2 and incrementing it by 1 each time. EC 02 49 produce 49 4A 4B
0xED: 3 bytes, same as 0xEC, but with P1+256
0xEE: 3 bytes, same as 0xEC, but with P1+512
0xEF: 3 bytes, same as 0xEC, but with P1+768


"Super" versions of the commands from 0x8n to 0xDn:
===================================================
0xF0: 4 bytes, 0xF0 P1 P2 P3. "Block copy": copies P1(+1) bytes from P3P2 location of decompressed track
0xF1: 4 bytes, same as 0xF0, but with P1+256
0xF2: 4 bytes, same as 0xF0, but with P1+512
0xF3: 4 bytes, same as 0xF0, but with P1+768
0xF4: without any doubt the "Super" version of 0xAn, with P1 replacing n
0xF5: ??? (same as 0xF4, but with P1+256?)
0xF6: ??? (same as 0xF4, but with P1+512?)
0xF7: ??? (same as 0xF4, but with P1+768?)
0xF8: 3 bytes, 0xF8 P1 P2. "Block copy": copies P1(+1) bytes from (currentLocation - P2) of decompressed track
0xF9: 3 bytes, same as 0xF8, but with P1+256
0xFA: 3 bytes, same as 0xF8, but with P1+512
0xFB: 3 bytes, same as 0xF8, but with P1+768
0xFC: ???
0xFD: ???
0xFE: ???
0xFF: 1 byte, end of track
GlitchCog
Posts: 28/29		 Thanks for your help. I figured out what was wrong. Ghost Valley 1 & 3 start with E5 compression. I was just treating it like an E4, so I was coming up two rows short and didn't notice since they were just two full rows of "00" missing from the tops of the tracks. And that's why I had to subtract 1 from the row value of the hex for them.

I guess E5 is just an additional 256 blocks. That's what I've got it set up to do for now, anyhow.

It looks like my 8n code does the same thing as yours, except that mine's all jumbled together and difficult to follow because I'm a terrible, terrible programmer.

case "8":
for ( $b=1; $b<=hexdec(substr($compressed_map,$n+1,1))+1; $b++) {

$decompressed_track = $decompressed_track . substr($decompressed_track,(2*hexdec(dechex(hexdec(substr($compressed_track,$n+4,2))+$temp) . substr($track_map,$n+2,2)))+(($b-1)*2),2);

}
$n = $n + 6;
break;
Stifu
Posts: 624/647		 Nope, no exception for Ghost Valley, or any other track...
In case that helps, here's our C code for 8n, I added a few quick comments to make it clearer... 

    unsigned int i = 0, j = 0, count, command; // "count" = "n" in our notes

    unsigned int lPosition = 0; // that's the position in the decompressed track

    int srcPosition; // that's for block copies



while (lPosition<(16384)) {

          value = buffer[i++];

// (...)

          command = value & 0xF0;

          command = command >> 4;

          count = value & 0x0F;



switch (command) {

// (...)

                 case 0x08 :

                      srcPosition = buffer[i]+buffer[i+1]*256;

                      i += 2;

                      for (j=0;j<=count;j++) {

                          track[lPosition+j] = track[srcPosition+j];

                      }

                      lPosition += j;

                      break;

// (...)

} // switch end

// (...)

} // while end


Let me know if that helps...
GlitchCog
Posts: 27/29		 Ah, I was wondering where the An compression was used since I couldn't find it in any of the existing tracks.

Is there any difference in the 8n compression on Ghost Valley tracks? I seem to need to subtract 1 from the 8n absolute hex address to get it to come out right, but it's only for Ghost Valley tracks.
http://matt.yanos.com/smktrk/?file=tracks/tr_map_gv1.bin&tiles=tiles/gv/&temp=-1 (Works perfectly)
vs.
http://matt.yanos.com/smktrk/?file=tracks/tr_map_gv1.bin&tiles=tiles/gv/&temp=0 (Blocks using 8n messed up)

It's probably a glitch in my code, but I just want to make sure you didn't find anything different about the Ghost Valleys too. Thanks.
Stifu
Posts: 623/647
Originally posted by HyperHacker
Are you finding this by tracing the code?

Nah, I don't know anything about that... I simply check how TrackDes behaves.

What we use to figure out stuff:

-The track decompressed with Lunar Compress, but still RLE (and stuff) compressed.
or the track recompressed with TrackDes (reinserting it into the ROM and decompressing it with Lunar Compress)
-The track totally decompressed (.mkt files generated by TrackDes)
-The track decompressed with our own decompressor, to compare...

The only case where I've found the 0xAn command so far was when I recompressed the Mario Circuit 1 track with TrackDes (which is AE 80 around the bottom of the track, IIRC) [Edit: wrong, that wasn't a 0xAn command...], but there's also the option of modifying the RLE compressed track manually, adding a certain command somewhere to check what's its behavior.
HyperHacker
Posts: 4938/5072		 Are you finding this by tracing the code? I could take a peek at it if I can get Geiger's debugger to work.
Stifu
Posts: 621/647
Originally posted by HyperHacker
Bouche once mentioned the idea of figuring out the maximum possible size a compressed map could be

That's what I thought of, basically... Except in the way I formulated the idea above, I thought I could allow more space than ever needed if I expand the ROM to 1 MB, so I wouldn't have to bother figuring out exactly what's the possible max size.
However, I may want to put other data at the bottom of each track, like their graphics... And maybe object and AI data, if it's also compressed.

I had another idea: somehow managing to pack all the track data together in the ROM, moving stuff around... All tracks being part of the same big pack would make space management easier... The difficulty to do this for me would be to change all the concerned pointers... Also, it may be confusing to users, as some things wouldn't be at the same place anymore (although I could explain what has been moved in a txt file)...

Thanks for having answered my PM, by the way... Command 5 (0xAn) is definitely a "copy command" though, so it's not what you suggested it might be... I'll spend more time on all that later.
HyperHacker
Posts: 4917/5072		 My Pokémon G/S editor handles saving maps in a simple but effective way: Zero out the map data in ROM, find a block of zeros large enough to hold the map, and save it there. This way if your new map is the same size or smaller than the old one, it gets put in the same place. Basically what TrackDes does except wiping out the old data first. (Those maps aren't compressed though.)

Bouche once mentioned the idea of figuring out the maximum possible size a compressed map could be (assuming it's compressed with the editor, not something dumb like repeating "1 byte uncompressed" commands over and over), and allocating each map that much space at the end of the ROM (basically assuming they will reach that size). This would allow you to save them in the same place each time, always using a specific amount of space.
Stifu
Posts: 619/647
Originally posted by Dirtbag
Welcome Midwife! Looks like the SMK scene is starting to pick up

Yeah, huge thanks to Midwife who helped me getting started with track decompression and all... He's a much better coder than me, too.
I've known him for years now, and he always makes the music for all of my game projects... Hopefully Epic Racers will be no exception.

Originally posted by Dirtbag
Stifu: Well done on finding the offsets for the demo macros. When I get some spare time I'm going to relase an update to my Mushroom Cup demo of Super Baldy Kart with the demo edited.

There's something I haven't figured out yet... See, replacing all the demo data with "00" of the first demo out of four (the Mario and Yoshi one) will leave drivers completely still, as you would expect... However, it's not the case of the other three demos. In the other demos, if you remove the data, drivers don't go forward anymore, but they still hop and turn their head from time to time, so there must be some data I haven't found yet...
Let me know what you manage to do. This shouldn't require pure hex hacking, it should be possible to copy data from a ZMV file, but I don't know the details...
Dirtbag
Posts: 98/105
Originally posted by midwife
Hi ! I'm Stifu's friend, helping him on SMK track codec ! let's go !


Welcome Midwife! Looks like the SMK scene is starting to pick up

Stifu: Well done on finding the offsets for the demo macros. When I get some spare time I'm going to relase an update to my Mushroom Cup demo of Super Baldy Kart with the demo edited.
midwife
Posts: 3/8		 Hi ! I'm Stifu's friend, helping him on SMK track codec ! let's go !
This is a long thread. Click here to view it.
Acmlm's Board - I3 Archive - ROM Hacking - RLE compression for SMK


ABII

Acmlmboard 1.92.999, 9/17/2006
©2000-2006 Acmlm, Emuz, Blades, Xkeeper

Page rendered in 0.012 seconds; used 377.16 kB (max 441.46 kB)