(Link to AcmlmWiki) Offline: thank ||bass
Register | Login
Views: 13,040,846
Main | Memberlist | Active users | Calendar | Chat | Online users
Ranks | FAQ | ACS | Stats | Color Chart | Search | Photo album
04-25-24 09:40 PM
0 users currently in ROM Hacking.
Acmlm's Board - I3 Archive - ROM Hacking - RLE compression for SMK New poll | |
Pages: 1 2 3Add to favorites | Next newer thread | Next older thread
User Post
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-01-07 09:50 AM Link | Quote
Originally posted by Alice
So in other words:
3n: Lay down 17+n tiles of the next texture value.

Personally, I'd say 16+(n+1), but that's just me.

Originally posted by Alice
Check you've opened the file in binary mode (fopen(blah, "rb")).

It works, awesome ! Thanks.


(edited by Stifu on 02-01-07 03:56 AM)
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-03-07 09:11 PM Link | Quote
I found out TrackDes does things the way I thought: in a quite dirty way.

First, when you modify a track, it actually leaves the original track data as is, and copies your new track at the bottom of the ROM (once expanded), and changes the concerned track pointer accordingly. Bouche made it this way to avoid compressed size problems...
The thing is, if you open a track and save it 10 times, it'll make 10 copies of it at the bottom of the ROM. It doesn't overwrite anything, but keeps adding things over and over again, until it reaches the ROM size limit. Some people reported that at some point, they couldn't change tracks anymore, TrackDes telling them their ROM is readonly or something... I'm pretty sure they quite simply reached the ROM size limit.

I'll try to do things in a much cleaner way, I have a few ideas...
GlitchCog

Goomba


 





Since: 01-31-06

Last post: 6287 days
Last view: 6287 days
Posted on 02-03-07 11:44 PM Link | Quote
That is pretty sloppy.

You could definately design tracks that would end up smaller than the originals. Especially the complex ones. If you turned a Donut Plains sized track into a Ghost Valley-like track where the off road area was all the same tile, it could be a lot smaller than all the bushes and lake tiles mixed together. And then there's a bunch of compression scemes that are only used a few times. If you designed your tracks specifically with the available compression in mind, you could probably come up with a really complicated track that doesn't take up space.

Did you figure out the 8n compression fully? The X part works for me. Like how when it's more than a full row's worth of tiles, it just skips to the next row, and I wind up in the right column, but I'm lost as to how the Y component works.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-04-07 12:05 AM Link | Quote
Yeah, I thought about making sure the newly compressed tracks fit right into the ROM, by messing with pointers, moving big tracks where there is a lot of space, and small ones in small spaces... You get the idea. Or even redefining the space attributed to each track, when possible (basically, when tracks are packed together), changing pointers accordingly... That'd probably be the cleanest way, although the most bothersome one. And I think it'd be difficult to implement in an automated editor, too...

My other idea was to expand the ROM like Bouche did (not to 2 MB though, 1 MB is enough), copying the new track data at the bottom of the ROM, but defining clear areas for each of the 24 tracks... so that each track has its own space, so no garbage would add up each time you edit one, unlike in TrackDes.

And yeah, we figured out the 8n compression fully. The 2 parameters following the 8n command actually form an absolute hex address. Example: 80 50 01 means "copy 81 bytes starting from location 0150 of the decompressed track".

We're almost finished figuring out all the compression commands, I'll post our updated notes (reformulated and clearer) once we're done... Command 0xAn is basically the last command giving us problems... It's another block copy command, but it's quite weird (and rarely used)...


(edited by Stifu on 02-03-07 06:19 PM)
midwife
Newcomer








Since: 02-04-07
From: Dijon / FRANCE

Last post: 6285 days
Last view: 6276 days
Posted on 02-04-07 12:32 PM Link | Quote
Hi ! I'm Stifu's friend, helping him on SMK track codec ! let's go !


(edited by midwife on 02-04-07 06:38 AM)
Dirtbag

Koopa








Since: 09-20-06
From: UK

Last post: 6279 days
Last view: 6275 days
Posted on 02-04-07 09:24 PM Link | Quote
Originally posted by midwife
Hi ! I'm Stifu's friend, helping him on SMK track codec ! let's go !


Welcome Midwife! Looks like the SMK scene is starting to pick up

Stifu: Well done on finding the offsets for the demo macros. When I get some spare time I'm going to relase an update to my Mushroom Cup demo of Super Baldy Kart with the demo edited.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-04-07 09:33 PM Link | Quote
Originally posted by Dirtbag
Welcome Midwife! Looks like the SMK scene is starting to pick up

Yeah, huge thanks to Midwife who helped me getting started with track decompression and all... He's a much better coder than me, too.
I've known him for years now, and he always makes the music for all of my game projects... Hopefully Epic Racers will be no exception.

Originally posted by Dirtbag
Stifu: Well done on finding the offsets for the demo macros. When I get some spare time I'm going to relase an update to my Mushroom Cup demo of Super Baldy Kart with the demo edited.

There's something I haven't figured out yet... See, replacing all the demo data with "00" of the first demo out of four (the Mario and Yoshi one) will leave drivers completely still, as you would expect... However, it's not the case of the other three demos. In the other demos, if you remove the data, drivers don't go forward anymore, but they still hop and turn their head from time to time, so there must be some data I haven't found yet...
Let me know what you manage to do. This shouldn't require pure hex hacking, it should be possible to copy data from a ZMV file, but I don't know the details...
HyperHacker

Star Mario
Finally being paid to code in VB! If only I still enjoyed that. <_<
Wii #7182 6487 4198 1828


 





Since: 11-18-05
From: Canada, w00t!
My computer's specs, if anyone gives a damn.
STOP TRUNCATING THIS >8^(

Last post: 6276 days
Last view: 6276 days
Posted on 02-06-07 02:03 AM Link | Quote
My Pokémon G/S editor handles saving maps in a simple but effective way: Zero out the map data in ROM, find a block of zeros large enough to hold the map, and save it there. This way if your new map is the same size or smaller than the old one, it gets put in the same place. Basically what TrackDes does except wiping out the old data first. (Those maps aren't compressed though.)

Bouche once mentioned the idea of figuring out the maximum possible size a compressed map could be (assuming it's compressed with the editor, not something dumb like repeating "1 byte uncompressed" commands over and over), and allocating each map that much space at the end of the ROM (basically assuming they will reach that size). This would allow you to save them in the same place each time, always using a specific amount of space.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-06-07 08:37 AM Link | Quote
Originally posted by HyperHacker
Bouche once mentioned the idea of figuring out the maximum possible size a compressed map could be

That's what I thought of, basically... Except in the way I formulated the idea above, I thought I could allow more space than ever needed if I expand the ROM to 1 MB, so I wouldn't have to bother figuring out exactly what's the possible max size.
However, I may want to put other data at the bottom of each track, like their graphics... And maybe object and AI data, if it's also compressed.

I had another idea: somehow managing to pack all the track data together in the ROM, moving stuff around... All tracks being part of the same big pack would make space management easier... The difficulty to do this for me would be to change all the concerned pointers... Also, it may be confusing to users, as some things wouldn't be at the same place anymore (although I could explain what has been moved in a txt file)...

Thanks for having answered my PM, by the way... Command 5 (0xAn) is definitely a "copy command" though, so it's not what you suggested it might be... I'll spend more time on all that later.
HyperHacker

Star Mario
Finally being paid to code in VB! If only I still enjoyed that. <_<
Wii #7182 6487 4198 1828


 





Since: 11-18-05
From: Canada, w00t!
My computer's specs, if anyone gives a damn.
STOP TRUNCATING THIS >8^(

Last post: 6276 days
Last view: 6276 days
Posted on 02-06-07 08:48 AM Link | Quote
Are you finding this by tracing the code? I could take a peek at it if I can get Geiger's debugger to work.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-06-07 08:57 AM Link | Quote
Originally posted by HyperHacker
Are you finding this by tracing the code?

Nah, I don't know anything about that... I simply check how TrackDes behaves.

What we use to figure out stuff:

-The track decompressed with Lunar Compress, but still RLE (and stuff) compressed.
or the track recompressed with TrackDes (reinserting it into the ROM and decompressing it with Lunar Compress)
-The track totally decompressed (.mkt files generated by TrackDes)
-The track decompressed with our own decompressor, to compare...

The only case where I've found the 0xAn command so far was when I recompressed the Mario Circuit 1 track with TrackDes (which is AE 80 around the bottom of the track, IIRC) [Edit: wrong, that wasn't a 0xAn command...], but there's also the option of modifying the RLE compressed track manually, adding a certain command somewhere to check what's its behavior.


(edited by Stifu on 02-10-07 04:49 AM)
GlitchCog

Goomba


 





Since: 01-31-06

Last post: 6287 days
Last view: 6287 days
Posted on 02-06-07 02:41 PM Link | Quote
Ah, I was wondering where the An compression was used since I couldn't find it in any of the existing tracks.

Is there any difference in the 8n compression on Ghost Valley tracks? I seem to need to subtract 1 from the 8n absolute hex address to get it to come out right, but it's only for Ghost Valley tracks.
http://matt.yanos.com/smktrk/?file=tracks/tr_map_gv1.bin&tiles=tiles/gv/&temp=-1 (Works perfectly)
vs.
http://matt.yanos.com/smktrk/?file=tracks/tr_map_gv1.bin&tiles=tiles/gv/&temp=0 (Blocks using 8n messed up)

It's probably a glitch in my code, but I just want to make sure you didn't find anything different about the Ghost Valleys too. Thanks.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-06-07 03:25 PM Link | Quote
Nope, no exception for Ghost Valley, or any other track...
In case that helps, here's our C code for 8n, I added a few quick comments to make it clearer...

    unsigned int i = 0, j = 0, count, command; // "count" = "n" in our notes

unsigned int lPosition = 0; // that's the position in the decompressed track
int srcPosition; // that's for block copies

while (lPosition<(16384)) {
value = buffer[i++];
// (...)
command = value & 0xF0;
command = command >> 4;
count = value & 0x0F;

switch (command) {
// (...)
case 0x08 :
srcPosition = buffer[i]+buffer[i+1]*256;
i += 2;
for (j=0;j<=count;j++) {
track[lPosition+j] = track[srcPosition+j];
}
lPosition += j;
break;
// (...)
} // switch end
// (...)
} // while end


Let me know if that helps...


(edited by Stifu on 02-06-07 09:27 AM)
GlitchCog

Goomba


 





Since: 01-31-06

Last post: 6287 days
Last view: 6287 days
Posted on 02-06-07 06:50 PM Link | Quote
Thanks for your help. I figured out what was wrong. Ghost Valley 1 & 3 start with E5 compression. I was just treating it like an E4, so I was coming up two rows short and didn't notice since they were just two full rows of "00" missing from the tops of the tracks. And that's why I had to subtract 1 from the row value of the hex for them.

I guess E5 is just an additional 256 blocks. That's what I've got it set up to do for now, anyhow.

It looks like my 8n code does the same thing as yours, except that mine's all jumbled together and difficult to follow because I'm a terrible, terrible programmer.

case "8":
for ( $b=1; $b<=hexdec(substr($compressed_map,$n+1,1))+1; $b++) {

$decompressed_track = $decompressed_track . substr($decompressed_track,(2*hexdec(dechex(hexdec(substr($compressed_track,$n+4,2))+$temp) . substr($track_map,$n+2,2)))+(($b-1)*2),2);

}
$n = $n + 6;
break;
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-06-07 07:02 PM Link | Quote
Originally posted by GlitchCog
I guess E5 is just an additional 256 blocks. That's what I've got it set up to do for now, anyhow.

Exactly. I'll just post our updated notes although they aren't complete yet...


Super Mario Kart - Track RLE format:

Px = Parameter x

Commands:

0x0n: X bytes, 0x0n etc. The next n(+1) bytes are not compressed. 02 F0 F4 63 produce F0 F4 63
0x1n: X bytes, same as 0x0n, but with n+16
0x2n: 2 bytes, 0x2n P1. Repeats P1 n(+1) times. 22 F0 produce F0 F0 F0
0x3n: 2 bytes, same as 0x2n, but with n+16
0x4n: 3 bytes, 0x4n P1 P2. Repeats P1 and P2 alternately until n(+1) bytes are laid. 43 22 23 produce 22 23 22 23
0x5n: 3 bytes, same as 0x4n, but with n+16
0x6n: 2 bytes, 0x6n P1. Lays n(+1) bytes, starting from P1 and incrementing it by 1 each time. 62 03 produce 03 04 05
0x7n: 2 bytes, same as 0x6n, but with n+16
0x8n: 3 bytes, 0x8n P1 P2. "Block copy": copies n(+1) bytes from P2P1 location of decompressed track. 80 50 01 means "copy 81 bytes starting from location 0150 of the decompressed track"
0x9n: 3 bytes, same as 0x8n, but with n+16
???0xAn: 2 bytes, 0xAn P1. "Block copy": repeats previous(?) data ranges until n(+1)+?? (?? = multiple de 16?) bytes are laid
0xBn: 2 bytes, same as 0xAn, but with n+16
0xCn: 2 bytes, 0xCn P1. "Block copy": copies n(+1) bytes from (currentLocation - P1) of decompressed track
0xDn: 2 bytes, same as 0xCn, but with n+16


"Super" versions of the commands from 0x0n to 0x7n:
===================================================
0xE0: X bytes, 0xE0 P1 etc. The following P1(+1) bytes are not compressed. E0 01 44 07 produce 44 07
0xE1: X bytes, same as 0xE0, but with P1+256
0xE2: X bytes, same as 0xE0, but with P1+512
0xE3: X bytes, same as 0xE0, but with P1+768
0xE4: 3 bytes, 0xE4 P1 P2. P2 is repeated P1(+1) times. E4 03 88 produce 88 88 88 88
0xE5: 3 bytes, same as 0xE4, but with P1+256
0xE6: 3 bytes, same as 0xE4, but with P1+512
0xE7: 3 bytes, same as 0xE4, but with P1+768
0xE8: 4 bytes, 0xE8 P1 P2 P3. Repeats P2 and P3 alternately until P1(+1) bytes are laid. E8 04 00 01 produce 00 01 00 01 00
0xE9: 4 bytes, same as 0xE8, but with P1+256
0xEA: 4 bytes, same as 0xE8, but with P1+512
0xEB: 4 bytes, same as 0xE8, but with P1+768
0XEC: 3 bytes, 0xEC P1 P2. Lays P1(+1) bytes, starting from P2 and incrementing it by 1 each time. EC 02 49 produce 49 4A 4B
0xED: 3 bytes, same as 0xEC, but with P1+256
0xEE: 3 bytes, same as 0xEC, but with P1+512
0xEF: 3 bytes, same as 0xEC, but with P1+768


"Super" versions of the commands from 0x8n to 0xDn:
===================================================
0xF0: 4 bytes, 0xF0 P1 P2 P3. "Block copy": copies P1(+1) bytes from P3P2 location of decompressed track
0xF1: 4 bytes, same as 0xF0, but with P1+256
0xF2: 4 bytes, same as 0xF0, but with P1+512
0xF3: 4 bytes, same as 0xF0, but with P1+768
0xF4: without any doubt the "Super" version of 0xAn, with P1 replacing n
0xF5: ??? (same as 0xF4, but with P1+256?)
0xF6: ??? (same as 0xF4, but with P1+512?)
0xF7: ??? (same as 0xF4, but with P1+768?)
0xF8: 3 bytes, 0xF8 P1 P2. "Block copy": copies P1(+1) bytes from (currentLocation - P2) of decompressed track
0xF9: 3 bytes, same as 0xF8, but with P1+256
0xFA: 3 bytes, same as 0xF8, but with P1+512
0xFB: 3 bytes, same as 0xF8, but with P1+768
0xFC: ???
0xFD: ???
0xFE: ???
0xFF: 1 byte, end of track



(edited by Stifu on 02-10-07 04:48 AM)
GlitchCog

Goomba


 





Since: 01-31-06

Last post: 6287 days
Last view: 6287 days
Posted on 02-06-07 10:44 PM Link | Quote
Awesome. I was just looking through the tracks that are already there to figure it out, so I wouldn't have gotten some that weren't used until I could step back and look at the overall patterns. Thanks.
midwife
Newcomer








Since: 02-04-07
From: Dijon / FRANCE

Last post: 6285 days
Last view: 6276 days
Posted on 02-07-07 02:12 AM Link | Quote
quite ( a lot ) of time to test each command ... Stifu made a great Job ! gratz !
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-10-07 07:10 PM Link | Quote
Alright, I made a bit of progress...

0xAn: 3 bytes, 0xAn P1 P2. Lays n(+1) bytes. Those bytes are calculated this way: (FF - byte found at P2P1 of the decompressed track)

I hope the explanation is clear enough. You just have to substract from FF what's already been decompressed at an earlier location.
If the bytes from the decompressed track are like 00 00 07 22 33, then the 0xAn command will produce FF FF F8 DD CC...

How would you call that command, HH ?

Anyway...

0xF4: 4 bytes, 0xF4 P1 P2 P3. Works like 0xAn, with P1 replacing n.
0xF5: 4 bytes, same as 0xF4, but with P1+256
0xF6: 4 bytes, same as 0xF4, but with P1+512
0xF7: 4 bytes, same as 0xF4, but with P1+768

The only commands we still need to figure out are FC, FD and FE...

Edit: Decompression 100% complete !

0xFC: 3 bytes, FC NN P1. Lays NN(+1) bytes. Byte calculation: FF - byte at (CurrentLocation - P1) of decompressed track
0xFD: 3 bytes, same as 0xFC, but with NN+256
0xFE: 3 bytes, same as 0xFC, but with NN+512


(edited by Stifu on 02-10-07 05:07 PM)
(edited by Stifu on 02-10-07 06:30 PM)
HyperHacker

Star Mario
Finally being paid to code in VB! If only I still enjoyed that. <_<
Wii #7182 6487 4198 1828


 





Since: 11-18-05
From: Canada, w00t!
My computer's specs, if anyone gives a damn.
STOP TRUNCATING THIS >8^(

Last post: 6276 days
Last view: 6276 days
Posted on 02-11-07 09:22 AM Link | Quote
I'd call it something like LZ77, Inverse Bits. Subtracting the value from 0xFF should invert its bits.

BTW, my PM should have explained what FC-FE did.
Stifu









Since: 11-18-05
From: Your mom's bed

Last post: 6277 days
Last view: 6275 days
Posted on 02-11-07 09:46 AM Link | Quote
Hmmm ? You mean the "Reverse-Bit Copy" thing, or something else ? I didn't quite understand that, even re-reading your PM, I can't see something along the lines of what FC-FE do... but I admit some parts kinda confused me.
Whatever, you've still been helpful.

Now to work on the graphic interface of the editor...


(edited by Stifu on 02-11-07 03:47 AM)
Pages: 1 2 3Add to favorites | Next newer thread | Next older thread
Acmlm's Board - I3 Archive - ROM Hacking - RLE compression for SMK |


ABII

Acmlmboard 1.92.999, 9/17/2006
©2000-2006 Acmlm, Emuz, Blades, Xkeeper

Page rendered in 0.023 seconds; used 461.52 kB (max 594.21 kB)