(Link to AcmlmWiki) Offline: thank ||bass
Register | Login
Views: 13,040,846
Main | Memberlist | Active users | Calendar | Chat | Online users
Ranks | FAQ | ACS | Stats | Color Chart | Search | Photo album
05-07-24 07:42 PM
0 users currently in ROM Hacking.
Acmlm's Board - I3 Archive - ROM Hacking - Another ROM hacking idea New poll | |
Pages: 1 2Add to favorites | Next newer thread | Next older thread
User Post
Legendre

Micro-Goomba


 





Since: 04-30-06

Last post: 6572 days
Last view: 6572 days
Posted on 05-01-06 07:49 PM Link | Quote
Hello again everybody,

How difficult would it be, assuming all unused RAM in a game were pinned down, to hack in a rudimentary memory allocation routine? Then SMW hackers, for instance, could agree on a specific block of unused RAM in SMW to hold a pointer table, and would jump to the memory allocation routine to populate the pointer table with pointers to however much RAM they need. This would be pretty inefficient RAMwise since the allocation routine itself would have to use extra amounts of RAM to store a linked list of blocks which have been allocated. However it would eliminate some of the problems discussed in this thread.

I imagine such a routine would be pretty difficult in raw ASM, especially since the free RAM is not nicely organized in one big chunk. But, just thought I'd share the idea... you guys are awesome
Disch

Red Cheep-cheep


 





Since: 12-10-05

Last post: 6567 days
Last view: 6567 days
Posted on 05-01-06 09:25 PM Link | Quote
I suppose something like this is possible, but as you pointed out, it's not very efficient with RAM. I don't know many details on the SNES -- but in the case of the NES something just isn't practical since RAM is extremely limited (only 2k available on the system -- though some mappers provide an additional 8k... MMC5 being a special case exception which can provide up to 64k).

Getting continuous blocks is another issue. Say you'd want to allocate 1k RAM for something -- and let's say there's even 10k available. There might not be a continuous block of 1k which you could allocate.

Not to mention having dynamic RAM means you'd have to access everything through a pointer, which means slightly slower code (again, probably not as big of an issue on the SNES as it would be on the NES).

And of course I have to ask myself "why". You said this would eliminate some problems discussed in a thread -- but I must have missed that thread. What problems were you referring to? And how would this approach help solve them?

The thing about these older consoles is -- the RAM is usually directly accessable in real-time (no cache levels, no abstraction... just 1 address = 1 variable), and it's a fixed, constant size. So doing something like dynamic RAM allocation (which is a great tool on a modern system where there are multiple cache levels, and where RAM is abstracted and a variable size -- and even where it's virtualized) doesn't really provide many benefits, and is just an extra chore. On these older systems you can just designate that X variable/array uses the RAM at X address -- no further abstraction required.


I don't mean to discourage you or anything. I know it probably seems like I'm always trying to shoot down your ideas but that's not my intent, really. ^^ It's good to get someone in here like you with a good head on their shoulders. I don't see much of that any more.
HyperHacker

Star Mario
Finally being paid to code in VB! If only I still enjoyed that. <_<
Wii #7182 6487 4198 1828


 





Since: 11-18-05
From: Canada, w00t!
My computer's specs, if anyone gives a damn.
STOP TRUNCATING THIS >8^(

Last post: 6288 days
Last view: 6288 days
Posted on 05-02-06 02:14 AM Link | Quote
He has the right idea but he's going about it wrong. A memory allocation system would really be useful for SMW especially, since a lot of hacks share addresses causing nasty results. However, the solution isn't trying to fit a memory allocator in the small, fragmented free memory in your average SNES game. What we need is a page in which people submit the disassembly of their ASM hacks (or raw binary for those who code directly in hex), and any addresses they use are added to a list. Other hackers can then check that their own hacks won't use these addresses if they want to be compatible with the existing hack. You could even do similar with ROM addresses, since many hacks require a few bytes in a certain bank.
Cellar Dweller +

Red Koopa









Since: 11-18-05
From: Arkansas

Last post: 6297 days
Last view: 6288 days
Posted on 05-02-06 03:44 PM Link | Quote
What is really needed is a linker. A database of symbols for the ROM would need to be created. Then, a linker could relocate the ASM hacks, assign RAM, and patch the hacks and existing game code to resolve the symbols. ASM hacks would be distributed as symbolic object files.

I really don't follow SNES stuff that much, so I don't know much about SNES development toolchains. I don't know what, if any, object file formats are already used that could be used for such a scheme, or what, if any, linker software is available that could be used, with or without modification.
Legendre

Micro-Goomba


 





Since: 04-30-06

Last post: 6572 days
Last view: 6572 days
Posted on 05-02-06 04:18 PM Link | Quote
Ah, excellent points raised all around.

Hmm, on SNES lots of games use save files... and some don't, or (in the case of SMW) the savefiles can't conceivably consist of much more than a few stats and a checklist of levels completed. Would it be possible, in games like this, to actually use the savefile space as a sort of virtual RAM? I imagine this would raise speed issues on a physical machine (would those evaporate on an emu?)

Thanks for the kind words Disch, and btw, sorry about the confusing remark about "this thread", I meant to post the message as a reply to the thread where people were talking about exactly the problem HyperMackerel described.
Cellar Dweller +

Red Koopa









Since: 11-18-05
From: Arkansas

Last post: 6297 days
Last view: 6288 days
Posted on 05-02-06 05:22 PM Link | Quote
On the SNES(and NES, but not the N64) save memory is normal RAM at a different address that is prevented from being erased when the power is turned off by a battery. It runs at the same speed as the main RAM. In fact games with a Super FX chip use the save RAM to pass information between the FX chip and the main CPU, in which hundereds or thousands of bytes are read from or written to save RAM each frame.

I see no way to use virtual memory on the SNES. The first Nintendo system that had any support for virtual memory was the N64. Even then, there was nothing suitable to use for swap.
Disch

Red Cheep-cheep


 





Since: 12-10-05

Last post: 6567 days
Last view: 6567 days
Posted on 05-02-06 05:34 PM Link | Quote
Linking while working around an already assembled product would be kind of a pain. I'd say if there's such a demand for it -- you'd think it's about time someone sat down and made a re-assembleable disassembly of SMW. This way instead of distributing patches for asm hacks, people could distribute source for them -- allowing people to combine several hacks, giving each their own unique RAM space and what not. It would also make creating those hacks much easier.

Game specific disassembly isn't as hard as people make it out to be. I've done one in the past with Final Fantasy 1 -- and there are several other NES disassemblies out there. It's generic "work on any ROM" disassemblers that are hard. If someone knowledgeable in SMW sat down and dedicated some time to it, I really think something like that could become a reality.


Would it be possible, in games like this, to actually use the savefile space as a sort of virtual RAM? I imagine this would raise speed issues on a physical machine (would those evaporate on an emu?)


It wouldn't be virtual. Saved games on NES/SNES are stored on actual, real RAM, backed with a battery so it maintains its data after power-off. There wouldn't be any speed issues either.

Many games already do this (especially on the NES. Final Fantasy has 8k of battery-backed RAM on the cartridge, only about 2k of which is actually save game stuff -- the rest it just uses as more work RAM since the NES provides so little).
Legendre

Micro-Goomba


 





Since: 04-30-06

Last post: 6572 days
Last view: 6572 days
Posted on 05-02-06 06:26 PM Link | Quote
Ah, very illuminating. I think this sort of thing should be studied by all serious coders just because it really opens your eyes to things you take for granted. I'm very impressed, for example, by these tales of desperate heroic acts of RAM usage by NES developers.

Well, continuing to use SMW as an example. Assuming you are willing to sacrifice two of the three save slots, you could disable them (so the game just has one save slot) and bang, lots of RAM freed up.

Do games with savefiles usually use compression of any sort when saving the game? Heh, wouldn't that be novel, incorporate 7z algorithms into games designed over a decade before 7z

Thanks again for all the great responses you guys give a newb like me!
Disch

Red Cheep-cheep


 





Since: 12-10-05

Last post: 6567 days
Last view: 6567 days
Posted on 05-02-06 07:18 PM Link | Quote
Originally posted by Legendre
I'm very impressed, for example, by these tales of desperate heroic acts of RAM usage by NES developers.


Heh... speaking of NES horror stories. Sit down and play Battletoads again sometime. And while you're playing... keep in mind that the NES only has 1 background (only 1 layer, and 64 sprites, no multiple layers or anything), runs at a wimpy 1.79 MHz, and has only 2k of RAM... about 300 bytes of which are more-or-less reserved for Sprite information and stack space (also note that Battletoads has no on-cartridge RAM -- so it does everything with just that 2k).

Yet Battletoads manages to pull off full paralax background effects, PCM audio streaming (for some of the sound effects), high scrolling speeds, two player coop, and many simultaeous on-screen sprites... all with no noticable slowdown (at least, I don't remember the game ever slowing down).

</offtopic>


Well, continuing to use SMW as an example. Assuming you are willing to sacrifice two of the three save slots, you could disable them (so the game just has one save slot) and bang, lots of RAM freed up.


Probably not as much as you'd think. How much does SMW actually save anyway? Player position on the map (3 or 4 bytes -- double that if it saves luigi's position too), remaining lives (1 or 2 bytes -- if it even saves this at all, it's been a while), and levels which have been completed (96 paths, right? probably bit-packed, so 96/8 = 12 bytes). Plus maybe the powerups/yoshi you have (maybe 2 or 3 bytes).

So clearing out 2 save files will probably only get you about 40 bytes of free RAM. Possibly 64 if it's wasteful.


Do games with savefiles usually use compression of any sort when saving the game? Heh, wouldn't that be novel, incorporate 7z algorithms into games designed over a decade before 7z


I doubt many games bother compressing the save RAM. Though most/all of them do some sort of CRC or checksum to make sure the data hasn't gotten corrupted.
Dragonsbrethren

440








Since: 12-01-05
From: New Jersey

Last post: 6475 days
Last view: 6475 days
Posted on 05-02-06 07:36 PM Link | Quote
Originally posted by Disch
Heh... speaking of NES horror stories. Sit down and play Battletoads again sometime. And while you're playing... keep in mind that the NES only has 1 background (only 1 layer, and 64 sprites, no multiple layers or anything), runs at a wimpy 1.79 MHz, and has only 2k of RAM... about 300 bytes of which are more-or-less reserved for Sprite information and stack space (also note that Battletoads has no on-cartridge RAM -- so it does everything with just that 2k).

Yet Battletoads manages to pull off full paralax background effects, PCM audio streaming (for some of the sound effects), high scrolling speeds, two player coop, and many simultaeous on-screen sprites... all with no noticable slowdown (at least, I don't remember the game ever slowing down).


They sure don't make 'em like they use'd to, do they? Speaking of which, Battletoads was fairly early in the NES's life, wasn't it? I wish developers would push consoles to their limits like that nowadays...
Mediocre Ibex?

Micro-Goomba








Since: 11-24-05
From: Illinois

Last post: 6336 days
Last view: 6336 days
Posted on 05-02-06 09:27 PM Link | Quote
1991, so late in its life.
HyperHacker

Star Mario
Finally being paid to code in VB! If only I still enjoyed that. <_<
Wii #7182 6487 4198 1828


 





Since: 11-18-05
From: Canada, w00t!
My computer's specs, if anyone gives a damn.
STOP TRUNCATING THIS >8^(

Last post: 6288 days
Last view: 6288 days
Posted on 05-03-06 03:13 AM Link | Quote
Originally posted by Disch
Probably not as much as you'd think. How much does SMW actually save anyway? Player position on the map (3 or 4 bytes -- double that if it saves luigi's position too), remaining lives (1 or 2 bytes -- if it even saves this at all, it's been a while), and levels which have been completed (96 paths, right? probably bit-packed, so 96/8 = 12 bytes). Plus maybe the powerups/yoshi you have (maybe 2 or 3 bytes).

SMW doesn't save your powerups, Yoshi, or lives. I believe it decompresses the overworld to SRAM too, so that events can simply modify it in memory as they run instead of having to re-apply them all each time (also providing a convenient way to save the passed event status).

BTW, some NES games such as Super Mario Bros 3 had on-cartridge RAM that wasn't battery-backed, just to use as work RAM.
BMF54123
WARNING: MOOD LEVEL CRITICAL








Since: 11-18-05
From: MOOGLES

Last post: 6288 days
Last view: 6287 days
Posted on 05-03-06 05:41 AM Link | Quote
Originally posted by Disch
Yet Battletoads manages to pull off full paralax background effects, PCM audio streaming (for some of the sound effects), high scrolling speeds, two player coop, and many simultaeous on-screen sprites... all with no noticable slowdown (at least, I don't remember the game ever slowing down).
It was also the only (?) NES game to feature real-time sprite zooming. No kidding. The white ship in the intro and the spinning 'Toad symbols on the title screen are both scaled in RAM and then copied to VRAM. In fact, a lot of effects in the game (like the vertically-scrolling cavern walls) were achieved by manipulating graphics in RAM.

On top of all that, nearly all the graphics are compressed, yet there's virtually no loading time between levels, or when new enemies appear in a level. Overall, it's a truly amazing (if horribly unbalanced and painfully difficult) game.

(okay, I'm done further derailing the topic...)
Glyphodon



 





Since: 11-18-05

Last post: 6328 days
Last view: 6309 days
Posted on 05-03-06 08:16 AM Link | Quote
Hyperhacker, I believe you're thinking of RAM bank 7F. Why would Super Mario's monstrocity of an overworld be loaded into limited SRAM? If the overworld is loaded into SRAM, why would it have to prompt players to save their game? It wouldn't make any sense.

I've done quite a bit of consideration and pages worth of post I won't include because of irrelevance, and I'd say doing any sort of memory allocation with an already assembled ROM just isn't a good idea. Here's why:


  • Disch was right; dynamic ram allocation in the ROM wouldn't solve any problems. I think it would create quite a few. Vying for RAM will still occur within the allocation table; who decides which code gets what part of the table? If a pointer to a spot in RAM is retrieved, that pointer must be saved in RAM in order to be used again, defeating the purpose of a table in the first place.

  • Saving a table of RAM addresses used uses up space in ROM or requires a separate file and has no guarantee everything will be listed and thus the system has a good chance of breaking.

  • Scouring a ROM for opcodes that use RAM requires a lot of parsing time and a compliant library listing all the possible calls to RAM that can be made using a processor. Due to the nature of some dynamic opcodes, not even after parsing the ROM could an accurate list be obtained through this method.



  • The most practical course of action is to make a hack that uses any memory it wants and to modify the patch as problems show up. Basically, the best change we should make to our current memory practices is no change.
    Legendre

    Micro-Goomba


     





    Since: 04-30-06

    Last post: 6572 days
    Last view: 6572 days
    Posted on 05-03-06 09:43 AM Link | Quote
    Originally posted by Disch
    It's generic "work on any ROM" disassemblers that are hard.

    Assuming by "any ROM" you mean "any ROM on a fixed system", why is a generic disassembler so hard?

    In my naive and newbieish mind it seems the hardest thing would be mapping the ROM, dividing it into code and data. Is this anywhere near the truth?
    MathOnNapkins

    1100

    In SPC700 HELL


     





    Since: 11-18-05

    Last post: 6288 days
    Last view: 6287 days
    Posted on 05-03-06 10:43 AM Link | Quote
    Mapping the rom into code and data is easy. Many debuggers already do it. It's knowing how it all works together so that it is reassembled that is hard. It's mostly proportional to the amount of indirection your processor is capable of. The more you can hide your program's execution through memory, the harder it is to know the exact structure of the program.
    HyperHacker

    Star Mario
    Finally being paid to code in VB! If only I still enjoyed that. <_<
    Wii #7182 6487 4198 1828


     





    Since: 11-18-05
    From: Canada, w00t!
    My computer's specs, if anyone gives a damn.
    STOP TRUNCATING THIS >8^(

    Last post: 6288 days
    Last view: 6288 days
    Posted on 05-03-06 11:06 PM Link | Quote
    Originally posted by Glyph Phoenix
    Hyperhacker, I believe you're thinking of RAM bank 7F. Why would Super Mario's monstrocity of an overworld be loaded into limited SRAM? If the overworld is loaded into SRAM, why would it have to prompt players to save their game? It wouldn't make any sense.

    You know when you start a new game, you do that little intro level with the "Bowser kidnapped Peach again!" message box before you actually get to the overworld? That's when the game is decompressing the overworld to memory. It takes a while, and to do this every time you start the game would be a pain. Then you'd have to step through all the passed events again, updating the map in memory. By keeping it in SRAM you avoid this. The save prompt is just to offer a choice. Saving the events passed and player position only takes a few bytes. This is also why you can't edit the overworld in LM, then load an existing save file.


    The most practical course of action is to make a hack that uses any memory it wants and to modify the patch as problems show up. Basically, the best change we should make to our current memory practices is no change.

    The problems with this should be obvious already. There are many hacks that use the same RAM/ROM addresses and thus aren't compatible with eachother. In many cases they don't get fixed, because of one or more of the following:
    1) The author didn't know of the problem and/or doesn't care.
    2) The author stopped hacking SMW ages ago, lost the source code, etc etc.
    3) It'd take a lot of work to use different addresses (especially ROM) and nobody's up to it. Especially in cases where other hacks, custom blocks etc interact with these hacks, and to change addresses would make them not work anymore.
    4) Both people think the other should be the one to change their hack.
    Legendre

    Micro-Goomba


     





    Since: 04-30-06

    Last post: 6572 days
    Last view: 6572 days
    Posted on 05-04-06 03:12 AM Link | Quote
    Originally posted by HyperMackerel

    You know when you start a new game, you do that little intro level with the "Bowser kidnapped Peach again!" message box before you actually get to the overworld? That's when the game is decompressing the overworld to memory.

    Hey HyperMackerel Thank you for posting this, I found this really interesting. I love this board to death, new eye openers every day

    The reason I'm replying is to ask: is there a similar explanation for the awkward pauses when you stomp on the big switches in the colored block switch palaces? I always wondered what was up with that!
    HyperHacker

    Star Mario
    Finally being paid to code in VB! If only I still enjoyed that. <_<
    Wii #7182 6487 4198 1828


     





    Since: 11-18-05
    From: Canada, w00t!
    My computer's specs, if anyone gives a damn.
    STOP TRUNCATING THIS >8^(

    Last post: 6288 days
    Last view: 6288 days
    Posted on 05-04-06 03:20 AM Link | Quote
    Not that I know of, but I wouldn't be too surprised. A lot of things in Nintendo games, especially on NES and SNES, are just to mask load time. The "Mario Start" screen hides the time it takes to render the level. The level name screen does the same on Yoshi's Island, as does the lives screen in Super Mario Bros. They're pretty clever when it comes to making things look faster than they really are.
    Glyphodon



     





    Since: 11-18-05

    Last post: 6328 days
    Last view: 6309 days
    Posted on 05-04-06 09:26 AM Link | Quote
    Originally posted by HyperMackerel
    You know when you start a new game, you do that little intro level with the "Bowser kidnapped Peach again!" message box before you actually get to the overworld? That's when the game is decompressing the overworld to memory. It takes a while, and to do this every time you start the game would be a pain. Then you'd have to step through all the passed events again, updating the map in memory. By keeping it in SRAM you avoid this. The save prompt is just to offer a choice. Saving the events passed and player position only takes a few bytes. This is also why you can't edit the overworld in LM, then load an existing save file.


    The reason you're warned against editing the overworld in LM and load an existing save file is because the exit directions and events are saved. Check the size of the .srm file that your emulator creates. Can you load the entire overworld into that space three separate times?

    [Edit: I see your point involving the allocation, but I doubt memory management systems would be any more compatible with different hacks than the original hacks they were designed to placate.]


    (edited by Glyph Phoenix on 05-04-06 09:09 AM)
    Pages: 1 2Add to favorites | Next newer thread | Next older thread
    Acmlm's Board - I3 Archive - ROM Hacking - Another ROM hacking idea |


    ABII

    Acmlmboard 1.92.999, 9/17/2006
    ©2000-2006 Acmlm, Emuz, Blades, Xkeeper

    Page rendered in 0.036 seconds; used 469.14 kB (max 597.57 kB)