From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 10303 invoked from network); 5 Sep 2023 16:57:22 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 5 Sep 2023 16:57:22 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 95FEE4104F; Wed, 6 Sep 2023 02:57:20 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuhs.org; s=dkim; t=1693933040; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=akPiv/EzgmOOHBKVaOvRC1F13Zw2ZFQDZSoPDMFVdZk=; b=gHVf09BZPsfcOYPtwf8gPp5HLGRGj7rxR8+Uh8lMn6UaKbHx5tgTBQJo6mgD6XCeW/Ivb1 BpBBwlwsk72qg/vMR8ZWWDvcgDVNk4RotSL005K/c5kim51nYo2fXSurQMGq0Xq5F0mXCr wk7CGCDkx7pWv98zxn+t/G1Fw4i2HE8= Received: from mail-4324.protonmail.ch (mail-4324.protonmail.ch [185.70.43.24]) by minnie.tuhs.org (Postfix) with ESMTPS id 2783A4104C for ; Wed, 6 Sep 2023 02:57:15 +1000 (AEST) Date: Tue, 05 Sep 2023 16:56:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1693933029; x=1694192229; bh=akPiv/EzgmOOHBKVaOvRC1F13Zw2ZFQDZSoPDMFVdZk=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=DVRO62p5LgxxPD5Ep+LXvIMQkB2U4Sqg+VlPnLFWGPWLLwAJ10k7VlVbd32F8ehra e4c3LdiaLX4OvSmHNm0ZYtr08902QHSzI90alHhjDgiRT0UiQ1KdaPjnYoM7faX/Qb u/fLnyKgxqR9EjS+jPB83ZixrjG+MVU/IyNAQlEOUrrH5PESfYBfdBhI2q0l4vL2HX CKPhO7oQoF215DqFkAGJt4289whe0aFmTmS2ZS+TtDtY3bPo8y7ZaiRfIdKCVzM3B/ E2GWGLCblyMFx+or30r4U5d1eanO1ExO6PlNB54No86UAAX2FpkiNByUORuRcmc72+ f5DNAyw0uluEg== To: COFF Message-ID: Feedback-ID: 35591162:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: GACSXNCHASAAPYW5RAGQCKTVJHKLNU2N X-Message-ID-Hash: GACSXNCHASAAPYW5RAGQCKTVJHKLNU2N X-MailFrom: segaloco@protonmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [COFF] Dragon Quest Disassembly List-Id: Computer Old Farts Forum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: segaloco via COFF Reply-To: segaloco Howdy all, I've made some mention of this in the past but it's finally at a= point I feel like formally plugging it. For the past few years I've been = tinkering on a disassembly of Dragon Quest as originally released on the Fa= micom (NES) in Japan. The NA release had already gotten this treatment[1] = but I wanted to produce something a little more clean and separated: https:= //gitlab.com/segaloco/dq1disasm Another reason for this project is I've longed for a Lions's Commentary on = UNIX-quality analysis of a scrollplane/sprite-driven console video game as = I find generally analysis of this type of hardware platform lacking outside= of random, chance blog posts and incredibly specialized communities where = it takes forever to find information you're seeking. In the longer term I = intend to produce a commentary on this codebase covering all the details of= how it makes the Famicom tick. So a few of the difficulties involved: The Famicom is 6502-based with 32768 bytes of ROM address space directly ac= cessible and another 2048 bytes of ROM accessible through the Picture Proce= ssing Unit or PPU. The latter is typically graphics ROM being displayed di= rectly by the PPU but data can be copied to CPU memory space as well. To a= chieve higher sizes, bank swapping is widely employed, with scores of diffe= rent bank swapping configurations in place. Dragon Quest presents an inter= esting scenario in that the Japanese and North American releases use *diffe= rent* mapper schemes, so while much of the code is the same, some parts wer= e apples to oranges between the existing and new disassembly. Where this h= as been challenging (and a challenge many other Famicom disassemblers haven= 't taken) is getting a build that resembles a typical software product, i.e= . things nicely split out with little concern as to what bank they're going= to wind up on. My choice of linker facilitates this by being able to defi= ne separate output binaries for different memory segments, so I can produce= these banks as distinct binaries and the population thereof is simply base= d on segment. Moving items between banks then becomes simply changing the = segment and adding/removing a bank-in before calling out. I don't know tha= t I've ever seen a Famicom disassembly do this but I also can't say I've go= ne looking heavily. Another challenge has been the language, and not necessarily just because i= t's Japanese and not my native language. So old Japanese computing is larg= ely in ShiftJIS whereas we now have UTF-8. Well, Dragon Quest uses neither= of these as they instead use indices into a table of kana character tiles,= as they are ordered in memory, as the string format. What this means is t= o actually effectively manage strings, and I haven't done this yet, is one = needs a to-fro encoder that can convert between UTF-8 Japanese and the part= icular positioning of their various characters. This says nothing of the d= ifficulty then of translating the game. Most Japanese games of this era us= ed exclusively kana for two reasons: These are games for children, so too m= uch complicated Kanji and you've locked out your target audience. The othe= r reason is space; it would be impossible to fit all the necessary Kanji in= memory, even as 8x8 pixel tiles (the main graphic primitive on these chips= .) Plus, even if you could, an 8x8 pixel tile is hardly enough resolution = to read many Kanji, so they'd likely need 16x16, quadrupling the space requ= irement if you didn't recycle quadrant radicals. In any case, what this me= ans is all of the strings are going to translate to strictly hiragana or ka= takana strings, which are Japanese, but not as easily intelligible as the g= roupings of kana then have to be interpreted into their meanings by context= clues often times rather than having an exact definition. The good news t= hough is again these are games for children, so the vocabulary shouldn't be= too complicated. Endianness occasionally presents some problems, one of which I suspect beca= use the chip designers didn't talk to each other...So the PPU exposes a reg= ister you write two bytes to, the high and low byte of the address to place= the next value written to the data register to. Problem is, this is a 650= 2-driven system, so when grabbing a word the high byte is the second one. = This means that every word has to be sent to the PPU in reverse when select= ing an address. This PPU to my knowledge is based on some arcade hardware = but otherwise was based on designs strictly for this console, so why they d= idn't present an address register in the same endianness I'll never know. = It tripped me up early on but I worked past it. Finally, back to the mappers, since a "binary" was really a gaggle of poten= tially overlapping ROM banks, there wasn't really a single ROM image format= used by Nintendo back in the day. Rather, you sent along your banks, layo= ut, and what mapper hardware you needed and that last step was a function o= f cartridge fab, not some software step. Well, the iNES format is an accom= modation for that, presenting the string "NES" as a magic number in a 16-by= te header that also contains a few bytes describing the physical cartridge = in terms of what mapper hardware, how many banks, what kind, and particular= jumpers that influence things like nametable mirroring and the like. Coun= tless disassembles out there are built under the (incorrect) assumption tha= t the iNES binary format is how the binary objects are supposed to build, b= ut this is simply not the case, and forcing this "false structure" actually= makes then analysis of the layout and organization of the original project= more difficult. What I opted towards instead is using the above-described= mechanism of linker scripts defining individual binaries in tandem with tw= o nice old tools: printf(1) and cat(1). When all my individual ROM banks a= re built, I then simply use printf to spit out the 16 bytes of an iNES head= er that match my memory particulars and then cat it all together as the iNE= S format is simply that header then all of the PRG and CHR ROM banks organi= zed in that order. The end result is a build that is agnostic (rightfully = so) of the fact that it is becoming an iNES ROM, which is a community inven= tion years after the Famicom was in active use. Note that this is still an ongoing project. I don't consider it "done" but= it is more usable than not now. Some things to look out for: Any relative positioning of specific graphic data in CHR banks outside that= which has been extracted into specific files is not dynamic, meaning movin= g a tileset around in CHR will garble display of the object associated. A few songs (credits, dungeons, title screen) do not have every pointer mas= saged out of them yet, so if their addresses change, those songs at the ver= y least won't sound right, and in extreme circumstances playing them with a= shift in pointers could crash the game. Much of the binary data is still BLOB-afied, hence not each and every point= er being fixed. This will be slow moving and there may be a few parts that= wind up involving some scripting to keep them all dynamic (for instance, m= aps include block IDs and such which are ultimately an index into a table, = but the maps make more sense to keep as binary data, so perhaps a patcher i= s needed to "filter" a map to assign final block IDs, hard to say.) Also, while some data is very heavily coupled to the engine, such as music,= and as such has been disassembled into audio engine commands, other data, = like graphic tiles, are more generic to the hardware platform and as such d= o not have any particular dependencies on the code. As such, these items c= an exist as pure BLOBs without worry of any pointers buried within or ID va= lues that need to track indices in a particular table. As such these flat = binary tiles and other comparable components are *not* included in this rep= ository, as they are copyright of their respective owners. However, I have= provided a script that allows you to extract these binary components shoul= d you have a clean copy of Dragon Quest for the Famicom as an iNES ROM. Th= e script is all dd(1) calls based on the iNES ROM itself, but there should = be enough notes contained with in if a bank-based extraction is necessary. = YMMV using the script on anything other than the commercial release. It i= s only intended to bootstrap things, it will not properly work to extract t= he assets from arbitrary builds. If anyone has any questions/comments feel free to contact me off list, or i= f there is worthwhile group discussion to be had, either way, enjoy! - Matt G. [1] - https://github.com/nmikstas/dragon-warrior-disassembly