ARC - Revision history

LagoLunatic: More accurate wording for file preloading

2020-04-24T02:44:44Z

More accurate wording for file preloading

LagoLunatic: Fix typo

2020-04-24T01:30:29Z

Fix typo

LagoLunatic: Document various new things, including how loading files to MRAM/ARAM/DRAM is determined

2020-04-24T01:24:28Z

Document various new things, including how loading files to MRAM/ARAM/DRAM is determined

LagoLunatic: Mention how WW loads files

2020-03-17T23:03:00Z

Mention how WW loads files

LagoLunatic: Document some more RARC fields

2020-03-17T22:07:41Z

Document some more RARC fields

LagoLunatic: Fix incorrect offset for one field

2020-03-08T03:41:17Z

Fix incorrect offset for one field

LagoLunatic: Added more information about some fields

2020-02-25T23:19:37Z

Added more information about some fields

Einstein95 at 04:25, 18 November 2016

2016-11-18T04:25:22Z

Gamma at 04:18, 18 November 2016

2016-11-18T04:18:54Z

ShimmerFairy: Add tiny section on string table, some header level changes

2016-11-18T04:12:52Z

Add tiny section on string table, some header level changes

@@ Line 21: / Line 21: @@
 | <center>0x0C</center> || <center>4</center> || File Data Offset || The offset to the file data (relative to the start of the data header, so add 0x20)
 |-
-| <center>0x10</center> || <center>4</center> || Total File Data Size || The combined size of file data in both the MRAM and ARAM blocks.
+| <center>0x10</center> || <center>4</center> || Total File Data Size || The combined size of all the file data, including both the MRAM and ARAM blocks as well as any files loaded directly from the DVD that are not in either block.
 |-
-| <center>0x14</center> || <center>4</center> || MRAM File Data Size || The size of the first block of file data, which has files to be loaded into main RAM. Can be 0 if no files have the MRAM flag set.
+| <center>0x14</center> || <center>4</center> || MRAM Preload File Data Size || The size of the first block of file data, which has files to be preloaded into MRAM as soon as the RARC is loaded. Can be 0 if no files have the MRAM flag set.
 |-
-| <center>0x18</center> || <center>4</center> || ARAM File Data Size || The size of the second block of file data, which has files to be loaded into ARAM. Can be 0 if no files have the ARAM flag set.
+| <center>0x18</center> || <center>4</center> || ARAM Preload File Data Size || The size of the second block of file data, which has files to be preloaded into ARAM as soon as the RARC is loaded. Can be 0 if no files have the ARAM flag set.
 |-
 | <center>0x1C</center> || <center>4</center> || Padding || Pads the header to 0x20/32 bytes
@@ Line 113: / Line 113: @@
 | 0x04 || Compressed File || This entry's data is compressed, either with Yaz0 or Yay0.
 |-
-| 0x10 || Load to MRAM || This file should be loaded to main RAM. In practice all non-REL files have this set.
+| 0x10 || Preload to MRAM || This file should be preloaded into MRAM. In practice all non-REL files have this set.
 |-
-| 0x20 || Load to ARAM || This file should be loaded to ARAM. In practice only REL files have this set.
+| 0x20 || Preload to ARAM || This file should be preloaded into ARAM, and then transferred to MRAM when it is needed. In practice only REL files have this set.
 |-
-| 0x40 || Load to DRAM || This file should be loaded to DRAM. This hasn't been observed in practice, so it's unclear exactly how it would work.
+| 0x40 || Load from DVD || This file should not be preloaded, it should be loaded directly from the DVD to MRAM when it is needed.
 |-
 | 0x80 || Yaz0 Compressed File || This entry's data is Yaz0-compressed, not Yay0 compressed. Bit 0x04 should be set as well.

@@ Line 117: / Line 117: @@
 | 0x20 || Load to ARAM || This file should be loaded to ARAM. In practice only REL files have this set.
 |-
-| 0x20 || Load to DRAM || This file should be loaded to DRAM. This hasn't been observed in practice, so it's unclear exactly how it would work.
+| 0x40 || Load to DRAM || This file should be loaded to DRAM. This hasn't been observed in practice, so it's unclear exactly how it would work.
 |-
 | 0x80 || Yaz0 Compressed File || This entry's data is Yaz0-compressed, not Yay0 compressed. Bit 0x04 should be set as well.

@@ Line 6: / Line 6: @@
 This contains data about the archive itself, such as the total size of the file and the counts and offsets of the nodes and file entries. It is 0x40/64 bytes long and has the following structure:
 {| class="wikitable"
@@ Line 15: / Line 17: @@
 | <center>0x04</center> || <center>4</center> || File Size || The size of the entire ARC file
 |-
-| <center>0x08</center> || <center>4</center> || Unknown Constant || Always 0x20/32. Same as the amount subtracted from the offsets in this header, possibly indicating a header size.
+| <center>0x08</center> || <center>4</center> || Data Header Offset || Offset from the start of the file to the data header. In practice this is always 0x20, the size of the RARC header. However, the game does properly read this, so it would have no issues loading RARCs with headers larger than 0x20 bytes as long as this value was updated to match.
 |-
-| <center>0x0C</center> || <center>4</center> || File Data Offset || The offset to the file data
+| <center>0x0C</center> || <center>4</center> || File Data Offset || The offset to the file data (relative to the start of the data header, so add 0x20)
 |-
-| <center>0x10</center> || <center>4</center> || File Data Size 1 || The size of the block of file data
+| <center>0x10</center> || <center>4</center> || Total File Data Size || The combined size of file data in both the MRAM and ARAM blocks.
 |-
-| <center>0x14</center> || <center>4</center> || File Data Size 2 || This must be equal to File Data Size 1 for most ARCs, but for RELS.arc this must be 0 instead.
+| <center>0x14</center> || <center>4</center> || MRAM File Data Size || The size of the first block of file data, which has files to be loaded into main RAM. Can be 0 if no files have the MRAM flag set.
 |-
-| <center>0x18</center> || <center>4</center> || File Data Size 3 || This must be equal to 0 for most ARCs, but for RELS.arc this must be equal to File Data Size 1 instead.
+| <center>0x18</center> || <center>4</center> || ARAM File Data Size || The size of the second block of file data, which has files to be loaded into ARAM. Can be 0 if no files have the ARAM flag set.
 |-
 | <center>0x1C</center> || <center>4</center> || Padding || Pads the header to 0x20/32 bytes
 |-
-| <center>0x20</center> || <center>4</center> || Node Count || The number of Nodes in the archive
+! Offset                !! Size               !! Name || Description
 |-
-| <center>0x24</center> || <center>4</center> || Node Offset || The offset to the start of the Node data
+| <center>0x00</center> || <center>4</center> || Node Count || The number of Nodes in the archive
 |-
-| <center>0x28</center> || <center>4</center> || File Entry Count || The number of File Entries in the archive
+| <center>0x04</center> || <center>4</center> || Node Offset || The offset to the start of the Node data (relative to the start of the data header, so add 0x20)
 |-
-| <center>0x2C</center> || <center>4</center> || File Entry Offset || The offset of the File Entry data
+| <center>0x08</center> || <center>4</center> || File Entry Count || The number of File Entries in the archive
 |-
-| <center>0x30</center> || <center>4</center> || String Table Length || The length of the String Table
+| <center>0x0C</center> || <center>4</center> || File Entry Offset || The offset of the File Entry data (relative to the start of the data header, so add 0x20)
 |-
-| <center>0x34</center> || <center>4</center> || String Table Offset || The offset of the String Table
+| <center>0x10</center> || <center>4</center> || String Table Length || The length of the String Table
 |-
-| <center>0x38</center> || <center>2</center> || Next Free File ID || If a new file is added to this ARC, this is the ID that it should be given. Then this value should be incremented by 1. Not read by the game itself.
+| <center>0x14</center> || <center>4</center> || String Table Offset || The offset of the String Table (relative to the start of the data header, so add 0x20)
 |-
-| <center>0x3A</center> || <center>1</center> || Sync File IDs and Indexes || Seems to be a boolean that, when true, indicates all the files will have file IDs equal to their index in the list of all the files in the ARC. When false, the file IDs will be out of sync with their indexes. This doesn't appear to be read by the game itself.
+| <center>0x18</center> || <center>2</center> || Next Free File ID || If a new file is added to this ARC, this is the ID that it should be given. Then this value should be incremented by 1. Not read by the game itself.
 |-
-| <center>0x3B</center> || <center>1</center> || Padding || Always 0x00
+| <center>0x1A</center> || <center>1</center> || Sync File IDs and Indexes || Seems to be a boolean that, when true, indicates all the files will have file IDs equal to their index in the list of all the files in the ARC. When false, the file IDs will be out of sync with their indexes. This doesn't appear to be read by the game itself.
 |-
-| <center>0x3C</center> || <center>4</center> || Padding || Pads the header to 0x40/64 bytes
+| <center>0x1B</center> || <center>1</center> || Padding || Always 0x00
 |}
 == Node ==
@@ Line 104: / Line 113: @@
 | 0x04 || Compressed File || This entry's data is compressed, either with Yaz0 or Yay0.
 |-
-| 0x10 || Non-REL File || This file is not an executable REL file, it's a normal data file.
+| 0x10 || Load to MRAM || This file should be loaded to main RAM. In practice all non-REL files have this set.
 |-
-| 0x20 || REL File || This entry is an executable REL file (relocatable object file).
+| 0x20 || Load to DRAM || This file should be loaded to DRAM. This hasn't been observed in practice, so it's unclear exactly how it would work.
 |-
-| 0x80 || Yaz0 Compressed File || This entry's data is Yaz0-compressed, not Yay0 compressed.
+| 0x80 || Yaz0 Compressed File || This entry's data is Yaz0-compressed, not Yay0 compressed. Bit 0x04 should be set as well.
 |}

← Older revision		Revision as of 23:03, 17 March 2020
Line 112:		Line 112:

	Note: Every directory contains two extra file entries in addition to the entries that represent the actual files it contains. These two file entries have the names "." and "..", and may stem from how Unix-like systems and Windows handle directories. In those cases, "." represents a link to the current directory, and ".." represents a link to the current directory's parent.		Note: Every directory contains two extra file entries in addition to the entries that represent the actual files it contains. These two file entries have the names "." and "..", and may stem from how Unix-like systems and Windows handle directories. In those cases, "." represents a link to the current directory, and ".." represents a link to the current directory's parent.
		+
		+	Note: The game can load files based off of file index, file ID, or file name. The most common is file index, used by almost all objects and a number of NPCs. File ID is used by most NPCs, as well as STB cutscenes. File name is used by only a handful of things, such as loading stages or dungeon doors.

	==String Table==		==String Table==

@@ Line 39: / Line 39: @@
 | <center>0x34</center> || <center>4</center> || String Table Offset || The offset of the String Table
 |-
-| <center>0x38</center> || <center>2</center> || File Entry Count || Seems to be the number of File Entries stored as a 2-byte value
+| <center>0x38</center> || <center>2</center> || Next Free File ID || If a new file is added to this ARC, this is the ID that it should be given. Then this value should be incremented by 1. Not read by the game itself.
 |-
-| <center>0x3A</center> || <center>1</center> || Unknown Byte 1 || Unknown, always 1?
+| <center>0x3A</center> || <center>1</center> || Sync File IDs and Indexes || Seems to be a boolean that, when true, indicates all the files will have file IDs equal to their index in the list of all the files in the ARC. When false, the file IDs will be out of sync with their indexes. This doesn't appear to be read by the game itself.
 |-
-| <center>0x3B</center> || <center>1</center> || Unknown Byte 2 || Unknown, padding for the previous byte?
+| <center>0x3B</center> || <center>1</center> || Padding || Always 0x00
 |-
 | <center>0x3C</center> || <center>4</center> || Padding || Pads the header to 0x40/64 bytes
@@ Line 77: / Line 77: @@
 ! Offset                !! Size               !! Name || Description
 |-
-| <center>0x00</center> || <center>2</center> || File ID || This is 0xFFFF if the entry represents a directory. Otherwise, the game uses this ID to load specific files
+| <center>0x00</center> || <center>2</center> || File ID || A unique identifier for each file. Directories don't use this, they usually have it set to 0xFFFF, but may have it set to 0x0000 instead in rare cases.
 |-
 | <center>0x02</center> || <center>2</center> || Name Hash || A hash of the file or directory's name. See "Filename Hashing Algorithm" below

@@ Line 3: / Line 3: @@
 ARC is used in nearly every first-party Nintendo game in the GameCube era, including ''The Wind Waker'' and ''Twilight Princess''. ''Luigi's Mansion'' typically used YAY0-compressed archives with the extension .szp, and ''Super Mario Sunshine'' used YAZ0-compressed archives with the extension .szs.
-==File Format==
+== Header ==
 This contains data about the archive itself, such as the total size of the file and the counts and offsets of the nodes and file entries. It is 0x40/64 bytes long and has the following structure:
@@ Line 25: / Line 21: @@
 | <center>0x10</center> || <center>4</center> || File Data Size 1 || The size of the block of file data
 |-
-| <center>0x14</center> || <center>4</center> || File Data Size 2 || Duplicate of File Data Size 1?
+| <center>0x14</center> || <center>4</center> || File Data Size 2 || This must be equal to File Data Size 1 for most ARCs, but for RELS.arc this must be 0 instead.
 |-
 | <center>0x18</center> || <center>4</center> || Padding || Pads the header to 0x1C/28 bytes
@@ Line 54: / Line 52: @@
 All of the offsets in this file need 0x20 added to them to reflect their actual offset within the file. This is the same value as the constant at offset 0x08. This could possibly mean that the first 0x20 bytes of this header are the "main" header, or otherwise ignored for offset counting in a way the next 0x20 bytes are not.
-===Node===
+== Node ==
 Nodes represent the directories stored within the archive. They are 0x10/16 bytes long, and are laid out like this:
@@ Line 73: / Line 71: @@
 |}
-===File Entry===
+== File Entry==
 These entries represent the actual hierarchy of files and directories. They are 0x14/20 bytes long and have the following structure:
@@ Line 85: / Line 83: @@
 | <center>0x02</center> || <center>2</center> || Name Hash || A hash of the file or directory's name. See "Filename Hashing Algorithm" below
 |-
-| <center>0x04</center> || <center>1</center> || Type || An indicator of whether this entry is a file or a directory
+| <center>0x04</center> || <center>1</center> || Type Flags || A bitfield of various flags indicating what type of entry this is. See below for details.
 |-
-| <center>0x06</center> || <center>2</center> || Name Offset || The offset of this entry's name in the string table
+| <center>0x05</center> || <center>3</center> || Name Offset || The offset of this entry's name in the string table
 |-
 | <center>0x08</center> || <center>4</center> || Data Offset || If this is a directory, the is the index of the directory's node. If it's a file, it's the offset to the file's data.
@@ Line 98: / Line 94: @@
 |}
-Every directory contains two extra file entries in addition to the entries that represent the actual files it contains. These two file entries have the names "." and "..", and may stem from how Unix-like systems and Windows handle directories. In those cases, "." represents a link to the current directory, and ".." represents a link to the current directory's parent.
+=== Type Flags Bitfield ===
-===String Table===
+==String Table==
 The string table is simply a series of null-terminated strings.
@@ Line 134: / Line 149: @@
 ==See Also==
-[https://github.com/LordNed/WArchive-Tools WAarchive Tools], a tool and library written by LordNed and Sage_of_Mirrors for extracting and packing ARC files
+Tools that can extract and repack ARC files:

@@ Line 66: / Line 66: @@
 | <center>0x04</center> || <center>4</center> || Name Offset || The offset to the directory's actual name
 |-
-| <center>0x08</center> || <center>2</center> || Name Hash || A hash of the directory's name. See "Hashing Algorithm" below
+| <center>0x08</center> || <center>2</center> || Name Hash || A hash of the directory's name. See "Filename Hashing Algorithm" below
 |-
 | <center>0x0A</center> || <center>2</center> || File Entry Count || The number of file entries that belong to this directory
@@ Line 83: / Line 83: @@
 | <center>0x00</center> || <center>2</center> || File ID || This is 0xFFFF if the entry represents a directory. Otherwise, the game uses this ID to load specific files
 |-
-| <center>0x02</center> || <center>2</center> || Name Hash || A hash of the file or directory's name. See "Hashing Algorithm" below
+| <center>0x02</center> || <center>2</center> || Name Hash || A hash of the file or directory's name. See "Filename Hashing Algorithm" below
 |-
 | <center>0x04</center> || <center>1</center> || Type || An indicator of whether this entry is a file or a directory
@@ Line 106: / Line 106: @@
 =Filename Hashing Algorithm=
-The NameHash field is a hash of the file's name. An example function for creating this hash is found below. It uses C# syntax.
+Nodes and File Entries have a field for a hash of their names. An example function for creating this hash from the name string is found below. It uses C# syntax.
   private ushort HashName(string name)