Reference · Formats

The DATA.BIN archive

solidfrom#3#5#20#242#324

The game's main asset archive: 1,023 gzip members laid end to end on sector boundaries, each holding one CCS scene file. The archive has no directory of its own; the index is compiled into the executable.

Layout

All little-endian.

DATA.BIN
  member    starts on a 2048-byte boundary
    gzip    RFC 1952, one member: 1f 8b 08, FLG 0x08 (FNAME only), MTIME,
            XFL, OS, NUL-terminated name, deflate data, CRC32, ISIZE
    padding zero bytes to the next 2048-byte boundary

index, in SLUS_202.67 .data
  categoryFDTbl   0x002fb750   u32[20]   pointer to each category's table
  cateCDOfsTbl    0x002fb7a0   u64[20]   byte offset of each category in DATA.BIN

category table    44-byte records, ended by one all-zero record
  char[32]  name      "XASC00.CCS", upper case, NUL-padded
  u32       offset    bytes from the category's start; a multiple of 2048
  u32       csize     exact length of the gzip member, padding excluded
  u32       usize     inflated length

A record's sector on the disc:

lba = lsn(\DATA\DATA.BIN;1) + cateCDOfsTbl[category] / 2048 + offset / 2048

where lsn is what sceCdSearchFile returns (12463 on the Infection disc). The code rounds offset up to a sector - (offset + 2047) >> 11 - which makes no difference, since every offset is aligned.

Categories

"Base sector" is cateCDOfsTbl[k] / 2048, relative to the archive. Categories 0 to 18 hold all 1,023 members, in archive order, with no gaps: each category starts where the previous one's last member ends, and the last record ends at the last byte of the file.

Fields

name - the gzip FNAME is the same name in lower case with .cmp in place of .CCS (XASC00.CCS / xasc00.cmp). The game only ever uses the record name.

csize - reading exactly csize bytes from the member's start reaches the end of the gzip stream in all 1,023 records.

usize - equals the gzip ISIZE in all 1,023. ccFileListLoad (at 0x00164ea8) chooses how to read a file by it:

• not 0: FileReadTh reads the member into one ring buffer, and UngzipTh inflates it from there into a second ring, which the scene loader (ccStream) reads.
• 0: FileReadTh reads straight into the loader's ring, and UngzipTh gets no ccUngzip, so it never calls Decode. The file's bytes reach the loader as they are: a file stored without compression.

Only category 19's entries have a usize of 0 (below).

Lookup

searchFname__FP8FILELIST (0x001651d0) gets a file-list entry (s32 category, then the name at +4), takes categoryFDTbl[category], and compares names with strcmp, 44 bytes at a time, until a match or a record named "NULL". On "NULL" it draws FILE NONE and hangs. Category 19 compares only the part of the name before the ., upper-cased.

File-list entries

The run-time file list (FILELIST, INF DWARF, filelib.cpp) holds one entry per requested file:

FILELIST    0x28 bytes, little-endian
  +0x00  s32       category
  +0x04  char[32]  name        ".ccs" appended when it has no '.', upper-cased
  +0x24  s16       deleteCnt   reference count
  +0x26  s16       addFlag     always -1

name - ccAddFileList (0x001639c0), ccAddFileListOne (0x00163bd0) and ccAddFileListName (0x00163db0) copy the requested name, strcat ".ccs" when strchr(name, '.') finds none, then strupr. A lower-case name in a static list (xddn_010.CCS, cdogboda.ccs) therefore matches its upper-case record.

deleteCnt - ccInitFileList sets 0. Every add increments it (0x00163a38), and so does fileConflictCheck when the file is already in the list (0x0016408c). ccFileListDeleteOne decrements it (0x00163664); at 0 it destroys the file's ccStream (ccsLoad[i]), decrements directNum for category 19, and renames the entry "NULL".

addFlag - written -1 at init and by each add (0x00163a50, 0x00163c58, 0x00163e4c), and otherwise only copied. Its one test, == 1 in ccFileExistCheck (0x001641a4), can never pass.

Every static file list on the four volumes (a run of {s32 category, char *name} pairs ended by a negative category or a null name, in the executable and the four overlays) names a file of its own category: 142, 152, 192 and 193 rows.

Category 19

Category 19 bypasses the archive. directCCSTbl (0x00383ec0, 16 records) is in .bss; ccInitFileList__Fv fills every slot's name with "NULL", and ccAddFileList / ccAddFileListOne add entries. Each new entry gets the name as the list gives it, offset 0, csize -1 and usize 0. ccFileListLoad opens cdrom0:\DATA\ + name with sceOpen, takes the size from sceLseek(fd, 0, SEEK_END) (the csize of -1), and reads the file without inflating it (the usize of 0). So category 19 loads a loose, uncompressed .CCS from DATA.

No volume uses it. A file list is an array of {s32 category, char *name} pairs, ended by a negative category or a null name, and all of them were checked on the four volumes:

• Lists in the executables and overlays. A scan of every word pair in the executable and the four overlays finds no category 19 followed by a pointer to a name.
• Lists made at run time. Their categories are constants (spcTempFileList 7 and 9, GateHackOutFileList 18, ccPuccigusoStart's 11, RequestCCS's calls 0 and 15 to 18, ccAddRequestFileListInu's 11), or come from rows of enemyTbl, gimmickTbl and npcTbl, which are in the scan above.
• The constant 19. None of the eight or nine places per volume that store it builds a file list: they are in sceMcRename, ccVoiceRequest, two event opcodes (a gimmick's entry type), ccAddRequestFileListSpc (a gateHackingOutID), MailList_control::PreviewRes and BOOK::PadControl03.

No disc has a loose .CCS in DATA either. Category 19 loads a scene file that is not in the archive, and the retail game never asks it to.

Notes

The static tables end with a record whose name is empty, not "NULL", so for categories 0 to 18 the "NULL" stop never fires: a name missing from its table would be searched for past the end, through the following categories' records.

categoryPathTbl (0x002fb700) names one archive per category - cdrom0:\DATA\cmn.bin through cdrom0:\DATA\event.bin - none of which is on the disc. Its only reader uses it for category 19, whose entry is cdrom0:\DATA\.

The game inflates with its own ccUngzip (Decode__8ccUngzipFUi, 0x00156ba0), whose methods carry the function names of gzip 1.2's inflate.c. A reader thread (FileReadTh) feeds it through ccRingBufferTh ring buffers, and it runs on its own thread (UngzipTh).

Counts

Measured over INF's whole file:

tools/fdtbl.py check verifies every record against the archive. It passes on all four volumes - 1,023, 1,031, 1,072 and 1,072 records - through the table names piney-gen syms carries to the stripped executables; the other volumes' category counts are on the four discs.

Unknown

None.