Entry 3
DATA.BIN is 1023 gzip members holding CCSF scene files
Revisited later. Something claimed here was corrected by CCSF chunk dispatch and the real texture chunk length.
Documented in The CCSF scene file, The DATA.BIN archive.
Filesdocs/formats/data-bin.mddocs/formats/ccs.md
INF DATA/DATA.BIN (136,665,088 bytes) has no header and no directory. It is
gzip members laid end to end, each starting on a 2048-byte boundary and padded
with zeros to the next. The first bytes of the file are 1f 8b 08 08, and the
gzip FNAME field right after the fixed header reads framebuf.cmp.
Scanning every sector boundary for 1f 8b 08 finds 1,023 members; the magic
occurs 6 more times off-boundary, all inside compressed data. All 1,023 inflate
with Python's zlib (wbits 31) with no errors, each ends with between 1 and
2,047 bytes of padding before the next member, and ISIZE matches the inflated
length every time. 135,599,928 compressed bytes become 338,646,324. Every member
has FLG 0x08 (FNAME only), every FNAME is lower case ending .cmp, and MTIME
runs from 973651006 (2000-11-08) to 1030068730 (2002-08-23).
The game does its own inflating: ccUngzip in main, whose methods -
huft_build, inflate_codes, inflate_stored, inflate_fixed,
inflate_dynamic, inflate_block, flush_window, updcrc - carry the
function names of gzip 1.2's inflate.c. It runs on a thread (UngzipTh)
fed by a reader thread (FileReadTh) through ccRingBufferTh ring buffers.
Every payload is CCSF
All 1,023 inflated payloads begin 01 00 cc cc. They are CyberConnect2's
chunked scene format - the engine calls it ccStream, with one class per chunk
kind (ccTexChunk, ccClutChunk, ccModelChunk, ccClumpChunk,
ccAnmChunk, ccMaterialChunk, ccCamChunk, ccLightChunk, ccHitChunk,
ccBboxChunk, ccEffChunk). A chunk is u32 type, u32 size in 32-bit words, payload.
What holds across all 1,023 files, each checked:
- The first chunk is type
0xcccc0001, size 13:"CCSF", a 32-byte name, and four u32s. The name equals the gzip FNAME without.cmpin all 1,023. - The second is
0xcccc0002, the name table:u32 file_count,u32 object_count, thenfile_count32-byte source-file names andobject_count32-byte object entries (30-byte name,u16file index). Its size is exactly2 + 8 * (file_count + object_count)words in all 1,023. Entry 0 of both tables is all zeros in all 1,023. All 316,528 object entries point at a valid file. - The third is type
0x00000003- note, not0xccccprefixed - in all 1,023.
The source-file names are the artist's paths with the root cut off:
x\window\tex\xasc00.bmp, c\w\dh\sw\02\max\cwdhsw02_6.max. Each is stored
with a leading ' ' (10,581 names) or '#' (184); what the # marks is not
known. The .max files are 3ds Max scenes, so the art pipeline was Max plus
a CyberConnect2 exporter.
Object names carry a type prefix, and the counts across the archive say what
the scenes are made of: OBJ_ 244,315, MDL_ 33,661, MAT_ 11,071, CMP_
7,254, CLT_ 5,538, TEX_ 4,306, ANM_ 4,091, DMY_ 2,529, HIT_ 2,388,
LGT_ 421, BLT_ 290, BOX_ 237, MPH_ 233, EFF_ 171, CAM_ 20, FBR_ 2,
PAG_ 1.
The header's first u32 after the name is 0x100 in 999 files and 0x90,
0x92, 0x95 or 0x96 in the other 24 - probably an exporter version. The
second varies (0 for xasc00, 242 for the x74*cam camera files) and looks
like an animation frame count; not checked.
Where a naive walk breaks
Walking chunks by the size field works through the header and name table and
through CLUT chunks (0xcccc0400), but fails on texture chunks
(0xcccc0300) and on 0xcccc0800. In xasc00 the texture chunk declares
0x2039 words, but its 9-word header plus 0x2000 words of 4-bit pixels
(256x256) leaves 48 declared words that are not in the file; in 552 files the
walk overruns the end by exactly 176 bytes; in others the pixel data runs on
past the declared end. The size field of a texture chunk evidently counts
something the file does not store - plausibly GS packet headers the loader
builds in place - and the loader (SetBltData__10ccTexChunkFP14ccTexChunkDesc)
is where the real rule is. That waits on the DWARF reader for
ccTexChunkDesc.
The index is not in the archive
The game asks for files by upper-case .CCS name: cmnFileList
(INF SLUS_202.67:0x00306de0) is {s32 kind, char *name} pairs,
XASC00.CCS through XWINDOW.CCS, ending with kind -1, and gcmnFileList
(0x00306e20) is the same shape with 31 entries. Nothing next to these lists
says where in DATA.BIN a name lives. A search of the executable, all four
overlays, ICON.BIN and OUTSIDE.BIN for the members' sector numbers - as
u16 or u32, relative to the archive or absolute (the archive starts at LBA
12463), at every stride from 4 to 64 - found two hits, both false: one in code
and one in Shift-JIS text. So either the table is computed, or it is somewhere
not yet looked at. searchFname__FP8FILELIST (0x001651d0) and
ccFileListLoad__FPv (0x00164540) will say, once there is a disassembler.
how a .CCS name is resolved to an offset in DATA.BIN; the texture chunk's real size rule; the meaning of the # file-name prefix; the header's frame-count guess; the kind field of the file lists.