Home Gallery Resources The Team Rules chat Login Register
Pages:  [1] 2 3 4 ... 9
Author Topic: - — File Parsing Boot Camp — - (Some file structures added)  (Read 25920 times)
0 Members and 1 Guest are viewing this topic.
Xiggah
Supreme Kitten
********
Offline Offline

Posts: 648

  • Awards Super Saiyan Topic Heart Container

  • View Profile WWW Awards
    « on: January 23, 2012, 11:26:26 PM »


    In this thread, we will train ourselves to parse Brawl's files, so Brawlbox will be able to edit more files and be capable of more hacks. In layman's terms, parsing a file means sorting out different parts of the file, similar to separating stacked lego blocks. We can do this through hexing Brawl's files and finding patterns in its code. There will be more tutorials, but until then feel free to use this thread to ask questions about file parsing and hexing.

    Here's a short list of file types that need parsed:
    Files needing parsed:
    - BLOC     File container, like ARC or BRRES, used commonly in Subspace Emissary files.
    - REFF     Normally seen in files with AI data, such as CPU controlled characters and items.
    - SCL0     No idea?...
    - SHP0     (SHaPe) Vertex morph, an animation file used to change a character's facial expression.
    - PAT0     (PATtern) Texture pattern, an animation file that switches textures to simulate an animated texture.
    - Havok Physics     Physics engine files, used for many things including hair and cape physics.

    Tutorials

        Intro to Hexing
    For those completely new to hexing, let's do a quick little exercise. Open Brawlbox, select File --> New --> BRRES. Right-Click the BRRES and select New --> Character Animation. Finally, Right-Click the animation and create a new bone animation. You should end up with this:

    Now you should have a basic animation file, 1 frame without any animation data. Right-Click and export the NEWCHR animation (CHR0) so it can be hexed. Open up the animation CHR0 in your hex editor, any hex editor will work, I'll be using Notepad++ but the results will be the same. Assuming you didn't rename the CHR0, it should look more or less like this, depending on your view settings:

    So you might be thinking "i could never read what these random numbers mean!" but let's get into that right now. Since there aren't any animations in this file, it's pretty much empty. Open the animation in Brawlbox and change the Translation X keyframe value to 1, export the CHR0 and open it in hex again.

    You can see some changes here and there, but how do we find out what they mean? Thanks to Brawlbox, we know that this animation file has a single keyframe with a value of 1. That number (keyframe value, not amount of keyframes) was translated into a 8-digit value called a Float value, which can be translated with this webpage. Tools like these are a hexer's best friend.

    Using that webpage, you can see that 1.0 in hexidecimal is 3F 80 00 00. You should be able to find that number in your hex editor: if not, then It's highlighted in the previous image. Any single number, even 99999 or 3.14, will be displayed as an 8-digit float value. If you change the keyframe value in Brawlbox, then the only difference will be that float value: everything else remains the same.

    Now let's do something cool with your hex editor: Click on the beginning of the float value and type 44 a7 20 00, save and open in Brawlbox. You just hexed the keyframe value from 1 to 1337. Pretty neat, right?

    This is what Brawl hackers do to change simple number values that aren't editable in Brawlbox, as well as hacking computer files in general. Try using your new knowledge of float values and hexing to change numbers in other Brawl files, or add more keyframes with numbers and see where each number ends up.

    In the meantime, I'll set up the next tutorial about identifying file structures.

    Download links:
    HXD, a commonly-used hex editor.
    Notepad++, a handy program based off of Notepad with a Hex Plugin.
    Hexidecimal Number Converter (website, not a download)


    File structures:

    BLOC
    0x00(4) - BLOC
    0x04(4) - # of entries
    0x08(4) - Unknown
    0x0C(4) - Nulls?

    Offsets (starts at 0x10)
    4 bytes * # of entries = size of string table

    DAT (Event Data)
    Header Size - 0x50

    0x08 = Match Type(Value is at 2 for Coin event match)
    0x0C = Time Limit
    0x1F = Stage ID
    0x38 = Game Speed
    0x3C = Camera Shaking control
    0x46 = Music ID

    PlayerData Size = 0x38
    ---------------------------------
    0x00  = byte CharacterID
    0x01  = byte Status (Normal,Metal,Invisible,BEEP?)
    0x04  = float Size(Scaling)
    0x08  = Team Flag
    0x0C  = ?????(Easy)
    0x0F  = Offense Ratio(Easy)
    0x11  = Defense Ratio(Easy)
    0x14  = Stock count(Easy)
    0x18  = Starting Damage(Easy)
    0x1B  = ?????(Normal)
    0x1D  = Offense Ratio(Normal)
    0x1F  = Defense Ratio(Normal)
    0x22  = Stock count(Normal)
    0x26  = Starting Damage(Normal)
    0x28  = ?????(Hard)
    0x2B  = Offense Ratio(Hard)
    0x2D  = Defense Ratio(Hard)
    0x30  = Stock count(Hard)
    0x34  = Starting Damage(Hard)

    SHP0
    unsafe struct SHP0
        {
            public BRESCommonHeader _header;
            public bint _dataOffset;
            public bint _stringListOffset; //List of vertex node strings
            public bint _stringOffset;
            public bint _unk1; //0
            public bshort _numFrames;
            public bshort _numEntries;
            public bint _unk2; //0x00, 0x01

    unsafe struct SHP0Entry
        {
            public bint _numEntries; //entries + 1 index part
            public bint _stringOffset;
            public bint _numIndices;
            public bint _unk;
            public bint _indiciesOffset;

    I'm pretty sure the keyframes are I12 Keyframe Entries except with no frame scale.

        public unsafe struct I12Header
        {
            public bushort _numFrames;
            public bushort _unk;
            public bfloat _frameScale; //This is the value that isn't present in SHP0

    public unsafe struct I12Entry
        {
            public bfloat _index;
            public bfloat _value;
            public bfloat _tangent;

    Havok Physics
    public unsafe struct PhysicsHeader
        {
            public const uint Tag1 = 0x57E0E057;
            public const uint Tag2 = 0x10C0C010;

            public uint _tag1; //0x57E0E057
            public uint _tag2; //0x10C0C010
            public bint _unk2; //0
            public bint _unk3; //4

            public byte _unk4; //4
            public bshort _unk5; //1
            public byte _unk6; //1
            public bint _unk7; //3
            public bint _unk8; //1
            public bint _unk9; //0

            public bint _unk10; //0
            public buint _unk11; //Size/Offset

            //Three sections of offsets:
            //__classnames__
            //__data__
            //__types__

            public OffsetSection ClassNames;
            public OffsetSection Data;
            public OffsetSection Types;

            public String Name { get { return new String((sbyte*)Address + 0x28); } }
            
            public VoidPtr ClassNamesData { get { return Address + *(buint*)(Address + 0x54); } }
            public VoidPtr DataData { get { return Address + *(buint*)(Address + 0x84); } }
            public VoidPtr TypesData { get { return Address + *(buint*)(Address + 0xB4); } }
    }

    public unsafe struct OffsetSection
        {
            public fixed byte name[0x10];

            public bint _unk0; //-1
            public buint _dataOffset; //Main header is the base
            
            //Offsets to indices struct. _dataOffset is the base for everything.
            public buint _dataOffset1;
            public buint _dataOffset2;
            public buint _dataOffset3;
            public buint _dataOffset4;
            public buint _dataOffset5;
            public buint _dataOffset6;

            //When offsets begin to repeat, stop reading
            //Indices count is ((next data offset) - (current data offset)) / 2
            //Indices are padded to 0x10 with 0xFF

            public String Name { get { return new String((sbyte*)Address); } }
        }

    public unsafe struct IndexGroup
        {
            //Base is the offset sections's _dataOffset
            //Still not sure exactly how these are used, but they're definitely a datalength or offset
            
            public buint _dataOffset; //Not necessarily...
            public buint _stringOffset; //Not necessarily...

    PAT0
    Header:
    0x00 (4) PAT0
    0x04 (4) String Offset Table
    0x08 (4) Version
    0x0C (4) Unknown1
    0x10 (4) Pipeline offset
    0x14 (4) "Texture Reference" (will describe this later)
    0x18 (4) Unknown2, one offset that leading to a emtpy space
    0x1C (4) Unknown3, often the same as Unknown2
    0x20 (4) String Offset Table (Copy)
    0x24 (4) Name of animation Offset
    0x28 (4) Unknown4
    0x2C (2) Frames
    0x2E (2) NumEntries/Materials
    0x30 (2) How many Textures (String Entries1)
    0x32 (2) String Entries 2, often set as "00"
    0x34 (4) Unknown5

    PipeLine:
    0x00 (4) How long PipeLine is
    0x04 (4) How many Entries/IDs
    0x08-1C IDs thingy
    dont exactly know how the ID's working but this is importiant:
    0x20 (4) It's name in String Offset (Pipelines offset + this offset = string offset)
    0x24 (4) "link" offset (Pipeline offset + this offset = "Link" offset)
    and the ID and Stirng/pattern offsets repeat on each entries

    "Link" Offset:
    0x00 (4) String Offset, linked to Materials name
    0x04 (2) What Texture it will using if Flag is set on "7". if the flag is "5" this is set on "00"
    0x06 (2) Flag: 5 using Pattern, 7 is Frozen
    0x08 (4) Pattern Animation Offset (Link Offset + this = Pattern Offset). this is set on "00" if the Flag is set on "7"

    Pattern Animation offset
    0x00 (2) Unknown, possibly flag
    0x02 (2) Unknown
    0x04 (4)  unknown, Possible many flags in "bits"
    0x08 (4) unknown, always(?) set on 00
    0x0C (2) Which Texture will be shown
    0x0E (2) Unknown, always(?) set on 00
    0x10 (2) Which frame and possible with a flag with it (if it saying 41B8, the 4 is a flag and 1B8 is frame)

    Repeat from 0x0C to 0x10 on how many frames

    "Texture refserence" Offset
    here is special and hard to explain, the textures here comes in order thanks by the "link" offset and it's the order the Pattern Animation in their "texture"

    0x00 (4) Offset to their texture name in String Offset

    String Table Offset:
    well, all the names are in here, always in the end of the PAT0s
    « Last Edit: January 26, 2012, 11:10:33 AM by Xiggah » Logged


    Xiggah
    Supreme Kitten
    ********
    Offline Offline

    Posts: 648

  • Awards Super Saiyan Topic Heart Container

  • View Profile WWW Awards
    « Reply #1 on: January 23, 2012, 11:26:39 PM »


    Also, I want this second post, just in case.
    Logged


    Xiggah
    Supreme Kitten
    ********
    Offline Offline

    Posts: 648

  • Awards Super Saiyan Topic Heart Container

  • View Profile WWW Awards
    « Reply #2 on: January 23, 2012, 11:27:07 PM »


    And I'll take this last post too. No more needed.
    Logged


    DSX8
    Stage/Character Importer
    Never Gonna Give You Up
    *
    Offline Offline

    Posts: 9288


    meow~

  • Awards Good Citizen >9000 King for a Day Heart Container

  • View Profile Awards
    « Reply #3 on: January 23, 2012, 11:31:43 PM »


    ill add some stuff bout PAT0 files in the morning Tongue

    mostly the header tho... im still going through the other sections in PAT0
    Logged

    Follow me on facebook and Twitter!!!
    https://www.facebook.com/DMNSLYRX8              https://twitter.com/Demonslayerx8

    3DS Friend Code: 0705-6436-8834              NNID: Demonslayerx8              PSN: Demonslayerx8



    Xiggah
    Supreme Kitten
    ********
    Offline Offline

    Posts: 648

  • Awards Super Saiyan Topic Heart Container

  • View Profile WWW Awards
    « Reply #4 on: January 23, 2012, 11:37:49 PM »


    Let's see..... we'll have this download link for the hex editor I use, HXD, and some basic tutorial writing for now.

    Thanks for the info Nymph, I could use some of those header informations. I'll probably have to use some pages from the Mario Kart Wiki page for now
    Logged


    DSX8
    Stage/Character Importer
    Never Gonna Give You Up
    *
    Offline Offline

    Posts: 9288


    meow~

  • Awards Good Citizen >9000 King for a Day Heart Container

  • View Profile Awards
    « Reply #5 on: January 23, 2012, 11:38:51 PM »


    no problem! i've like fixed alot of V4 PAT0 files and they were able to open file in BB once replaced with the unfixed file... so yeah Tongue

    edit:
    btw i've added u on msn... so ya hope u got my invite lol
    Logged

    Follow me on facebook and Twitter!!!
    https://www.facebook.com/DMNSLYRX8              https://twitter.com/Demonslayerx8

    3DS Friend Code: 0705-6436-8834              NNID: Demonslayerx8              PSN: Demonslayerx8



    DSX8
    Stage/Character Importer
    Never Gonna Give You Up
    *
    Offline Offline

    Posts: 9288


    meow~

  • Awards Good Citizen >9000 King for a Day Heart Container

  • View Profile Awards
    « Reply #6 on: January 24, 2012, 12:35:23 AM »


    this post is reserved by me


    SCN0 Discussion:
    Stage lighting, fog, and camera for stages! More soon to come

    Header for brawl SCN0's:
    Quote
    Version# Offset: 00x8
    Frame Offset:
    Loop Offset:
    ---------------------------------------------------------------------------------------------------
    PAT0 Discussion:
    (PATtern) Texture pattern, an animation file that switches textures to simulate an animated texture.

    Header for brawl V3 PAT0's:
    Quote
    Version# Offset: 00x8
    Frame Offset:
    How many sub-files offset:
    Loop Offset:


    Header for TvC (and other wii games) V4 PAT0's:
    Quote
    Version# Offset: 00x8
    Frame Offset:
    How many sub-files offset:
    Loop Offset:

    ---------------------------------------------------------------------------------------------------
    From Pikazz helpful Info:
    Quote from: pikazz
    okay, here we goes:


    Header:
    0x00 (4) PAT0
    0x04 (4) String Offset Table
    0x08 (4) Version
    0x0C (4) Unknown1
    0x10 (4) Pipeline offset
    0x14 (4) "Texture Reference" (will describe this later)
    0x18 (4) Unknown2, one offset that leading to a emtpy space
    0x1C (4) Unknown3, often the same as Unknown2
    0x20 (4) String Offset Table (Copy)
    0x24 (4) Name of animation Offset
    0x28 (4) Unknown4
    0x2C (2) Frames
    0x2E (2) NumEntries/Materials
    0x30 (2) How many Textures (String Entries1)
    0x32 (2) String Entries 2, often set as "00"
    0x34 (4) Unknown5

    PipeLine:
    0x00 (4) How long PipeLine is
    0x04 (4) How many Entries/IDs
    0x08-1C IDs thingy
    dont exactly know how the ID's working but this is importiant:
    0x20 (4) It's name in String Offset (Pipelines offset + this offset = string offset)
    0x24 (4) "link" offset (Pipeline offset + this offset = "Link" offset)
    and the ID and Stirng/pattern offsets repeat on each entries

    "Link" Offset:
    0x00 (4) String Offset, linked to Materials name
    0x04 (2) What Texture it will using if Flag is set on "7". if the flag is "5" this is set on "00"
    0x06 (2) Flag: 5 using Pattern, 7 is Frozen
    0x08 (4) Pattern Animation Offset (Link Offset + this = Pattern Offset). this is set on "00" if the Flag is set on "7"

    Pattern Animation offset
    0x00 (2) Unknown, possibly flag
    0x02 (2) Unknown
    0x04 (4)  unknown, Possible many flags in "bits"
    0x08 (4) unknown, always(?) set on 00
    0x0C (2) Which Texture will be shown
    0x0E (2) Unknown, always(?) set on 00
    0x10 (2) Which frame and possible with a flag with it (if it saying 41B8, the 4 is a flag and 1B8 is frame)

    Repeat from 0x0C to 0x10 on how many frames

    "Texture refserence" Offset
    here is special and hard to explain, the textures here comes in order thanks by the "link" offset and it's the order the Pattern Animation in their "texture"

    0x00 (4) Offset to their texture name in String Offset

    String Table Offset:
    well, all the names are in here, always in the end of the PAT0s

    This is my work and I have done this in more than a year Im srs here
    « Last Edit: January 26, 2012, 01:27:11 PM by Nymph » Logged

    Follow me on facebook and Twitter!!!
    https://www.facebook.com/DMNSLYRX8              https://twitter.com/Demonslayerx8

    3DS Friend Code: 0705-6436-8834              NNID: Demonslayerx8              PSN: Demonslayerx8



    deadname mcredactedface
    Lol Kitten
    *********
    Offline Offline

    Posts: 773


    Kind of want to start making mods for this again

  • Awards Heart Container

  • View Profile Awards
    « Reply #7 on: January 24, 2012, 01:48:43 AM »


    Posting to subscribe.
    Logged


    Xiggah
    Supreme Kitten
    ********
    Offline Offline

    Posts: 648

  • Awards Super Saiyan Topic Heart Container

  • View Profile WWW Awards
    « Reply #8 on: January 24, 2012, 02:37:41 AM »


    There's a tutorial for hexing, I'll write one for file structures next.
    Logged


    pikazz
    Heroic Kitten
    **
    Offline Offline

    Posts: 2281


    Machinimator!

  • Awards Pin Collector Good Citizen Helping Hand Super Saiyan Topic

  • View Profile Awards
    « Reply #9 on: January 24, 2012, 02:57:09 AM »


    PAT0 is already complete by me  *looking for my files*
    when a "nice looking form" by E_Y

    EDIT: dont tell me that Smashboard deleted one section ;_;
    « Last Edit: January 24, 2012, 03:02:38 AM by pikazz » Logged






    DSX8
    Stage/Character Importer
    Never Gonna Give You Up
    *
    Offline Offline

    Posts: 9288


    meow~

  • Awards Good Citizen >9000 King for a Day Heart Container

  • View Profile Awards
    « Reply #10 on: January 24, 2012, 03:01:49 AM »


    PAT0 is already complete by me  *looking for my files*

    EDIT: dont tell me that Smashboard deleted one section ;_;
    Im srs here
    dont tell me u dont keep a backup saved in a .txt file not impressed
    Logged

    Follow me on facebook and Twitter!!!
    https://www.facebook.com/DMNSLYRX8              https://twitter.com/Demonslayerx8

    3DS Friend Code: 0705-6436-8834              NNID: Demonslayerx8              PSN: Demonslayerx8



    pikazz
    Heroic Kitten
    **
    Offline Offline

    Posts: 2281


    Machinimator!

  • Awards Pin Collector Good Citizen Helping Hand Super Saiyan Topic

  • View Profile Awards
    « Reply #11 on: January 24, 2012, 03:07:10 AM »


    Im srs here dont tell me u dont keep a backup saved in a .txt file not impressed
    nope cause it was over one year old Oh shi~ cause it was on smashboard and I never though they would delete that section but I can redo it cause I still remember 70-75% of it without using an PAT0 file as a reference D:
    « Last Edit: January 24, 2012, 03:10:56 AM by pikazz » Logged






    DSX8
    Stage/Character Importer
    Never Gonna Give You Up
    *
    Offline Offline

    Posts: 9288


    meow~

  • Awards Good Citizen >9000 King for a Day Heart Container

  • View Profile Awards
    « Reply #12 on: January 24, 2012, 03:11:40 AM »


    nope cause it was over one year old Oh shi~ cause it was on smashboard and I never though they would delete that section but I can redo it cause I still remember 70-75% of it without using an PAT0 file as a reference D:
    hmm.... well give me wat u kno... and ill add it then, but with major credit to u Tongue
    Logged

    Follow me on facebook and Twitter!!!
    https://www.facebook.com/DMNSLYRX8              https://twitter.com/Demonslayerx8

    3DS Friend Code: 0705-6436-8834              NNID: Demonslayerx8              PSN: Demonslayerx8



    DarkPikachu
    Angel Kitten
    ***
    Offline Offline

    Posts: 3069


    complexity == fun

  • Awards Fiery Topic Heart Container KCMM Veteran Tutorial Writer

  • View Profile Awards
    « Reply #13 on: January 24, 2012, 03:57:44 AM »


    I've joined your camp simply by being cool "Wherever Vyse goes, cool follows."

    all I can contribute to here is my REFF template for HexEdit 4.0 =3=
    (I'm on wii so I can't do anything other than tell you about it) X(

    EDIT:
    I could use some help in the Melee area >_>
    (Me and Milun can't do it all)

    you can DL UMC 2.6d and my last (crappy) HexEdit 4.0 template (it works) if your interested in learning the files
    NOTE: everything in my template needs to be renamed...

    I think I have a download over at Brawl Imports...
    if not, then my src is on gist.github.com
    (search for 'DAT Template' on google and look for the gist link)

    EDIT2:
    btw, I only keep backups of my informative format posts:
    UMC TMP format
    MDL0 format (v9 and v11 (soon to merge))
    Melee DAT format (needs an update BADLY)
    others that I can't quite rmbr atm Tongue
    « Last Edit: January 24, 2012, 04:15:19 AM by Tcll » Logged


    Quote: Friedslick6
    you have been through a lot of hassle. I've watched every topic you posted on this, and most of them seemed to disintegrate gradually.
    But the coolest part was that you didn't stop working on it despite that.

    Quote: Internet Explorer
    you're doing more with your life right now than probably most other people around you. You're a valuable asset to the Smash community. So yeah, you should be proud.

    quote: Greg
    You do have a gift which I've seen many developers use to their advantage. You can become a great coder, and with all of those ideas I think you can really build something great.

    ForOhFor Error
    Holy Kitten
    *
    Offline Offline

    Posts: 1472


    DOG is your destiny

  • Awards Super Saiyan Topic Pin Collector Starstormer Famous Hacker

  • View Profile Awards
    « Reply #14 on: January 24, 2012, 04:16:08 AM »


    I might join. Also, there's a whole bunch of SSE filetypes to sift through.
    Logged


    FC: 2191-7379-6272

    Pages:  [1] 2 3 4 ... 9
    Print
    Jump to: