- — File Parsing Boot Camp — - (Some file structures added)

[pikazz]

You guys... REFT and REFF rebuilding is COMPLETE. All I need figured out is the actual DATA contained in the REFF. Not the string relocation offsets. Just the data. REFT is 100% rebuildable, but the texture importer needs a few tweaks to work right with it. I've replaced Peach's hearts with stars as a test and it looked fine in BB, but they appeared swizzled in-game.

so you are not proud? D;

will do some research in the REFF about the data

[DarkPikachu]

OMG D:
I hope I didn't release my last MDL0 template like this :|

I never noticed it because I havn't been able to revise it until now, but

1 - the vertex attributes don't work as the command var isn't referenced in the switch.
2 - the entire 2nd half of the VtxAttrHi is labled as UV_7 in the primitives

but, I was trying to be quick and release it, and the area had nothing on it, so...
at least now it's fixed
(and I sped it up a little (can only do a minor speedup))

the requirements of the MDL0 structure causes it to load extremely slow >_>

I've suggested a template buffer for HexEdit to Andrew though,
so he should hopefully implement that.
(would give a 10x speedup)

the template takes over an hour to load when loading Pachirisu (v11) in design mode D:
(about 15 minutes out of design mode)
and yes, I've fixed the v9 and v11 implementations, so that will be in my next release

I'll disable alot so it won't take forever to load...
(best I can limit is 3-5 minutes for small files)

EDIT:
forgot to mention...
the registers (shaders and nodes), attributes (objects) and primitives (objects) are what drastically increase loading time...

the constant repeat in structures is what causes it >_>

[DarkPikachu]

D-Post... IK... shoot me.

I've just UD'd BJ's thread with important info about BP registers.

IK my template is wrong, but I had the wrong idea...
I'm working on correcting it right now

btw, I looked at my last template release...
it was before I've finished re-defining everything XDD
(my slowest release yet)

hopefully this next one will be alot faster than that

[Xiggah]

It's all right, share your progress. By the way, are there any specific defined structures you want me to post from Brawlbox?

[DarkPikachu]

It's all right, share your progress. By the way, are there any specific defined structures you want me to post from Brawlbox?

just what I'm interested in

that being:
Resource Groups (still needed)
Materials (because of the shaders)
Nodes (somewhat still interested in these)
Objects (I'm sure there's been new findings)

that's it

btw...
I don't really have a beef with multi-posting...
-was flammed and warned for someone else's complaints... (I kept multi-posting)
*cough* NSM *cough cough*
I blame the admins for removing the "show signature" option...
I'd at least do that >:O

*sigh* and now I rmbr why I hate this forum -.-*
but I'm not gonna rant about it...
this is about file data,
not why people are P'd at forums

[Xiggah]

All right, I'll go gather up some of that source code. By the way, do you know anything about nulling normals or similar things like that?

[DarkPikachu]

if I'm right...
n = (0,0,0)*count

I think that's what does it >_>

[Xiggah]

Resource Groups (still needed)
Materials (because of the shaders)
Nodes (somewhat still interested in these)
Objects (I'm sure there's been new findings)

Let's see if I can do this right.... I'm just posting the headers here, I don't know if you wanted me to lay any of it out or anything like that.

Resource Groups: Do you mean the node trees or the Definitions?

Materials:
        public bint _dataLen;
        public bint _mdl0Offset;
        public bint _stringOffset;
        public bint _index;
        public buint _isXLU; //0x00 0x80000000 for XLU textures
        public byte _flag1; //Texture count
        public byte _numLayers; //Not exactly
        public byte _flag3; //Repeated on shader
        public byte _flag4;
        public bint _cull; //0x02, XLU = 0
        public byte _flag5;
        public byte _flag6;
        public byte _flag7; //Texture layers
        public byte _flag8;
        public bint _unk3;
        public bint _unk4; // -1, 0xFFFFFFFF
        public bint _shaderOffset;
        public bint _numTextures;
        public bint _MatRefOffset; //1044, or 1048 for v11 MDL0
        public bint _part2Offset; //0
        public bint _dlOffset; //Offset to display list(s).
        public bint _unk6; //0


Nodes.... there are 5 different versions. Which ones do you want?


Objects
        public const uint Size = 0x64;

        public bint _totalLength;
        public bint _mdl0Offset;
        public bint _nodeId; //Single-bind node

        public CPVertexFormat _vertexFormat;
        public XFVertexSpecs _vertexSpecs;

        public bint _defSize; //Size of def block including padding? Always 0xE0?
        public bint _defFlags; //0x80, sometimes 0xA0
        public bint _defOffset; //Relative to defSize field

        public bint _dataLen1; //Size of primitives
        public bint _dataLen2; //Same as previous
        public bint _dataOffset; //Relative to dataLen1

        public XFArrayFlags _arrayFlags; //Used to enable element arrays?

        public bint _unk3; //0
        public bint _stringOffset;
        public bint _index;

        public bint _numVertices;
        public bint _numFaces;

        public bshort _vertexId;
        public bshort _normalId;
        public fixed short _colorIds[2];       
        public fixed short _uids[8];

        public bint _nodeTableOffset;

[DarkPikachu]

ok umm...

I prbly should just call them Resource Lists since you keep getting confused over them XD

and Nodes == Material-Nodes
they're currently called Shaders in Brbx...
(the term needs correction) >_>

here's some useful info BJ doesn't seem to want to listen to: <_<

Shaders are located w/in the material and control things such as fur, grass, smoke, or water
they can handle multiple textures (up to 10 in blender)
and can also control things such as bump-maps, texture displacement, multi-texturing (with an alpha)

Material-Nodes are a seperate instance from the material, and control material color (of it's linked material), texture color/brightness, light effects and color, among other things I've yet to look into <_<

I have more info in my old MDL0 thread >_>

I've also animated Pikachu's eyes with a dynamic node
although you have to do double the work to animate it with the script <_<

yeh... I've still got a ways to go in game development yet -.-
but I'll get there

NOTE:
I'm actually describing this stuff as it is in Blender...
(JIC people want to think I'm making this stuff up) >_>
^you have no idea how much I've been through Xiggah D:

[Xiggah]

ok umm...

I prbly should just call them Resource Lists since you keep getting confused over them XD

and Nodes == Material-Nodes
they're currently called Shaders in Brbx...
(the term needs correction) >_>

here's some useful info BJ doesn't seem to want to listen to: <_<

Shaders are located w/in the material and control things such as fur, grass, smoke, or water
they can handle multiple textures (up to 10 in blender)
and can also control things such as bump-maps, texture displacement, multi-texturing (with an alpha)

Material-Nodes are a seperate instance from the material, and control material color (of it's linked material), texture color/brightness, light effects and color, among other things I've yet to look into <_<

I have more info in my old MDL0 thread >_>

I've also animated Pikachu's eyes with a dynamic node
although you have to do double the work to animate it with the script <_<

yeh... I've still got a ways to go in game development yet -.-
but I'll get there

NOTE:
I'm actually describing this stuff as it is in Blender...
(JIC people want to think I'm making this stuff up) >_>
^you have no idea how much I've been through Xiggah D:

Lots of work, that's what you've been through. Much more than I at any rate. Those materials and shaders, we need a nice guide on how to make nice effects with them. I'm specifically interested in decreasing the amount of lighting, I suppose that would be with the shader.

Shaders: There are also TEVblocks, do you want to see those?
        public const int Size = 32;

        public bint _dataLength; //Always 512
        public bint _mdl0Offset;
        public bint _index;
        public byte _flag; //Same as material
        public byte _res0, _res1, _res2; //Always 0. Reserved?
        public sbyte _ref0, _ref1, _ref2, _ref3, _ref4, _ref5, _ref6, _ref7;
        public int _pad0, _pad1; //Always 0

        //Display list
        public ShaderTevBlock* DisplayList { get { return (ShaderTevBlock*)(Address + Size); } }
        //There are 8 structures max following the display list, each 0x30 in length.
        //Each structure has 9 commands.


Definitions: I don't know if this is the right Resource Entry or not, but here they are. It mostly refers to other node types.
public byte _type;

        private VoidPtr Address { get { fixed (void* ptr = &this)return ptr; } }

        public MDL0NodeType2* Type2Data { get { return (MDL0NodeType2*)(Address + 1); } }
        public MDL0NodeType3* Type3Data { get { return (MDL0NodeType3*)(Address + 1); } }
        public MDL0NodeType4* Type4Data { get { return (MDL0NodeType4*)(Address + 1); } }
        public MDL0NodeType5* Type5Data { get { return (MDL0NodeType5*)(Address + 1); } }

        public MDL0DefEntry* Next
        {
            get
            {
                switch (_type)
                {
                    case 2: return (MDL0DefEntry*)(Type2Data + 1);
                    case 3: return (MDL0DefEntry*)(Type3Data + 1);
                    case 4: return (MDL0DefEntry*)(Type4Data + 1);
                    case 5: return (MDL0DefEntry*)(Type5Data + 1);

[DarkPikachu]

here's a small example of rendering hair with shaders:


decreasing the lighting??
are you talking about the specular part of the materials??
(I always use black to get something towards Pika's skin look)
^eg: my ava

although I think that can be controlled through nodes as well...

TBH, I really don't think actual materials are used anymore...
(I think it's all dynamically controlled through the shader/node)

and the resource lists are what the MDL0 header contains offsets to

but yur OK with the Defs being posted XD

[Xiggah]

I don't know the difference between specular or diffuse or ambient yet... so no idea

Brawlbox's source code has BJ's name definitions, that's all. Should I still post those tidbits of code?

[DarkPikachu]

I don't know the difference between specular or diffuse or ambient yet... so no idea

Brawlbox's source code has BJ's name definitions, that's all. Should I still post those tidbits of code?

yea, go ahead an post em

and for the colors:
diffuse is your main model color
specular is the light reflection color
ambiant *forgets how it works*
(ambiant light would be like light in a room (not reflected from a direct source))

EDIT:
ambient *correction* color is the reflected color from the object to another object

[Xiggah]

yea, go ahead an post em

and for the colors:
diffuse is your main model color
specular is the light reflection color
ambiant *forgets how it works*
(ambiant light would be like light in a room (not reflected from a direct source))

I didn't even know Brawl had light reflection, I thought it was all direct light. Anyways, I might just have to toy around with the Shader settings until I find the right values. Too bad I have idea what any of it means

Here, have some.... node types. Here, have the entire code snippet. now complete without accidental italicization.

Code:

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe abstract class MDL0NodeClass
    {
        public static object Create(ref VoidPtr addr)
        {
            object n = null;
            switch (*(byte*)addr++)
            {
                case 2: { n = Marshal.PtrToStructure(addr, typeof(MDL0Node2Class)); addr += MDL0Node2Class.Size; break; }
                case 3: { n = new MDL0Node3Class((MDL0NodeType3*)addr); addr += ((MDL0Node3Class)n).GetSize(); break; }
                case 4: { n = Marshal.PtrToStructure(addr, typeof(MDL0NodeType4)); addr += MDL0NodeType4.Size; break; }
                case 5: { n = Marshal.PtrToStructure(addr, typeof(MDL0NodeType5)); addr += MDL0NodeType5.Size; break; }
            }
            return n;
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe class MDL0Node2Class : MDL0NodeClass
    {
        public const uint Size = 0x04;

        public bushort _boneIndex;
        public bushort _parentNodeIndex;

        public ushort BoneIndex { get { return _boneIndex; } set { _boneIndex = value; } }
        public ushort ParentNodeIndex { get { return _parentNodeIndex; } set { _parentNodeIndex = value; } }
        
        public override string ToString()
        {
            return string.Format("Type2 (Bone Index:{0}, Parent Node Index:{1})", BoneIndex, ParentNodeIndex);
        }
    }

    public unsafe class MDL0Node3Class
    {
        public bushort _id;
        public List<MDL0NodeType3Entry> _entries = new List<MDL0NodeType3Entry>();

        public unsafe MDL0Node3Class(MDL0NodeType3* ptr)
        {
            _id = ptr->_id;
            for (int i = 0; i < ptr->_numEntries; i++)
                _entries.Add(ptr->Entries[i]);
        }

        public ushort Id { get { return _id; } set { _id = value; } }
        public MDL0NodeType3Entry[] Entries { get { return _entries.ToArray(); } }

        public int GetSize() { return 3 + (_entries.Count * MDL0NodeType3Entry.Size); }

        public override string ToString()
        {
            return string.Format("Type3 (ID:{0})", Id);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType2
    {
        public const int Size = 0x04;

        public bushort _index;
        public bushort _parentId;

        public ushort Index { get { return _index; } set { _index = value; } }
        public ushort ParentId { get { return _parentId; } set { _parentId = value; } }

        public override string ToString()
        {
            return string.Format("Type2 (Index:{0},ParentID:{1})", Index, ParentId);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType3
    {
        public const int Size = 0x03;

        public bushort _id;
        public byte _numEntries;

        private VoidPtr Address { get { fixed (void* ptr = &this)return ptr; } }

        public MDL0NodeType3Entry* Entries { get { return (MDL0NodeType3Entry*)(Address + 3); } }

    }
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType3Entry
    {
        public const int Size = 0x06;

        public bushort _id;
        public bfloat _value;

        public ushort Id { get { return _id; } set { _id = value; } }
        public float Value { get { return _value; } set { _value = value; } }

        public override string ToString()
        {
            return String.Format("Type3Entry (ID:{0},Weight:{1})", Id, Value);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType4
    {
        public const uint Size = 0x07;

        public bushort _materialIndex;
        public bushort _polygonIndex;
        public bushort _boneIndex;
        public byte _val4;

        public ushort MaterialId { get { return _materialIndex; } set { _materialIndex = value; } }
        public ushort PolygonId { get { return _polygonIndex; } set { _polygonIndex = value; } }
        public ushort BoneIndex { get { return _boneIndex; } set { _boneIndex = value; } }
        public byte Val4 { get { return _val4; } set { _val4 = value; } }

        public override string ToString()
        {
            return string.Format("Type4 (MatID:{0},PolyID:{1},BoneIndex:{2},Unk:{3})", MaterialId, PolygonId, BoneIndex, Val4);
        }
    }

    //Links node IDs with indexes
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType5
    {
        public const uint Size = 0x04;

        public bushort _id; //Node Id
        public bushort _index; //Node Index

        public int Id { get { return _id; } set { _id = (ushort)value; } }
        public int Index { get { return _index; } set { _index = (ushort)value; } }

        public override string ToString()
        {
            return string.Format("Type5 (ID:{0},Index:{1})", Id, Index);
        }
    }

[DarkPikachu]

I didn't even know Brawl had light reflection, I thought it was all direct light. Anyways, I might just have to toy around with the Shader settings until I find the right values. Too bad I have idea what any of it means

Here, have some.... node types. Here, have the entire code snippet. now complete without accidental italicization.

Code:

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe abstract class MDL0NodeClass
    {
        public static object Create(ref VoidPtr addr)
        {
            object n = null;
            switch (*(byte*)addr++)
            {
                case 2: { n = Marshal.PtrToStructure(addr, typeof(MDL0Node2Class)); addr += MDL0Node2Class.Size; break; }
                case 3: { n = new MDL0Node3Class((MDL0NodeType3*)addr); addr += ((MDL0Node3Class)n).GetSize(); break; }
                case 4: { n = Marshal.PtrToStructure(addr, typeof(MDL0NodeType4)); addr += MDL0NodeType4.Size; break; }
                case 5: { n = Marshal.PtrToStructure(addr, typeof(MDL0NodeType5)); addr += MDL0NodeType5.Size; break; }
            }
            return n;
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe class MDL0Node2Class : MDL0NodeClass
    {
        public const uint Size = 0x04;

        public bushort _boneIndex;
        public bushort _parentNodeIndex;

        public ushort BoneIndex { get { return _boneIndex; } set { _boneIndex = value; } }
        public ushort ParentNodeIndex { get { return _parentNodeIndex; } set { _parentNodeIndex = value; } }
        
        public override string ToString()
        {
            return string.Format("Type2 (Bone Index:{0}, Parent Node Index:{1})", BoneIndex, ParentNodeIndex);
        }
    }

    public unsafe class MDL0Node3Class
    {
        public bushort _id;
        public List<MDL0NodeType3Entry> _entries = new List<MDL0NodeType3Entry>();

        public unsafe MDL0Node3Class(MDL0NodeType3* ptr)
        {
            _id = ptr->_id;
            for (int i = 0; i < ptr->_numEntries; i++)
                _entries.Add(ptr->Entries[i]);
        }

        public ushort Id { get { return _id; } set { _id = value; } }
        public MDL0NodeType3Entry[] Entries { get { return _entries.ToArray(); } }

        public int GetSize() { return 3 + (_entries.Count * MDL0NodeType3Entry.Size); }

        public override string ToString()
        {
            return string.Format("Type3 (ID:{0})", Id);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType2
    {
        public const int Size = 0x04;

        public bushort _index;
        public bushort _parentId;

        public ushort Index { get { return _index; } set { _index = value; } }
        public ushort ParentId { get { return _parentId; } set { _parentId = value; } }

        public override string ToString()
        {
            return string.Format("Type2 (Index:{0},ParentID:{1})", Index, ParentId);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType3
    {
        public const int Size = 0x03;

        public bushort _id;
        public byte _numEntries;

        private VoidPtr Address { get { fixed (void* ptr = &this)return ptr; } }

        public MDL0NodeType3Entry* Entries { get { return (MDL0NodeType3Entry*)(Address + 3); } }

    }
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType3Entry
    {
        public const int Size = 0x06;

        public bushort _id;
        public bfloat _value;

        public ushort Id { get { return _id; } set { _id = value; } }
        public float Value { get { return _value; } set { _value = value; } }

        public override string ToString()
        {
            return String.Format("Type3Entry (ID:{0},Weight:{1})", Id, Value);
        }
    }

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType4
    {
        public const uint Size = 0x07;

        public bushort _materialIndex;
        public bushort _polygonIndex;
        public bushort _boneIndex;
        public byte _val4;

        public ushort MaterialId { get { return _materialIndex; } set { _materialIndex = value; } }
        public ushort PolygonId { get { return _polygonIndex; } set { _polygonIndex = value; } }
        public ushort BoneIndex { get { return _boneIndex; } set { _boneIndex = value; } }
        public byte Val4 { get { return _val4; } set { _val4 = value; } }

        public override string ToString()
        {
            return string.Format("Type4 (MatID:{0},PolyID:{1},BoneIndex:{2},Unk:{3})", MaterialId, PolygonId, BoneIndex, Val4);
        }
    }

    //Links node IDs with indexes
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public unsafe struct MDL0NodeType5
    {
        public const uint Size = 0x04;

        public bushort _id; //Node Id
        public bushort _index; //Node Index

        public int Id { get { return _id; } set { _id = (ushort)value; } }
        public int Index { get { return _index; } set { _index = (ushort)value; } }

        public override string ToString()
        {
            return string.Format("Type5 (ID:{0},Index:{1})", Id, Index);
        }
    }

all I can say is google image search: 'specular color'
you should get the idea right off
(it's the lightest color)

EDIT:
Samus has a specular on top of a texture reflection