[Python] All-in-one import hexer. Version 2! GUI and multiple file support!

[ForOhFor Error]

We already have working programs to hex imports from games with MDL0s into usable models. However, these programs are fairly slow, only fix one MDL0 at a time, and, well, there's 3 of the things thatn eed to be run separately.

I am doing my first full program, based in python, to remedy that situation. It will combine the 3 steps (header, materials, polygons) into one program, which will hex every model in a brres file.

And, since it's in Python instead of Gamemaker, it's much faster than the existing programs.

It will be useful for, say, hexing those bothersome okami models.

UPDATE LOG:

V 2.0:
-Added a GUI
-Added multi-BRRES support
-Now overwrites input files


Program completed, download here:
http://dl.dropbox.com/u/5869687/Python%20Model%20Hexer%202%20%28GUI%29.zip

Old version:
http://dl.dropbox.com/u/5869687/Python%20Model%20Hexer%20-%20Complete.zip

TO DO BEFORE FULL RELEASE:
Done
In progress
Bug fixing
Not started

- Opening file, translating into a usable string
- Locating all MDL0s and logging their locations
- Creation of a module to store long functions
- MDL0 header fixing function
- MDL0 material fixing function
- MDL0 polygon fixing function
- Launching these functions through a main program
- Translating usable hex back to a savable format
- Saving the file

I may need some help along the way, just for the locating of specific values in the hex data.

[BlackJax96]

Nice. This will help a lot.
Good luck, if you need it

[ForOhFor Error]

Thanks!
(Also, note that, not only being my first utility program, I've never really manually hexed anything before )

I've made some progress, but have a small problem:

How a certain block should look like after the header fix (From the existing fixing program):

4D444C30 00066040 00000009 FFFFFF80 000000A0 000000E8 00000140 00000258 00000340 00000448 00000560 00000648 00000730 00000848 00000000 00826C44 00000040 FFFFFFC0 00000000 00000000 00005C8E 00003294 00000000 00000004 01000000 00000040 C24EAAEF BFC2D6FE C394B4AF 437DC28A 40E99CA8 42B804D7 00000004 00000001 00000002 00000003 00000000 FFFFFFFF FFFFFFFF FFFFFFFF

How it does look:

4D444C30 00066040 00000009 FFFFFF80 000000A0 000000E8 00000140 00000258 00000340 00000448 00000560 00000648 00000730 00000848 00000000 00000000 00000000 FFFFFFC0 00826C44 00000040 FFFFFFB4 00000000 00000000 00000000 00003294 00000000 00000003 01000000 00000040 C24EAAEF BFC2D6FE C394B4AF 437DC28A 40E99CA8 42B804D7 00000004 00000001 FFFFFFFF FFFFFFFF FFFFFFFF

No clue what's causing it...
but that's the only difference once both have been header fixed.

[DarkPikachu]

how are you handeling the values??
are the values being handeled floats or ints??

it's possible you could be handeling a float as an int

[ForOhFor Error]

as a string...

[DarkPikachu]

ah

here, the full mdl0 header should look like this:
note: this is reading from it

        #<<header>>
        magic = fr.read(4)
        if magic == 'MDL0': #check if valad
            print 'MDL0 file is valad' #check
            #--header (part1)--
            strtbl_offset = ui(fr.read(4))
            sections = ui(fr.read(4))
            nodes_offset = si(fr.read(4))*(0-1)
            #list offsets to relocation table groups
            definits_list = ui(fr.read(4))
            bones_list = ui(fr.read(4))
            vertices_list = ui(fr.read(4))
            normals_list = ui(fr.read(4))
            colors_list = ui(fr.read(4))
            uv_points_list = ui(fr.read(4))
            materials_list = ui(fr.read(4))
            shader_conf_list = ui(fr.read(4))
            objects_list = ui(fr.read(4))
            textures_list = ui(fr.read(4))
            pallets_list = ui(fr.read(4))
            #--header (part2)--
            string_offset = ui(fr.read(4))
            header_length = ui(fr.read(4)) #64
            header_offset = ui(fr.read(4))
            #--displayed in brawlbox:--
            unknown_1 = fr.read(4)
            unknown_2 = fr.read(4)
            NumVertices = ui(fr.read(4))
            NumFaces = ui(fr.read(4))
            unknown_3 = fr.read(4)
            NumNodes = ui(fr.read(4))
            version = hex(fr.read(4)) #no idea how to handle
            unknown_4 = fr.read(4)
            MinX,MinY,MinZ = fl(fr.read(4)),fl(fr.read(4)),fl(fr.read(4))
            MaxX,MaxY,MaxZ = fl(fr.read(4)),fl(fr.read(4)),fl(fr.read(4))
            NumNodes2 = fr.read(4) #(Copy?)
        #<</header>>


here's the common functions being used:

#--<<common functions>>--
import struct as St
def L(this): return St.unpack('>l', this)[0] #return a long int
def hex(this): return this.encode('hex') #return the hex value of a char
def ui(this): return int(this.encode('hex'),16) #return an unsigned 8bit, 16bit, or 32bit int (most common)
def si(this): return int(long(this.encode('hex'),16)-2**32) #return a signed 8bit, 16bit, or 32bit int
def bfl(this): return St.unpack("<f", this)[0] #return a big endian 32bit float
def fl(this): return St.unpack(">f", this)[0] #return a 32bit float (highly common)
#--<</common functions>>--

[ForOhFor Error]

That code from your Universal Model Converter?
Thing is, I'm just using this here (the tutorial):
http://www.smashboards.com/showthread.php?t=263127
as a guide for what to change, using code to do the steps.
:/

[DarkPikachu]

if you're having any problems understanding how to code for the MDL0,
you can look into my WIP template for HexEdit:
http://brawlimports.proboards.com/index.cgi?board=templates&action=display&thread=12

[DarkPikachu]

That code from your Universal Model Converter?

yea...

you can use it to see how to write the values

btw,
use St.pack
not St.unpack as in the functions

use the L() function when writing something like '00 00 00 09'
if you use ui(), you'll just get '09'
I like to use chr(int(h,16)) alot,
and I use chr(0) for padding

[ForOhFor Error]

Well, here's the code I'm using in the MDL0 fixing module:

Code:

import string
def fixHeader(Data,indexNum):
    data2 = Data
    logThis = Data[indexNum+88:indexNum+96]
    newDat = Data[0:indexNum+16]
    newDat += "00000009ffffff80"
    newDat += Data[len(newDat):len(newDat)+(8*6)]
    newDat += Data[len(newDat)+16:len(newDat)+56]
    newDat += Data[len(newDat)+8:len(newDat)+24]
    newDat += "ffffffc0"
    newDat += Data[len(newDat):len(newDat)+40]
    newDat += logThis
    newDat += Data[len(newDat):]
    return newDat
def fixHeaderPt2(Data,indexNum):
    savey = Data[indexNum+(4*8):indexNum+(5*8)]
    startOfDefHead = int(savey,16)-12
    newDat = Data[0:indexNum+startOfDefHead*2]
    newDat += "ffffffff"*3
    newDat += Data[len(newDat):]
    return newDat

I call fixHeader() first, with my logged position of the MDL0 file (indexNum) and the data of the brres file, saved as a string (Data) as arguments (This does all the steps in the text tutorial, excluding adding 3 'FFFFFFFF's), set a new data string (newdata) to the returned string then call fixHeaderPt2() to add the 3 'FFFFFFFF's and set newdata to that. Here's the main program's code:

Code:

import os
import string
import ___MDL0Fix___
from tkFileDialog import askopenfilename
from tkFileDialog import asksaveasfilename
newData = ""
brres = askopenfilename( filetypes=[
('Brawl Resource Archive','*.brres')
] , )
f = open(brres,'rb')
data = f.read().encode('hex')
if data[0:8] == "62726573":
    print("Valid BRRES confirmed")
else:
    print "Not a valid file. Passing exception to end program."
    x = 1/0
numMdl0s = data.count("4d444c30")
indexLoc = []
start = 0
print numMdl0s,"MDL0 format models found"
for x in range(0,numMdl0s):
    offset = data[start+8:len(data)].find("4d444c30")
    start += offset+8
    indexLoc.append(start)
    print "MDL0 found at",start
print numMdl0s,"MDL0 format models found"
newdata = data
for x in range(0,numMdl0s):
    print "Fixing MDL0 header...(1/2)"
    newdata = ___MDL0Fix___.fixHeader(newdata,indexLoc[x])
    print "Fixing MDL0 header...(2/2)"
    newdata = ___MDL0Fix___.fixHeaderPt2(newdata,indexLoc[x])
    print "MDL0 header fixed!"
f.close()
outputName = asksaveasfilename( filetypes=[
('Brawl Resource Archive','*.brres')
] , )+'.brres'
f = open(outputName, 'wb')
print "Saving Output..."
f.write(newdata.decode('hex'))
f.close()
print "Output Saved!"
quity = input("Press enter to quit!")

Of course, that's just what I have so far.
The problematic part is that the output from this program differs from the fixed product of the existing MDL0 header fixer :/

[DarkPikachu]

that's some complicated code you got there
it'd be a shame if something were to happen to it... lol

but I see you're storing the file in a var...
that might be your complication...

you'll notice I'm just using:
fr.read(4)
fr.read(4)
fr.read(4)

you could use:
fw.write(var1)
fw.write(var2)
fw.write(var3)

I find it alot more simpler and organized to do than all that fancy code-work with a stored data var

I'm not too sure of your prob since I'm seeing that :/
that would take me a while to figure out...

I'm on a minecraft server right now,
so I'm not too into the coding stuff...

doesn't mean I can't lend over some info though

use my code to figure out how to write the specific data types to the header
(weather to write int(), float(), long(), etc...)
glad your using len() for alot of it

btw, been wanting to say this for so long:
your ava is freakin WIN!!! <3 <3

[Difegue]

I don't know if it can be of any use to you, but I have the Game Maker source files of FortWaffles' original programs >.> (GM8 decompiler works wonders)

[DarkPikachu]

I don't know if it can be of any use to you, but I have the Game Maker source files of FortWaffles' original programs >.> (GM8 decompiler works wonders)

I can haz plz =D

I've been wanting to look at FW's src codes for quite a while

[Difegue]

Sure, I'll copy-paste the scripts... They're written in Game Maker Code, so ask me if you don't understand a function or something:

mdl0headerfix

Code:

//Initialisation
aaa=0
bbb=0
global.mdl0file=file_bin_open(get_open_filename('Brres File|*.brres',m),2)
curpoly=0
curmatr=0
//script_execute(getoffs)
//script_execute(getnumbers)
state=0
curbone=0
curvertex=0
vvv=file_text_open_write(program_directory+"/game.txt")

Code: (Getoffset)

//def
file_bin_seek(global.mdl0file,mdl0spot+16)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.defoffset=aaa+bbb+ccc+ddd
//bones
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.boneoffset=aaa+bbb+ccc+ddd
//vertexs
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.vertexoffset=aaa+bbb+ccc+ddd
//material1
file_bin_seek(global.mdl0file,mdl0spot+40)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.matoffset=aaa+bbb+ccc+ddd
//polygon
file_bin_seek(global.mdl0file,mdl0spot+48)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.polyoffset=aaa+bbb+ccc+ddd
state=2

Code: (Getnumber)

//material1
file_bin_seek(global.mdl0file,global.matoffset+4+mdl0spot)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.nummat=aaa+bbb+ccc+ddd
//polygons
file_bin_seek(global.mdl0file,global.polyoffset+4+mdl0spot)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
global.numpoly=aaa+bbb+ccc+ddd
tempcount=0
tempcount2=0


state=6

Code: (getmdl0)

aaa=file_bin_read_byte(global.mdl0file)
if(aaa==77){
tempcount=file_bin_position(global.mdl0file)
aaa=file_bin_read_byte(global.mdl0file)
if(aaa==68){
aaa=file_bin_read_byte(global.mdl0file)
if(aaa==76){
aaa=file_bin_read_byte(global.mdl0file)
if(aaa==48){
mdl0spot=tempcount-1
set_automatic_draw(true)
state=1
}
}
}
}

Code: (Checkmove)

tgt=0
tempcount=0
movetype=0
file_bin_seek(global.mdl0file,mdl0spot+40)
aaa=0
bbb=0
while(bbb<8){
aaa+=file_bin_read_byte(global.mdl0file)
bbb+=1
}
if(aaa>0) tgt=1
file_bin_seek(global.mdl0file,mdl0spot+68)
bbb=0
while(bbb<4){
aaa+=file_bin_read_byte(global.mdl0file)
bbb+=1
}
if(aaa>0) movetype=1
state=3

Code: (Fixheader)

file_bin_seek(global.mdl0file,mdl0spot+8)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,9)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,128)
file_bin_seek(global.mdl0file,mdl0spot+80)
if(movetype==1) file_bin_seek(global.mdl0file,mdl0spot+76)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,192)

state=4
if(tgt==1) state=5

Code: (moveheader)

qqq=file_bin_position(global.mdl0file)

file_bin_seek(global.mdl0file,mdl0spot+48+tempcount)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
file_bin_seek(global.mdl0file,mdl0spot+40+tempcount2)
file_bin_write_byte(global.mdl0file,aaa)
file_bin_write_byte(global.mdl0file,bbb)
file_bin_write_byte(global.mdl0file,ccc)
file_bin_write_byte(global.mdl0file,ddd)
tempcount+=4
tempcount2+=4
if(tempcount==20){
if(movetype==0) tempcount2-=4




}
if(movetype==0){
if(file_bin_position(global.mdl0file)==global.defoffset+mdl0spot-12){
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
state=5
}
}
if(movetype==1){
if(file_bin_position(global.mdl0file)==global.defoffset+mdl0spot-8){
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
file_bin_write_byte(global.mdl0file,255)
state=5
}
}

Code: (Fixpoly)

file_bin_seek(global.mdl0file,mdl0spot+global.polyoffset+36+(curpoly*16))
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
curpolyoffset=aaa+bbb+ccc+ddd

file_text_write_real(vvv,mdl0spot+global.polyoffset+curpolyoffset)
file_text_writeln(vvv)
file_bin_seek(global.mdl0file,mdl0spot+global.polyoffset+curpolyoffset+96)
aaa=file_bin_read_byte(global.mdl0file)
if(aaa==255){
while(eee<>2128){
file_bin_seek(global.mdl0file,mdl0spot+global.polyoffset+curpolyoffset+100+tempcount)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)

file_bin_seek(global.mdl0file,mdl0spot+global.polyoffset+curpolyoffset+96+tempcount2)
file_bin_write_byte(global.mdl0file,aaa)
file_bin_write_byte(global.mdl0file,bbb)
file_bin_write_byte(global.mdl0file,ccc)
file_bin_write_byte(global.mdl0file,ddd)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
eee=aaa+bbb+ccc+ddd
tempcount+=4
tempcount2+=4
}
file_bin_seek(global.mdl0file,file_bin_position(global.mdl0file)-4)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
}
curpoly+=1
tempcount=0
tempcount2=0
if(curpoly==global.numpoly) state=8

Code: (FixMat)

if(curmatr<global.nummat){
file_bin_seek(global.mdl0file,mdl0spot+global.matoffset+36+(curmatr*16))
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
aaa=aaa*256*256*256
bbb=bbb*256*256
ccc=ccc*256
curmatoffset=aaa+bbb+ccc+ddd
file_bin_seek(global.mdl0file,mdl0spot+global.matoffset+curmatoffset+56)
while(file_bin_position(global.mdl0file)<mdl0spot+global.matoffset+curmatoffset+1044){
file_bin_seek(global.mdl0file,mdl0spot+global.matoffset+curmatoffset+60+tempcount)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
file_bin_seek(global.mdl0file,mdl0spot+global.matoffset+curmatoffset+56+tempcount2)
file_bin_write_byte(global.mdl0file,aaa)
file_bin_write_byte(global.mdl0file,bbb)
file_bin_write_byte(global.mdl0file,ccc)
file_bin_write_byte(global.mdl0file,ddd)
tempcount+=4
tempcount2+=4
}
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
file_bin_write_byte(global.mdl0file,0)
curmatr+=1
tempcount=0
tempcount2=0
}
if(curmatr==global.nummat) state=7

Code: (FixLast)

file_bin_seek(global.mdl0file,mdl0spot+128)
aaa=file_bin_read_byte(global.mdl0file)
bbb=file_bin_read_byte(global.mdl0file)
ccc=file_bin_read_byte(global.mdl0file)
ddd=file_bin_read_byte(global.mdl0file)
file_bin_seek(global.mdl0file,mdl0spot+92)
file_bin_write_byte(global.mdl0file,aaa)
file_bin_write_byte(global.mdl0file,bbb)
file_bin_write_byte(global.mdl0file,ccc)
file_bin_write_byte(global.mdl0file,ddd)
game_end()

And the script which is repeating every step:

if(state==0) script_execute(getmdl0)
if(state==1) script_execute(getoffs)
if(state==2) script_execute(checkmove)
if(state==3) script_execute(fixheader)
if(state==4) script_execute(moveheader)
if(state==5) fixlast()

If it's of any use to you, I'll post the codes for the Matrix and Polygon fixers  

[DarkPikachu]

If it's of any use to you, I'll post the codes for the Matrix and Polygon fixers  

all of them plz

hmm, GM code looks like a mix of C++ and Py
interesting