[2000-07-04]

Baldur's Gate: File Format Hacking Project

Site news

2000-07-03 This page is now officially dead. The project has changed its name to the Infinity Engine File Format Hacking Project and is now lead by Jed Wing. This page will remain here for historical reasons only. I repeat, the information on this page is to be considered extremely outdated. For current information go to the new homepage.
2000-06-03 This page is now extremely outdated. Jed Wing is about to take over as maintainer. For the latest on BG-hacking check out Infinity Explorer. 2000-01-12 Small additions.
1999-09-10 First pass of table optimization. Contact info changed. New tools linked.
1999-09-02 Petr updated the page in the sections AREA (huge update), BAM and ITM.

[ file formats ] [ in-depth discussions ] [ credits & contact info ] [ tools / downloads ] [ tables ]

Introduction

If you are looking for the Baldur's Gate Mapping Effort - BAGME - you have the wrong address. That page has moved here. This page will instead focus on what gamehacking ought to really be about, namely hexeditors, calculators and lot's and lot's of notes with hexadecimal offsets :-)

I (Eddy L O Jansson, I'm authoring this page) and my friend Robert Risberg will here try to disclose everything we know about the inner workings of Baldur's Gate. Since 1999-05-10 we have also been joined by Petr Zahradnik who's been injecting many fresh ideas into this project, and who deserves equal credit.

Note that we will often sound as if we are stating a fact when we're actually just stating what we currently hold for true. This is just a convention so that we can keep the technical parts of this paper reasonably brief and to the point without lotīs of "we believe", "as far as we know", et cetera. You will also see me go back and forth between the we and I form as I move from something we know/don't know to my personal opinions which are to be held separate from those of the other participants of this project..

It has been our intention from the very start to hack the internals of BG. We talked about it even before the game was out. Personally though, I didn't want to start hacking until after I'd completed the game as looking through the datafiles will ruin the overall experience. Now (1999-04-04), I'm done and can proceed to hack.

Naturally, I took the time to look at the current state of BG-hacking, only to find lot's of executables and no real useful information. I think it's sad that this community seem to be so focused on thinking "proprietary" (there are some notable exceptions; see the credits section), when sharing information would be much more efficient and fun. To counter this we will take the other path; less executables and more information. All we would ask in return is that - if you use this information to build a tool - that you please greet us somewhere in the documentation and/or in the program itself.

Another exception is the fact that we are not doing this in order to cheat - which should be obvious if you study which structures we've worked on so far - we do this because it's a challenge, because it's fun and because we really enjoy games, game development and reverse-engineering. If you simply want to cheat, please go somewhere else, and please don't bother us with questions about cheating. Thank you.

General

Baldur's Gate is scripted internally using the language Lua, developed by TeCGraf at the Pontifical Catholic University of Rio de Janeiro in Brazil. Some knowledge of this language will be useful for the serious hacker. You can read the reference-manual online, it will take you a couple of hours if you read it all, much less if you just browse the highlights; datatypes, how to adress tables, the preprocessor, comments. I am not sure if the scripting available to players is also based on Lua, but it seems probable.

When we adress individual bits, we count with bit 1 being the least significant bit at the far right, with increasing value as we move from right to left.

There's much research yet to be done.

File Formats

BG use a number of different files, and each can contain may different resources. We will go thru them and disclose what we know so far.

    BIFFV1

    One of the primary file formats are that of BIFFV1 (extension .bif). These files are resource-files of the type that became      popular with the .WAD files of Doom et al, in other words, the act as a container for other (much smaller) resources. It is obvious that the bif-files of BG are to remain static and that they will not be affected by patches. For one thing, they mostly remain on your CD :-), but I was thinking more on their structure; having the catalog at the top of the file, something which is a really bad idea if you want to add entries. Instead there's an override directory which when a resource is sought will get searched first, only if the resource is not found there the game will reach for the bif-file.

    One common BIFF is the AREAxxxx.BIF. These files contain the basic information about an area, three smaller BMP images of which is probably used for the sound-system (very smooth but crude approximation of the area layout), one is used as a map over where you can walk (looks hand drawn) and the last one.. well, we don't know. You will also find a couple of other chunks, and at the very end, all the graphics for the map as seen by the player (we've got some info on the format of that, stay tuned).

What BIFF stands for? If it is an acronym, it could stand for Baldurīs Gate Interchangable File Format

This is what we know about these files.

    The following structure is unique to BIF's containing ARxxxx resources. The starting offset of this list must be calculated using the formula 0x14+(TBIFHeader.CatEntries*0x10). Not yet fully researched. See also MOS.

[ back to index ]

    TLK V1

    Resource containing dialogue, descriptions and names of objects; items, NPC, etc. Various items and functions will then reference these strings using the zero-based index of the TTLKEntry.

[ back to index ]

    KEY V1

    An index over where a resource file (.bif) can be found, and what particular resources it contains. Only 2+4+4 bits remain unaccounted for.

[ back to index ]

    SAV V1.0

    Holds the state of the world. Consists of a short header followed by one or more TSAVResourceEntries. Typically found in a 'baldur.sav' file, located in the path of a save-game. We believe this one contains local data, that is, information in one of these chunks are only accessed when you actually enter the area in question.

    Each of the entries of this file stores the state of one area or store. One of the things stored here is what parts of the area you have uncovered.

[ back to index ]

    GAME V1.1

    Another resource found in save-games, this one is found in the file 'baldur.gam'. We believe this one stores global data, that is, data that must be available to the engine at all times. Your party is saved out in this file, and it also contains a list of all major actions you've taken throughout the game.

    These global variables below are set using code like SetGlobal(ActionID,"GLOBAL",value) or SetGlobalTimer(ActionID,"GLOBAL",EIGHT_DAYS).

[ back to index ]

    CHR V1.0

    Character resource. Found as a small header preceeding the CRE V1.0 resource of exported characters. Believed to be fixed size; 0x64 (100) bytes.

    CRE V1.0

    Creature resource. Holds information about stats, inventory and everything else that is your character. Variable in size depending on the spells and inventory of the creature. This will probably result in this resource being split in five or more different TCRE-types (note: this is a low-priority structure for us).

[ back to index ]

    ITM V1

    Items. Used for anything you can carry.

    Only valid for weapons? No. Describes item abilites (if any). Detected ability types: Weapon,Spell,Missile,Launcher,and 0-'InGuiAbillity - Learn Spell etc.' . This may be valid only for Baldur's Gate and not for TOTSC, because of the new resource  type EFF :-)

    TITMEffectEntry is probably variable record, which depends on Effect.

[ back to index ]

    AREA V1.0

    An important resource. Defines the contents of an area; the location of exits (doors), signs, triggers, and other things that interact with your character in one way or another.

[ back to index ]

    DLG V1.0

    This resource controlls the flow of dialogue.

[ back to index ]

      BAM V1

    This format is used for storing multiple animations, which could share the same frames and same palette. Even though this is a format for animations, some files contain only one frame, or a set of static images.  

    After the header comes FrameCnt of TBAMFrameHeaders as specified below.

    The Xpos/Ypos are coordinates of top left hand corner of the frame, relative to the reference point (usually that cross between foots). That means that both Xpos and Ypos should be negative. Width and height is given only for each frame, this means you must calculate the maximum width/height of the animation.

    After those frame headers comes AnimCnt of TBAMAnimHeaders:

    After those animations headers comes one TPalette (standard 0x400 bytes). Then comes TBAMAnimArray. 

    TBAMAnimationArray contains a lots of FrameID's. Exactly Sum(TBAMAnimHeader.Frames). To play animation simply
display frames from TBAMAnimArray[Start] to TBAMAnimArray[Start+Frames]. (maybe :-)

    And then finally comes the graphics data itself, which uses a special kind of run length transparency encoding where the byte 0x00 is special and codes for the byte after it meaning either a) the number of transparent pixels to emit or b) if the following byte is zero also, emit the entries for palette[0].

[ back to index ]

    WAVCV1.0

    Compressed sound resource..

[ back to index ]

    STORV1.0

    Stores and what they sell. Called up using code like: StartStore("tav0114",LastTalkedToBy())

    List of items available in the store.

    List of drinks available for order.

    List of spells available. Temples only (see StoType).

[ back to index ]

    MOS V1.0

    This structure contains the graphics for the mini-map of an area, and also for various other parts of the game interface. The format have some things in common with the 0x3EB format used for the huge background graphics. The format is one TMOSHeader, followed by a number of palettes, followed by an index of pointers to TGraphicsBlocks. These blocks shall then be arranged in a left-to-right, line by line order when presented. Like this:

    The size of the 'border' blocks must be calculated from the width and height fields available in the TMOSHeader.

    After this header comes the array of TPalette-structures (RGBA*256). There is one such for each graphical datablock in the file as can be computed from columns*rows. This is the format of the palettes used in MOS and other structures. (for n=0 to 0xff):

    Immediately after that comes an index of pointers to TGraphicBlocks, relative from the start of data chunk. One such block consists pixel-data, usually 64*64 bytes but it can be less (see diagram above). This means the maximum size of one of these blocks are 0x1000 bytes. (I may expand on all this later as I have my own implementation written).

[ back to index ]

    WMAPV1.0

    Not much to say, this structure is used for the world map and your progress (areas discovered) in the world is stored here. You find it primarily in a file called worldmap.wmp in your savegame directory, which is built from the worldmap available in default.bif. As usual, all offsets are relative to the root-header, in this case the WMAPV1.0 structure.

    I am not sure these should really be split up, but let's work from that assumption for now since it explains the header above and also makes sense from a design-point-of-view.

    Work in progress.

[ back to index ]

    WED V1.3

    This structure defines various things about an area, for instance, it contains a list of which graphical 0x3EB resources to load.

[ back to index ]

In Depth Discussions and Clarifications

Under this heading I will dwell deeper into some areas that may need explaining.

    Encryption

    For some reason some of the 0x3F0 and 0x3F4 resources are encrypted. I am not sure why Bioware did this, especially      why they chose not to encrypt all resources, but just a selected few. I have as of now (1999-05-03) modified biftool to decrypt all encrypted resources of those types on-the-fly. You can disable this behaviour using the option -nodecrypt.

    The files are encrypted using a polyalphabetic cipher, using the exclusive-or operator and a running 512-bit key. Here is the key in hexa for you: "88 A8 8F BA 8A D3 B9 F5 ED B1 CF EA AA E4 B5 FB EB 82 F9 90 CA C9 B5 E7 DC 8E B7 AC EE F7 E0 CA 8E EA CA 80 CE C5 AD B7 C4 D0 84 93 D5 F0 EB C8 B4 9D CC AF A5 95 BA 99 87 D2 9D E3 91 BA 90 CA". A correct binary key will CRC32 to 0xEAA0399F using the standard polynomial. Any questions? Just mail me.

    If you use this, don't forget to credit us (with a link, thank you) for the research. You know you will feel very bad if you don't and instead try to pass this off as your own work.

    Time

    In the world of the Infinity-engine one day is 7200 units. This means that one hour is 7200/24=300 units, and thus 60 game seconds, which is one round, represents 5 units. We come to the conclusion that the engine is working with a 60/5 = 12s in-game-exposed-to-the-outside granularity, which in real world time is about 1.2s. I'm sure I got something wrong there, but the basic fact of 300 units/hour is correct, you can derive the rest yourself. The thing when computing the time from the TGAMHeader is that there's an extra 7 game-hours (=2100 units) for which you must account. Let me explain; Say the header reads 0x030201 and you want to calculate the time as shown by the load screen. First you must subtract the extra seven hours which gives you 195021. The number of days is then (195021/7200)=27 and the number of hours is ((195021/300) mod 24)=2. 27 days and two hours. See, magic! I'm sure you are very happy that I took the time to explain these higher maths for you ;-)

Tools

Here is a link to Petrs newer utilities. Below are my older command-line tools.

biftool v0.0.62 (~14Kb) Simple *.bif resource extractor. Example usage: 'biftool -extr 1 GUI.bif'
keytool v0.0.1 (~8Kb) Use this to reverse the bitmaps of the chitin.key file. Usage: 'keytool >keytool.txt'
talktool v0.0.4 (~8Kb) For us hackers only :-) 'talktool >talktool.txt'
itmtool v0.0.2 (~8Kb) Dumps the descriptions of an .itm file.

Various Tables

Some relevant tables.

    The table below describes who cannot use an item.