.rle format

[The Hairy Beast From The East]

Here's what I've managed to discern of the .rle format. It's quite vague, and there are several fields that I haven't succeeded in locking down. Also, the palette and colour format is giving me no end of grief - I have no idea where the master palette is stored, or how it's created. It has nothing (as far as I can tell) to do with the colorization*.bmp or kingdom_colors.bmp.

However, I'll present what I know in the hopes that someone can build on this.

I'm currently writing this while fighting off starving lions, so there may be a mistake. Or two.




.rle format

Terms used :

byte 1 byte
short 2 bytes
int 4 bytes
stream sequence of bytes

all unsigned, LSB.


Header - 20 bytes
short magic 'RL' magic bytes identifying this file as RLE
int dataLen Length of data (ie, file length - length of header)
int width Width in pixels
int height Height in pixels
int type Image type, or flags (Probably bpp)
short nCol Number of colours in internal palette, or 0. This is usually 0 unless bpp==9.

Palette - nCol * 4 bytes
if nCol > 0
{
int ??? Unknown, followed by nCol entries of
byte,byte,byte,byte pal Palette entry (possibly an RGBA variant)
}

Index - height * 4 bytes
for each line
{
int lineOff Offset to start of line data, from the end of the header
}


Line data - varies
Each line consists of a series of (byte skip, stream data), ending with an optional skip. More formally, (skip, data)...[skip]

skip is simply the number of pixels to jump along the line, can be 0.

data consists of (byte plotType, stream plotData)

if plotType > 0x80 (ie has bit 7 set)
{
This indicates an alpha plot (for example, shadows).
plotLength = plotType - 0x80

If type (in the header above) == 8, then
{
for each of plotLength
{
plotData = (byte colour1, byte colour1, byte alpha).
I'm uncertain as to the exact format of colour1 and colour2. They appear to be pointers into a palette, but I'm unable to find that palette
}
}
else if type == 9
{
for each of plotLength
{
plotData = (byte colour, byte alpha), where colour is probably an index into the palette defined earlier
}
}

}
else if not set,
{
This is a flat color pixel (ie, alpha = 255).
plotLength = plotType

if type (in the header above) == 8, then
{
for each of plotLength
{
plotData = (byte colour1, byte colour1).
}
}
else if type == 9
{
for each of plotLength
{
plotData = (byte colour)
}
}
}


Edit : Note that this holds true only for files where type = 8 or 9. They're by far the most numerous of the files. There are a handful of files of other types (ranging from 3 to 7), and the format is approximately the same (there are fewer colour bytes in plotData, maybe only 1).

.img format
Not much to say here other than rows and cols define how many 'sub-images' are stored in a file. No idea what boundRect is.


Hope this is useful to people. I could really use some pointers with regard to the colours/palette. I'll keep working on it, but I won't be able to spend as much time on it from now on.

Stay beastly, y'all.

[w0lfshad3]

thancks, unfortunately i can't research on it now, nice job :)

[Shealladh]

Cheers, a good start, many thanx :go:

[Olliminatore]

*push*
I could retrace and confirm your instructions!

The first bytes in the header are the data size(but file size -(1024+24)).
The header is 24 bytes, then comes the RGBA-Palette (Red,Green,Blue,Alpha)*256(2^8)
1048=0x418 (0x means HEX)

Than beginns a mystery Block (perhaps a Gray Alpha-Palette, I'm not sure yet).
The first bytes of this Block are equal to his Size.
I found 124(0x7C)=(4*31) with step (31+2)33 * 30(31-1) Image-size 31*31
and 492(0x1EC)=(4*41*3) with step (41+2)43 * 122(3*41-1) Image-size 41*41
and 492(0x1EC)=(4*123) with step (123+2)125 * 122(123-1) Image-size 123*123

So this Blocks are completely equal (for .rle) in same Image-size !!!
Than the DataBlock begins, wich subsist width*height plots.

each plot begins with this 2 bytes (=plotLength)
The plot length is so 2 bytes + WidthPixel bytes
Example PixelWidth = 31, PixelHeight = 31,
so plotLength = 33 and DataBlockLength = plotLength * PixelHeight = 1023

Then finally the File-size is so: 24+1024+124+1023 = 2195 bytes (for a 31*31 *.rle).

Here this example file: \images\units\Icons\battle_view\small_size\spearma n.rle

But I also not completly understand the RGBA Palette yet.

I've also not many time more. I became try program a converter in next days.

[Doux]

By the way, I don't know if anyobdy noticed this already, but there is an invalid shortcut to "ImgViewer.exe" in the images folder, which was probably used to see (&edit?) the .RLE/.IMG files.

[aco]

I hear from acdsee, that can use this formats.

[Olliminatore]

Nonono @aco :wink: only the non standard BMP.

I also searched ImgViewer.exe + RLE + IMG in google.
And found a tool like this, but it's also only a normal picture viewer.

But thx for the tips specially @Dous for IMG(another thread).
My converter is still at the beginning.

[edit] What I forgot to say this .rls from BSS is definitively not a runtime length encoded picture.

[Winrich von Kniprode]

But what your tool will be capable off? A simple converter/decompresser, or an editor too?

[Olliminatore]

I hope that's not requisitely. First only is schemed a converter which can decode and encode rle or convert an reconvert bmp.
So you can use another tool to edit.

[aco]

With this tool i can´t open rle or img files.

[[TGH]Silver Surfer]

is it possible to extract the unit into a spritesheet or many frames of the unit like in AOK and AOE games' SLP extractor/editor? and can you get them out in bmp format one frame by one frame?

[Doux]

We don't have a clue yet of how the .RLE files are built, so we can't extract anything from it yet, [TGH] Silver Surfer, except that the second 8 bytes give the image height and the next 8 the image width. But that can be found in the .IMG files too. It could also be first width, then height, I'm not sure.

[The Hairy Beast From The East]

So, by chance I happen to return to the .rle problem that I started on in 2005, and I discover that, not only am I terribly hairy, I'm also terribly stupid.

Of course it's 565! I have no idea why it didn't click the first time.

So, with that said, here's the second version of what I've managed to discern of the .rle format. It's more solid than the first version, but there are still fields and values that I've not managed to work out.

The format is now detailed enough that a decent programmer could write a converter. I've written an rle2png (and png2rle) converter, and not only do the images look correct when they've been converted from .rle to .png, they still look correct when going .rle to .png to .rle to .png. That'll be up in my next post.

I'm currently writing this while hanging upside-down from a biplane flying over the Atlantic, so there may be a mistake. Or two.


.rle format

Terms used :byte 1 byte
short 2 bytes
int 4 bytes
stream sequence of bytes

all unsigned, LSB.

colour16 is a 565 16-bit colour defined as: rrrrrggggggbbbbb
converted to 24-bit RGB, this is rrrrr000 gggggg00 bbbbb000


Header - 20 bytes
short magic 'RL' magic bytes identifying this file as KoH RLE
int dataLen Length of data (ie, file length - length of header)
int width Width in pixels
int height Height in pixels
int type Image type, or flags (probably bpp). Otherwise, unknown.
short nColours Number of colours in internal palette, or 0. This is usually 0 unless bpp==9.

Palette - (nColours * 4) bytes
if nColours > 0
{
int Unknown. Maybe an alpha component, maybe gamma, maybe nothing.

Followed by nColours entries of
{
byte (blue)
byte (green)
byte (red)
byte (alpha?) I'm still not certain this is an ordinary alpha value
If you treat it as a straight 0-255 alpha, the images come out very dark and 'washed out'
If you ignore it, and force alpha=255, images come out with the correct colours, but no soft shadows.
If you invert it (ie, transparency instead of opacity), the colours are correct, but again, very dark.
At the moment, I think it's a threshold value related to the Unknown int above

*** Aha! It appears we should ignore it, and instead use the alpha specified in the plotData stream (see later)
}
}


Index - (height * 4) bytes
for each line
{
int lineOff Offset to start of line data, from the end of the palette (if palette present), or header (if no palette).
}


Line data - varies
Each line consists of a series of (byte skip, stream data), ending with an optional skip. More formally, (skip, data)...[skip]

skip is simply the number of pixels to jump along the line without plotting anything, can be 0.

data consists of (byte plotType, stream plotData)

paletteIndex is an index into the palette defined earlier

if plotType >= 0x80 (ie has bit 7 set)
{
This indicates a translucent series of pixels (for example, shadows).
plotLength = plotType - 0x80 (note, can be 0 after the subtraction)

If type (in the header above) == 8, then
{
for each of plotLength
{
plotData = (byte alpha, short colour16).
}
}
else if type == 9
{
for each of plotLength
{
plotData = (byte paletteIndex, byte alpha)
}
}
}
else if not set,
{
This is an opaque pixel sequence (ie, alpha = 255).
plotLength = plotType

if type (in the header above) == 8, then
{
for each of plotLength
{
plotData = (short colour16).
}
}
else if type == 9
{
for each of plotLength
{
plotData = (byte paletteIndex)
}
}
}


Edit : Note that this holds true only for files where type = 8 or 9. They're by far the most numerous of the files. There are a handful of files of other types (ranging from 3 to 7), and the format is approximately the same (there are fewer colour bytes in plotData, maybe only 1).


In summary, this is what I don't understand. If anyone is feeling adventurous, they can have a go at answering some of these questions.

The 'type' int in the header. What values can it take? How does it relate to the number of colours? What sort of .rle uses which type of image? Is it a bitfield?
The Unknown int in the palette. What is it for? How does it vary? What effect - if any - does it have on the following palette values?
The alpha bytes in the palette. Are they really connected to alpha? Why don't they produce 'real' coloured images? If they're to be ignored, why are they there? Are they related to gamma?


Stay beastly, y'all.

-- -- -- --

(Hmm. It appears I can't post attachments. Someone give the beast a hand?)

Here as promised, is my converter. It will take a (type = 8 or 9) .rle file, and create a 32-bit .png file, or take a 32-bit .png file and create a (type = 8, ie, non-paletted) .rle file.

Bear in mind this is a beta, it has many flaws and limitations, but I wanted to post it now to allow people to experiment. There may be another version.


To use the converter, you'll need a version of Java installed. To see if you already have this installed (you probably have), go to a command prompt (Start -> Run, and type in 'cmd.exe' (minus quotes)), and type in java -version

You should see something like java version "1.4.2_12"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_12-b03)
Java HotSpot(TM) Client VM (build 1.4.2_12-b03, mixed mode)

The important thing is the version number in the first line. It needs to be equal to, or greater than, 1.4.2.

If the version number is too low, or you get an error "'java' is not recognized as an internal or external command, operable program or batch file.", go here http://java.com/en/download/index.jsp

Right. Good. Download the .zip file containing the compiled version of the converter (not the file that ends with '.java'), and open it. Inside you'll find 3 .class files, and 1 .bat file. Save all the files somewhere. Open the .bat file with your favourite editor, and replace replace_this_with_the_directory_of_the_class_files with... well, with the directory containing the .class files.

You can now copy the .bat file to whatever directory you're working in, and it will be easier to use than the more complicated java command line.

All the following instructions assume you're using the rle2png.bat file.

Is it working?
Now, from your command window, go into your KoH directory (I suggest somewhere like \Knights Of Honor Data\images\units\Varangian).

General format is rle2png -v (name of the rle or png file you want to convert)
ie,
rle2png -v varangian_Taunt.rle

You should see something like this appearinput=varangian_Taunt.rle
output=varangian_Taunt.png
Reading header...
Reading palette... ... done
Reading index... doneCheck the directory, and you should find a varangian_Taunt.png file there. Load it into your favourite image editor. (Most editors these days support .png. If yours doesn't, check for a later version).

To see brief instructions on how to use it, type rle2png

I recommend you don't use -vvv unless you want to compare output against your own code. -vv is fine for most uses.



I've also included the source code, but no instructions. If you want the source code, you'll know how to compile it. It compiles fine under 1.4.2, with the standard classes. Source is (barely) commented, and completely free to use with the condition that if you fix any bugs, or implement any extensions, you post details to this thread. I'll probably post a neater, better-commented version in a few days.


Disclaimers! Oh, so many!

Consider this a first draft - it's really, really, really late here, and I just want to get this out into the world. A better version may follow.

The code was written while juggling old dynamite, and it's HORRIBLE. Do not, under any circumstances, use this to learn Java.

It has been tested - only briefly - but I'm posting it here now so that people can use it.

As mentioned in my previous post, it converts from rle to png fine, but I strongly doubt that it creates .rle files that will work in KoH. Do NOT spend hours and hours drawing new units only to find that you cannot import them back into KoH.

It does not handle all types of .rle files. It does deal with the majority though.

It doesn't handle wildcards. You can't convert *.rle, for example. Maybe in the next version.



Please post if you find bugs. I'll need to know the error message, and the file you tried to convert.

Finally, while I'll occasionally post newer versions with fixed bugs, I will almost certainly not come back to this code again. It's presented here in the hope that someone can understand and use it. Hopefully someone will recode it in C/C++ to build a native rle2png.



Stay beastly.

[Edit by Angryminer]
Attached files rle2png.zip (http://forum.sunflowers.de/attachment.php?attachmentid=3417) and rle2png.java (http://forum.sunflowers.de/attachment.php?attachmentid=3418) (appended .txt to allow it to be uploaded to the forum)

[Doux]

Excellent! The questions you mention seem difficult, but maybe some experiments could answer them.

I compliment you with your technical, though humouristic style of writing :go:

You can ask and send your file to any mod that will be active later today: the Webmaster, Angryminer or Dobber, and they'd undoubtedly be willing to attach it to your post.

[Angryminer]

Looks good.
I think you can also ask Frujin about hints on the missing parts. He may take a while to respond, but he's always been very helpful.
For loading the file up, either contact me and I can attach the file, or load it up to a hosting service. :go:

Angryminer
--
Attachments uploaded to the post above.

Angryminer

[Doux]

Ah - beautiful. It works very well! There are only a few that do not (e.g. wolf, maybe the only one). Finally we can see the peasants with swords and shields shouting their warcry. Strange that the wolf doesn't work. -vvv option holds at the ncolours output (no output).

[[TGH]Silver Surfer]

Hi there,

I have tested the rle tool and it is a great progress from what I expected. would like to suggest a few points which might make the next version of the RLE tool better hopefully

1) Add in a player colour feature - right now modding the units is quite a pain as the palettes' colours are not very useful. I mean that the shadows of the units and the units' "player colours" are off the same colour value. so I cannot do much colour editing because of this. would be good if the player colours could be changed in the program initially during extraction say to purple (as it is used in most games as initial player colour)

2) Frame extractor - It might be helpful to have a program or tool in the RLE tool which is able to cut out the movements in single BMPs for editing or creation of gifs. Otherwise, we have to cut them out one by one ^_^.

3) Would buildings be able to be used with this tool? I have not tried it yet but if it is possible, It would be great

4) additional help in adding in customized created units say from AOK or AOE or even games like LEGION and CHARIOTS of WAR.




thanks for the great tool ^_^

[Partizanac]

I don't understand
Is this program finished? I've downloaded the files from this thread, and they don't have the icon for starting the program :(

[Doux]

I don't understand
Is this program finished? I've downloaded the files from this thread, and they don't have the icon for starting the program :(It is finished partly: it only works for type 8 and 9 .rle files as stated in this thread. For instructions on use see http://forum.sunflowers.de/showthread.php?p=268592&highlight=use+converter#post268592

[Partizanac]

OK I've done everything from instruction, and an error occurs:
D:\Rle Editor\rle2png -v slav axeman_Fight.rle
Exception in thread "main" java.lang.NoClassDefFoundError: Editor
Please help