Deciphering the SHP format...

[MarkSheppard]

I have some of fred Chlanda's old notes that came packed with SHPEd, plus the source code for SHPEd v0.50 (see attached).

Here are my notes:

Main File Header (Offset 0) (4 Bytes)
The SHP file itself starts with four bytes that are always "1.10".

SHP Image Count (Offset 4) (4 Bytes)
4 Byte variable giving the number of SHP images in the file.

Table List (Begins at Offset 8)

This lists the address(es) where each image begins and the address of the palette information for that image (this part is generally ignored.)

Essentially, it is a repeating table of two values:

SHP Offset (four byte long integer)
SHP Palette Offset (four byte long integer)

If the SHP file has 10 images, there would be 10 sets of the above, for 20 total values.

Here's an example from a SHP:

256
0
7930
0
10757
0
10853
0

Basically, SHP #1 begins at offset 256, SHP #2 at offset 7930, etc.

The Table List offsets that concern us for Steel Panthers are:

Image #1 Address Offset: 8
Image #2 Address Offset: 16
Image #3 Address Offset: 24
Image #4 Address Offset: 32
…
etc.

A simple equation to calculate where to get the offset from is this:

8 * ImageNumber = Offset Location

Thus, Image 35 would be 8 * 35 = 280 offset.

Image Information (Begins at offset indicated in table for that slot)

Each SHP image begins with a header which is 24 bytes long and stores various parameters such as the image size.

The header is uncompressed and is arranged in the following order:

Height (aka lines) (two byte integer) (this is really the number of lines -1)
Width (two byte integer)
var1 (four byte long integer)
xstart (four byte long integer)
ystart (four byte long integer)
xend (four byte long integer)
yend (four byte long integer)

The header is followed by the actual image data. These headers are very easy “tells” by which to locate SHP data; especially if all the icons in a SHP file are the same standard size: 88 x 88 – then you'd look for the following sequence that indicates the start of a new SHP Icon+Header: 88 0 88 0

*************

This is where I get to the tricky part. SHP raw image data is stored in a compressed format (I think it's Run Length Encoding); same as most other data in Steel Panthers, going back to SP1 itself.

Fred's SHP Decompression/Image Write algorithm in Main.cpp is:

//-----------------------------------------------------------
// read the shp data (for icon n) into the pseudo image
void __fastcall read_shp(FILE *inf, int n)
{
int l; // first line the counter
int lf; // last line
int ch,b,r,i;


fseek(inf,icon_add[n]+HEADER_SIZE,0);
if (header.ystart<0)
{
// a trial
// lf=header.yend+(header.var1>>16);
// l=header.ystart+(header.var1>>16);
l=0; lf=header.yend+abs(header.ystart);
}
else
{
l=header.ystart;
lf=header.yend;
}
if (header.xstart<0)
// a trial
// pix_pos=header.xstart+(header.var1&0xFFF);
pix_pos=0;
else
pix_pos=header.xstart;
do
{
// skip to xstart
// for (i=0; i // put_pix(l,BACK_COLOR);

// read data and decode
ch=fgetc(inf);
r=ch%2;
b=ch/2;
if (b==0 && r==1) // a skip over
{
ch=fgetc(inf);
for (i=0; i put_pix(l,BACK_COLOR);
}
else if (b==0) // end of line
{
++l;
if (header.xstart<0)
{
pix_pos=0;
// a trial
// pix_pos=header.xstart+(header.var1&0xFFF);
}
else pix_pos=header.xstart;
}
else if (r==0) // a run of bytes
{
ch=fgetc(inf); // the color #
for (i=0; i put_pix(l,ch);
}
else // b!0 and r==1 ... read the next b bytes as color #'s
{
for (i=0; i {
ch=fgetc(inf);
put_pix(l,ch);
}
}
} while (l<=lf);
}

I think it works by:

Gets one CHARACTER from the raw SHP file.

Subjects that CHARACTER to a series of tests:

R = CHARACTER Modulus of 2.
B = CHARACTER divided by 2.

Then runs a series of logical tests:

If B = 0 AND R = 1: (Paint one pixel the background color.)

ELSE IF B = 0: (End of that Line of pixels?)

ELSE IF R = 0: (Single Pixel paint)

It gets a bit confusing past that point, as I'm not familiar with C++ and Fred nested things a bit deeply...